爬虫第四课---网页解析

壹小俊发布时间：2019-04-10 21:31:56 ，浏览量：2

BeautifulSoup4的使用(文档https://www.crummy.com/software/BeautifulSoup/bs4/doc/index.zh.html)

1.安装 pip install BeautifulSoup4

'''
bs4的使用
'''
import re
from bs4 import BeautifulSoup

#测试的html
html_doc = """
The Dormouse's story

The Dormouse's story

Once upon a time there were three little sisters; and their names were
Elsie,
Lacie and


    
        
        
        
            最近更新
            
                深拷贝和浅拷贝的区别（重点）
【Vue】走进Vue框架世界
【云服务器】项目部署—搭建网站—vue电商后台管理系统
【React介绍】 一文带你深入React
【React】React组件实例的三大属性之state，props，refs（你学废了吗）
【脚手架VueCLI】从零开始，创建一个VUE项目
【React】深入理解React组件生命周期----图文详解（含代码）
【React】DOM的Diffing算法是什么？以及DOM中key的作用----经典面试题
【React】1_使用React脚手架创建项目步骤--------详解(含项目结构说明)
【React】2_如何使用react脚手架写一个简单的页面？
            
        
        
        
            热门博客
            
                优秀的代码都是如何分层的？
Spring 最常用的 7 大类注解，史上最强整理！
别再用currentTimeMillis统计耗时了，太 Low，试试StopWatch吧！
HTTP 3.0彻底放弃TCP，TCP到底做错了什么？
为什么有些大公司技术弱爆了？
聊聊8 种架构模式，你经过几种？
同事写了一个责任链模式，bug无数...
那些让你起飞的计算机知识。。
3行代码写出8个接口，开挂了？
AI 加持实时互动｜ZegoAvatar ⾯部表情随动技术解析





        
        [ 申请 ]友情链接：
        
            ClashX教程
            绘画宝宝
            配音宝宝
        
    


    
        
            关于我们
            服务条款
            广告服务
            联系我们
            网站地图
            免责声明
            WAP
        
        技术支持：
            武汉快勤科技有限公司
            XML网站地图 
            备案号：鄂ICP备18027844号-9
            
        
    




    
        立即登录/注册
        
    
    
        
        微信扫码登录
    













	    基本
        文件
        流程
        错误
        SQL
        调试
    

		    
    
	请求信息 : 2025-04-22 09:42:22 HTTP/2.0 GET : /home/article/detail/id/386828.html
运行时间 : 0.0394s ( Load:0.0095s Init:0.0010s Exec:0.0182s Template:0.0107s )
吞吐率 : 25.38req/s
内存开销 : 1,923.19 kb
查询信息 : 18 queries 0 writes 
文件加载 : 36
缓存信息 : 5 gets 0 writes 
配置加载 : 132
会话信息 : SESSION_ID=fcrc3i5ite6hsrgid8hns0jr0d
    
    
        
    
	/www/wwwroot/www.chaojiit.com/index.php ( 1.33 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/ThinkPHP.php ( 4.71 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Think.class.php ( 12.32 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Storage.class.php ( 1.38 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Storage/Driver/File.class.php ( 3.56 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Mode/common.php ( 2.82 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Common/functions.php ( 51.07 KB )
/www/wwwroot/www.chaojiit.com/Application/Common/Common/function.php ( 6.52 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Hook.class.php ( 4.02 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/App.class.php ( 12.44 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Dispatcher.class.php ( 15.15 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Route.class.php ( 13.38 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Controller.class.php ( 10.95 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/View.class.php ( 7.96 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/BuildLiteBehavior.class.php ( 3.69 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ParseTemplateBehavior.class.php ( 3.89 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ContentReplaceBehavior.class.php ( 1.93 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Conf/convention.php ( 11.18 KB )
/www/wwwroot/www.chaojiit.com/Application/Common/Conf/config.php ( 1.81 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Lang/zh-cn.php ( 2.57 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Conf/debug.php ( 1.51 KB )
/www/wwwroot/www.chaojiit.com/Application/Home/Conf/config.php ( 0.05 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ReadHtmlCacheBehavior.class.php ( 5.62 KB )
/www/wwwroot/www.chaojiit.com/Application/Home/Controller/ArticleController.class.php ( 6.10 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Model.class.php ( 67.27 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Db.class.php ( 5.70 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Db/Driver/Mysql.class.php ( 8.73 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Db/Driver.class.php ( 41.60 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Cache.class.php ( 3.84 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Cache/Driver/File.class.php ( 5.90 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Template.class.php ( 28.35 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Template/TagLib/Cx.class.php ( 22.62 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Template/TagLib.class.php ( 9.19 KB )
/www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php ( 15.51 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/WriteHtmlCacheBehavior.class.php ( 1.43 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ShowPageTraceBehavior.class.php ( 5.27 KB )
    
    
        
    
	[ app_init ] --START--
Run Behavior\BuildLiteBehavior [ RunTime:0.000009s ]
[ app_init ] --END-- [ RunTime:0.000036s ]
[ app_begin ] --START--
Run Behavior\ReadHtmlCacheBehavior [ RunTime:0.000392s ]
[ app_begin ] --END-- [ RunTime:0.000417s ]
[ view_parse ] --START--
[ template_filter ] --START--
Run Behavior\ContentReplaceBehavior [ RunTime:0.000074s ]
[ template_filter ] --END-- [ RunTime:0.000103s ]
Run Behavior\ParseTemplateBehavior [ RunTime:0.008454s ]
[ view_parse ] --END-- [ RunTime:0.008475s ]
[ view_filter ] --START--
Run Behavior\WriteHtmlCacheBehavior [ RunTime:0.000148s ]
[ view_filter ] --END-- [ RunTime:0.000160s ]
[ app_end ] --START--
    
    
        
    
	[2] session_save_path(): open_basedir restriction in effect. File(/var/lib/php/session) is not within the allowed path(s): (/www/wwwroot/www.chaojiit.com/:/tmp/) /www/wwwroot/www.chaojiit.com/ThinkPHP/Common/functions.php 第 1239 行.
[8192] Array and string offset access syntax with curly braces is deprecated /www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Cache/Driver/File.class.php 第 59 行.
[8] Undefined variable: user /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 34 行.
[8] Undefined variable: user /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 134 行.
[8] Trying to access array offset on value of type null /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 134 行.
[8] Undefined variable: user /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 135 行.
[8] Trying to access array offset on value of type null /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 135 行.
[8] Undefined variable: pinglun_list /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 144 行.
[8] Undefined variable: top_list /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 179 行.
    
    
        
    
	SHOW COLUMNS FROM `configuration` [ RunTime:0.0004s ]
SELECT `value` FROM `configuration` WHERE `name` = 'site_name' LIMIT 1   [ RunTime:0.0002s ]
SHOW COLUMNS FROM `menu` [ RunTime:0.0002s ]
SELECT * FROM `menu` WHERE `fid` = 0 AND `status` = 1  [ RunTime:0.0002s ]
SELECT * FROM `menu` WHERE `fid` = 1 AND `status` = 1  [ RunTime:0.0001s ]
SELECT * FROM `menu` WHERE `fid` = 2 AND `status` = 1  [ RunTime:0.0001s ]
SELECT * FROM `menu` WHERE `fid` = 3 AND `status` = 1  [ RunTime:0.0001s ]
SELECT * FROM `menu` WHERE `fid` = 4 AND `status` = 1  [ RunTime:0.0001s ]
SHOW COLUMNS FROM `article` [ RunTime:0.0003s ]
SELECT * FROM `article` WHERE `id` = 386828 LIMIT 1   [ RunTime:0.0002s ]
SHOW COLUMNS FROM `bloger` [ RunTime:0.0002s ]
SELECT * FROM `bloger` WHERE `id` = 598 LIMIT 1   [ RunTime:0.0002s ]
SELECT COUNT(*) AS tp_count FROM `article` WHERE `bloger_id` = 598 LIMIT 1   [ RunTime:0.0002s ]
SHOW COLUMNS FROM `article_content` [ RunTime:0.0002s ]
SELECT `content` FROM `article_content` WHERE `article_id` = 386828 LIMIT 1   [ RunTime:0.0001s ]
SHOW COLUMNS FROM `article_cate` [ RunTime:0.0003s ]
SELECT `name` FROM `article_cate` WHERE `id` = 40 LIMIT 1   [ RunTime:0.0001s ]
SELECT * FROM `article` WHERE `bloger_id` = 598 ORDER BY view_count desc LIMIT 0,10   [ RunTime:0.0039s ]
    
    
        
    
	    
    
    



0.0394s