您当前的位置：首页 > 爬虫

爬虫第四课---网页解析

壹小俊发布时间：2019-04-10 21:31:56 ，浏览量：3

BeautifulSoup4的使用(文档https://www.crummy.com/software/BeautifulSoup/bs4/doc/index.zh.html)

1.安装 pip install BeautifulSoup4

'''
bs4的使用
'''
import re
from bs4 import BeautifulSoup

#测试的html
html_doc = """
The Dormouse's story

The Dormouse's story

Once upon a time there were three little sisters; and their names were
Elsie,
Lacie and


    
        
            
                
                
                    壹小俊
                    暂无认证
                
            
            
                
                    
                        3浏览
                        0关注
                        873博文
                        0收益
                    

                    
                        0浏览
                        0点赞
                        0打赏
                        0留言
                    
                
            
            
                私信
                关注
            

        
        
            热门博文
            
                我是如何走上运维岗位的?
如何从生命周期的视角看待应用运维体系建设?
标准化体系建设(下):如何建立基础架构标准化及服务化体系?
为什么Netfix没有运维岗位?
了解移动App的持续交付生命周期
持续交付中有哪些宝贵数据?
计算资源也是交付的内容
利用Mock与回放技术助力自动化回归
越来越重要的破坏性测试
代码静态检查实践






    [ 申请 ]友情链接：
    
        快连vpn
        快连
        快连vpn
        搜外友链
        笔趣阁
        爱思助手
        ClashX教程
        绘画宝宝
        配音宝宝
    


    
        
            关于我们
            服务条款
            广告服务
            联系我们
            网站地图
            免责声明
            WAP
        
        技术支持：
            武汉快勤科技有限公司
            XML网站地图 
            备案号：鄂ICP备18027844号-9
            
        
    




    
        立即登录/注册
        
    
    
        
        微信扫码登录
    












	    基本
        文件
        流程
        错误
        SQL
        调试
    

		    
    
	请求信息 : 2025-08-19 15:23:56 HTTP/2.0 GET : /home/article/detail/id/386828.html
运行时间 : 0.3414s ( Load:0.0565s Init:0.0020s Exec:0.2561s Template:0.0268s )
吞吐率 : 2.93req/s
内存开销 : 1,923.96 kb
查询信息 : 18 queries 0 writes 
文件加载 : 36
缓存信息 : 5 gets 0 writes 
配置加载 : 132
会话信息 : SESSION_ID=8ghfbogiq4teaebetnv67j95j3
    
    
        
    
	/www/wwwroot/www.chaojiit.com/index.php ( 1.30 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/ThinkPHP.php ( 4.71 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Think.class.php ( 12.32 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Storage.class.php ( 1.38 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Storage/Driver/File.class.php ( 3.56 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Mode/common.php ( 2.82 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Common/functions.php ( 51.07 KB )
/www/wwwroot/www.chaojiit.com/Application/Common/Common/function.php ( 6.83 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Hook.class.php ( 4.02 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/App.class.php ( 12.44 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Dispatcher.class.php ( 15.15 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Route.class.php ( 13.38 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Controller.class.php ( 10.95 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/View.class.php ( 7.96 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/BuildLiteBehavior.class.php ( 3.69 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ParseTemplateBehavior.class.php ( 3.89 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ContentReplaceBehavior.class.php ( 1.93 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Conf/convention.php ( 11.18 KB )
/www/wwwroot/www.chaojiit.com/Application/Common/Conf/config.php ( 1.81 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Lang/zh-cn.php ( 2.57 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Conf/debug.php ( 1.51 KB )
/www/wwwroot/www.chaojiit.com/Application/Home/Conf/config.php ( 0.05 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ReadHtmlCacheBehavior.class.php ( 5.62 KB )
/www/wwwroot/www.chaojiit.com/Application/Home/Controller/ArticleController.class.php ( 6.71 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Model.class.php ( 67.27 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Db.class.php ( 5.70 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Db/Driver/Mysql.class.php ( 8.73 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Db/Driver.class.php ( 41.60 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Cache.class.php ( 3.84 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Cache/Driver/File.class.php ( 5.90 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Template.class.php ( 28.35 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Template/TagLib/Cx.class.php ( 22.62 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Template/TagLib.class.php ( 9.19 KB )
/www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php ( 14.55 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/WriteHtmlCacheBehavior.class.php ( 1.43 KB )
/www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Behavior/ShowPageTraceBehavior.class.php ( 5.27 KB )
    
    
        
    
	[ app_init ] --START--
Run Behavior\BuildLiteBehavior [ RunTime:0.000031s ]
[ app_init ] --END-- [ RunTime:0.000101s ]
[ app_begin ] --START--
Run Behavior\ReadHtmlCacheBehavior [ RunTime:0.000677s ]
[ app_begin ] --END-- [ RunTime:0.000754s ]
[ view_parse ] --START--
[ template_filter ] --START--
Run Behavior\ContentReplaceBehavior [ RunTime:0.000109s ]
[ template_filter ] --END-- [ RunTime:0.000178s ]
Run Behavior\ParseTemplateBehavior [ RunTime:0.022268s ]
[ view_parse ] --END-- [ RunTime:0.022367s ]
[ view_filter ] --START--
Run Behavior\WriteHtmlCacheBehavior [ RunTime:0.000602s ]
[ view_filter ] --END-- [ RunTime:0.000680s ]
[ app_end ] --START--
    
    
        
    
	[2] session_save_path(): open_basedir restriction in effect. File(/var/lib/php/session) is not within the allowed path(s): (/www/wwwroot/www.chaojiit.com/:/tmp/) /www/wwwroot/www.chaojiit.com/ThinkPHP/Common/functions.php 第 1239 行.
[8192] Array and string offset access syntax with curly braces is deprecated /www/wwwroot/www.chaojiit.com/ThinkPHP/Library/Think/Cache/Driver/File.class.php 第 59 行.
[8] Undefined variable: user /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 37 行.
[8] Undefined variable: user /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 97 行.
[8] Trying to access array offset on value of type null /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 97 行.
[8] Undefined variable: user /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 98 行.
[8] Trying to access array offset on value of type null /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 98 行.
[8] Undefined variable: pinglun_list /www/wwwroot/www.chaojiit.com/Application/Runtime/Cache/Home/3c8a1a47a3534a7b1252c226abfc3928.php 第 107 行.
    
    
        
    
	SHOW COLUMNS FROM `configuration` [ RunTime:0.0013s ]
SELECT `value` FROM `configuration` WHERE `name` = 'site_name' LIMIT 1   [ RunTime:0.0076s ]
SHOW COLUMNS FROM `menu` [ RunTime:0.0016s ]
SELECT * FROM `menu` WHERE `fid` = 0 AND `status` = 1  [ RunTime:0.0132s ]
SELECT * FROM `menu` WHERE `fid` = 1 AND `status` = 1  [ RunTime:0.0002s ]
SELECT * FROM `menu` WHERE `fid` = 2 AND `status` = 1  [ RunTime:0.0002s ]
SELECT * FROM `menu` WHERE `fid` = 3 AND `status` = 1  [ RunTime:0.0003s ]
SELECT * FROM `menu` WHERE `fid` = 4 AND `status` = 1  [ RunTime:0.0001s ]
SHOW COLUMNS FROM `article` [ RunTime:0.0263s ]
SELECT * FROM `article` WHERE `id` = 386828 LIMIT 1   [ RunTime:0.0066s ]
SHOW COLUMNS FROM `bloger` [ RunTime:0.0186s ]
SELECT * FROM `bloger` WHERE `id` = 598 LIMIT 1   [ RunTime:0.0121s ]
SELECT COUNT(*) AS tp_count FROM `article` WHERE `bloger_id` = 598 LIMIT 1   [ RunTime:0.0010s ]
SHOW COLUMNS FROM `article_content` [ RunTime:0.0148s ]
SELECT `content` FROM `article_content` WHERE `article_id` = 386828 LIMIT 1   [ RunTime:0.0083s ]
SHOW COLUMNS FROM `article_cate` [ RunTime:0.0112s ]
SELECT `name` FROM `article_cate` WHERE `id` = 40 LIMIT 1   [ RunTime:0.0093s ]
SELECT * FROM `article` WHERE `bloger_id` = 598 ORDER BY view_count desc LIMIT 0,10   [ RunTime:0.0137s ]
    
    
        
    
	    
    
    



0.3414s