<jupyter_start><jupyter_text>需要用到的库requests库用于请求htmlBeautfiSoup解析html<jupyter_code>from bs4 import BeautifulSoup
import requests
import re<jupyter_output><empty_output><jupyter_text>函数定义用requests库得到html文档用BeautifulSoup解析html文档，并得到 类为 ‘bs4.element.ResultSet' 的对象结果<jupyter_code>def get_html(url):
    response = requests.get(url)
    html = response.text
    return html
def parse_html(html):
    soup = BeautifulSoup(html, 'lxml')
    picUrls = soup.find_all('a','wrap-img')
    return picUrls<jupyter_output><empty_output><jupyter_text>确定爬取页面这里只爬取一个页面的内容我选择 **简书首页** 作为爬取的目标网页爬取首页的图片<jupyter_code>#爬虫入口（单个页面爬取入不入口其实无所谓）
url = 'https://www.jianshu.com/'

html = get_html(url)
imgs = parse_html(html)

#用来存结果的list
urls = []<jupyter_output><empty_output><jupyter_text>得到图片链接 **tag**，**ResultSet** 是 BeautifulSoup 中的对象，详细可见[官方文档](https://www.crummy.com/software/BeautifulSoup/bs4/doc/index.zh.htmlfind-all)<jupyter_code>for tag in imgs:
    #得到ResultSet中的tag
    
    img = tag.find_all('img')
    #得到img标签
    
    for src in img:
        #得到src中，图片的url
        
        url = src['src']
        urls.append('https:'+url)<jupyter_output><empty_output><jupyter_text>以下是一些url<jupyter_code>urls[:3]<jupyter_output><empty_output><jupyter_text>存储图片<jupyter_code>import os 
os.makedirs('./img/', exist_ok=True)
#用 os 模块新建一个img文件夹，如果存在的话就直接使用已经存在的文件夹

for i in range(len(urls)):
    r = requests.get(urls[i])
    #用requests库得到图片
    
    description = 'picture-{}'.format(i)+'.jpg'
    with open('./img/%s'%description,'wb') as f:
        f.write(r.content)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup