Weibo_Hot_Search

Weibo_Hot_Search

It is said that people on the Internet only have seven seconds of memory, but I want to record these seven seconds of memory.

The project has been deployed on the server. It will crawl Weibo's hot search list regularly at 11 am and 11 pm every day, save it in Markdown file format, and then upload and backup it to GitHub. You can download and view it at will.

Don't ask me why I chose the two time points of 11, because I always feel that big events will happen around these two time points.

No matter what the hot searches on Weibo are about family affairs, national affairs, world affairs, or entertainment gossip, I just want to faithfully record it...

Operating environment

Python 3.0+

 pip install requests

pip install lxml

pip install bs4

or execute

 pip install -r requirements.txt

Environment required for installation and operation

run

• Please make sure you have prepared the required running environment
• Run method (choose one)
  1. Run weibo_Hot_Search_bs4.py (new) or weibo_Hot_Search.py ​​in the warehouse directory
  2. Execute python weibo_Hot_Search_bs4.py (new) or python weibo_Hot_Search.py ​​in cmd
• Automatically run: Use Windows Task Scheduler to achieve this

Generate files

After running, a folder named with time will be generated in the current folder, as follows:

 2019年11月08日

(Updated) and a Markdown file named with a specific time in specific hours will be generated, as follows:

 2019年11月08日15点.md

(Continue to update) and a csv file named with a specific time in specific hours will be generated, as follows:

 2020年08月27日00点.csv

Interface source

The public hot search list link on Sina Weibo is used: https://s.weibo.com/top/summary/

statement

All data sources for this project come from Sina Weibo. The data content and its interpretation rights belong to Sina Weibo.

Update information

July 31

1. Added bs4 method
   • Added new file weibo_Hot_Search_bs4.py
2. Optimize data storage format
   • The data of the bs4 method is stored in the ./bs4版数据/ directory. The storage data format is序号-标题-热度（或置顶） . This format is easy to process and facilitates subsequent data visualization and other analyses.
3. Notice
   • Data files are all stored in .md format. It is recommended to use Notepad or similar software to open them.

August 5

1. The bs4 version is modified to .csv storage format, which is more conducive to later data analysis.
2. The new .csv files are stored in bs4[.csv]版数据folder.

August 9

1. Added explanation: bs4[txt]版数据and lxml版数据have been stopped. All new data are saved in bs4[.csv] version data files.

License

GNU General Public License v3.0