- Green versionCheck
- Green versionCheck
- Green versionCheck
Note: Version 1.2 of the data collector is not compatible with configuration scripts from previous versions.
It has strong flexibility and is capable of handling various websites with different structures. If you have any questions during use, please contact: Discussion group: 601169305 QQ: 2146651351 WeChat: AIChangeLife
How to use:
1. First analyze the web page structure of the website to configure the crawling strategy (there is a comprehensive process description, if you don’t understand, refer to the video tutorial).
2. Run the script to perform data processing.
3. Scrapy development auxiliary support, including generating Scrapy code snippets and directly inputting the Scrapy Xpath path into the Scrapy script.
4. HTTP request log data output methods include:
1. Output through CSV files. Editing software such as EXCEL and most database software support importing from CSV files.
2. Output SQLITEBrowser through the program’s built-in database (SQLITE) to support SQL statement query of SQLITE database: http://sqlitebrowser.org/ https://github.com/sqlitebrowser/sqlitebrowser/releases/download/v3.10.0/DB.Browser. for.SQLite-3.10.0-win32.exe has extensive support and many options
3. Output through the MYSQL database (after downloading the official MYSQL installation package, you can install it directly), and the data is saved in UTF-8.
it works
it works
it works