bee collector

bee collectorSoftware features

Extremely low resource usage, excellent operating performance, and long-term stable operation

Based on QT and written in native C++, the program consumes very little resources and runs extremely fast, thus supporting more concurrent tasks.

Rich managers, secure account management

The software has four managers: basic manager, collection manager, release manager, and other managers.

Basic managers are mainly account-related managers, which are used to store various account information that needs to be used in the software. The data is encrypted and stored. Including: Cookie management, User-Agent management, database connection management, FTP connection management, SSH connection management, SMTP mailbox account, proxy server management, proxy configuration management, and external program management.

Collection managers are mainly collection-related managers used for collection work. Includes: list page URL plug-in, data processing plug-in, image watermark management, Chinese word segmentation settings, and synonym database management.

Release managers are mainly release-related managers used for release work. Including: publish to site, site publishing module, publish to database, database publishing module, publish to file, publish to plug-in, content publishing plug-in, publish to mailbox, file upload configuration management, file upload plug-in, message notification configuration management, message notification plug-in.

          Other managers, mainly some miscellaneous items. Including: task running management, scheduled tasks, task running statistics, task running log files, etc.

Powerful and flexible functions

Database connection access supports SSH tunnel mode.

Supporting agents makes collection more convenient.

Supports multi-account collection and multi-account publishing.

​ ​ Rich plug-in types. Supports plug-ins in four programming languages: PHP, Python, Nodejs, and Go.

​ ​ ​Powerful image watermark function.

File downloads support resumed downloads.

​ ​ Rich file upload methods. FTP, SFTP, plug-in method, file upload method in site publishing module. Supports uploading files to Alibaba Cloud OSS, Tencent Cloud, and Qiniu Cloud. The FTP method supports resumed downloading from breakpoints.

​​​​​​​​​​​​​​​​​​​​​​​​​​​​​JSON parsing tools, form scraping tools, and more.

Each manager supports import and export.

​​ 1. Added: the execution timeout parameter of the plug-in.

​ ​ 2. Added: URL extraction wildcard rules for Excel file sources.

​ ​ 3. Optimization: Allow manual input of content test URLs.

​ ​ 4. Optimization: The re-extraction content function of secondary processing of label data supports circular labels.

​​ 5. Fix: When the data processing plug-in fails to execute, the task does not stop automatically.

​​ 6. Fixed: The problem that the drop-down box of the new version of Qt drop-down selection box may pop up misplaced.

​ 7. Fix: The task did not stop automatically when the commercial agent API failed too many times.

​​ 8. Optimization: Fixed several other bugs and optimized some functions.