crawler

Scrapy, a fast high-level web crawling & scraping framework for Python.

👾 Fast and simple video download library and CLI tool written in Go

Elegant Scraper and Crawler Framework for Golang

Python爬虫代理IP池(proxy pool)

A Powerful Spider(Web Crawler) System in Python.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、微信读书、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

A scalable web crawler framework for Java.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Incredibly fast crawler designed for OSINT.

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

基于搜狗微信搜索的微信公众号爬虫接口

Distributed crawler powered by Headless Chrome

A collection of awesome web crawler,spider in different languages

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Redis-based components for Scrapy.