Async Python 3.6+ web scraping micro-framework based on asyncio（Python3.6+异步爬虫框架）

Web Scan Lazy Tools - Python Package

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

Crawlzone is a fast asynchronous internet crawling framework for PHP.

🌌 High productivity semi-automatic crawler generator 🛠️🧰

Easily crawl news portals or blog sites using Storm Crawler.

A Python framework to build polite, but tenacious crawlers / scrapers with a MariaDB backend

Serritor is an open source web crawler framework built upon Selenium and written in Java. It can be used to crawl dynamic web pages that require JavaScript to render data.

🔍 A powerful web-crawling framework, based on aiohttp.

基于python协程池、用法灵活的高性能爬虫框架

Useful functions for connecting to the network in the PHP based applications.

A framework incorporating ropensci modules and several API's to crawl bibliographic data

A crawler program to extract all of the data and the price for symbols in the global stock exchange.

Demonstration on how to use the Crawling Framework to setup a simple science news crawler and store results in ElasticSearch. Use this configuration to set up your own crawler.

✨ NodeJs crawling & scraping framework heavily inspired by Scrapy

Domain Discovery for the Sparkler Crawl Environment

A Crawling Framework Based on Data Flow and Decorators

An Android app crawling framework, making automatic crawling mobile apps super easy! (if possible, iOS will be supported after Android version)

python crawl projects, including selenium crawl with requests and crawl with scrapy and all esay for redesign

⚡Lightweight web crawler with powerful plugins!

Phrawl: A web crawling framework in PHP (or it seems so)

A web crawler