Scrapy project

scrapyd-client Public
Command line client for Scrapyd server

Python 632 132 10 1 Updated Dec 28, 2021
scrapyd Public
A service daemon to run Scrapy spiders

Python 2,392 538 50 20 Updated Dec 28, 2021
scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 42,409 9,447 492 (31 issues need help) 310 Updated Dec 25, 2021
scrapy.org Public
The scrapy.org website

HTML 45 148 1 (1 issue needs help) 1 Updated Dec 23, 2021
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python 731 108 28 (1 issue needs help) 14 Updated Dec 22, 2021
w3lib Public
Python library of web-related functions

Python 339 89 15 (3 issues need help) 10 Updated Dec 16, 2021
itemadapter Public
Common interface for data container classes

Python 29 BSD-3-Clause 6 2 2 Updated Oct 12, 2021
protego Public
A pure-Python robots.txt parser with support for modern conventions.

DIGITAL Command Language 23 BSD-3-Clause 18 1 (1 issue needs help) 1 Updated Oct 7, 2021
cssselect Public
CSS Selectors for Python

Python 246 53 13 10 Updated Aug 30, 2021
queuelib Public
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

Python 224 49 2 2 Updated Aug 26, 2021

Popular repositories