Create agents that monitor and act on your behalf. Your agents are standing by!

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Ruby gem to inspect completely a web page. It scrapes a given URL, and returns you its meta, links, images more.

Crawl instagram photos, posts and videos for download.

A survey of Apple developer documentation.

IF (接口/网页 有变化) THEN (提醒你)

📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.

MangaScrap is a Ruby based scrapper that generates a local website with the downloaded mangas

🕸 Builds and serves RSS feeds via HTTP. Generate your own feeds or start instantly with the included configs.

Fetch latest fuel prices for major Indian cities

🧐 Scrape your friends' Facebook photos

Crawls Media Bias/Fact Check and saves output to JSON.

Proof of concept for a security issue (in my opinion) that I found in accounts.google.com

A fairly simple gem that will help you simplify the parsing of web pages.

The program is implementing a scraper for search results of a Job offer website. This scraper helps you to (i) Search by keywords in https://www.indeed.com web page (ii) Scrape the results of the search (iii) Display the scraped data in terminal or to export them into a CSV or a Text files.

New and improved TrakTrain beat downloader.

Simple guide to working with Scrapinghub

A bunch of website scraping scripts

An unofficial DuckDuckGo search API.

rails app for web scraping purposes. It scrapes a given page (by the URL), and returns you all informations about that page.

A smart distributed crawler that infers navigation models of structured websites, used to cluster pages based on their structure and extract data from them.

Web Scraper - Final project of Ruby section inside Microverse's course - Project built to get all information related to the products inside Disney Store website. Information of the search is exported into a CSV file.

📷 Instagrammer lets you fetch Instagram user info and posts

🇯🇵📰🗻 NHK News Web (Easy) word frequency (core list) scraper for Japanese language learners.

A scraping/binding tool for Worm

A Simple Web Automation project to scrap and store real-estate ads in a Sqlite and ElasticSearch

Scrapes the entire Maven Central to find all artifacts, their dates, and versions

Scrape web content with configuration options including Javascript support and Tor.

A web scraper to get a Marmiton, 750g or cuisineaz's recipe

Scrape Instagram posts from an user or hashtag 👀