| Nov | DEC | Jan |
| 23 | ||
| 2019 | 2020 | 2021 |
COLLECTED BY
Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
History is littered with hundreds of conflicts over the future of a community, group, location or business that were "resolved" when one of the parties stepped ahead and destroyed what was there. With the original point of contention destroyed, the debates would fall to the wayside. Archive Team believes that by duplicated condemned data, the conversation and debate can continue, as well as the richness and insight gained by keeping the materials. Our projects have ranged in size from a single volunteer downloading the data to a small-but-critical site, to over 100 volunteers stepping forward to acquire terabytes of user-created data to save for future generations.
The main site for Archive Team is at archiveteam.org and contains up to the date information on various projects, manifestos, plans and walkthroughs.
This collection contains the output of many Archive Team projects, both ongoing and completed. Thanks to the generous providing of disk space by the Internet Archive, multi-terabyte datasets can be made available, as well as in use by the Wayback Machine, providing a path back to lost websites and work.
Our collection has grown to the point of having sub-collections for the type of data we acquire. If you are seeking to browse the contents of these collections, the Wayback Machine is the best first stop. Otherwise, you are free to dig into the stacks to see what you may find.
The Archive Team Panic Downloads are full pulldowns of currently extant websites, meant to serve as emergency backups for needed sites that are in danger of closing, or which will be missed dearly if suddenly lost due to hard drive crashes or server failures.
Collection: Archive Team: The Github Hitrub
Fix CodeIgniter scraperde33aef
git clone https://github.com/freeCodeCamp/devdocs.git && cd devdocs gem install bundler bundle install bundle exec thor docs:download --default bundle exec rackupFinally, point your browser at localhost:9292 (the first request will take a few seconds to compile the assets). You're all set. The
thor docs:download command is used to download pre-generated documentations from DevDocs's servers (e.g. thor docs:download html css). You can see the list of available documentations and versions by running thor docs:list. To update all downloaded documentations, run thor docs:download --installed. To download and install all documentation this project has available, run thor docs:download --all.
Note: there is currently no update mechanism other than git pull origin master to update the code and thor docs:download --installed to download the latest version of the docs. To stay informed about new releases, be sure to watch this repository.
Alternatively, DevDocs may be started as a Docker container:
# First, build the image git clone https://github.com/freeCodeCamp/devdocs.git && cd devdocs docker build -t thibaut/devdocs . # Finally, start a DevDocs container (access http://localhost:9292) docker run --name devdocs -d -p 9292:9292 thibaut/devdocs
localStorage are used to speed up the boot time, while memory consumption is kept in check by allowing the user to pick his/her own set of documentations. The search algorithm is kept simple because it needs to be fast even searching through 100,000 strings.
DevDocs being a developer tool, the browser requirements are high:
●Recent versions of Firefox, Chrome, or Opera
●Safari 11.1+
●Edge 17+
●iOS 11.3+
This allows the code to take advantage of the latest DOM and HTML5 APIs and make developing DevDocs a lot more fun!
Docs module.
There are currently two kinds of scrapers: UrlScraper which downloads files via HTTP and FileScraper which reads them from the local filesystem. They both make copies of HTML documents, recursively following links that match a set of rules and applying all sorts of modifications along the way, in addition to building an index of the files and their metadata. Documents are parsed using Nokogiri.
Modifications made to each document include:
●removing content such as the document structure (<html>, <head>, etc.), comments, empty nodes, etc.
●fixing links (e.g. to remove duplicates)
●replacing all external (not scraped) URLs with their fully qualified counterpart
●replacing all internal (scraped) URLs with their unqualified and relative counterpart
●adding content, such as a title and link to the original document
●ensuring correct syntax highlighting using Prism
These modifications are applied via a set of filters using the HTML::Pipeline library. Each scraper includes filters specific to itself, one of which is tasked with figuring out the pages' metadata.
The end result is a set of normalized HTML partials and two JSON files (index + offline data). Because the index files are loaded separately by the app following the user's preferences, the scraper also creates a JSON manifest file containing information about the documentations currently available on the system (such as their name, version, update date, etc.).
More information about scrapers and filters is available in the docs folder.
thor list from the project's root.
# Server rackup # Start the server (ctrl+c to stop) rackup --help # List server options # Docs thor docs:list # List available documentations thor docs:download # Download one or more documentations thor docs:manifest # Create the manifest file used by the app thor docs:generate # Generate/scrape a documentation thor docs:page # Generate/scrape a documentation page thor docs:package # Package a documentation for use with docs:download thor docs:clean # Delete documentation packages # Console thor console # Start a REPL thor console:docs # Start a REPL in the "Docs" module # Tests can be run quickly from within the console using the "test" command. # Run "help test" for usage instructions. thor test:all # Run all tests thor test:docs # Run "Docs" tests thor test:app # Run "App" tests # Assets thor assets:compile # Compile assets (not required in development mode) thor assets:clean # Clean old assetsIf multiple versions of Ruby are installed on your system, commands must be run through
bundle exec.
| Project | Description | Last commit |
|---|---|---|
| Chrome web app | Chrome Web App which adds a shortcut to DevDocs apps page. | N/A |
| Ubuntu Touch app | Application for devices running Ubuntu Touch. | N/A |
| Sublime Text plugin | Sublime Text plugin to search DevDocs by selection or by input. |  |
| Atom plugin | Atom plugin adding the doc command to search DevDocs. |
 |
| gruehle/dev-docs-viewer | Brackets extension for searching and viewing DevDocs content. |  |
| naquad/devdocs-shell | GTK shell with Vim integration. |  |
| skeeto/devdocs-lookup | Quick Emacs API lookup on DevDocs. |  |
| yannickglt/alfred-devdocs | Alfred workflow for DevDocs. |  |
| waiting-for-dev/vim-www | Vim search plugin with DevDocs in its defaults. |  |
| vscode-devdocs for VS Code | VS Code plugin to open and search DevDocs inside VS Code. |  |
| devdocs for VS Code | VS Code plugin to open the browser to search selected text on DevDocs. |  |
| egoist/devdocs-desktop | Cross-platform desktop application for DevDocs. |  |
| qwfy/doc-browser | Native Linux app that supports DevDocs docsets. |  |
| hardpixel/devdocs-desktop | GTK3 application for DevDocs with search integrated in the headerbar. |  |
| dteoh/devdocs-macos | Native macOS application for DevDocs. |  |
| Merith-TK/devdocs_webapp_kotlin | Android application which shows DevDocs in a webview. |  |