Focused crawls are collections of frequently-updated webcrawl data from narrow (as opposed to broad or wide) web crawls, often focused on a single domain or subdomain.
Unless I missed something, the documentation doesn't explain how to query document metadata (searching "site:montferret.dev metadata" through Google returned nothing, neither did grepping the source code).
As an example, I tried to query the og:url metadata.
I tried variations of //meta[property='og:url']::attr(content), with or without the leading //, and with or without the `attr(conte
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?
Is your feature request related to a problem? Please describe.
A new contributor will feel overwhelmed when they will try to contribute to this project.
Describe the solution you'd like
Add a architecture.md as described in this blog post
Describe alternatives you've considered
Not having architecture.md.
When looking up an attribute with .attr(), the name of the attribute should be lowercased before looking up in .attribs object.