Focused crawls are collections of frequently-updated webcrawl data from narrow (as opposed to broad or wide) web crawls, often focused on a single domain or subdomain.
A Burp Suite Extender that try to find sub-domain, similar-domain and related-domain of an organization, not only a domain! 利用burp收集整个企业、组织的域名（不仅仅是单个主域名）的插件
I tried crawling with a couple of sites using the nutch crawler. It shows that it has crawled ~13000 pages. When i click on the visualize button, the kibana dashboard says i have to configure an index pattern.
"logstash-* " doesn't seem to work.
Does this need log.io to work? What sort of values can i give in the index field to see the visualization?
This repository contains the Domain Discovery Tool (DDT) project. DDT is an interactive system that helps users explore and better understand a domain (or topic) as it is represented on the Web.
Domain Discovery Operations API formalizes the human domain discovery process by defining a set of operations that capture the essential tasks that lead to domain discovery on the Web as we have discovered in interacting with the Subject Matter Experts (SME)s.
When the user performs a search query, show a progress bar or something similar rather than showing the infinity gif. This is because the gif does not indicate if firefox crashed or the parsing is taking longer.
We need to have a /status endpoint that the UI can query to get the status of the system, eg working, exception, idle etc.
I tried crawling with a couple of sites using the nutch crawler. It shows that it has crawled ~13000 pages. When i click on the visualize button, the kibana dashboard says i have to configure an index pattern.
"logstash-* " doesn't seem to work.
Does this need log.io to work? What sort of values can i give in the index field to see the visualization?
Is there any documentation for this part?