BlackWidow

BlackWidow is a website scanner developed specifically for helping you look for and download specific files such as pictures and videos, identify all types of links, be they broken or external, and apply powerful filters.

The GUI doesn’t look particularly enticing but the robust look proves its functionality as you may easily switch between the key features of the utility, namely web browser, filters, scanner, broken links, extracted email addresses, grabbed external links, web site structure and Internet sniffer.

The utility lets you make use of a built-in web browser that allows you to input custom URLs, go to the previous or next webpage, stop the loading process, refresh the current page, view source and take a snapshot.

BlackWidow is able to automatically look for links each time it grabs a document or HTML page, or scans a website. You may set up filters in order to modify a document or HTML page before the program has fetched it (prior to parsing it for new links), define a link mask that the utility should not follow, filter out data by file types, as well as substitute a link for another one using regular expressions.

What’s more, you can make the application scan the whole site or only the startup URL, enter the page transition depth, include or verify external links, specify the default index page name, and add the startup referring URL.

The app allows you to download data while scanning a specific website, enter the maximum number of connections, limit the scan speed, specify the saving directory, preserve folder structure, as well as ignore duplicates. You may start, pause or stop the scan process.

During a scan, the tool is also able to identify broken links, which are no longer available. They are listed in a dedicated panel along with the code and description of the error. BlackWidow can identify email addresses and external links to other websites.

You can make use of an Explorer-like layout in order to browse throughout the content of all the files and directories found by the scanner. The structure can be saved to WEB file format so you can continue scanning and downloading data later, and you may download the selected files.

The Internet Sniffer is used for capturing all URLs and listing them in a dedicated window. You may select the network adapter to spy on and filter the information using wildcards or regular expressions.

The program gives you the possibility to export all files or only the selected ones and include items in subfolders. You may save the information to plain text, HTML or XML file format.

BlackWidow reveals simple looks backed up by a powerful suite of configuration settings. The app uses multi-threading for grabbing all files and testing the links, and may prove to be of great use for webmasters that need to identify broken links and error codes.

On the downside, the utility doesn’t include support for a help manual (only some tips are displayed in its panel) so rookies may encounter some difficulties in understanding the function of each parameter.