VovSoft Collect URL 3.2.0

Collect URL is an internet crawling bot designed to collect URLs similar to search engines like a spider would do, listing every link it finds at each URL entered into Collect URL's field before scanning every one recursively. Users can limit Collect URL to collect single domain only; file extensions ignored can also be edited; response timeout values can also be edited as is user agent value used with HTTP and HTTPS connections; as well as edit user agent values as needed for any connection type (HTTP and HTTPS connections, respectively). Additionally users can change user agent values used with HTTP and HTTPS connections by editing user agents value editable fields on each field's editable field for the file extension ignored list editable field before doing recursive scans on all links contained there.