Brad Werth

Spiders

Robotspider

I enjoy developing Webbots, Spiders, Crawlers, Screen Scrapers, and all other forms of Automated User Agents. Below are a few of the spiders that I have worked on recently.

Job Spider

Job Spider searches online job banks for listings. Using advanced analysis of the job descriptions, Job Spider automatically applies for relevant jobs using a custom cover letter and resume. Job Spider also maintains a database of keywords, for statistical analysis. The Job Spider works extremely well. Now have a job that I love, working at a great internet design company as an Applications Developer. Like most of my spiders, Job Spider is not available to the public (except for close friends and family members).

Cigar Spider

The Cigar Spider crawls cigar retailers' websites and adds their prices into a database. It started out as a small project that I could use to save a few bucks on cigars. The Cigar Spider has quickly grown to be the largest online cigar price indexing site.

Proxy Spider

Proxy Spider parses numerous proxy lists all over the internet and compiles the output into a single text file. The text file is useful to load into a proxy checker program to generate a list of fully anonymous proxies. While my Proxy Spider is not available to the public, you can always download the proxy list.

Servers.ini Spider

Servers.ini Spider collects and attempts to verify a listing of IRC servers and channels. It then writes the resulting output to a servers.ini file for use with Avarice (or any XDCC client that uses the Bottler servers.ini format). Servers.ini Spider is not available to the public, but you can download the generated servers.ini files.

Album Spider

Album Spider compiles a list of soon to be released albums, then downloads the torrent when it becomes available. Album Spider has the option to either download or ignore Advance Releases, ensuring that new music is downloaded as soon as it is available. Album Spider is not available to the public.