Awesome-list of crawlers

A awasome collection of web crawlers ,spiders and resources in different languages. Tested by myself on Mac OS X with Python 3.

Scrapy: The King of Web Crawler Framework written by Python.  A very fast high-level html-content scraping and web crawling framework.

A Complete List of all crawlers can be found under:

1 thought on “Awesome-list of crawlers”

Leave a Comment