Awesome-list of crawlers
A awasome collection of web crawlers ,spiders and resources in different languages. Tested by myself on Mac OS X with Python 3. Scrapy: The King of Web Crawler Framework written by Python. A very fast high-level html-content scraping and web crawling framework. A Complete List of all crawlers can be found under: GitHub: https://github.com/BruceDone/awesome-crawler