crawlers list with github repos – python, go, java, php & co

“There are only two hard things in Computer Science: cache invalidation and naming things” Phil Karlton on my last post from 30 March 2022, I started with same crawlers to finding unique hostnames and then collecting them on a mysql database. Example Crawler: dcrawl – searches hostnames from given start url. A free open-source project … Read more

How to build a Search Engine with Laravel & MongoDB and Scrapy [PHP, NoSQL, Python on Linux OS]

This article is currently being revised and expanded. “The amateur software engineer is always in search of magic” Grady Booch, father of UML In this article we will are planing to build a search engine with Laravel 9, Python 3 and MongoDB. Using the Laravel Scout Package and MongoDB for Backend and also MySQL on … Read more