crawlers list with github repos – python, go, java, php & co

“There are only two hard things in Computer Science: cache invalidation and naming things” Phil Karlton on my last post from 30 March 2022, I started with same crawlers to finding unique hostnames and then collecting them on a mysql database. Example Crawler: dcrawl – searches hostnames from given start url. A free open-source project … Read more

How to build a Search Engine with Laravel & MongoDB and Scrapy [PHP, NoSQL, Python on Linux OS]

This article is currently being revised and expanded! Last update on 01.07.2022: getting more then 100 Million of unique domain name on a MySQL Database.Using MeiliSearch for indexing is cool, but after 10 Million Index it will be very slow to indexing new rows.Because of this I am looking for a faster way to indexing … Read more

python string replace in python 3.x

Python String replace() Method There are two ways for replacing a String for python 2.x and also 3.x, use str.replace(). The replace() method in python 3 is used simply. Example for a string replace: or calling the method replace() of  <class ‘str’> in python 3.x: Notes from Python Docs: Method 1: use builtin str’s replace -> str.replace(strVariable, old, … Read more

how to split a string in python

How to split a string into a list of characters in Python?Answer is: You need Lists. Docs: http://docs.python.org/library/functions.html#list You take the string and pass it to list() or You can also do it in this very simple way without list(): If you want to process your String one character at a time. you have various … Read more

python yield keyword

What does the “yield” keyword do in python? First of all you must to understand what generators [ Generators – Python Wiki ] are. And before you understand what generators are, you must understands what iterables [Iterables – PLYMI] are. Iterables Definition: An iterable is any Python object capable of returning its members one at a time, … Read more