search engine working technology

How does search engine work ?

Easy to understand (?)

• Stores information of the web

• Classify information and rank them

• Bring us with the most relevant content related keywords

Hard to do it (!)

• Billions of Web Pages

• Billions of Gigabytes data

• Every milisecond millions of new data

• Characteristic differences with languages ( localization )

• Requires huge amount of investment

• Respond query less them 1 second

Google Data Centers

• 11 Data centers in the world

• Each data center consist of 10.000 Computers

• Last data center cost 600 Million U.S Dollar

Google Data Centers

Google Data Centers

3 Main aspects of Search Engines

• Web crawling

• Indexing

• Searching

Search Engine Scheme

Web Crawling

• Walks Around the web

• Follows link from every site

• Retrieves information from web pages html

• Stores information in Data Centers

Indexing

• Analyze the stored information by web crawler

• Classify them

• Rank them

Searching

• Keyword query on search engine

• Fetch related links of the keywords

• List them according the relavance