How Do Search Engines Actually Search

It’s estimated that as many as 547,200 new websites are created every single day, and that isn’t going to slow down.

18:43 27 July 2020

As new sites try to stand out in a web full of competition, how do search engines like Google and Bing go through these sites and direct them to their eventual audiences?

 

The primary function of a search engine is to direct a user to a specific webpage based on keywords they input. The webpages they get directed to are dependent on Search Engine Optimization, which specifies what results are shown. For the end-user, they’ve seen this process countless times. After all, there probably isn’t a single person on the internet who hasn’t used Google at some point. So what kind of tools are in place to make sure these web pages get displayed at all? Let's take a look at how search engines actually search.

 

The Steps to Searching

 

Search engines like Google typically follow a process that leads them to discover a webpage. The first step is known as crawling, where the engine will begin to scour the internet to find new sites. These can be discovered by following newer links on old web pages or looking over submitted lists, called sitemaps, specifying where on the internet the search engine should look. Sitemaps are important for discovery because they will give the search engine the entire outline for a website, helping them crawl more effectively. Sites can rely on services to handle this process for them, using specific indexes like robots.txt for Magento.

 

Once crawling is complete, the next phase begins where a search engine tries to process what it just found and assign it to a specific location. This is the indexing phase. Algorithms are used to analyze everything in the webpage, from the text, to images, to videos and compared to similar entries already existing in the database.

 

Finally, once the meaning of a page has been deciphered, the last phase, or serving, can begin. This is where the search engine will determine what kind of search will be most appropriate for that specific result. Many factors are taken into account in this phase to determine what gets ranked were, such as location. These factors, plus the webpages specific Search Engine Optimization all pool together to find the perfect spot.

 

 It’s important to note that these results constantly change as more recent and relevant information is acquired. For example, someone who googles “Tom Hanks Newest Movie” will find new results every year because Tom Hanks will make new movies, so if you’re searching for his newest one, you’re probably not looking for 1999’s Toy Story 2. Someone Googling “Tom Hanks Birthday” on the other hand will probably get linked to the same page because Tom Hanks will always have the same birthday.

 

That’s the process of how search engines like Google find new websites on the internet. It’s a complex system with many moving parts, but understanding how it works can lead to a much greater appreciation for how we found the websites we use every day.