Google is now announcing their new indexing system called Caffeine. Google says Google Caffeine search engine indexing system is much faster than their older system. One advantage is that news stories, blog posts and forum posts are indexed better and faster allowing users to find links to relevant content much sooner after it is published than was possible before.

Google Caffeine provides 50 percent fresher results for web searches than their last indexing system. According to Google this is the largest collection of web content Google has ever offered.

Why did Google create a new indexing system instead of using the current index?

Content on the web is continually expanding.  This content might be text, images, videos, news and some real time updates. Also present web pages are richer and more complex. From the Searches perspective it is also important to provide users with the latest relevant content . Publishers need content to quickly appear on the search engines shortly after their posts in search results. To meet these expectations Google implemented this new search indexing system.

The image below explains the difference between the old indexing system and the new caffeine.

Before comparing the old index and caffeine, lets look how search engines crawl the entire web and store the data found there. When you search Google, you’re not searching the live web, you’re searching your query through the Google database, or index, or registery like we do in a library. If we want to find a specific book , first we go to the librarian or search the index of the library. After the librarian finds the specific place for that book we go to that rack to get the book. This index system, just like a card catalog in a library, helps you to find that specific content. Google does the same for you with it’s index of the web it shows the path to find relevant data for your search query.

Google’s old indexing system had several layers, and some layers refreshed in a faster rate than others. The main layer updates every couple of weeks, this means there is some kind of significant delay from the published date. News for example though it is highly relevant shortly after it happens, tends to become history rather than news in a very short time. We live in a world of instant gratification and get news immediately on our cell phones or computers – even a week delay isn’t really acceptable.

In Caffeine, Google analyzes the web in small portions and updates their search index rapidly, continuously, and also globally. If the “bot” finds some new information on existing pages it is directly added to their index, so for both searchers, and for publishers, if you need fresher information, here you have it. Publishers also can also publish their hot information and get it prioritized for indexing making it appear in searches much faster.

According to Google in every second caffeine processes hundreds of thousands of pages in parallel, so the new indexing system is much busier than the old one, getting content indexed and available faster.

Google – “We’ve built Caffeine with the future in mind.” Google’s idea is to build a faster and more comprehensive search engine that delivers more and more relevant results. Google engineers are still working on this great system, there will be some more improvements in coming months. We have to wait to see what those are.