In these times when social media dominates the digital conversation, search engines still holds position of prominence. The gargantuan volume of information that is spawned everyday (content from websites and live user-generated content through social media) is astonishing to say the least. Sorting and finding relevant information from this ocean of content, finding answers to myriad questions is the job of the search engine. When it comes to search, Google is ubiquitous with search engine and holds a lion's share of the market, despite Bing and others trying desperately to break the dominance.
The ubiquity comes from the reputation that Google algorithm has gained over the years for presenting the most relevant results and even predicting what you might search! The Google search algorithm is getting better by the year, and it comes from employing the smartest computer scientists who have refined the algorithm to a level of sophistication that is difficult to replicate (hence it is unlikely that Google will ever have serious competition). The algorithm however is a black box, the exactitude of which is unknown since it is Google's secret recipe. All knowledge of how it functions and the implications of various updates to the algorithm are speculations based on observations by search marketing experts, and with a little help from Google.
Whether you are digital marketer or just curious, understanding the evolution of the algorithm over the years helps you put things in perspective about the impact its had on search rankings. The date of updates to the algorithm will also help you correlate variations in rankings that you may have observed for your search listings.
To start with, let's look into the definition of some key updates before we look at chronological list of updates to the algorithm. For the sake of brevity, I've included only the most important updates to the algorithm since 2010.
Google Hummingbird Update Google started using Hummingbird around the end of August 2013, and announced it to the world on September 26th on the eve of their 15th anniversary.
The Hummingbird update puts emphasis on the context of the query rather than just interpreting it unidimensionally for synonyms. This makes the algorithm's interpretation of the search query more human, capable of understanding intent and relationship between keywords. In effect, the search results places greater importance on page content that answers the user's query rather than just return pages that match the keywords in the query.
The Hummingbird update has been a major update to Google's search algorithm, probably the most important update to the algorithm since 2001. In fact, it wouldn't be wrong to call it an overhaul instead of an update considering the substantial impact it has had on search results.
Google Panda Update (also known as the Farmer update) The Panda update was introduced in February 2011, in a drastic attempt to combat web spam seeping into Google search results. In effect, the Panda update penalizes sites that are low quality in content, or "thin" websites that usually have a large amounts of advertising and very little content. The good news is that websites with high quality content are elevated in search rankings.
Interestingly, the Panda update was named after Navneet Panda, an Indian engineer who was one of the key guys behind the update. It is also called the Farmer Update because the update had a substantial negative impact on "content farms". Content farms are companies that employ freelance writers to churn out content by the thousands (content that matches popular search queries), specifically to generate advertising revenue.
The Google Penguin Update (also known as the over optimization penalty) The Penguin update was launched in April 2012, and like the Panda update, is an important update in Google's efforts to improve the quality of search results. The Penguin update is an attack on websites that violate Google's Webmaster Guidelines by using black-hat search engine optimization (popularly called SEO) techniques that were manipulative to enhance their search rankings.
Black-hat SEO techniques in the past involved link spamming(buying links from link farms or link networks to boost ranking), content manipulation, keyword stuffing etc. that artificially raised the profile of a website in search results.
As a result, websites that were in violation were significantly downgraded in their search ranking.
- Google's "Mobile Friendly" Update This update favors "mobile friendly" websites that are optimized for mobile devices (responsive or adaptive design). Dubbed "Mobilegeddon" by many, this update to the algorithm is said to have as significant an impact on the search results as Panda or Penguin. According to Google, "Starting April 21, we will be expanding our use of mobile-friendliness as a ranking signal". In a departure from the past, Google has been telling its users exactly what to expect from the update way back in February, and integrated mobile usability reports and testing tools as part of Google Webmaster Tools.
- Wikipedia Results See a Decline Wikipedia and Google have shared a symbiotic relation in the past, Google drove a bulk of Wikipedia's traffic since it ranked a Wikipedia article in the Top 5 and Google in turn benefitted from higher user retention (because of the quality of Wikipedia articles). Though core and popular topics on Wikipedia maintain a high standard of quality, topics that are less popular around the fringe are bad in quality or biased. This, along with "Google's Knowledge Graph" is probably the reason why Wikipedia has lost some of its sheen in the recent update.
- Continuous, Incremental Updates for Penguin Rather than a major release to the Penguin update, Google has confirmed that it will receive continuous, incremental updates. This will ensure that optimizations are continuous and not clubbed together to be launched at a fixed date.
- Scraper Websites Penalized Websites that copy content, "scrape" content off websites for which Google has received numerous request for removal (DMCA) will be penalized.
- Google Panda Update This update to to Google Panda improves the algorithm and also improves the search ranking for small or medium websites with high quality content.
- Google Pirate Update This update is an attempt to tackle online piracy, and also acting on a barrage of crticism that Google isn't doing enough to fight piracy. Consequently torrent websites, free movie streaming or download websites have been hit hard, losing a significant portion of their search ranking and listing.
- End of the Authorship Markup This marks the end of the authorship markup (rel=author tag), and to many in the search community it wasn't much of a surprise - the writing was on the wall. The reason for the removal was the dismal adoption of the authorship tags by the community, even among the early adopters. Google's proprietary tag will give way to more standardised data structures that can also be interpreted by other search engines.
- Google HTTPS/SSL Update - Secure Websites Get a Boost Google announced that encrypted websites (SSL secured) now have a higher chance of a better ranking than their unencrypted counterparts. Encrypted websites according to Google secure a user's browsing and transaction experience and hence must be given a boost. It is likely to become a more significant ranking signal in time to come.
- The Pigeon Update With the usage of mobile device for search, and searches on the go, localization of results is becoming important for Google. This latest update is testament to that, and improves relevance for results that are local. Traditional ranking signals like links from local sources are treated as positive.
- Author Photo Dropped from Search Results In a move that received mixed reactions from the webmaster community, and to the surprise of many, Google announced that author photos will no longer be included in search results. The photos were an important element of Google authorship where photos of the author appeared alongside the search listing. Interestingly the authorship is linked with Google+, from where the author photo was sourced and its elimination may signal that Google+ is losing out in significance.
- New Panda Update Dubbed the Panda 4.0 update by industry insiders, this is a major update to the Panda algorithm and the repurcussions have been felt. According to sources ~7.5% of English language search queries have been affected because of the update. Panda 4.0 also seems less harsh on small and medium websites
- "Payday Loan" Update This is an update to the algorithm that targets websites or pages specifically designed to appear in search results for spam queries like payday loan, pornographic queries etc. These websites contain little or no content and are made with the intent of making money from advertising.
- Scraper Sites Penalized Google launched a scraper report form to target scraper sites (websites that copy content from websites). This is a reaction from the community that scraper websites have better ranking than the sites they copy. Funnily, the most denounced website is Google itself thanks to data it fetches from various sources as part of its Knowledge Graph!
- Page Layout Update This is a refresh to an update in 2012 that penalized websites that were "top heavy", or having too many ads above the fold making it difficult for users to find content. Though sites with intrusive or sub-optimal ad positioning were penalized, the overall impact on websites wasn't substantial.
- The Google Hummingbird Update As explained in the definition earlier in the article, Google announced the Hummingbird update on September 26th on the eve of their 15th anniversary. Unlike the Panda, Penguin and other updates that were alterations to the algorithm, this was an overhaul of the Google algorithm, akin to a new engine upgrade. The Hummingbird upgrade addresses the semantics of a query rather than focusing on keywords that form a query. This aspect of Hummingbird "humanizes" the search, especially since search requests are getting more conversational (Google voice search for e.g.). Hummingbird also has the ability to string in queries without losing context. Let me cite an example; you can now search for "What is the temperature in Bangalore?" and follow it up with "Will it rain tomorrow?", without mentioning Bangalore in the second query.
- Panda Softens It's Stance for Niche Websites The latest Panda update seems to have gone soft on niche websites, having recovered lost traffic and ranking. The update favours authority of a niche website as a signal to avoid being being penalized.
- Google Knowledge Graph Update The number of search queries that included the Knowledge Graph was increased substantially. The Knowledge Graph introduced by Google to it search results in May 2012 changed the dynamics of search, collating interconnected facts and figures hence improving the utility of the search. According to various sources, about 25% of all searched now include entries from the Knowledge Graph. Brands have particularly benefitted from the update and have seen a increase in search results with Knowledge Graph entries, indicating that Google wants users to have easier accesss to brand related information.
- Google Payday Loan Update "Payday Loan", pornographic and other scammy websites that have benefitted in the past from unscrupulous exploitation of keywords have been the target of this update. According to official sources, the clean-up for websites in this particular vertical that are spammy in nature will take a few months.
- Penguin 2.0 Update This update had harsher implications for websites that missed the wrath of Penguin in its early avatar, especially the ones with artificial or manipulative back links (usually with optimized anchors). The new iteration affected all pages of the website, not just the home page which was the case in the earlier version.
- Google Panda to be Absorbed into the Primary Algorithm According to Matt Cutts from Google, the next Panda update may be the last before it is assimilated with the core algorithm. This implies that you may not see manual refreshes of updates to the Panda, one that is a large scale consolidated version. Instead it will be gradually built into the index in real-time.
- Penalty for Outbound Links in the Footer According to recent changes in the Google Webmaster Guidelines, links that are widely distributed in the footer of various sites are in violation of their guidelines. An unintended casualty of this guideline could be design agencies who usually leave a link back to the parent website in a client's website.
- Google Page Layout Update This update was in continuation with the Page Layout Update (in January) that penalized websites with too much of ad above the fold. Essentially too many ads above the fold were considered obstructive to the user experience.
- Google Exact Match Domain (EMD) Update Websites with domain names that match exact keywords or phrases, popularly called EMD (Exact Match Domains) will now be penalized. EMDs have been exploited by black hats to gain better search ranking, and many of them are rife with low quality content. For e.g. if a business sold cheap steel sprockets, then the domain cheapsteelsprockets.com would qualify as an EMD and is likely to be penalized. This update affects about 1% of queries.
- The Pirate Update - Penalty for DMCA Violations In a move that was welcome by the entertainment industry in general, Google announced the "Pirate"update. The pirate update penalized websites that were plagued with copyright infringement complaints possibly via DMCA takedown requests. According to Google, more requests for takedown maybe considered as a signal that affects search rankings. However, search entries won't be removed unless Google received a valid copyright removal notice from the copyright owner.
- Google Knowledge Graph Update The Knowledge Graph update is an enhancement over the regular search results, and an important progress in semantic search. The Knowledge Graphs supplements the search result with additional information about people, places and things hence making the search result more relevant. The update isn't just about serving pages that match search queries but provides answers, and in cases like calculations, the direct answer. There were discussions in the community that this would mean loss in traffic (and eventually revenue) for certain publishers because Google now provides the knowledge directly to the end-user. Information provided in the Knowledge Graph panels were fetched from publicly available sources like Wikipedia, CIA World etc.
- The Google Penguin Update - Penalty for Over Optimization If Panda was an attack on low quality and thin content, it was now time to introduce Google Penguin an update to the algorithm to fight a more vicious issue - over optimization. Historically, webmasters and SEO agencies have taken advantage of certain optimization factors to rank higher in search results, and that has worked well for them in the past. The issue however is that of over optimization, using black hat techniques that makes a website of little use to the end user but favours the website to search engines, often ranking higher than more useful results. Some of the over optimization techniques involve link spamming (especially from links that aren't the same topic as that of the website), keyword stuffing, excessive links in an article with keywords in the anchor text. An entire industry has been spawned around these manipulative techniques where black hatters have made millions by diverting traffic to their website and making money out of ad revenue. This is a major update and has been long anticipated considering the effect it has had on the quality of search results. The Penguin update had a significant effect on search results, affecting over 3% of English queries worldwide.
- Google Changes the Way Links are Analyzed Link optimization is an important component of SEO, but also often misused to get the better of search results. To counter this perhaps, Google announced recently that it will alter the way links are evaluated. In a statement by Google, "We often use characteristics of links to help us figure out the topic of a linked page. We have changed the way in which we evaluate links; in particular, we are turning off a method of link analysis that we used for several years.". However, Google wouldn't reveal which of the link analysis methods were turned off. Discussions amongst the community and polls speculate that PageRank, Age of the link and text surround a link are the ones that have been turned off.
- Reaffirmation of Penalty for Above the Fold Ads In contination of an update in November last year that punished websites that were top heavy, Google now devalues websites with excessive ads above the fold. This is an affirmation from Google that websites that forgo user experience for the sake of advertising will in the cross hair. Ironically most of the websites that have been in violation have been using Google Adsense.
- Google+ integration into Search Results In a controversial move, this update integrated Google+ social data into search results. This was an attempt to push adoption and relevance of Google+, that had failed to gain the traction enjoyed by Facebook, Twitter and other social networks despite much efforts by Google. The move infuriated a lot of users and webmasters who felt that Google+ was being forced on them, and they had deviated from its path of serving relevant, unbiased results.
- Detection of Parked domains In this update, parked domains were penalized and excluded from search results. Parked domains are basically placeholders for the domain without any actual content in them, and usually there to leverage revenue from ads (ironically from Google Adsense).
- Boost for Official Websites In this update, websites that have been identified (explicitly claimed) as official or belonging to a person will now receive priority in search ranking over unofficial results. This may look like an obvious choice considering the intent of the user, however there are instances where the unofficial sites are more informative and receives more clicks than the official.
- Excessive Ads Penalty Excessive number of ads in a page is now penalized, as confirmed by Google in this recent update. This has always been an impediment from a user experience perspective, and now its been integrated as a factor in the algorithm. However one needs to keep in mind that more than the number of ads, its the proportion of ads in relation to the size of the page. The chances of penalty are higher if there are more ads above the fold.
- Google Search - Now Served Fresh Freshness of the pages (recency) is taken into consideration for ranking pages in search results. This update rewarded websites that were infused with fresh content and acted as an incentive for websites to keep their website updated frequently. A massive 35% of search queries were affected by this update and the repurcussions were felt across. Categories or content that had a longer shelf life like cooking recipes, DIY weren't affected by the update.
- Google Queries are now Encrypted In an effort to protect the privacy of users (since search keywords could reveal sensitive personal information), Google decided to encrypt (SSL search) search queries by default. As a result of this update, Google Analytics now masked keywords from organic searches with the "(not provided)" term. This had a drastic impact on how SEO agencies or webmaster perceived the effect of their SEO efforts, since the vital keyword information wasn't available.
- Google Pagination Update This update addressed the issue of pagination, using the link elements rel="prev" and rel="next". The elements indicate the relationship between URLs in the pagination series; for e.g., rel="next" indicates that the linked page is next in the series of pages and vice versa with rel="prev". With the pagination update, Google is now able to present relevant results for content split across pages and also avoids cases of content duplication. If the pagination has a "view all" link, where the content is consolidated in one page, Google gives preference to that in search results as opposed to the paginated version.
- Scraping the Scrapers As discussed earlier, the Panda update caused havoc among websites that scraped content. However, that was not always the case; there were instances of scraper websites that duplicated content having better positions than the original website from where it was copied. Google was reported to work on an algorithm to fix that and also asked webmasters to report such instances.
- Nihao Panda - Panda is Now Available in All Languages After the successful rollout of Panda for Engligh language websites across the world, Panda was now extended to all languages around the world. The impact was reported to be upto 10% of search queries across these languages.
- Google Expanded Site-links Update As a value add, and an enhancement to search results, Google now includes more links to the site beneath the main sitelink. Initially there were 12 sitelinks displayed which was reduced to 6 links shortly after the launch.
- Post-Panda Recovery Since June 15, some sites recovered from Panda penalty, when they were modified to remove duplicate content. This seems to address mainly site hits because duplicate content. The sites which were victim of scrapers recovered and the latter now often removed from SERPs.
- The Author Attribute Myself . This will help to classify the pages per author. The profile page so designated must be on the site that contains this attribute.
- Google Schema.org Update To structure data on the Internet in a manner that was universally accepted, Google along with Microsoft and Yahoo collaborated on a vocabulary that culminated into schema.org. The schematics makes it easy to add semantics to different kinds of data in a web-page, and hence making it easy to classify and extract.
- Panda Goes Global The effects of Panda would now be felt globally. After the rollout in March, Google has announced the extension of Panda to all English language queries across the world, not limited to English speaking countries. Also, sites blocked by users as a result of manual intervention from search results were taken into consideration (though it was a minor factor).
- The Google Panda Update (Also Known as the Farmer Update) Refer to the explanation about the Panda update at the start of the article. Addendum: The vicious Panda update was a direct attack on low quality content farms that were in violation of Google's Quality Guidelines. Google confirmed that about 12% of search results were affected by the Panda update.
- Content Attribution Update Google launched this update to combat cases of content misattribution or websites that scrape content from other websites (also called scraper websites). This update affected 2% of search queries according to Matt Cutts, but that was enough to feel the repurcussion for a lot of websites, especially the ones without original content. Many in the webmaster community speculate this update as a precursor to the Panda update.
- Taking Stock of the Situation - Google Takes Overstock.com to Task Google, the benevolent dictator for webmasters have taken certain websites to task for flouting guidelines and employing black hat SEO tricks. In an article by New York times (yet again) it was outlined that the web retailer Overstock.com paid universities and colleges to post links back to Overstock in exchange for discounts. This trick obviously had the advantage of boosting the SEO profile of Overstock who ranked high on many shopping related terms. Subsequently, Google took action and Overstock was heavily penalized and to the extent that it adversely impacted revenue (as it was declared in their annual fiiling). Others to be hit by this penalty was the retailer JCPenney who employed similar tactics.
- Google Strikes Back - The DecorMyEyes Incident In an unprecedented step in the history of its updates, Google decided to act based on a scathing report about a flaw in its algorithm that was reported in the New York Times. The article was based on the infamous incident of an unscrupulous merchant who ran the website DecorMyEyes, whose website ranked high on search results based on negative reviews. The website benefitted from a flaw where irrespective of the nature of the backlink (even if they were rants), the website gained in prominence. Post the article, and to prevent such cases, Google included sentiment analysis as a factor that analyzes text around the link to determine if the sentiment is negative. Google also advised usage of the "nofollow" attribute when adding a link to negate passing on any influence/weightage to the destination link.
- Social Signals Update Google has confirmed the inclusion of social signals as a factor in the algorithm. This wasn't much of a surprise considering the prominence and growth of social networking websites like Facebook and Twitter. Subsequently, if a website is part of the social conversation it has a good chance of appearing up in the results.
- SVG is Now Indexed SVG (Scalable Vector Graphics) files are now indexed by the algorithm. Unlike a JPG image which is rasterized, an SVG image has its attributes defined in a text based XML file which makes it easier for crawlers to derive information about the image.
- More Results from the Same Domain Google now features multiple results from the same domain in a search result. In the past Google allowed only one or two results from the same domain in a search result. This particular update helped brands get more mileage from search results, and eventually more traffic.
- The Google Caffeine Update - Freshly Brewed On the 8th of June, Google finalized the roll-out of Caffeine, a new indexing engine. Caffeine now produced 50% fresher results, thus increasing its relevance to the user. It was high-time this was done considering the amount of fresh information that is churned every day thanks to blogs and various social media sites. In the past the index was updated in waves, and hence there was always a lag between when information was crawled and when it appeared in search results. In Caffeine, the index is updated incrementally and has significantly reduced the gap between crawling and indexing.
- The Google Mayday Update The dreaded update from Google (infamously named the Mayday update) is here, and it has rattled websites to the core. This update attacks websites with low quality content that exploits "long tail" keywords (explained earlier). The repurcussions were felt earlier in April when websites saw the effect of traffic being hit around long tail keywords, but this time it has been confirmed officially as a change in the core algorithm. This can also be seen as the precursor to the "Panda" update in 2011.
- The Long Tail Wags the Dog This update affects the ranking of websites that attract a substantial amount of traffic from "long tail" keywords. Long tail keywords are longer queries or phrases that attract little traffic, but as an aggregate attract huge traffic. The term "long tail" is a statistical term to denote a a long section of the probability distribution that thins away from the head of the distribution. Because of the update, many sites have seen a change in their traffic with some experiencing a drastic drop in their overall traffic.
- The Site Speed Factor The speed of a site is now officially a ranking signal. The speed of a website (the time it takes to load) affects user retention, even if it is a few seconds more than the standard deviation. and Google has taken cognizance of that. After all if a user is dissatisfied with the website experience, it may also reflect poorly on Google for having led them there.
Since important updates to the Google Algorithm are announced (or deduced) every other month, this article will be refreshed accordingly with the latest. If you wish to point out an important ommission or wish to share feedback, please mail [email protected]