In this post, Myriam Jessier, takes us through the history of search. Expect expert knowledge, dinosaurs, jokes and all-around chaos.
Knowing the past of the search industry can help you be an impactful part of its future
Knowing the history of your industry better than anybody else is a huge career advantage, which is wild because even though it’s so simple, few people read up on it.
Today, we are going to be talking about the history of the SEO industry. Gather round kids, we have things to cover. Knowing the history of your industry will help you understand where it’s going.
Talk like a T-Rex (but don’t walk like one)
Hi, my name is Myriam and I specialize in SEO. Not CEO. Most of my clients call it CEO but my paycheck doesn’t match that title (yet). I’m an ambitious woman.
Having ambition and speaking with authority are not aggressive acts. When a woman speaks her mind, she is simply asserting her view and expressing her opinions, however in turn she becomes labelled aggressive. – Jennifer Hoffman.
So now that we established that I will be labelled aggressive, let’s embrace it. I’m the most adorable French T-Rex ever. Aggressive with short arms, this defines me and the T-Rex. Little known fact, you never ever want to strut like a T-Rex. Do you know what struts like a T-Rex? A chicken.
Let’s not go there. We shall glide with elegance and poise through a short history of SEO, you know, before someone else declares it dead for the third time this week, like every week since 2006*.
A quick word on the age of dinosaurs, aka going beyond Jurassic Park
The ‘Age of Dinosaurs’ (the Mesozoic Era) included three consecutive geologic time periods :
- 🐊Triassic period: 252-201 million years ago
- 🦕Jurassic period: 201-145 million years ago
- 🦖Cretaceous Periods: 145-66 million years ago
So now that you know this, let’s draw a parallel with the SEO industry by defining our Triassic, Jurassic and Cretaceous periods. But before that, I must really show you what scientists think the T-Rex strut looked like back in the day:
Not really, this is a video of chickens with 3D-printed T-Rex arms. Make sure you check your sources folks, here is the official paper on the topic: Walking Like Dinosaurs: Chickens with Artificial Tails Provide Clues about Non-Avian Theropod Locomotion. This study was peer reviewed.
The triassic period: Early Years: Mid 1990s – Early 2000’s
Here is the TL;DR provided by Google about the real Triassic:
The start of the Triassic period (and the Mesozoic era) was a desolate time in Earth’s history. Something—a bout of violent volcanic eruptions, climate change, or perhaps a fatal run-in with a comet or asteroid—had triggered the extinction of more than 90 percent of Earth’s species.
The early days of the internet were a bit like that. Many things happened and if you are curious to understand more about the beginnings of this incredible adventure, I highly recommend checking out the work of Howard Rheingold. He is one of the most influential university references that I can recommend to other folks who do not have the patience to take classes on the history and sociology of the world wide web. Here is his official Wikipedia bio: “Howard Rheingold is an American critic, writer, and teacher, known for his specialties on the cultural, social and political implications of modern communication media such as the Internet, mobile telephony and virtual communities.”
When did SEO begin?
If you work in SEO, you may think that this industry has been around forever but all signs point to the term SEO originating around 1997. Source I was there to witness it all as a naive pre-teen. I didn’t know the internet would be a part of my job. I didn’t know my job already existed in the 1990s. I just *knew* that I wanted to make a website and use cool (incredibly slow to load) GIFs.
So what happened in 1997?
- Steve Jobs returns to run Apple
- Hong Kong returns to Chinese rule
- Princess Diana dies in Paris
- An Avian flu in Hong Kong terrifies the world (if we only knew)
- French truck drivers block all the things and José Bové, French farmer, politician and syndicalist, member of the alter-globalization movement, destroyed a McDonalds which was a huge deal back in the day (like burning an American church for us French people)
- Stock markets crash
- Scientists in New Mexico have created a sound that hasn’t been heard in 75 million years: the low-pitched roar of the parasaurolophus dinosaur*
I remember 1997, dialup, Geocities, ICQ and every online search query getting you porn results. I was a child and learned quickly:
- My mother and the internet were not to be trusted when it came to washing machine instructions. One would turn my white shirts pink and the other would just show me porn.
- You can find anything online if you know how to look for it properly, in between all the porn results.
- Also, never, ever, make the mistake of looking for “cute girls bathing suits” – because those queries will lead to even more porn, more than the washing machine instructions.
In the olden days, exact keyword match was based on the meta keywords of each page. Obviously, as webmasters figured out they could stuff whatever keywords they wanted into the meta keywords tag, search results became pretty terrible.
So what did the triassic period look like online?
The actual Triassic period has its superstars: the Sauropods. Sauropods had very long necks, long tails, small heads and four thick, pillar-like legs. They are superstars because they were enormous. They were some of the largest animals to have ever lived on this planet. Sauropods have their own emoji: 🦕. Here are some of the Sauropods of the World Wide Web:
- Geocities: GeoCities included GeoCities Marketplace, a commercial website. It included the GeoStore, which sold GeoCities-branded merchandise. This is something you didn’t know about GeoCities even if you know what GeoCities was. If you don’t know what GeoCities is and you think this sounds weird, go Google it, have fun and enjoy this chaotic piece of internet archeology.
- ICQ: developed in 1996, this messaging platform was bought by AOL in 1998. I can’t explain the pure joy and social anxiety generated by ICQ and AIM (AOL messenger). Online messengers were very important for most of the youth but what was paramount wasn’t what you said to people, it was what you said in your “away” message. I know this makes no sense but it was a lot angstier than a Facebook post from that one aunt that can’t keep her drama out of your timeline.
- The first search engines that people took notice of were Alta Vista, Ask Jeeves (now called ask.com). AltaVista was the first really fast search engine that also covered much of the Web. AltaVista had the first successful Boolean search options. You can quote me on this, I was there. You don’t understand how amazing it was to be able to search combining keywords with operators like AND, NOT and OR to get better results. Actually, you can, you read my pre-teen odyssey online a few lines before reading this.
My first search engine optimization wasn’t for Google. I was simply pissed off that my wonderful GeoCities website was not showing up when people were looking for content in my “niche”. Did I truly have a niche back then? Of course, I did! Everyone with a website was an expert. 9 year old me absolutely felt like I belonged there, telling people all about Eric Cartman from South Park.
Everything was directory driven
You had to be listed to even exist online. It made sense in a world where you had Yellow Pages and White Pages to find businesses and people. Yahoo was known for its ability to find the best websites out there according to its editors. I was not in the target demographic of whatever editors deemed to be one the “best websites” out there. This made no sense whatsoever in my mind. I set about fixing that in my own little way.
Hindsight: I could have never predicted that backlinks would get so huge. It never made sense to me. That’s like saying you are wonderful based on who you are friends with. Which…is pretty much why you are reading this article. You probably know my name because someone from Google’s search team liked something silly on Twitter. But at the end of the day, I have come to learn: what you do and who you do it with is an important trust signal. Even today, one of the key takeaways from the SEO Triassic period is that links (internal or external) matter.
Dinosaur break time: 🦕vs🦖
I know some of you are wondering: did the T-Rex eat Sauropods? Well, we cannot say for certain that T-rexes never tasted Sauropod meat but we know for a fact that Tyrannosaurs never evolved to effectively kill a Sauropod since there were none in their territory. They coexisted on the same planet, at the same time but not in the same environment. Source: Reddit (go read the thread, it’s good stuff).
Next instalment, we will discuss the early naughties. Paris Hilton, Google Backrub and other Black Hat unmentionables.
Black Hat Gets a Bad Rep: Early 2000s – 2005
Before the World Wide Web climate changed and pushed many early questionable marketing practices into extinction, we had wild fauna and flora online. But from the ashes, life began to emerge on the giant C-shaped supercontinent as Pangea. Oops, I mean, but from the ashes, search began to emerge on the search engine known as Google.
Google Backrub Leads to a Happy Ending
That subtitle was super gross to write. But it’s true; Google has its origins in “BackRub”, a research project that was begun in 1996 by Larry Page and Sergey Brin. For his dissertation theme, Page considered exploring the mathematical properties of the World Wide Web, understanding its link structure as a huge graph*. The project focused on the problem of finding out which web pages link to a given page, based on the consideration that the number and nature of such backlinks was valuable information about that page.
*See, if I knew that back in the day, I would have known that backlinks would be very important for my job.
Here are some the best black hat hits used to increase PageRank and get to #1:
- Cloaking is the practice of presenting different content or URLs to human users and search engines. You can achieve this by putting white text in white margins (margins were common for websites back in those days). Another way to do this would be serving a page of HTML text to search engines, while showing a page of images to users.
- Link Farms, also known as link schemes (private link networks) were used to manipulate PageRank or a site’s ranking in Google search results may be considered part of a link scheme and a violation of Google’s Webmaster Guidelines.
- Keyword stuffing: it’s just as awkward as stuffing for the Thanksgiving bird but your on-page SEO is the turkey. It makes you sound like a robot and a bad one to boot.
- Content Automation
- Doorway Pages.
- Hidden Text or Links.
- Keyword Stuffing.
- Reporting a Competitor (or Negative SEO) the concept was to make sure that if you can’t be the best with what you’ve got, may as well bring others down a peg underneath you. That’s not levelling the playing field, that’s playing dirty and ruining the fun for everyone.
- Sneaky Redirects
Search engines then innovated on their matching technology to try to understand the actual webpage and parse out words that matched users’ searched keywords. This would render a lot of these tactics useless or seriously damaging to SEO efforts.
Dynamic Search Takes Off: Mid 2000s – 2009
But alas, webmasters were still not complying. Instead, they adapted to the changes and were able to sculpt results by buying links using specific anchor text, putting keywords that were highly searched into title tags, and optimizing content with metrics like “keyword density”.
2000: Introducing PageRank
An algorithm named the PageRank (PR) was used by Google Search to rank web pages in their search engine results pages. PR became a metric to measure the importance of website pages.
“Guaranteeing SEO arguments for years to come, Google launched their browser toolbar, and with it, Toolbar PageRank (TBPR). As soon as webmasters started watching TBPR, the Google Dance began.” – Moz
Important fact: J. Lo’s dress caused Google to create Google images in 2001. She had her moment in 2000 and in 2001, Google launched this new product. I was online and delighted when this happened (I was a teen by then, of course, I cared about J. Lo’s infamous dress). But don’t take my word for it, GQ wrote this whole thing about it. Today, image search is big business because as they say “a picture is worth a thousand words”. Complex ideas (and e-commerce products) travel better when expressed as an image rather than in 3000-word articles.
Around 2002, the Google Dance refers to the phenomenon of website pages jumping ranks as Google adjusts its rankings and re-evaluates pages. Well, this is the original meaning. If you asked me in 2018, the Google Dance was a conference at Google headquarters in Zurich.
2004: Fighting against SEO spam and introducing Latent Semantic Indexing
Here are the new changes introduced: Latent Semantic Indexing (LSI), emphasis on anchor text relevance, and link “neighbourhoods.” Latent Semantic Analysis is an efficient way of analysing textual content to find hidden topics by understanding the context of the content. Basically, it looked like Google was making an effort to crack down on dubious on-page tactics like text cloaking and meta-tag stuffing. Some of you may not know this but Google used to have a magical meta tag called meta keywords that enabled SEOs to provide the keywords they wanted the page to be ranked on for.
2005
Again, the search engine matching turned out not to be ideal because webmasters were able to sculpt results by buying links using specific anchor text, putting keywords that were highly searched into title tags, and making headlines that matched those keywords.
- Personalized search tapped directly into users’ search histories to adjust results. We all know the company has grown in leaps and bounds from that moment.
- That year, sitemap.xml became an option. Before that, you had to submit a handmade, artisanal, organic, free range sitemap in HTML format. I had to make those and they were not fun. Ever try to specify HREF Lang in a sitemap by hand, for an entire website? Even less fun (but that’s another, later, story).
- Google put forth more efforts to target low-quality links: link farms, paid links and reciprocal link exchanges. Update on this: nobody gave a crap, these practices were still going strong way beyond 2005.
- Google also merged Google Maps data and the main search results. In my opinion, 2005 is the official local SEO genesis.
- Bonus: Google had an update named “Big Daddy”, I remember it was something to do with the way redirects were handled but I mainly remember giggling at the name.
2007 – 2009 : Getting into gear with autosuggest and real time results
- 2007 : Google integrated traditional search results with News, Video, Images, Local, and other verticals, dramatically changing their traditional 10 listing format in SERPs.
- 2008 : Suggest is now displaying suggested searches in a dropdown below the search box while queries are being typed.
- 2009 : we complained that an update favored big brands more than the little guys. Google denied it. Real-time search came to results: say hello to Twitter showing up in results.
The Jurassic period: Quality Becomes the Focus: Early 2010s – mid 2010s
The Jurassic period (199.6 million to 145.5 million years ago) was characterized by a warm, wet climate that gave rise to lush vegetation and abundant life. Many new dinosaurs emerged—in great numbers. Among them were stegosaurus, brachiosaurus, allosaurus, and many others. But what did the SEO Jurassic period look like?
The turn of the decade was a big deal in search
In 2010, just like the Jurassic movies, some of these big search engine changes became foundational to the way we optimize websites for SEO today.
- Google Places is launched. It will be rebranded later as Google My Business Center with local search gaining ground.
- Google officially rolled out Caffeine, an update that happened over many months in order to boost speed, and to improve how crawling and indexation interact to get a fresher index.
- Multiple pages from the same website can now rank at the top of the results. Many of us SEOs used this to ensure competitors didn’t even show up in the top 3 results…and this is why we can’t have anything nice (Google would go on to yank this later on).
- Using Google Suggest, Google Instant displayed search results as a query was being typed. I remember my team losing their everloving minds over this. I just annoyed me because I have a slight handicap that does not play well with stuff moving all over the place.
- Social signals began to be considered by search engines (Google and Bing at the very least). 2009: Google indexes fresh tweets and a year later, it considers social media as a signal.
- Bonus: Google thought Google Instant was a cute idea. I did not. Showing results changing live as you type your query is not my idea of fun.
2011-2013: Panda and Penguin : the updates that keep on giving
To address specific search engine problems, Google had two major algorithm updates. In 2011, after some big penalties were dolled out to major retail brands, Google unveiled Panda. Panda hit sites with thin or low quality content, affecting up to 10/12% of search results based on some scribbled notes in one of my SEO notebooks (yes, I take handwritten notes, I’m old school). Our copywriter was excited because it meant her contract would be renewed since she got way more hours than usual. Another big deal that year was the appearance of expanded sitelinks and schema.org which enabled richer search results. Our developers were not pleased to hear about that, it added hours to their day and they, unlike the copywriter, were salaried. Google+ also happened that year, but let’s not talk about that. 2011 is also the year Google decided to rob us of our organic keywords. This is the beginning of “not provided” data. I’m still salty about that.
2012 was the year of the 1001 and SERP design tests. Panda is still being tweaked and the Venice update favours local results. But the star of the show is Penguin, the webspam update that aimed to weed out keyword stuffing. I remember people in my agency crying and I was laughing because as a college grad, I knew my way around a thesaurus. Synonyms allowed me to say the same thing five ways to meet content requirements set by my professors at the Sorbonne. The knowledge graph starts showing, DMCA requests start taking down content and Google decides to hate on exact domain matches (EDM). If you owned german-chocolate-cake.com, you were no longer king! Google was onto SEO driven domain decisions.
2013 can be summarized by: panda, panda, panda, “payday loan” update penalizing porn and phishy financial platforms. A lot more queries start showing Knowledge Graphs as well. In-depth articles and evergreen content became the biggest SEO content strategy. Authorship also gave authors more visibility and more impact. It would be dropped the next year.
2014 was panda, panda, payday loan and pigeon. Pigeon was a dramatic shift toward local SEO. That’s evolution before our eyes folks! We saw it coming over the years and here it is.
The Cretaceous period: Faster, Better, Contextual: Late 2010s – present
This is the last period ending with the extinction of the dinosaurs (except some birds). This era offered us the Parasaurolophus, a duck face type of herbivore, fondly referred to as the Lana Del Rey of dinosaurs by my friend Ezechiel. The Triassic Period was the shortest period of the Mesozoic Era. And that’s pretty much who this last decade has felt for our industry. AI, ML, Big Data: Google uses everything at its disposal to break down a query and really get at what the user is seeking. And our industry has once again had to adapt. But the cycles are getting shorter and shorter.
We had the Mobilegeddon: mobile rankings would differ for mobile-friendly sites. We had ghost updates, machine learning took over, RankBrain freaked out everyone. And then, Google threw up in for a loop in 2016 by removing the right sidebar full of ads to replace it with knowledge panels. Penguin and Panda were joined by Possum to impact local rankings this time.
In 2017, I published something about intrusive interstitials on Smashing Magazines because this update was just so complicated and murky. We had “Fred”, the randomly named major update, with reports of widespread impacts across the SEO community. Google launched the jobs portal. Yes, there are specs to show up in those results. At the end of the year, the knowledge panel results started taking over rich snippets in the results. We complained about it. The one test I won’t forgive was the 300 character meta description. This is why many of my clients, years later, are stuck with the false impression that they must put 300 characters in the meta description field in Shopify (the team left the microcopy with this 300 car guideline for years).
“In other words, trying to wrap an understanding of how Google works into one simplistic algorithm ignores the nuances of how complex search rankings really are.”
Schwartz, Eli. Product-Led SEO: The Why Behind Building Your Organic Growth Strategy (p. 26). Houndstooth Press. Kindle Edition.
I like this quote because it really explains how complex things have gotten. You can’t think of Google as a singular, all-encompassing démi-urge. If we continue this religious metaphor, Google is a pantheon of deities. If we plunge back into the Cretaceous ambience, we need to discuss all the animals that make up the ornithopods. This group of bird-hipped dinosaurs started out small but grew in size and numbers until they were the most successful Cretaceous herbivores. Well, think of each Google algorithm as a small-bipedal grazer coalescing into one big market-dominating search engine.
Will SEO exist in 5 years?
Yes. Just like we have animals still alive from dinosaur times, we will have SEO still alive from back in the black hat days. SEO will never cease to exist. What changes are the efforts that make up SEO.
Here are some animals that made it through:
- Crocodiles
- Snakes
- Bees
- Sharks
- Horseshoe crabs
- Sea stars
- Lobsters
- Duck-billed platypuses (they glow in the dark)
Here are some SEO practices that made it through:
- Meta titles
- Content hierarchy
- …
A word on keywords
For the last twenty years, keywords have been the most important element. As such, an entire industry of keyword-research tools popped up. At first, spelling, phrasing, order, and words of a query were incredibly important. This made no sense to me because if you truly care about user intent, you should focus on the meaning of the query, not its exact form. Today, Google has location data, time of day, and past search-history information, and now, machine learning is applied to the query experiences.
To be continued…?