A History of SEO, Part 1: Excite, Yahoo, and Jefferson Starship

Over the past quarter century, SEO (search engine optimization) has evolved considerably. In its earliest days, the idea was to trick the search engines into listing websites whether or not they were pertinent to the search. Common strategies included hiding unrelated keywords on the page, buying and farming links, stuffing content with nonsensical keywords, and even stealing huge chunks of content from other (higher ranking) websites.

Sadly, these skeevy tactics *did* work for a while. Search engines were much less sophisticated in the mid-late 90s. Most relied on a simple scan of on-page content, directory structure, and website popularity to deliver their results. When SEO practitioners used these “black hat” strategies to skew the results, users would end up with a list of websites that were, at best, barely relevant to what they were searching for.

These days, SEO is less about gaming the search engines and more about working *with* the search engines to ensure your content is available to the people who are searching for it. It’s about making sure your website is properly formatted and organized so it can be read, parsed, and delivered by the search engines. In short, SEO is no longer about exploiting loopholes; it is about best practices.

But what a long, strange trip it’s been. What follows are some of the highlights, lowlights, and odd moments in the history of SEO.

Dawn of the Search Engines

Although there is some contention regarding which search engine was first, the most likely candidate is Excite (which was launched as Architext in 1993). Developed by six Stanford students, this search engine was the first to crawl the web and index results based on keywords found in the content.

On the tail of Architext/Excite was Yahoo, which was developed by another pair of Stanford students and launched in 1994. In its original incarnation, Yahoo was manually indexed and required users to submit their websites in order for the to show up in the search results.

The year 1994 also gave us AltaVista, Infoseek, and Lycos, all of which operated in a manner similar to Excite. These search engines all provided results through ranking factors such as keywords, domain names, and site popularity (i.e. number of visitors). 

“Page #@&! Four, You #@&! Morons!”

In 1995, the rock band Jefferson Airplane was touring to promote its new live album Deep Space/Virgin Sky. Eager to capitalize on the burgeoning World Wide Web, the band’s manager, Bill Thompson, hired the technology company Cybernautics to develop their website.

Thompson wanted to show the new website to an event promoter, but he couldn’t remember the URL. He did a quick Lycos search… and was embarrassed to discover that the site wasn’t appearing on the first page of results.

So Thompson called Bob Heyman, a senior VP of Cybernautics, at 3:00 in the morning. Heyman answered the phone and Thompson tore into him with a profanity-laden rant. “Why the #@&! don’t we come up before page four on this #@&! thing? Page #@&! four, you #@&! morons!”

The next morning, Heyman dragged into the office, gathered his staff together, and explained to them that the company’s new priority was to figure out the art of search engine ranking. After some experimentation, the designers figured out that the reason the page was ranking so poorly is because other websites (primarily fan pages) were using the keyword “Jefferson Starship” far more frequently. To remedy this, they simply pasted dozens of instances of “Jefferson Starship” in black text against the black background. In a matter of hours, the page rank vaulted to the top listing on page one.

Heyman and his partner Leland Harland referred to this brand-new service as “search engine optimization."

Next: A History of SEO, Part 2: The Rise of Google