The Role of APIs in the Future of Search

AppId is over the quota AppId is over the quota The author's posts are entirely his or her own (excluding the unlikely event of hypnosis) and may not always reflect the views of SEOmoz, Inc.

People talk a lot about APIs in the SEO industry (me especially) - the tools you can build with them, the competitive analysis data you can access, the reports you can automate. However, we tend not to discuss the wider picture, the thousands of APIs out there for other things, and, most importantly, the profound effect that APIs are going to have on the web, and thus the SEO industry, in the coming decade.

I believe that over the next 10 years there will be a huge decline in the number of users visiting websites, and that APIs (and structured data) are going to play a pivotal role in that shift.

To understand why I think this, lets take a look at the evolution of the web over the last 20 years.

(Note - I'm not going to explain APIs in this post - but you only need a broad understanding of them to get my point.)

The (public) web has been around for a little over 20 years, and in that time it has changed a lot. The first website, built by Tim Berners-Lee, looked like this:

(see it here)

With black and white text and no images, it was a pretty static experience, but was the birth of something amazing. Over the next 5 years, although the web grew and moved forward, it didn't see any drastic changes. Over the next 10 years the web got more colourful, with images appearing and layout being improved:

(Image courtesy of archive.org)

However, it was still a pretty static experience, and a long way from the web we know today, which looks more like:

Good old Bear Grylls. His site (which he updated this week!) shows off animation, high-res imagery and video, is interactive, has embedded social media, and is representative of the dynamic web we know and love today.

Obviously, this was a crude history of the web, but my point here is that in 20 years the web has accelerated from something akin a simple set of interlinked text pages to an interactive multimedia system. Let's tie this together with how the web is consumed and searched.

We've seen the web has become a far richer experience to interact with in the last 20 years. We can see that it has also undergone drastic changes in the way that we consume it. In the beginning, it was almost universally accessed via desktop machines:

Some readers won't even remember the days of having to turn on the external modem, dial up to connect and wait whilst the modem sung its song as it connected for a while. Using the web back then was something you made a slot of time for - you'd decide you need to check your email, and you'd head off and maybe spend 30 minutes or an hour using the painfully slow connection, before disconnecting and turning off the modem.

That slowly changed, and by the early 2000s you were quite possibly looking at the internet on a laptop:

(Image: Wikimedia Commons)

Your laptop maybe had a built in modem giving you 'mobile internet' you could use by a phone line, but we were starting to see 'broadband' connections and the advent of wifi. This was the beginning of always being connected for some people, but still the internet wasn't truly mobile, and there certainly wasn't widespread availability.

Nowadays, the majority of readers here will be accessing the web on their mobile:

There is prolific availability of wifi, most of us have 3G internet built into our phones, and using the internet is completely without thought; on the occasion that I'm not connected I'll inevitably try to do something on my phone before remembering "Yep, that also requires internet."

We have moved from interacting with the web via infrequent and prolonged interactions of minutes or hours to many short interactions of only a few seconds. In 2002, the average internet user used the web for 46 minutes a day, but in 2012 that has gone up to 4 hours a day. We're using the web for more time, more frequently, and in shorter bursts. Using the web has moved from being a task in and of itself and has become ubiquitous tool that we don't even notice. 

Finally, let me briefly discuss the way the web has been searched over the last 10-15 years. Very broadly speaking the general process, which has been changing in the last 2-3 years, has been searching in a generic fashion to searching in a specific fashion. Crudely this could be thought of as 'searching to find the place where you can search for the answer'.

You traditionally have started with the '10 blue links' result, which has taken you to a place where you could often refine your search in a context specific fashion. What do I mean by context specific? I mean that this specific search allows you to specify attributes unique to the type of thing you are searching for.

For example, in the image above, the search page on Amazon allows me to specify attributes such as author, ISBN and others which are specific to the context: book. Whereas the search box on Google / Bing is a single text entry box to cover all types of search for all types of results.

However, we have been seeing a shift in this generic -> specific model. Google has started to detect the context of your search from keywords, and then provide additional context specific inputs to refine your search. The classic example is the hotel search:

Now, beyond the general text field, Google allows me to enter a Check-In and Check-Out (attributes, unique to the context: hotel) date to refine my search, and see real-time prices from a variety of vendors. What is notable is that these prices I'm seeing from sites such as ebookers.com are right here in the search results - I'm not visiting their site to see them. I'm not seeing any of their marketing bumpf, any of their special offers or deals, I'm just seeing their price.

Google has also began customising the display of the results based on context, with various examples from maps to weather:

Rather than the 10 blue links of days gone by, we are increasingly seeing the a context specific SERP with a customised display. With emerging search technologies such as Siri, this is often the norm rather than the exception:

With Siri, there never was 10 blue links, but only ever context specific results, each presented in a fashion appropriate to that context.

Siri's weather results above are provided by Yahoo!, but again the user never visits the Yahoo! Weather site. Likewise, the results about the distance to the sun are from Wolfram Alpha, but the user never visits their website. The results are being server via APIs, allowing Siri to leverage Yahoo! and Wolfram Alpha's services directly.

If we look over this history of the web we can see that the web has become progressively more dynamic, progressively more interactive, and richer and richer. Along the way internet use has become increasingly mobile, increasingly ubiquitous, used increasingly often and in shorter and shorter interactions.

You'll notice above that I distiguish between 'the web' and 'internet use' - not all internet use is via web pages. The internet existed for many years before the web was invented, where email and bulletin boards reigned. We can see above that more and more often APIs are serving people's search requests in place of the web.

Beyond search, wildfire adoption of Apps in the last few years has seen another avenue which has replaced people web use with something else; people are Facebooking and Tweeting, doing internet banking, finding routes, sharing photos and more all directly via apps instead of via the web. Masses of these apps are using publicly available APIs, with others using private APIs.

If we extrapolate from the changes to the way we consume the web, the shift in the way search results are being delivered, the uptake of apps I can't see how there is any conclusion other than the web will continue to give way to these other technologies.

The technological changes that push this are going to keep on coming:

Will the web die? I don't imagine so, and certainly not any time soon. However, it is going to be ever more superceded, and progressively relegated to be where your marketing lives, and less where your customers interact with you.

The majority of the services that will replace the web will be connecting via APIs, whether they are public or private. Having a private API, with an App and other ways to access it is going to be increasingly important. Having a public API allows other people to build on top of your platform; Paul Graham of the startup accelerator YCombinator says "APIs are self serve business development" (Paraphrased from this tweet).

I can imagine a lot of you are thinking 'We don't need an API!' / 'What would we do with an API?' / 'We have a website, so no need for an API.' These all sound strangely familiar to the same arguments we heard 15 years ago with the web. In the 90s hoardes of companies were unconvinced by the web, but later came to regret it. It's how companies such as Nissan ended up spending millions in later trying to secure their domain names (nissan.com remains owned by a small computer company with a handful of employees), and how other companies went out of business to their competitors who did embrace the web.

I'm not sure that all companies need an API, and we are in early days still. We're going to see the landscape change more still, and certainly many smaller companies might not need an API. However, my point is that companies need to be thinking about this, we as SEOs need to be thinking about how this will affect the industry over the next few years, how we can be forward looking to help out clients, and I'm mostly hoping this post kicks off some discussion about it all.

View the original article here