A good API is not enough

[The post below was written for the JISC “Good API’s” project, but I like to get as much mileage out of my work as possible so I’m posting it here too]

We often argue about whether we can trust service X or whether we can hold our data in service Y. This usually boils down to whether the provider of the service is likely to survive, whether the data is secure and whether we can access it in an open format via a documented API. We usually fail to consider the influence we will have on the providers of the service or software exposed by the API, we therefore fail to consider our own futures.

For this post I’ll limit myself to thinking about web based APIs (which we should not do, but since the UK HE and FE communities tend to think API means access to a web service I’ll do the same here). In this sitution we need to consider access to the source code that implements the API. This should be an important part of our decision making process for many reasons , the most important of which is that it increases the options available.

As an example we can consider microblogging, an area that is getting a fair amount of attention in the education sector, see Brian Kelly’s WebFocus Blog for examples.

Twitter, probably the most popular of microblogging platforms, is web based and provides a clearly documented and reasonably complete API. Institutions and researchers are currently considering solutions based on microblogging systems like Twitter. The focus of such efforts include questions like “will Twitter be here in 5 years?”, “what is their profit model?”, “will they start charging soon?”. These are important questions because the instituion will have no control over their own systems if they depend on an external service like Twitter.

For small organisations with no IT support, the use of Twitter is attractive. It is low cost, feature rich and popular. Should the model of use within Twitter change a small organisation can adapt quickly. However, for larger organisations like Universities and Colleges something like Twitter can embed itself, almost unseen, into many different systems across the organisation. For example, it could be integrated into institution wide news networks, the student VLE and and researchers VRE, the student tracking systems and the lecturer feedback mechanisms (to name just a few of the ideas I’ve heard). Clearly an institution can quickly become dependent on a microblogging service and thus the longevity of Twitters support, at current pricing levels, is very important to medium and long term planning.  Unfortunately, there is no way of knowing where Twitter is currently heading and so planning becomes very difficult.

Some commentators argue that depending on any third party solution is too risky and inflexible. Others argue that third party solutions can provide significant cost savings with only a limited sacrific of flexibility. What we really want is a middle ground. A solution in which we can take advantage of third party solutions for as long as the opportunity cost of doing so falls below the cost of developing our own, independent, solutions. Fortunately, there is such a “third way”, that third way is open source software.

Continuing our example of a microblogging service,  an alternative to Twitter is identi.ca. Like Twitter, Identi.ca provides an API for accessing the system and so it can be integrated with in-house systems. It provides all of the features of Twitter and content can be bridged between the two (via the respective APIs). There is, at least on the surface, no significant difference between the two systems.

However, the Identi.ca platform is built using Laconica an open source microblogging tool. This means that if the hosted identi.ca service becomes unsuitable for any reason one can “simply” install Laconica on a suitable server and continue as before. Of course, it really isn’t that simple, by moving away from the Identi.ca server you are taking on the responsibility of maintaining your own server. You may also be forcing your users to rebuild thier social networks on yet another system, which presents a barrier to use. Fortunately, there is a middle ground in this case too. Since Laconica is open source, any third party could set up a company providing Identi.ca like services, including, for example, a group of collaborating universities.

It is tempting to say that an API is good if it provides the functionality we need and the ability to export our data if we decide to move on. However, as we have seen, this is not always enough, not when we need to plan strategically. Web based APIs are, in some cases, merely a way to provide the flexibility to customise systems whilst still locking you into a single provider solution. Experience has shown us that a monopoly in any domain is a dangerous thing.

We must be able to avoid lock-in to any individual providers services. Simply being able to export data from our current service provider is not sufficient. We also need to be able to find an alternative provider. Whilst open source does not guarantee the existence of an alternative it certainly increases the chances of one being created where demand exists. This in turn puts additional pressure on other providers to satisfy the needs of their existing customer base. This pressure often manifests itself through the provision of a more complete and flexible API. This can be seen in the example of the Identi.ca API which not only implements the full Twitter API but goes much further.