Archive for the 'Standards' Category

UK Government to level the playing field?

Published by Ross Gardler on February 25, 2009 in Discussion, Standards and Strategy and Policy. 5 Comments

In May 2008 OSS Watch published a workshop report with the title “Levelling the playing field: developing a mixed economy for software procurement”. This report focussed on procurement in the Higher and Further Education sectors and recomended that we work to:

  • ensure all solutions use open standards and provide protection against vendor lock-in
  • facilitate better communication with senior managers across HE/FE as to the potential benefits and pitfalls of making use of open source solutions
  • encourage educational ICT bodies with an overview of the sector such as UCISA and BECTA to assist institutions with open source related training and knowledge
  • work to improve the ITT and PPQ processes within institutions

OSS Watch has been funded by the JISC since 2003, part of our remit has been to facilitate the appropriate adoption of open source in the sector, yet the recomendations made in our workshop were largely the same as they were in 2003. Did this mean that OSS Watch was having no effect?

I’m pleased to say that OSS Watch have had some influence on the adoption of open source in the  education sector. For example, we worked with our own funders on an open source policy which was adopted in 2004. Similarly, we worked with BECTA during the creation of the Open Source Schools project (as well as helping BECTA understand what the goals of this project were we continue to provide advice, guidance and materials to the company running the site).

However, when it comes to influencing individual procurement decisions we have struggled to have any significant impact. Put bluntly, there is very little our small team can do when faced with procurement policies and staff that are predisposed towards the incumbent suppliers products.

I was therefore encouraged when the Cabinet Office published “Open Source, Open Standards and Re–Use: Government Action Plan“. This document is intended to put open source software onto an equal footing with proprietary forms for procurement.

As with the 2004 government policy on open source we are assured that “Procurement decisions will be made on the basis on [sic] the best value for money solution to the business requirement, taking account of total lifetime cost of ownership of the solution, including exit and transition costs, after ensuring that solutions fulfil minimum and essential capability, security, scalability, transferability, support and manageability requirements” and “The Government will use open standards in its procurement specifications and require solutions to comply with open standards. The Government will support the development of open standards and specifications.”

However, this new document goes a small, but important, step further.

It states “Where there is no significant overall cost difference between open and non-open source products, open source will be selected on the basis of its additional inherent flexibility.” Many commentators have, so far, missed the importance of this statement. The key is in the acknowldgement of “additional inherent flexibility”. This is over and above the flexibility provided by the adoption of open standards.

This “additional inherent flexibility” is a result of having access to the source code. Closed source software can adopt open standards, but they still provide a form of lock-in since there is only one source of customisation and maintenance for that product. When the source is freely available one is able to shop around various support providers in addition to selecting from various interoperating products.

The introduction of competition through open standards is clearly a step in the right direction. However, competition between software providers is also desirable. This is a topic I cover when presenting at procurement related events, and is something OSS Watch believe is very important given that requirements for software usually change as an organisation matures. These changes may not be aligned with the business model of the current support provider.

I’m also encouraged to see that the document identifies a number of actions including “develop clear and open guidance for ensuring that open source and proprietary products are considered equally” (action 1). This is a very complex issue and is something OSS Watch have been trying to do in the education sector for some time.

The problem is that open source and closed source solutions cannot be compared using the same techniques. Whilst the software products themselves can be compared on a feature by feature basis, the softer aspects, such as quality of support, security, flexibility and sustainability of the solution cannot be easily compared like for like. Consequently, it is necessary to change the procurement process itself before any real impact will be seen. Simon Phipps of Sun Microsystems suggests one potential model for a level procurement playing field through adoption led approaches, and warns about how the existing process can be gamed.

Further to the need to change the procurement process we must also ensure our workforce has the necessary skills to evaluate and engage with open source software. Without this skillset policies and action plans will fall on deaf ears, who is going to implement them? I discuss this in my November post “We have an open source future – or do we?“

Despite these concerns, I welcome this document from the Cabinet Office and encourage those with an understanding of open source and, in particular, how it should be evaluated to actively review and comment on the document using the CIO defined tag of #ukgovOSS so that it gets picked up and syndicated on a special public FOSS Aggregation page.

Microsoft, POI and odd distinctions

Published by Rowan Wilson on April 9, 2008 in Development, Legal and Standards. 0 Comments

In the run-up to the ISO vote on the controversial OOXML specification, Microsoft – OOXML’s creator – announced that they would be funding development of the open source Java API to access Microsoft Office formats Apache POI to support the new standard. Information Week reported on this announcement, and made the following statement:

For patented protocols, Microsoft said it would offer licenses on “reasonable and non-discriminatory terms.” Open source developers can access the protocols for free for noncommercial use without fear of lawsuits, Microsoft said.

Now, as we mentioned a week ago Microsoft accompanied their submission of the OOXML standard to the OSI with an ‘Open Specification Promise‘ in the following words:

Microsoft irrevocably promises not to assert any Microsoft Necessary Claims against you for making, using, selling, offering for sale, importing or distributing any implementation to the extent it conforms to a Covered Specification (“Covered Implementation”), subject to the following. This is a personal promise directly from Microsoft to you, and you acknowledge as a condition of benefiting from it that no Microsoft rights are received from suppliers, distributors, or otherwise in connection with this promise. If you file, maintain or voluntarily participate in a patent infringement lawsuit against a Microsoft implementation of such Covered Specification, then this personal promise does not apply with respect to any Covered Implementation of the same Covered Specification made or used by you. To clarify, “Microsoft Necessary Claims” are those claims of Microsoft-owned or Microsoft-controlled patents that are necessary to implement only the required portions of the Covered Specification that are described in detail and not merely referenced in such Specification. “Covered Specifications” are listed below.

This promise is not an assurance either (i) that any of Microsoft’s issued patent claims covers a Covered Implementation or are enforceable or (ii) that a Covered Implementation would not infringe patents or other intellectual property rights of any third party. No other rights except those expressly stated in this promise shall be deemed granted, waived or received by implication, exhaustion, estoppel, or otherwise.

This would seem to be a blanket promise (or covenant) to avoid taking patent infringement action against anyone implementing the current OOXML standard or using such an implementation – for example Sourcesense and the users of POI who will be receiving their Microsoft-sponsored OOXML code. There is no mention of a different deal for commercial use of open source implementations, as the Information Week story seemed to imply. Does this discrepancy matter? Well it has certainly caused anger and confusion among some in the free and open source community. Michael Tiemann President of the Open Source Initiative (OSI) and Vice President of Open Source Affairs at Red Hat Inc. quickly responded to the article with a blog post on the OSI site entitled ‘Microsoft’s new weapon against open source: stupidity‘. In it, Tiemann laments the unquestioning attitude of the media in reporting Microsoft’s seemingly discriminatory attitude to open source as cosy non-discriminatory affection. Trouble also kicked off on the Apache developer mailing list for POI, with a quotation of the Information Week article and a call for the Sourcesense code to be rejected from the project as it was ‘encumbered’ by Microsoft’s seeming insistence that patent licenses must be obtained for commercial use.

The mismatch between the Information Week article and the Open Specification Promise puzzled me, so I contacted Microsoft here in the UK to see what the truth of the matter was. The query found its way to Microsoft US’ Public Relations firm Waggener Edstrom who replied as follows:

Apache libraries are open source code, and available through broad licensing. Any required Microsoft patent rights relative to Office Open XML are available on a royalty-free, perpetual basis to all implementers, as outlined within the Microsoft Open Specification Promise.

So it would seem that the indirect quote from Microsoft in the Information Week article was either misreported or inaccurate, and the dismay in the free and open source community is the inevitable mistaken result of this mistaken account.

The question remains, though, how did the error occur? Certainly it could have been a typo somewhere, but I wonder if it is perhaps a result of the phraseology Microsoft adopts when discussing free and open source. To return for a moment to the web page of Microsoft’s ‘Open Specification Promise‘ we find that there is also a lengthy FAQ to elucidate the effects of the covenant. In this FAQ, Microsoft twice draws a distinction between ‘commercial’ and ‘open source’ software:

The Open Specification Promise is a simple and clear way to assure that the broadest audience of developers and customers working with commercial or open source software can implement specifications through a simplified method of sharing of technical assets, while recognizing the legitimacy of intellectual property.

and later

The Open Specification Promise is a simple and clear way to assure that the broadest audience of developers and customers working with commercial or open source software can implement the covered specification(s).

This is an odd way to speak, given that there are large commercial open source companies out there of whom Microsoft must be aware. While one can only speculate about Microsoft’s reasons for giving the impression that open source and commerciality are mutually exclusive, it certainly seems possible that this odd linguistic tic is the root cause of the confusion in Information Week and the resulting screams of protest from the free and open source community.

Notice: Sourcesense have provided speakers for OSS Watch events in the past, and a member of Sourcesense sits on our Advisory Committee.

Microsoft’s OOXML Wins ISO Approval

Published by Rowan Wilson on April 2, 2008 in Community, Development, Legal, Software, Standards, Strategy and Policy, e-Administration, e-Learning and e-Resources. 3 Comments

Perhaps wary that the date might detract from the news, ISO – the International Organization for Standards – waited until today before announcing that Microsoft’s Office Open XML (OOXML) document description schema has finally been accepted as an ISO standard as of April 1, 2008. There has been a long and bitter battle over whether this schema should be adopted. For one thing, an ISO-approved XML standard for describing office documents already exists in the form of OpenDocument created in association with Sun Microsystems by the Organization for the Advancement of Structured Information Standards or OASIS. Many argue that having multiple standards for the same objects defeats the purpose of establishing standards in the first place. While this is on the face of it a reasonable argument, it seems a little Utopian to expect complete global unanimity on these subjects, particularly where such valuable commercial interests are at stake. After all, the world has not even managed to agree on a standard standards body, so expecting agreement at any lower level seems over-optimistic. Microsoft’s OOXML has been a standard according to ECMA International since 2006, while OASIS approved OpenDocument back in 2005.

So why is there such bitterness over this issue? Well, some of it comes from the perception that OOXML is in itself an inadequate standard which has triumphed through Microsoft’s expertise at lobbying ISO member bodies for their votes. Critics point out that the standard is itself is incredibly long and complex – over six thousand pages. It has also been widely observed that rather than trying to select a set of characteristics that need to be described in order to define a document minimally and efficiently, OOXML instead describes a huge set of overlapping characteristics that define the many different ways Microsoft has described documents over the almost twenty year life of the Microsoft Office product. It is easy to see why they have done this; it greatly facilitates conversion of all legacy documents into the new format. Still, it also results in a swollen specification that competitors will find very difficult to implement in their products. For example, OOXML defines many functions such as shapeLayoutLikeWW8, which instructs a rendering application to arrange text around a shape in the same way as Microsoft’s Word 97. Clearly Microsoft will have an advantage over competitors in making their products reliably behave in these ways.

Back in September 2007 OOXML lost an adoption vote at ISO, partly as a result of muscular lobbying from the free and open source communities, and hundreds of changes to the standard were requested by the voting members. While many of these were implemented by Microsoft and ECMA, the majority remained unimplemented at the time of OOXML’s approval.

Another controversial aspect of the OOXML standard is Microsoft’s patent non-enforcement promise that accompanies it. International standards must at the very least include fair and non-discriminatory terms for the licensing of patents that their use might infringe. Generally the standards bodies prefer that associated patents are licensed at no cost, and this is essentially what Microsoft has done with their Open Specification Promise. It promises that Microsoft will not enforce their patents against anyone as a result of their activities implementing OOXML readers, writers or renderers. However Microsoft make no explicit promise that subsequent versions of OOXML will also be covered by such a promise, merely saying that they aim to continue the promise in areas where they continue to engage with open standards bodies. This has alarmed many people, pointing to a possible future where everyone has adopted OOXML only to find that Microsoft withdraw from engagement with standards bodies and also withdraw their patent promise for subsequent versions. In comparison, Sun’s Non-Assertion Covenant for OpenDocument offers a perpetual promise not to sue for both version 1.0 and all subsequent versions. In the run-up to ISO’s decision, the Software Freedom Law Center (SFLC), a free-and-open-source-supporting public interest legal practice, released a document filled with dire warnings about Microsoft’s Patent Promise, and telling anyone writing software under the GNU General Public License to shun it. SFLC’s argument is twofold. Firstly they argue that, despite the promise, a piece of multi-purpose code might be protected when used to implement the standard but infringing when used for something else. Secondly, they argue that Microsoft’s failure to extend the promise to future revisions of OOXML means that projects attempting to progressively implement newer and newer versions of the standard may hit a legal brick wall down the line.

Are these worries justified? Certainly the SFLC’s first point is well taken, given the propensity of free and open source developers to repurpose code. The second point is less persuasive, I think, and a little opaquely worded in their document. To be clear, implementations of the current version of OOXML will always be protected from patent action by Microsoft, whether they withdraw the promise from future versions or not (provided the code in question is actually used to implement the standard). As to whether Microsoft will actually withdraw the promise from future versions, it is a difficult issue to predict. Microsoft got into the open standards game in the first place in order to win procurement contracts – often in the public sector – where open standards are listed as pre-requisites. While it may be notionally possible for Microsoft to partially re-enclose their format by either withdrawing the promise from a future version or withdrawing from the open standards process altogether, the practicality of such a move would depend heavily on how Microsoft’s users would respond to it. Thus the future of the standard really depends less of Microsoft’s whim and more on ourselves and the organisations for which we work.

Open Standards are not enough to prevent lock in

Published by Ross Gardler on November 30, 2007 in Development, Discussion, Standards and Strategy and Policy. 4 Comments

Many people claim that open standards are the answer to lock in problems of software. Even our government can be heard to claim open standards are the answer:

There can sometimes be a danger of lock-in with some proprietary providers, and we must avoid developing an over-reliance on individual suppliers. The Government, via the Office of Government Commerce, work hard to avoid that by using open standards to ensure that different suppliers’ software can be used interchangeably. (Angela Eagle, The Exchequer Secretary to the Treasury during a parliamentary debate)

However, there is much more to the lock in problem than the format the data is stored in. We also have to consider how this data is stored and processed in any given business process.

Until recently I’ve been reading about peoples concerns over the closed nature of Microsofts Sharepoint with a pinch of salt. I have to admit I just didn’t get the problem. If the data was in an open format you could just take your data and run, right?

Well no, that is not the case. Thanks to my old boss Randy Metcalfe, I now realise the lock in comes in the form of business processes tied to the repository. Matt Asay explains in an interview with lwn.net:

Let’s assume you store data in ODF in a Sharepoint repository. It doesn’t matter that ODF is an open format. The repository holding it is proprietary, and that proprietary lock-in is doubled by the fact that the enterprise will build (proprietary, non-standard) workflows to manage that content which keeps content a prisoner to Microsoft.

This may be true, but the fact is that Sharepoint makes it possible to build these workflows. I’m aware of no other single tool, open or closed, that is as complete. Almost certainly this is why many of the people I speak to in the education sector report an interest in Sharepoint.

What worries me is not that these people are considering Sharepoint, it’s that they think that a move to Sharepoint, coupled with an adoption of open standards will prevent a lock-in to a single vendor. This does not appear to be the case.

So, if you are concerned about vendor lock-in what can you do?

Firstly, you should recognise that no software tool can be rolled out across an organisation without significant configuration and optimisation for the (often fluid) local business practices of that organisations. Buying any off the shelf product will always result in the need to also buy consultancy and/or staff training to provide ongoing support . As a representative of a major UK university recently told me “we thought we could buy the licences and pay some consultants and that would be it. Unfortunately it’s not as simple as that.”
Secondly, we must recognise that it is possible to create a software stack using mature and successful open source software that will do everything Sharepoint will do, and more. Sure, it takes effort to do this, but it can be done.

Finally, we must ensure that we evaluate any closed source solutions against any open source alternatives, taking into account all strategic, technical and resource objectives.

Why must we consider open source? There are many reasons, the most relevant to this post is that open source, coupled with open standards prevent lock-in.

Is Open Social an open standard?

Published by Ross Gardler on November 15, 2007 in Community, Development, Discussion and Standards. 1 Comment

The initial flurry of activity over open social is over. Now people can settle down and consider its merits and its warts. This very topic came up yesterday in a session I ran on open development at UKOLN. We discussed isues such as what makes it a standard? and what makes it open?

The initial open social partners will have us believe that it is a standard because they say it is one (a de facto standard). They also tell us it is open because anyone can implement it. But why should we listen? How can we influence its design?

The people behind open social are not stupid. They are also realistic enough to know that open social will only become a genuine de facto standard if it is used beyond the initial group. The easier it is to implement the more likely it is to be used, the more it is used the more “standard” it becomes.

Accordingly, Ning have offered to donate their implementation of open social, complete with an initial set of committers to support it, to the Apache Incubator. Assuming the project is accepted this will result in an easy route to implementation for almost any project.

So that’s initial take-up pretty well sorted then (unless the users think open social is a bad idea of course). What about the “open” part?

The ASF has a history of managing reference implementations of standards and of protecting users in standards definition processes. They are often held in high regard for this important work. I hope this means that the open social partners are going to allow the ASF to create a valid user community around open social. That is a community with a voice and an ability to influence the standard.
Lets see what smart developers can do with this. If you want to turn your project into a container you probably want to save yourself lots of effort and wait for the Ning donation to hit the Apache Incubator.

Open access bill in the USA looks likely to pass

Published by Stuart Yeates on November 8, 2007 in Community, Standards and Strategy and Policy. 1 Comment

Open access looks about to pass a significant milestone with a bill in USA Congress which requires open access to National Institutes of Health (NIH) funded outputs. While NIH funded research is only a small fraction of the peer review funded research globally, it’s one of the largest coordinated research programs with huge inertia, both externally and internally. It seems likely that all significant medical and genetics peer review publication forums will be open access in the near future. NIH also funds work in a whole range of disciplines which impact on human health, so they’ll receive an open access boost too.

Such a big win is built on the work of a whole lot of individuals and groups world-wide, including Stevan Harnad (long term open access über-evangelist) and the JISC funded Sherpa, OpenDOAR and ePrints projects. Congratulations guys.

The end of “walled garden” social networks?

Published by Ross Gardler on November 3, 2007 in Community, Development, Software and Standards. 0 Comments

I worry about attempting to build communities in walled gardens.

Is OpenSocial going to change the world? (If you need a quick intro to OpenSocial then watch this short video, or if your prefer reading try the API site).

Now all we need is a set of open source social network tools that allow us to build the niche social networks we need for real work whilst not expecting the participants to be totally cut off from the larger (and some would say more fun) networks that are currently popular.

FOAF + OpenID

Published by Stuart Yeates on November 1, 2007 in Community, Discussion and Standards. 1 Comment

FOAF + OpenID is a semantic web attempt to solve the problem of blog spam. The idea is that those people who have FOAF files and OpenID identities can identify each others networks of friends, colleagues and acquaintances using their FOAF files and authenticate the individuals using OpenID.

I’ve found a flaw in this: I could have (and probably should have) a link in my FOAF file to a semantic wiki representation of myself, which is (in the way of wikis) world writeable. Spammers could easily edit the wiki to insert a link from myself to them which would let them become part of the group and spam us.

There are a number of fixes for this:

  • Check the metadata in each FOAF file to ensure that it claims to be written by the subject of the file (which wouldn’t be the case for the wiki). This would require many FOAF/RDF generation tools to be updated.
  • Add trust attributes to external links in FOAF files. This would also require many FOAF/RDF generation tools to be updated.
  • Compile a list of known world-writable RDF sources and use it to black-list them. This would always be playing a game of catch-up and there some sites might slip through.
  • Require trusted users not to link to world-writeable RDF sources (or sources of RDF that harvest from the wider web). This requires that the semantic web workers work in a walled garden and not link outside it into the wider web.

None of these are easy.

Somehow this whole thing reminds me of the OpenPGP web-of-trust, without the cryptographic underpinnings.

Communities can’t flourish in walled gardens

Published by Ross Gardler on August 21, 2007 in Community, Information Environment, Software and Standards. 3 Comments

I recently posted on the dangers of using closed Social Networking sites to develop community. Since writing that post I have entered into numerous discussion about my position. These discussions have taken place in both “walled garden” tools as well as within open communities such as mailing lists, direct emails and the blogsphere. In that time I have listened to, and learnt from, many different views and I’ve started to come to the conclusion that, well, I’m only partially right to when I say:

I predict only one or two of the current Social Networking sites will survive, and they will be the ones that share their network data first.

Randy Metcalfe pointed me at a BBC story in which Michael Geist agrees with my “sharing data” point:

 The better approach – for users and the sites themselves – would be to work towards a world of interoperable social networking.

However, Micheal disagrees, like many others, with my claim that only one or two Social Networking sites will survive:

Some services may believe that it is in their economic interest to stick to a walled garden approach; however, given the global divisions within the social networking world, the mix of language, user preferences, and network effects, it is unlikely that one or two services will capture the global marketplace.

I agree with Michael, and others. There is lots of room out there for niche players. What we need is open standards for creating interoperable networks.

I recently started reviewing open source social networking tools and the standards they adopt. This will be published as an OSS Watch briefing paper sometime in the next couple of months. Please let me know, via your comments, of any social networking/news/bookmark tools you think I should look at.

XCRI: standard course information

Published by Stuart Yeates on July 30, 2007 in Community, Development, Standards and e-Administration. 1 Comment

At the recent IWMW, I went to a session on XCRI. Unfortunately I was too busy listening to take detailed notes and the presentation slides don’t appear to be in the web.

XCRI is a new standard for exchanging post-compulsory course information. Universities, further education, adult learning centres, vocational agencies and continuing professional development providers can all publish information about their courses, enabling careers advisers, institutions and government agencies to find the relevant information on courses in order to encourage people to enrol in them.

Previously there was no standard format for such information and the main consumers of it all require it in different forms. UCAS is a major consumer, as are any number of different government schemes aimed at increasing the take-up of educational opportunities and regional development programs aiming to tackle unemployment by retraining and upskilling. Institutions also typically have their own course catalogue of some description too. Keeping all of these in sync, both with each other and with what students of the course actually get taught is a significant challenge.

XCRI is an XML standard similar in nature to Atom: (a) it’s plain XML (for those people who want to keep things simple) with a mapping to RDF (for those wanting generalised knowledge representation); (b) it’s got a small number of tags as possible, and where ever possible those tags reuse definitions widely used elsewhere; (c) a feed is a list of items.

To make publishing XCRI easier, the standard assumes (but doesn’t enforce) that the feed is merely a text file on a webserver representing all an institutions forthcoming courses. This is to explicitly encourage batch export and validation of XCRI from legacy systems, which is expected to the dominant form of generation for most institutions for some time.

XCRI is a new standard, and their website is still under construction, but some of the community members have websites with decent information on XCRI. Indeed the community building around XCRI is very impressive, with support from a wide variety of institutions.

If you’ve got an open source or open development project you’re trying to build a community around, why not join the new community-development mailing list that we at OSS Watch have recently started? Unfortunately, no, we can’t claim the success of XCRI had anything to do with us, but we can certainly answer your questions and give you pointers.