Author Archive for Sander van der Waal

Contribute to the OSS Watch National Survey 2010

Published by Sander van der Waal on September 10, 2010 in Strategy and Policy. 0 Comments

Here at OSS Watch we have just started our National Software Survey for 2010 and we are in the data collecting phase. Everybody active in Higher or Further Education in the UK is invited to take part. This survey, commissioned by JISC for the fourth time, will assess the state of software policies and usage in Further and Higher Education.

In previous years this survey has been sent out to ICT directors across FE and HE institutions in the UK. This year, however, we have decided to expand the survey, in order to gain a broader view of the state of open and closed source software in FE and HE. Therefore, if you work in the UK HE/FE sector, we encourage you to contribute. Your data will provide invaluable background on the status of open source use as depicted by ICT directors, and will make an important contribution to the future planning of ICT procurement and support by FE/HE institutions and the JISC.

Please access this year’s survey here.

We appreciate that you may not know the answers to all of the questions as you may not be dealing with all aspects of policy planning and implementation. We would, however, appreciate any information you can provide, so please don’t hesitate to give this survey a go.

The previous survey, conducted in 2008, found that there awareness and usage of open source software has increased significantly, although closed source software remained more popular than open source in both HE and FE institutions. The survey pinpointed the main obstacles to engaging with open source software as being a perception of difficulty and a need for more resources and highly skilled staff, as well as a perception that open source software is not supported.

On the OSS Watch website you can read the detailed findings from the 2008 report (pdf) and the 2006 report.

Open innovation tactics and incentives applied to software

Published by Sander van der Waal on August 27, 2010 in Community and Strategy and Policy. 3 Comments

A very interesting blog post was published on the 100% Open website about 7 tactics and incentives for open innovation. It struck me how well these all apply to open source software projects. So I’ll discuss all 7 of them from the perspective of open source, but make sure you’ll also read the original post for the original, more generally applicable view on these tactics and incentives.

1. Share both Risks and Rewards

When participating in an open source project you are largely in the same boat as all the other contributors to the project, therefore sharing the risks among each other. If a release is delayed or major bugs are introduced in the software, everybody suffers. However, some open source licences allow you to add your own private rewards by building your own customization of the software without contributing it back to the project. It is a bad idea to do so because when you let your code deviate from the project’s code you always end up with more complex migration paths which makes it harder to keep profiting from the efforts of the community.

2. Tap into Intrinsic Incentives

Intrinsic incentives are extremely important for open source software projects. There is still a widespread misconception that open source software is being developed by hobbyists where there is no money involved. This is not the case, because a large majority of the code in open source software projects is being developed by people who are paid by their employers to do so. This is also true in the educational sector in the UK, where software projects are being fund by the likes of JISC and the research councils. Nevertheless, for any sustainable open source community intrinsic incentives are very important. For example in the Apache Software Foundation, when a contributor becomes a committer to an ASF project they personally become one and never as an employee of some company X. Being part of a community that builds cool software is just great and having a culture within the project that feeds into that is therefore extremely important. A nice illustration of this Dan Plink’s TED talk on motivation. He shows in a very powerful way that highly skilled people are not mainly motivated by money, but by being challenged and by the opportunity to develop a mastery.

3. Don’t Expect Something for Nothing

For an open source software project to be truely sustainable, external contributions and engagement from new participants are extremely important. Usually, a public mailing list or forum is the first entry point for potential contributors. Although it is likely that people first ask questions on these lists rather than answering them, in a healthy project all participants help out each other. This makes the project scalable and is one of the reasons why it does not necessarily takes a lot of time to open up a software projects to the outside: if you manage to engage new people they will help out others and that way a truly sustainable community can develop.

4. Ask Engaging Questions

People or companies that are involved in open source projects never have completely overlapping problems and therefore it is not always clear which solution is the most appropriate for all of them. Moreover, if you encounter a project that provides a lot of the functionality you need but not all of it, there are very effective mechanisms to discuss the features of the project. Mailing lists and forums are used widely to engage in discussion and find ways of merging features different people need. Of course, if you require a specific piece of functionality, it is up to you to build it and contribute it to the project. But discussing the requirements and problems of different people can lead to interesting insights that can be valuable to the whole project. Due to the distributed nature of open source software projects people with very different backgrounds will bring their own viewpoints, which can lead to more creative solutions and spark new ideas.

5. Build Business Empathy

Open source projects can thrive or be damaged by reputation just like businesses. The plea in the original post for an honest and human approach is very well applicable to open source projects. But in many cases it comes more natural to open source projects to have that approach because, as mentioned earlier, there is already a focus on individual contributions incorporated in the dna of many projects. For new projects or projects that are working towards sustainability it is important to define processes that support this approach and to fix it in a governance model document, so it is clear to everybody what they can expect from the project, thereby providing a more level playing field.

6. Target Quantity before Quality

This tactic is well-known in software where it is more commonly known as the ‘Release early, release often’ mantra. If you are active in a young open source software project that is still in its infancy, getting a release out is a very effective way of engaging new contributors and is therefore a huge opportunity to let your project grow to become sustainable. Releasing early makes the barrier to entry lower for new users, albeit that the first few releases will be of lower quality and contain less features. As long as this is clearly communicated to the (prospective) this need not be a problem but can help the project as a whole move forward more quickly.

7. Find Your Top 1%

In the original post the 100% open team explains that out of 100 users, there are usually only 10 who are really engaged and just 1 who will provide a substantial contribution. Although the percentages may vary, also in open source software projects it is very important to identify the users of today that are most likely to become the contributors of tomorrow. It is essential for any open source project to engage those users and try to have them contribute to the project and perhaps even become a committer to help achieving sustainability in the long run.

OSS Watch community development manager Gabriel Hanganu published an excellent briefing note recently, in which he explains how the sustainability lessons can be appied to research infrastructure. Gabriel’s analyis shows that a lot of the tactics and incentives for open innovation are also important in that space.

Why it makes sense to sustain your project beyond its initial funding

Published by Sander van der Waal on August 3, 2010 in Community, Development and Strategy and Policy. 2 Comments

Scott Wilson from CETIS, University of Bolton showed in a very compelling way at TransferSummit/UK 2010 how it can be strategically important to sustain your publicly funded software project beyond its initial funding period. The figures in Scott’s slides say it all: by investing a tiny survival budget to sustain their Wookie project after the funding would run out they managed to secure about £700k of new funding from two European (FP7) projects.

How they achieved this? Their overall project, although being a bit specific, implemented the emerging W3C widget standard which is relevant to a wider community. They managed to attract some interest from outside the initial project group. OSS Watch helped them with community development and identifying potential sources of value and funding. A good home for the project was found at the Incubator of the Apache Software Foundation, thereby attracting much more interest and contributions from parties inside and outside the academic sector.

Currently, Apache Wookie (Incubating) is a thriving project and has seen many bugfixes and new features contributed by the community. It resulted in a lot of visibility for the University of Bolton outside the regular channels, leading to new partnerships with the commercial sector and universities inside and outside of the UK. Last but not least they managed to secure a lot of new project funding from European sources.

Sustaining your software project beyond funding is not just morally right or something that should be done so your money is not spent wastefully. Scott’s example shows that it is very much in the interest of the institutions and the project team to sustain the project. So think about how your software development project can be sustained after the funding has run out or which part of it is most potential to generate a viable community. And get in touch with OSS Watch; we are here to help.

The increasing importance of open source for the EU

Published by Sander van der Waal on July 22, 2010 in Community, Discussion and Strategy and Policy. 1 Comment

An interesting video message from Neelie Kroes, European Commissioner for Digital Agenda, was published last week. The message was recorded in support for GNOME and its events, such as the upcoming GNOME Users’ And Developers’ European Conference.

YouTube Preview Image

In this video, she reflects on times when open source was not considered in public organisations, because of issues that were perceived such as lacking technical support and worry about IP infringement issues. The consequence was that officials were discouraged from selecting software on merit. There may have been open source products out there sufficiently fulfilling the requirements, but this was not enough to be considered by public organisations.

But times are changing and it’s meaningful that Kroes, who is also the Vice-President of the European Commission, acknowledges this. She mentions a few developments to illustrate this point. For example, OSOR provides unbiased advice and guidance on the use, development, and licensing of free and open source software (sounds familiar?). It also contains a ‘forge’ providing a home to open source software projects.

Kroes also mentions the European Interoperability Framework (EIF) as an example. This is interesting because there is a lot of debate and lobbying going on regarding version 2.0 of EIF. A few months ago Glyn Moody analysed a leaked draft of EIF 2.0 highlighting the confusing and vague references to openness and open source software. More recently, the New York Times wrote how companies like Google, IBM and Red Hat are lobbying for inclusion of open source software in the document, which is perceived as a strategy to break Microsoft’s hold. It probably won’t be until the end of the year before the final version 2.0 of the EIF document is released. Although the closeness around the drafting process of this document does not look very promising, it may be a good sign that Kroes includes the framework in this speech.

Additionally, Kroes stresses the importance of strong communities and the role they play in shaping Europe’s digital future. And now the EU commission has the opportunity to put the money where their mouth is, as it recently announced to fund projects worth 1.2 billion Euros to be launched in 2011. This is a genuine opportunity to invest in open source software and in open source companies to make sure that the open source offering can compete better with companies that offer proprietary alternatives.

Why TransferSummit appeals to a broad audience

Published by Sander van der Waal on June 21, 2010 in Community and Development. 0 Comments

We are only two days away from the start of TransferSummit/UK, the conference on open innovation and innovation in open source software that is sponsored by OSS Watch. TransferSummit is the perfect opportunity for academia to meet business and vice versa. We hope that everybody will find something valuable to take away from this conference, whether it’s about innovation, collaboration or development on open source software.

The three themed tracks provide a very diverse programme which allows attendees to pick and choose. If you’re coming for inspiration, you will have to make sure not to miss the keynotes by Steven Pemberton and Roland Harwood. And hear about the ‘thoughts from the frontline’ by Simon Phipps.

Are you more interested in open source software foundations? We’ve got speakers from the Mozilla Foundation, GNOME Foundation, LiMo Foundation and The Apache Software Foundation. Not to mention the new CTO of the CodePlex foundation, Stephen Walli, who will be talking about the direction CodePlex is heading.

Perhaps you have a more technical background and are running your own open source software project. In that case you may want to know more about ‘Managing IP’ which will be the topic of OSS Watch’s Rowan Wilson’s talk, and go hear more about project governance by Ross Gardler.

Or are you from a business background and would you like to know more about how to make money out of open source? Then the session on ‘FOSS business models’ by Sirius’s Mark Taylor is for you, and you should hear from Bertrand Delacretaz why his company Day Software open sources most of their code.

And this is only scratching the surface of what the programme at TransferSummit has to offer. There are still a few places available so come down to Oxford on Thursday and Friday.

In case you are not able to come to TransferSummit, make sure you bookmark our live page where you will find two live blogging streams as well as the Twitter feed (hastag #TS10).

Using open innovation to meet ambitious carbon emission targets

Published by Sander van der Waal on May 18, 2010 in Community. 0 Comments

The ICT sector faces big challenges in lowering their carbon footprint and there are many initiatives to make technology more energy efficient. The JISC has created a separate Green ICT programme for ICT projects that address these issues.

In Oxford (where OSS Watch is based) one of these projects has been quite successful. The Low Carbon ICT project has developed several tools to lower the carbon emissions at the University of Oxford. One of these is the creation of Wake On LAN software project to remotely manage desktops in a computer network. By remotely shutting off computers at night, institutions can save a lot of energy.

The University of Oxford has recently reconfirmed that it is serious about lowering their carbon emissions by signing up to the 10:10 initiative. The University’s 10:10 target is to reduce carbon emissions by between 3 and 10 per cent compared to 2009-2010 levels by 31 March 2011.

In order to meet this challenge it is very important for the University of Oxford to work together with other universities and the private sector. A lot of innovation is needed and all institutions face this challenge. The model of open innovation, whereby ideas and solutions are shared as widely as possible, is perfectly suited to make sure a good solution developed at one university is used elsewhere. This model, perfected in open source software projects, is already working well for the Wake On LAN software that Oxford has developed. The same software is now also in use at Liverpool University and they are developing it into a national service for other HE/FE institutions in the UK.

To foster open innovation in the Green ICT sector OSS Watch has recently submitted a project proposal for a call on open innovation. Together with Sirius, PC Power Down and Data Synergy we aim to bring together academic and commercial partners to create innovative Green ICT solutions on an open platform. If funded, this platform will be based on our existing public registry of open source software projects, which currently already contains over 1500 projects.

Open innovation will also be a central theme of TransferSummit/UK that takes place on 24/25/26 June in Oxford. The TransferSummit provides a forum for business executives and members of the academic and research community to discuss requirements, challenges, and opportunities in the use, development, licensing, and future of open source technology. Registration is now open and if you work on academic projects don’t forget to contact us for a discount code.

Making your open source software releases more secure

Published by Sander van der Waal on April 21, 2010 in Community and Development. 0 Comments

When you’re writing code in an open source software project, you are generally using some version control system (you should!). Hosting websites like SourceForge and Google Code usually provide one free of charge, eg. a Subversion repository. All developers or other people interested in the source code use a client application to download the code and synchronize with the repository on the server. Users can download the source code using http or https.

Making your code available for external parties by using a version control system is a very good idea. But if you want to attract more kinds of users than just developers, you should also periodically create a release of your source code and make these separately available for download. Less or non-technical users will then find it easier to use your software and get involved in the project more easily if they don’t have to build the executables themselves.

However, if you release executables through your own (or a hosting) website, this is not entirely risk-free. If someone tempers with your binaries, eg. by adding malware to the code, your users may be downloading malicious code to their computers by using your software, and of course you would want to prevent that. Two ways of doing this are to hash the download and make the checksum available as a separate file, or by signing the release with a PGP signature.

Adding a checksum to your download
Several algorithms have been developed to generate a hash checksum for a file, eg. MD5 or the SHA family (SHA1, SHA128, SHA512). The purpose of creating this message digest is to assure that the file you downloaded is exactly the same file that was offered, so no one byte can be different. This does not only help against infringement from outsiders, but also helps you detecting technical errors in the transmission of the file. An easy, simple-to-use tool to generate hashes is the open source application GNU Privacy Guard. It supports MD5 as well as several SHA algorithms. In general you should use the algorithm with the longest hash key, as they are more secure. With brute force it is possible to break all hash keys, but a short key like from MD5 is much easier to crack than one like SHA512. The process is fairly simple. You create the checksum file for the generated download and upload them both to your website. The user interested in the download also downloads the checksum and uses a similar tool as GNU PG to check if the checksum is correct.

Signing your release with a PGP signature
Another way to protect your released binaries is to sign them with a PGP signature. You can use the same tool GNU Privacy Guard for that. When you start using it, you first need to generate a public/private key combination. You will have to publish the public key in a file, commonly this is one KEYS file for the whole project that contains the public keys of all relevant developers. Next, you can sign a release binary using your own key, which will result in a signature file, which is quite similar to a hash key file. Now anybody that can download the binary, your public key and the signature file, can check whether the signature matches with the public key and the binary. This will ensure that the file that has been downloaded is the same file that has been created by you, the release manager.PGP signatures can provide an extra level of protection by identifying that the public/private key combination that signed the binary indeed is linked to the individual. It works in a decentralised way (as opposed to the PKI method that needs a certifying authority). Once you have created your own keys you can exchange your public key with other users and thereby add them to your web of trust. You do that by signing each other’s keys, but you should really do this in person to make sure you know who you are dealing with, eg. at a conference.

Finally, if you’d like to know more the Apache page on signing is an excellent resource. For more technical details about signing Henk Penning’s page is very informative.

Then, as a really final note, I should add that these mechanisms of securing your releases are not error-free and don’t fully guarantee that people with malicious intentions can do harm, because they usually always can. However, the described mechanismsdo make your releases saver than not using any security and it’s therefore a good idea to use them.

Opportunities for scientific research in open source projects

Published by Sander van der Waal on March 3, 2010 in Community and Development. 1 Comment

There are many interesting open source projects that can be beneficial to academic research. As OSS Watch’s recent article on e-Research by Gabriel Hanganu shows there are social and organisational problems in adopting open source for e-Research, but there are many open source software projects there to be joined. Some projects are suited very well to be used in scientific research and I feel that this is especially true in the realm of big data databases.

Google showed the way, really, with the MapReduce paper in 2004. They published their programming model for processing large amounts of data in parallel and although publishing it, they did not neglect to apply for a patent as well, which was recently granted. Hadoop, which originates from a project at Yahoo!, also implements the MapReduce pattern, but is completely open source being a project of the Apache Software Foundation. And now recently Apache Cassandra has joined the mix. Cassandra originates from Facebook, but has become open source in July 2008. It recently promoted from the Apache Incubator and is now an official top-level Apache project.
Work has been initiated to facilitate integration between Cassandra and Hadoop, which simplified means the Hadoop database HBase is replaced with Cassandra. There has been discussion of this on the list and a feature has recently been implemented. So there’s Yahoo! working on Hadoop and Facebook working on Cassandra, and recently also Twitter has announced that it is working towards using Cassandra for their backend. Also worth mentioning is the open source implementation of Amazon’s Dynamo database which is named Voldemort. This project is used and actively developed by LinkedIn and is therefore another example of how you can benefit from the work this large company is investing by engaging with this project.

To me, this all shows that there will be large investments in NoSQL databases from major companies in the coming years, and it will all be in open source software. This means that there is a lot of opportunity for anybody who has to deal with big data to profit from this investment. All you have to is try out the software and engage with these projects. Researchers also have to cope with more and more data, so I think they have good reason to follow these developments closely and step in to benefit.

Building W3C widgets on the Wookie training day

Published by Sander van der Waal on February 15, 2010 in Community, Development, Event and Standards. 2 Comments

Last week OSS Watch organised its first training day in Oxford. We got together with about 15 people to gain hands-on experience with Apache Wookie (Incubating). Wookie provides an implementation of the W3C widget specifications, so a lot of emphasise was put on building these kinds of widgets. We succeeded quite well in getting to know the spec and how to build widgets and ended the day with a nice collection of newly built widgets and even a submitted patch to the Wookie source code.

Scott Wilson, the Wookie guru from Bolton University, where it all started, started the day off with a presentation (pdf) of what widgets and Wookie are all about. Widgets are basically small mini applications that are designed to work in a small view area. Many platforms have created their own format for it, but the W3C is working on a set of specifications for it with a consortium of partners from both traditional computing and mobile platforms, which will lead to a true cross-platform standard which will hopefully lead to widespread adoption. A minimal W3C widget consists of nothing more than a config file and an HTML file, zipped up as an archive with file extension .wgt. The config file contains basic configuration such as the name, description and preferred dimensions of the widgets. The widget can furthermore include as much HTML, CSS, images and JavaScript files as one would like.

Apache Wookie (Incubating) is an application that provides a W3C-compliant widget server. You can use Wookie to deploy widgets and you can serve W3C widgets from the Wookie server in third party applications. Plugins have already been written for Moodle, LAMS, Sakai and Google Wave. Wookie also has a REST API that can be used to get, or create widgets.

After Scott’s intro it was time to get dirty. Ross handed out CDs containing the latest sourcecode of Wookie (which can be downloaded by anybody from Subversion) and prerequisites like a JDK and Apache Ant. His presentation (pdf) was about ‘how to build your first widget’. I was surprised to see that there were 10 people with Mac-books in the room, amongst 4 Windows machines and one Linux netbook. Apple surely knows how to impress the developer these days! After some initial troubles with environment settings etc. most people got up-and-running fairly quickly and were ready to build their first widget. Wookie provides handy Ant tasks for building and deploying widgets, which means that generating a hello-world skeleton widget is as easy as typing ant seed-widget and answering some questions about the name, description and dimensions of your widget. After you have started up the Wookie server using ant run you can deploy the widget using ant deploy-widget. That was it, quite easily. I must say, having moved away from Ant and using Maven2 for the last few years, it’s nice to be remembered of the powerful features Ant has to offer. Especially since Wookie uses Ant in combination with Apache Ivy, the dependency management alternative for Maven2. (To be precise, you can also use Ivy with Maven2 repositories). Ross also demonstrated how you can make use of OpenStreetMap JavaScript APIs to embed cool navigational features in your widget quite easily. You can check out his presentation (pdf) or directly check out the source code of the tutorial including the example JavaScript.

In his second presentation (pdf) Scott focussed on some design principles behind the widget specification and gave a walk-through of how you can design a more advanced widget by making use of features of the W3C widget object API and integrate with the Google Wave Gadgets API. No Wave server is needed to get this working, as Wookie can handle everything for you. Scott demonstrated a Task widget with collaboration features, that can be used by different users concurrently using State and Participants.

After the break it was high time for everybody to create their own widget and some interesting ideas had come up. One of us decided it would be much cooler to hack directly in the server code instead of building widgets and he submitted a patch to Wookie to allow hot deployment of a widget to ease the development/deployment cycle. That’s very cool, thanks Matthew!

The rest of us built some widgets for a wide variety of purposes. One of the nice things about the widgets was that we could easily merge them all together on one Wookie instance and show all widgets there. These were some of the widgets that resulted from this 1.5 hour hack-fest:

  • Video player embedded in a widget with fallback to other formats depending on the user agent
  • Display a list of links using output from one of the Yahoo pipes
  • Display the last.fm playlist of a user and show what that user is currently listening to
  • Show a canvas drawing where multiple people can collaborate by working on the same drawing using HTML5
  • Cool kids’ game where the user can name his pet dinosaur
  • Currency converter that (eventually) would use an external currency conversion provider

It was fun to see how easily you can create functional widgets. If you make use of external JavaScript APIs or data feeds it is also quite simple to create a useful (or not so useful…) widget. This was a nice conclusion of the day and seeing all the widgets we had created we thought we had deserved our beer and headed off to the pub. Thanks to Scott and Ross for making this a successful Wookie training day!

The power of community put into practice

Published by Sander van der Waal on January 26, 2010 in Community and Development. 1 Comment

At OSS Watch, we actively promote that there is more to open source software than just a licence. Open source projects should use not just an OSI-approved licence but practice the open development method and if they want to become sustainable they should be building a community around their project. Once in a while, we come across a nice example of how the power of the community can be beneficial, and recently one of these examples occurred.

It started with an application that has been built by Nick Burch at the Apache Software Foundation to facilitate the search of geographically ‘nearby people’. He made this little Django application available via a Subversion repository with an Apache licence.

Linking people and projects is also one of the aims of the project registry framework Simal that OSS Watch is involved in. On Simal’s public demo site there is a collection of projects and people working on these projects. Besides doing development work on the Simal application OSS Watch is starting to use the registry more often in our daily work. Unfortunately, we recently failed to find out about a project that was run at our institution, Oxford University, even though it was present in our public registry.

When I realised Simal was lacking functionality that had been useful for OSS Watch, i.e. to find nearby projects based on location, I created issue 263 for Simal, dumping my thoughts about possible solutions, among which the ASF application on nearby people.

A key problem in adding this functionality was to have the geo-location data of the institutions that are in involved in the projects. This prompted Ross to reach out to his wider community to see whether anyone had tackled this issue.

The first and very useful suggestion on this matter was from Paul Stainthorp who pointed to a list of UK universities and their geo-location, which is maintained at Wolverhampton university.

The second one was from Sam Easterby-Smith who pointed to a list on Wikipedia. That was a good one, as Wikipedia is quite complete and geo-tagged, so we would have the data from that source if only we had a convenient way of extracting it.

The solution to that problem is to use DBpedia and it was suggested both by James, who added a comment to the issue in the tracker, and by Wilbert Kraan on Twitter. DBpedia is a community effort to extract structured information from Wikipedia and it provides a public SPARQL endpoint for querying Wikipedia data. We can conveniently query that endpoint for a list of the geo locations of al UK universities and add that data to our Simal repository.

So within one working day we have a solution to the main problem, getting the geo-location data. But Ross’s discussion with someone already doing this revealed that they are manually creating the data, so they can potentially benefit from our search and automate it, if they want to. Furthermore, someone on Twitter noticed our search and he indicated that he would be interested in the solution, so potentially more people and/or projects can benefit. Furthermore, since everything happened completely in the open, even more people have the opportunity to find our solution and use it in their own problem space.

To me, this is a perfect illustration of the power of community. There is just so much that we all collectively know and by having your project run out in the open, freely accessible by everyone, enables you to tap into the collective knowledge of many experts. If this is not a reason to use the open development method, I don’t know what is.