Its a common dilemma facing a funded project: do you deliver the software you’ve promised in your workplan by building it from scratch, or do you work with an existing open source project? If you build it from scratch, you keep control of the direction of the software, you can plan it according to your project workplan and schedule, and you know how you can report the effort involved to your funders. However, you’re likely to struggle with making that software sustainable after the life of the project.
On the other hand, engaging with an existing open source project brings a different set of risks and benefits; you can benefit from the contributions to the codebase from the wider community, and the software is far more likely to be sustained in the longer term as it doesn’t rely on the project funding. However, the project consortium is only one group of contributors to the community, and so the software roadmap and release schedule is no longer under control of the project.
This was the situation faced by the OMELETTE project, funded under the European Framework 7 Programme. OMELETTE had a need for a mashup delivery platform – in essence, a web platform for combining web apps created by mashup tools into shared workspaces. Requirements had been written, and there was a workplan identifying when functionality had to be delivered. The obvious route would be to go and build the platform, and then make some justification in the exploitation plan as to how the partners would make the platform sustainable when the funding ran out. However, the consortium decided to take a different approach, and work directly with the Apache Rave and Apache Wookie projects, which already implemented many of the requirements identified by the consortia.
Again, another obvious strategy at this point would have been to create a private fork of the software. That would enable the consortia full control over the direction and delivery of the platform; the drawback is that a fork can fall rapidly out of step with the original project, creating additional effort in porting important bug fixes and new features into the fork. This was going to be a big problem with Wookie and Rave, as both projects were evolving rapidly. Rave in particular as a fairly new project was getting new features implemented at a rapid pace.
So the approach taken was to work directly with the communities on the core codebase wherever possible rather than to create a project fork. This meant that the project could set up continuous integration (CI) using the current trunk of Apache Rave, ensuring that mashups developed in the project could be tested with the latest running code.
One immediate benefit was that some of the key requirements of OMELETTE, that could potentially have required considerable development effort, were being implemented as part of the Rave community roadmap. For example, inter-widget communication, a core OMELETTE requirement, was implemented in Rave without needing any real input from the consortium at all; all the project partners had to do was make use of it in their web apps when it turned up in the daily build.
For other project requirements, a more proactive approach was needed. One feature that OMELETTE needed to implement was a way of connecting workspaces with an external repository developed within the project; this could have been done as an OMELETTE-specific module, however when the project partners discussed the functionality it was found that it could be expressed as a more generic feature for connecting Rave with widget marketplaces (for example, Edukapp and Widgr). The internal requirement was rewritten using external language (terminology can be an issue when working across projects) and submitted to the Rave community as a new feature request. Almost immediately, other community members began commenting on their own need for such a feature. (This is now being implemented by OMELETTE partners and contributed back to Rave).
Of course not everything in OMELETTE involves open development; some aspects of OMELETTE for one reason or another haven’t lent themselves to working with existing open source communities, and have been developed entirely within the project. Some other components are closed-source. However the platform used to integrate the whole project is Apache Rave, including contributions from the project.
So could this approach work for other projects? I think it depends on a number of factors, not least the willingness of the project consortium to make a trade off between control and sustainability.
Realistically, most software developed in funded consortium projects is not sustained beyond the end of funding, although naturally every project claims it will do its best, sometimes for larger projects even setting up their own foundations. However, I think privately most organisations involved would admit that often such claims of sustainability are over-optimistic, and project officers and programme managers are I think increasingly skeptical unless the consortium members have the track record to back it up.
Loss of control might be a problem for consortia, particularly where there isn’t a close alignment between the consortium objectives and how the wider community sees the future of the software. In this case, OMELETTE were using Rave and Wookie in ways that were clearly complementary with the use being made by other members of the wider community, and so the control issue never arose. As a backup plan, if there was a requirement needed by OMELETTE that didn’t fit the Rave or Wookie roadmap, the project could have created a fork, but this hasn’t happened yet – and seems very unlikely.
It helped that in the OMELETTE consortium there was already a partner engaged in both ASF communities, in this case the University of Bolton. Developers from Bolton were able to bridge the gap between the consortium and the communities, bringing other partners to ASF hackathons, and introducing developers from other partners in the consortium to open development.
If there was one clear improvement that could have been made, it would have been to have conducted internal capacity building activity much sooner and with more resources; it can take a lot of mentoring and encouragement to get developers used to open development practices, and there can be plenty of opportunities – and incentives – along the way for going back into a closed way of working. If more effort had been made from the beginning in mentoring developers, the project could have been contributing actively to the community much sooner than it did.