The Sahana Software Foundation are looking for additional developers to help with Sahana, a disaster management tool. They need to continue to build an information portal that is seeing increased interest and usage to assist organisations responding to events in Haiti. Information on the work underway can be found on the Sahana Haiti response Overview page.

The skills they are primarily looking for are (you don’t need them all, any will do):

•  Python - the main development for SahanaPy is Python (we’re not using PHP for this instance)
•  web2py - Sahana uses the web2py enterprise framework for SahanaPy (I’m told it is fairly easy to learn if you’re used to Django)
•  OpenLayers
•  jQuery

To find out more and offer your help please jump onto #sahana on freenode. Please give as much or as little as you want.Some of the core devs have been working really long hours the past 5-6 days, and any additional resource would assist us greatly, particularly as the effort is really starting to get some interest, traction and coverage, which leads to more and more feature requests.

Thanks for reading this far - please consider giving a few hours of your time to help out, if you can.

This has been a very hard piece to write. We felt we needed to talk to as many people as possible, we needed to sift through  significant amounts of Fear Uncertainty and Doubt along with unnecessarily emotional responses.

Things weren’t made any easier by the fact that every time we felt ready to publish something else heppened that seemed to change the story somewhat and we had to return to our sources for more observations.

During our research for this article OSS Watch have been accused, by an OSI board observer and ASF Member, of being “surrogates” for Microsoft, whilst Tony Hey (Corporate Vice President of External Research, Microsoft) privately expressed concern that OSS Watch was “encouraging academics to use the GPL.” Simultaneously, various free software representatives have pointed out how “naive” they believed us to be by even considering the idea that Microsoft may have genuine intentions with respect to engaging with the free and open source community.

As a non-advocacy advisory service we tend to think that if all sides in a debate believe we are in the wrong, yet all are still talking to us, we are probably doing something right. Certainly none of them can claim us as their own.

Given all this input what did we conclude?

Well, as you would expect, the conclusion is far from clear. On the one side we have the Stallman’s (Free Software Foundation) view that “these free programs are meant specifically to prevent the world from freeing itself from non-free software”. On the other side we have Erenkrantz’s (The Apache Software Foundation) view that “every positive and constructive engagement Microsoft has with the open source community (and vice versa) … will continue to chip away at the old perceptions”.

Furthermore, whilst Microsoft may be making concessions to open source and are happy to play with open source when it suits their needs they are also willing to use other methods where it best suits their business. For example, on patents Darren Strange (Head of Open Source Engagement, Microsoft UK) says “Patents drive innovation and they drive openness actually.”

Our own conclusion is that “Microsoft is not simply an unchanging monolith.” The article demonstrates that things within Microsoft are changing. Naturally they are changing in ways that benefit Microsoft as a business, but the good news is that some of these changes also benefit the world of free and open source software.

Over the years I have often quoted Ghandi when looking at Microsoft and their relationship with Free and Open Source Software: “First they ignore you, then they ridicule you, then they fight you, then you win”. FOSS has not “won” yet, but the frontline is moving and it is open source software that is winning.

Apache Wookie passes W3C Widgets conformance

After a marathon code sprint [well done Scott - RG] Apache Wookie (Incubating) now passes all 166 W3C Widgets conformance tests, the third application to reach a 100% pass rate.

Two other applications - the Aplix Web Runtime engine and the BONDI reference implementation for Windows Mobile - have also been able to successfully pass all the conformance tests. Several others are also approaching a full pass rate, as can be seen on the W3C implementation report.

Not only is this good news for Wookie its also good news for W3C, as more successful implementation helps the progress of the specification. Also, open source implementations can also help other developers build interoperable applications by reusing code. I hope in future we’ll be able to make the widget parser in Wookie distributable as a standalone library as well as part of the Wookie widget engine, to help with this process.

Useful links:

• W3C Widgets 1.0: Packaging and Configuration
• Apache Wookie (incubating)
• A good list of resources on W3C widgets development

I guess this is one reason why the ASF board recently approved a resolution to create a new Community Development project and appointing me as Vice President of Community Development. I look forward to this new activity in the ASF allowing me to further cross-fertilise between the ASF and the UK HE/FE sector, whilst the launching of a new EC project focussing on a wider engagement of computer scientists in open source projects will allow us to reach out to other software foundations.

For example, Greg Kroah-Hartman reported, in a Google Talk, that of 2399 unique contributors active on the Linux Kernel project during a year and a half, roughly 75% of the work was done by people who are paid to do so. All of those people are considered volunteers within the project, but they are not there for the good of their health (although open source work is often fun). Furthermore, half of those contributors only contributed a single patch, whilst thirty people (1.25%) do nearly a third of the work.

My own anecdotal evidence, collected from over 8 years of working in projects at The Apache Software Foundation, leads me to assert that the popular “army of volunteers” myth is a result of the chatter of the masses (the other 98.75%). In surveys, interviews and snapshot observations the one patch contributors far outnumber the visionary leaders in a project and as a result it is their voices and opinions that are heard. At the same time, the leaders know that they need to play down their own importance in order to credit the community, since the contributions of the community are vital to the survival of the project, I’ll look at why later in this post.

Dan Woods, in “The Myth of Crowdsourcing” appears to agree that leadership and vision is what is needed in a successful project:

There is no crowd in crowdsourcing. There are only virtuosos, usually uniquely talented, highly trained people who have worked for decades in a field…What really happens in crowdsourcing as it is practiced in wide variety of contexts, from Wikipedia to open source to scientific research, is that a problem is broadcast to a large number of people with varying forms of expertise…There is no crowd of open-source developers ready to attack every problem. In fact, most open-source projects are the product of one obsessed individual who wrote the software to meet his own needs. Often this individual was joined by other programmers who shared the founder’s vision and, under his direction, created great software.

However, I believe Dan, at least partially, misunderstands the importance of the crowd in crowdsourcing.

The real importance of community

The community is vital to the sustainability of an open project, and, increasingly as I’ll discuss later, closed projects too.Note that Dan talks about the “broadcasting of ideas”, this is where the “army of volunteers” comes into its own. They validate what is happening in the project, they fix bugs, make suggestions for improvements and provide sounding boards for new ideas. Without these contributions a project is completely reliant on a handful of individuals in leadership roles. This dependency presents two main problems.

The first is that the project relies on the limited knowledge of the leadership. Even the best of the best don’t know everything, or as Mark Twain puts it “to succeed in life, you need two things: ignorance and confidence.” An effective and successful leader has the ability to step back and listen to those with different experiences, make the right (or best) call and ensure it is actioned. That is, the best leaders encourage, support and reward an army of contributors so that their project continues to excel.

The second problem of this dependence on a small group of leaders is that things change. Over time, one or more of those leaders will move on. Where will their replacement come from? If the leaders have already cultivated an “army of volunteers” in an open and transparent way there should be no shortage of people willing and able to step up and take on the role.

What about closed development?

To ignore your community is a mistake in almost every case. Community and the power of crowdsourcing is not, as Dan explains, the sole domain of open source software. Open innovation is becoming increasing popular in most industrial sectors, including closed source software. Microsoft, for example, has its Most Valuable Professional programme which is specifically designed to:

recognize the best and brightest from technology communities around the world…These exceptional community leaders come from a wide range of backgrounds. They are teachers, artists, doctors, engineers, as well as technologists, who actively share their high-quality, real-world technical expertise with the community and with Microsoft.

Is openness for me?

Most people starting new projects assume that either their project is too specialised to be of interest to third parties or that opening up to the community will result in them being swamped by well meaning contributors. As a result, they don’t engage, support and reward their community.

Popular communities do not grow from one member to hundreds overnight, there is little chance of you being swamped. They grow organically and, assuming the community is empowered to self-govern under strong leadership, they grow in self-sufficient way. As for niche projects, an “army” of just one person can still be extremely valuable, who knows what just one person can do - after all, you are just one person, right?

If you care about software sustainability you should never ignore your community.

The goal of the #JISCRI projects is not  to have perfect software solutions, but to develop expertise in a range of potential solutions. By sharing this expertise across the whole community we not only increase the skills base of all, but we also bring ideas together - it is this converging of ideas that results in innovation, or, as Marten Mikos puts it:

innovation happens … when you encounter other people and also when you step over some boundary and you combine ideas that haven’t been combined before.

By adopting an open source licencing model and an open development governance model we allow anyone with an idea to bring that idea to the table, the governance model defines the mechanism for evaluation and subsequent acceptance or rejection of the idea.

However, at this point we hit another problem. People are often adverse to contributing their ideas because they also desire to control the latest and greatest innovations. A good governance model will reward the best ideas with a stake in the overall project. That is, it recognises that if we want to benefit from other peoples ideas we need to ensure they can benefit from our own. At this point please allow me to misquote Kahlil Gibran:

 If you love somebody a project, let them go, for if they return, they were always yours. And if they don’t, they never were.

If you create and manage a truly innovative project then the rewards will always come back to you. By allowing people to experiment with your project you encourage those people to dream up new innovations and to invest new resources into implementing those innovations. These resources and the expertise they bring is something that your project would never have if you failed to let it go in the first place. The trick is to make it more beneficial for the third party to work with you rather than to work independently, for this you need a good governance model.

This kind of collaboration happens a great deal in the commercial sector, but it is much rarer in the academic sector (although it does happen). The reason for this lack of openness is a cultural mismatch with two distinct causes. The first is that scientists are encouraged by the peer review system of publication to keep things close to their chests until the point of publication. The second is that universities are geared up to exploit innovations through patent licencing, which is not compatible with the idea of openness during the act of software innovation (this post is getting to be quite long, so I’ll save the exploitation issue for another post).

The fact that the peer review system prevents the kind of peer recognition demanded in open source projects is ironic since that system was devised to encourage openness. Micheal Nielson explains it well:

The value of cultural openness was understood centuries ago by many of the founders of modern science; indeed, the journal system is perhaps the most open system for the transmission of knowledge that could be built with 17th-century media. The adoption of the journal system was achieved by subsidizing scientists who published their discoveries in journals. This same subsidy now inhibits the adoption of more effective technologies, because it continues to incentivize scientists to share their work in conventional journals and not in more modern media.

My hope is that initiatives such as the JISC Rapid Innovation projects will help increase openness in the academic software development sector, but my concern is that very few of these projects understand the importance of being open from day one. I’ll be challenging people to prove me wrong in my lightning talk at the #JISCRI event and OSS Watch will be exploring the need for openness in our Engaging Developers workshop in October.

As with all JISC calls, bidders are expected to consult with OSS Watch in order to understand how the JISC open source policy and the open development approach to project sustainability apply to their project (see paragraph 57 of the call details).

We look forward to helping you write your bids.

In January Scott Wilson attended an OSS Watch workshop on software sustainability at which I and my colleague Rowan Wilson presented a number of models for managing open source software in a sustainable way. After my presentation Scott approached me to ask what was involved with entering the Apache Incubator.

I’m pleased to announce that the Wookie proposal has now been accepted into the incubator. There is still some work to do before the code actually appears there, but it won’t be long

This is great news for all involved. OSS Watch hope that Wookie will go on to provide our first significant use case demonstrating that software developed in the UK academic sector can, and should, become useful beyond our sector. I congratulate the Wookie team on clearly separating the domain specific features of their work from the generic code that will be useful beyond their core team. It is this, and the teams awareness that restricting their code to a niche market would restrict the pool of contributors, that has enabled OSS Watch to assist Wookie as it worked towards entry into the Apache Incubator.

I truly believe that the University of Bolton, and any other project or institution that adopts the Wookie code, will benefit significantly from this move. Already we are seeing interest from many third parties, indeed, it is a long time since a proposal passed with such a large number of positive votes.

[DISCLAIMER: I am a Member of The Apache Software Foundation, but this has no bearing on the choice of the ASF as a home for Wookie. The justification for the ASF is that it has a track record of producing reference implementations for open standards]

The article identifies Ken Saunders, a 41-year-old, legally blind volunteer for Mozilla as a prime example of the kind of third party contributions that are key to the success of open development projects:

Saunders is among hundreds of people who donate time and skills to Mozilla, the Mountain View (Calif.) company that releases Firefox and other open-source software. Even as Mozilla’s internal staff has grown to 250, from 15 in 2005, an army of volunteers still contributes about 40% of the company’s work, which ranges from tweaks to the programming code to designing the Firefox logo.

Innovation in open development can happen within the core project team, but it can also happen at the edges, where people with different objectives and interests see value in the projects outputs. According to the Business Week article Mike Beltzner, who runs the Firefox project, says that open development projects need to “allow people to innovate and to explore and [give them] the freedom to do what they want along those edges—that’s where innovation tends to happen in startling and unexpected ways.”

The article also discusses some less successful attempts at open development, highlighting some of the good and bad approaches to rewarding and recognising volunteer contributions. A recent case study on TexGen, published by OSS Watch, illustrates how the open development model can be used to apply to even the smallest niche area project.

Collaboration is the key

OSS Watch have been acutely aware for some time that the “standard” approach to software sustainability in our sector is to “throw more money at it”. Indeed, most of the bids into this call, that we are aware of, take this approach. At OSS Watch we take a longer term view. We believe that sustainability is about reducing long term cost through collaborative development, this requires a short-medium term investment in developing collaborative software development skills. Unfortunately, our sector is more accustomed to competition for grants than to collaboration for value for money.

Evidence of this can be seen in the OSS Watch experience during the writing the our EPSRC bids. We were approached by a number of people who refused to allow us to talk to others about their plans, a ludicrous situation since some of the gaps in experience in one proposal could be closed by bringing in another member of our network. In other cases we invited representatives of potentially competing bids to view and discuss our own bid in order to find points of collaboration. Some joined our open mailing list  and viewed our draft proposal in the wiki but not once did they contribute, they didn’t even acknowledge our invite. In other words they wanted to take, but not to give.

The end result of this competition is, in this case, a number of less than ideal bids. This prompted OSS Watch to sumbit a bid that explicitly called for the EPSRC to match our own bid with another with, in the funders eyes, a suitable domain specific expertise. Assuming that our bid is seen as valuable in itself we look forward to sharing our expertise with one or more of the other bid teams.

How do we collaborate?

It is our belief that to create sustainable software that survives between one development team (i.e. grant) and the next (i.e. another grant) we need to change this culture of competition into one of collaboration. In the non-academic sectors there are significant examples of this working on both large and small scales.

The problem with this approach is that project team currently expect their own chances of survival to be dependant on their “ownership” of the outputs and knowledge acquired during the first grant. Indeed OSS Watch are an example of this. As OSS Watch manager I am responsible for ensuring we have enough grant funding for over five full time staff. If I encourage others to enter our space am I putting my team at risk?

I don’t beleive so, my team have a very specific set of experiences and skills. Collectively we cover most aspects of open source software development. Our network reaches far and wide into the academic and non-academic open source communities, but we are not omnipresent. By being clear about our specialisms and recognising other groups expertise we should be able to increase the amount of grant money we receive as we will get access to more varied pots through our cooperation. What the sector needs to realise and accept is that by being the expert one will be invited to participate in more projects not compete for less.

Whilst OSS Watch’s experience in this EPSRC bid would seem to simultaneously support and undermine this argument, it is our belief that in the long run cooperation is where value and sustainability lies. There  are examples of this happening with niche software produced in the academic sector too. For example TexGen, a project at University of Nottingham has recieved significant additional funding as a direct result of the openness of their software outputs.

Sucessful collaboration means fully open collaboration

Unfortunately, TexGen is the exception rather than the rule. Even where collaboration is the goal, our sector still tends to artificially limit membership of the community to those who are funded. This can be seen in the Community Source model, popular in large US projects which excludes some parties from participation and therefore limits the sharing of both costs and rewards to a smaller group than is necessary.

The community source model aims to add a level of control to open development projects that it is believed does not exist in true open source. However, I would argue that this is a lack of understanding of the open development model rather than fact.

My own experience shows me that there are decision making models in open development that allow for the level of central planning and control needed in major projects whilst still ensuring the doors are open to anyone who wishes to contribute in a sensible and constructive way. Indeed, if this were not the case then open source, and sustainable, products such as Eclipse, which is found at the heart of a large number of varied third party tools, would not exist.

In order to demonstrate this to be the case OSS Watch have been working with some experienced community source practioners in the Matterhorn Project. Together we have drawn up a governance model for this new project which appears to satisfy the control concerns of the funders whilst also satisfying OSS Watch’s concerns about keeping the community open.

At first glance this model may seem somewhat heavyweight for a smaller project. Therefore, OSS Watch have also worked with other parties to develop a much lighter touch governance model suitable for “rapid innovation” projects that are funded for just six months.

Those of you who read these models will realise that they are essentially the same model, it’s just that one defines more formal roles than the other since it has more initial stakeholders. This means that should the rapid innovation project be succesful it’s model can grow to accommodate a growing, and as yet unknown, community. It is this growth that is the start of sustainability through a collaborative development model.

The practice of open development

Now that OSS Watch have started to show that closed communities are not required, the next step is to encourage people to develop their software in such a way as to ensure non-community members are empowered to participate. It is these third party contributions that spread the cost of development across multiple financial pots and thus reduces centralised risk.

We recognise that policy is all very well but it is practice that matters. The adoption of open development practice requires a significant cultural change, but this is a change that our research shows is desired by enough practitioners to make it viable.

OSS Watch argues that funding projects to experiment with software ideas and then spending more money to “harden” the few that have promise is wasteful. We believe all software should be developed openly from the outset. By doing this the cost of sustaining software that successfully meets user needs is decreased since initial outputs, of all kinds, are reusable. Where software does not meet user need, as can often be the case in research software development,  the increased level of collaboration and awareness within the community serves to increase the chances of success in future projects. This is because the community is sharing not only the “hardened” successes, but also the lessons learnt in other projects.