Monthly Archives: March 2010

Importance of open source screen readers for accessibility: Interview with Joanie Diggs

Right at the end of the CSUN conference I cornered Joanie Diggs, newly appointed module maintainer of the GNOME Orca screen reader. We flopped down in a quiet corner to chat about open source and screen readers.

Topics covered include Joanie’s journey to becoming Orca maintainer (03:10), the benefits of open source development for accessibility (07:08), collaboration between projects to fix problems (13:40) and how others can be encourage to take a more active role (25:30). We also discussed Joanie’s thoughts on the important issue of gender imbalance in open source development (18:10).

Listen to the audio below or download it in mp3 format.

Running time: 28 min

Open innovation vs trade secrets

A few days ago I came across this nice quote by a Nokia senior manager:

“We believe the world is changing and the competitive advantage comes from how many others can you get from participating in this network. This network becomes more important than trade secrets.”

The quote appears in the context of a study presenting the collaboration initiated by Nokia a few years ago during the development of its internet tablet. Through a series of interviews with Nokia senior managers and developers, contracted open source businesses and independent developers, we are shown how Nokia opened up software, leveraged externally developed open source technologies and encouraged contributions by both independent developers and competing businesses.

“We have evidence that some of our competitors are now looking at our code and they are investigating if they could use our code in their products. You might say that we help them now to get their products out fast.[…] But if we had not put it out there we could not have used the OSS communities who have already helped us to develop that code.”

Does this sound familiar to those of you who are aware of open development? The study points out that in the process of opening out the source code and encouraging external collaboration Nokia created a new market for the internet tablet, but more importantly it learned how to cooperate with a diverse community of employers, volunteers and contractors:

“It’s all about the process… You develop this openly within the communities and you try to synchronize your own work with the heartbeat of the communities. Some companies now understand this better than others. We certainly have done our learning. We have made some mistakes too on this front”

Moreover, by allowing external developers experiment with the software, Nokia enabled innovations previously seen as unrealistic by its own engineers:

“I think from my point, if you let people change things […] and document them and open them up so people can hack their own stuff, you never know what is going to happen, what kind of things people are going to write for your device which ultimately could make it sell millions if someone writes the killer application for it”

Again, this is likely to be common sense to open source communities, who are familiar with governance models that document ways in for potential external contributors. The interesting bit is that increasingly the corporate sector is looking at open development as inspiration for their R&D and innovation policies.

Two common innovation models described in the study are the so-called “private investment” and “collective action” models. In the “private investment” model innovators commit their time and resources if they can get appropriate returns from these investments. By contrast, the “collective action” model assumes that publicly subsidised innovators work for the public domain, generally associated with non-rivalry and non-exclusivity in consumption. More recently a third, middle ground model called  “private-collective” was identified, where the innovator uses private resources for public good innovation.

As is the case with open source development,  this model seems counter-intuitive in the first instance. Why should one make one’s innovations available to all, and why should one pay for something that anyone else can use for free? In fact, the study points out, the innovator working within such a model often receives higher benefits by contributing to the public goods creation then by only free riding on its production by others.

This appears to be true beyond independent innovation. As the Nokia example demonstrates, companies themselves can benefit by encouraging the creation of publicly available innovations. The quote at the top of this post suggests that networking within appropriate communities can be more important than jealously guarding the secrets of the trade. This is a new type of asset for the companies, which if explored in the open development spirit can become an important source of competitive advantage.

And yes, in this respect OSS Watch’s expertise in open development becomes more valuable by the day.

Report from the CSUN conference in San Diego

I’m taking a quick break from the whirlwind of activity to mention some of the open source highlights at the CSUN 2010 conference, or the 25th Annual International Technology & Persons with Disabilities Conference, to give it’s full title. I’m here wearing several ‘hats’ in addition to the main OSS Watch ‘topper’, and already I have met a very wide selection of people and had many interesting conversations. The common factor has been open accessibility, as you might well expect.

As a member of the GNOME accessibility community I’m helping to man the booth and supporting presentations (although I’ve not actually been at the booth that much yet). So far Willie Walker and Eitan Isaacson have presented. I missed Will’s but Eitan did an excellent and humorous presentation of how open source allows engagement with a project in order to fix problems. A couple of weeks work, including audit, turned the largely inaccessible user interface into something with working theme support and keyboard access. He clearly demonstrated how he worked with the Banshee media player and Orca screen screen reader teams to audit and fix the accessibility issues. There was a lively discussion, including how to get institution IT departments to accept open source.

The GNOME booth is festooned with the contents of the large GNOME event box, including the necessary swag collection. We’re running presentations and demos of the complete accessibility stack that GNOME provides. We also have a stack of OpenSUSE CDs and shirts form Bryen is a on the board and a GNOME a11y member. As Eitan tweeted today there is some Buzz around GNOME and I suspect that is partially due to the increased interested in Open Source. It’s 4 years since Mozilla first mildly perplexed the CSUN visitors by flying the open source flag through running a booth, thus preparing the ground for GNOME’s arrival this year. The use of Twitter to raise profile no doubt also helps. Another contributing factor is likely to be the CSUN team’s interest in and support of Project:Possibility SS12 competition students; one of the teams worked on GNOME Caribou with first class mentoring from Ben Konrath, module lead.

So far I spoken to 2 people who are keen to introduce open source participation to their students as part of the course work, and with an accessibility angle. One is a CS lecturer from CSUN Northridge itself, the other is introducing a new education technology PhD in Michigan. Both are excited about the possibilities that GNOME accessibility offers.

On Tuesday, before the main conference started, we held the GNOME accessibility hackfest, which despite is name was more of a face to face meet-up in order to discuss critical matters. The GNOME accessibility team are under resourced and Willie Walker has had to step down as lead due to being made redundant by Oracle. In Willie’s words he will ‘turn into a pumpkin’ from next week as he needs to focus on job hunting and family. Ben Konrath is also unsure how long he will be able to maintain Caribou. Thus a large part of the agenda was taken up with how the community will move forward. Another large discussion was around what to do for the next release of GNOME 3. The issue is there is a huge list of work to do and very few people to do it. To give a flavour of the problems, a large part of the accessibility plumbing has been rewritten, with obvious knock on effects. Another high risk factor is the new GNOME Shell redesign of the desktop which is being done with almost no reference to accessibility. A boost is that couple of developers said their employers have given them around 20% time each to work on GNOME a11y. At the end of the day we had a lot of useful discussion and the future looks OK, if not exactly ‘rosy’.

A real high at the end of the day was a surprise presentation ceremony organised by Peter Korn (Oracle, AEGIS). Peter had organised a large framed ‘graduation’ certificate for Will Walker and signed by members of the community. Willie was suitable ‘choked but soon recovered as we all enjoyed the accompanying bottle of 16 yr old Lagavulin single malt whisky, served in cups embossed with Braille (it was German Braille so no one could quite work it out).

I’m now heading back to the GNOME booth and to track down Willie Walker for an interview. In my next post I’ll provide more details of the Project:Possibility students activity here at CSUN.

[Update] Bryen and Eitan also posted reports.

An open source community at work: first impressions

As a content editor for OSS Watch, I’ve been preparing one of our forthcoming briefing notes, ‘Roles in open source’, for publication. An offshoot of our recent document on governance models, it describes ways in which you can contribute to an open source project. These include:

  • providing user feedback
  • supporting new users
  • marketing
  • designing and implementing software
  • carrying out quality assurance
  • providing graphics and art
  • writing FAQs, how-tos, tutorials and developer guides
  • translating
  • making monetary donations
  • So you don’t have to be a software developer to get involved. In fact, it’s one of my aims, as a non-techie, to experience open source first hand – instead of just reading and writing about it – and get involved in a real-life project. So far, I’ve joined the Matterhorn mailing list, followed some of the less technical threads and explored the website.

    Matterhorn was initiated by the Opencast Community for creating software to produce, manage and distribute academic audio and video content, especially lecture recordings. It’s been intriguing to see how far-flung contributors are, and the discussions I have followed have been interesting. One, for example, debated in some detail which terminology to use: is a single recording of a lecture an ‘episode’, an ‘item’, a ‘lecture’ or a ‘recording’; is a group of such recordings a ‘collection’, ‘series’ or ‘season’?

    Following discussions on the mailing list has given me an insight into the workings of an active open source community. I’ve been impressed by how committed people are to the project, the attention they pay to detail and the priority they clearly give to user experience. I’ve also been struck by the fact that, without exception, issues are raised and contributions are handled with respect, courtesy, humility and often humour. This should give any would-be contributor lurking on the mailing list with something to say the encouragement they need to take that first step.

    GNOME and Project:Possibility: collaboration in open source accessibility

    In my last post I mentioned how satisfying it is to watch the results of making introductions between people or groups. Recently I was involved in a link up between students and GNOME accessibility and though I’d explain how events panned out, as well my hopes.

    Project:Possibility organise team coding competitions for computer science students. The the goal of these SS12 events is to create accessibility software that will be of benefit to real users. These weekend ‘code-a-thons’ bring the students together with a mentor from industry and provide an exciting chance to learn something about accessibility, while creating open source software. Thus students get exposure to topics and skills that are still sadly absent from many computer science courses. Plus they get to have fun and win prizes.

    Project:Possibility was set up by Chris Leung while he was at USC so it is fitting that the latest SS12 was a face off between USC and rivals UCLA. This added an extra element of competition, and the winning team from each campus have been invited to present their work at the 25th CSUN conference (#csun10) next week. They also get the opportunity to explore the conference, getting a flavour of the many accessibility offerings. CSUN have been very supportive of Project:Possibility and are keen to see students learn about accessibility and take the knowledge along with them as they start their careers.

    The Project:Possibility board decided to concentrate our energy on the weekend competitions, at least for the short term. While accessibility is clearly a key feature of the events, I felt the experience of open source is less compelling. The code is given a open source licence and placed on Google Code but there is not much opportunity to provide an in depth experience of open development or to support subsequent maturing of the new software. OSS Watch are very aware that employers are seeking open source development experience, and yet there is a shortage of skills as they are not often being taught in computer science course. This further influenced my thinking.

    I have previously worked with the friendly and hard working GNOME accessibility community, so I therefore proposed that we offer students the opportunity to work with them. The idea was to support students in adding features to existing GNOME accessibility projects, working with community members. This would introduce students to the project, hopefully exposing them to best practices. My hope was also that this introduction will lead to longer term engagement with GNOME accessibility or other projects and continuing contributions. Some concern was voiced that students would not want to climb the extra learning curve while competing in a weekend event. The GNOME team, however, were supportive of the idea and we all hoped the right students might be ‘up for’ the greater challenge and potentially greater rewards. We came up with a few coding ideas and I’m pleased to say a team from each of USC and UCLA chose to work on GNOME Caribou, the new GNOME on screen keyboard that provides operation for non keyboard users (via pointer, switch or other devices).

    Ben Konrath, the Caribou lead, was available on IRC during the competition and answered questions from the students, helping them to get up to speed. With his help, both sets of students got through to the finals and came 2nd and 3rd place overall. Ben will now help the students go through the submit and review process and hopefully commit their code into Caribou. From there it will be part of GNOME and so will appear in many Linux distributions. Something the students can rightly be proud of.

    If you are interested in seeing what the USC students have done with Caribou then you can meet the teams at CSUN next week on 26th and 27th. There is already considerable interest in the students work, including from Microsoft. CSUN have promoted their attendance in conference announcements, and this has also raised GNOME accessibility profile, not bad seeing it is their fist year at the conference. If you would like to find out more about GNOME open accessibility the team are there all week at the booth.

    I’m really looking forward to the conference, catching up again with GNOME (and Mozilla), meeting the students and, hopefully, hacking a little on Caribou. I hope to see you there.

    Life is wonderful

    Whilst this blog allows OSS Watch members and guest posters to express personal opinion we don’t usually use this blog for personal items. However, I’m making an exception today to remind myself, and hopefully some others, that whilst life can sometimes be horrible it is more often than not wonderful

    .Loving brotherDad and newborn Saskia Proud MumSaskia Frances Gardler

    On Sunday at 17:17 (GMT) March 7th my wife gave birth to a beautiful and healthy baby girl, Saskia Frances Gardler at 7lb 10.5 oz. Mum and baby are doing very well and are now home with a proud Dad.

    I’d like to publicly acknowledge the superb  care that my wife and I received during this pregnancy. We only truly appreciate the National Health Service when we really need it, and Heidi and I have  needed it on far too many occasions over the last few years. The care we have received has been exceptional when compared to what I have seen in other countries.

    I should also mention Oxford University Computing Services. I could not have hoped for a more understanding employer, it was clear from the first day that as long as I worked hard when I was able OUCS would respond by giving me as much space and time as I needed to support my family during this difficult period. I’m truly grateful to my boss Lou Burnard and to my exceptional team here at OSS Watch.

    As for extended family and our friends – we hope they already understand how appreciative we are of their combined efforts in recent years.

    Like I said in the intro, sometimes life is horrible, but more often than not life and the people around us are wonderful. We should never forget that in the difficult times.

    Guest post: 2010 – Threats to copyleft

    This post is by Patrice-Emmanuel Schmitz, Director for European institution studies at Unisys Belgium (Brussels). His team is in charge of the (Open Source Observatory and Repository), the Free/Libre/Open Source information platform and forge launched by the European Commission for public sector projects.

    Combining freedoms and copyleft in the Gnu GPL licence (invented by Richard Stallman) was the cornerstone of free software. This is now questioned due to the proliferation of incompatible copyleft licences.

    After counting 1,800 free software licences used in hundreds of thousands of projects, the Black Duck company patented (Patent US 7,552,093 B2) the technology for controlling the use of open source licensing in a multi-source development process (meaning combined works, elaborated from multiple free components under different licences).

    No need to say that patenting proprietary technology to solve copyleft licences incompatibility may not be seen by everyone as a major achievement!

    Lamenting on licence proliferation or blaming new licence authors – who all call upon the best reasons of the world, looks useless. It would certainly be reasonable, as recommended by Bruce Perens, to deal with only four permissive and copyleft licences, but this is wishful thinking. New licences are presented every week by FLOSS authors and communities, and no benevolent dictator will limit human innovation regarding licensing.

    The heart of the problem does not lie in the number of licences, but in their incompatibility. I do not think that licence proliferation is a failure of the FLOSS movement, it is rather the entire contrary: a testimony of the attractiveness of FLOSS models. In reality, licence proliferation illustrates the failure of a certain model of strong copyleft, as it was initiated by the GPL in the ’80s and – unfortunately – reproduced by nearly all subsequent copyleft licences. Once necessary and successful, this model looks not adapted anymore because it was copied and – seeing the Black Duck patent – one may question (like Ernest Park has done) if the way copyleft is applied does not generate today more jails than freedom.

    According to my first study for the European Commission, the Gnu GPL v2 was used in 85% of the FLOSS projects in 2001. With a copyleft that was – maliciously – said “viral” by some, meaning that compatibility is always “upstream” (to itself) and never “downstream” (to other licences), the adoption rate of the GPL should have been universal in 2010, confirming analysts’ assumption that “it is good for the community if people use a single copyleft license [1]”. However, the exact reverse happened: the GPLv2 (reducing) is still used in 50% of projects, the new GPLv3 reaches little more than 5% and other licences are proliferating.

    The fact a dozen of licences are used by 90% of the FLOSS projects does not help very much, as the implementation of free solutions (which are often combined works) is done through integrating many components. It is enough to find only one incompatible licence to compromise the distribution of these solutions.

    The current situation is damaging for other reasons: it creates endless discussions on what could be considered as integration in combined works (dynamic or static linking) and it feeds disputes. To preserve their communities from schisms, gurus and acolytes urge followers not to use any other copyleft licences, whatever their specific merits or advantages could be. It is time to admit that the strategy of keeping a “captive asset” of licence users was not successful for avoiding proliferation, and that it is not the most appropriate way to reinforce the freedom, collaboration spirit and consistency within the fragmented FLOSS world.

    In Europe, the recent – OSI approved and copyleft – European Union Public Licence (EUPL) meets some initial successes due to its compliance with Member States’ law and because it has equal value in the 22 languages of the Union. It has been selected by the German Federal Agency for Information Technologies, it is the licence of choice in schemes published by the Dutch NOiV (see on the site an English version of this scheme, translated by a member of the Swiss administration), a dedicated EUPL site was created in Italy, etc. In Spain, the Ministry of Industry, Tourism and Commerce (where the public agency is located, in charge of information technology) provides the following in call for tenders (software specifications):

    “In case the contractor integrates in the development that is the object of the contract with modules or elements owned by third parties, he must first obtain from the legal owners the licences and rights necessary to transfer the ownership of the development to , which will submit it, including the elements that are performed under the contract (such as fonts, dll, scripts, etc..) to the public licence EUPL. In any case the total and final result of the development and the overall project will be subject to a licence EUPL.”

    Applying such provisions excludes strong copyleft components from the delivered combined work: all original developments are allowed; all “permissive” components are allowed (BSD, MIT, etc.); all “weak copyleft” components are allowed (i.e. LGPL), but no Gnu GPLv2 (or V3) components (except if the copyright owner is entitled to dual license the component under LGPL-like terms, for the purpose of addressing the contractual specifications).

    The EUPL itself has an innovative approach to solve copyleft licence conflicts: it publishes a downstream compatibility list (to other licenses). It is allowed to integrate EUPL components in a combined work that will be distributed under a compatible licence. The concept is not new, as the FSF applied it with the LGPL a long time ago. The LGPL (now LGPLv3) is convenient for software libraries aimed to produce combined or derivative works: if the library is propagated on its own, it must be under the provision of its original licence (LGPL), but if the library components are part of a derivative work, this work can be licensed under another licence, while the original library remains LGPL. The EUPL compatibility is exactly the same, but its copyleft effect is stronger than in the LGPL (because compatibility is restricted to a limited and published list of other copyleft F/OSS licences). Therefore it is not a weak but – say, a “tolerant copyleft”.

    Such flexibility removes incompatibility barriers and restores developers’ freedom, while keeping it in the limits of the desired copyleft effect.

    While the EUPL solution may be considered as a conceptual progress, its tangible impact will stay very limited as long other copyleft licences will not give some reciprocity: the quantity of available EUPL-ed material is quite small today, compared to the mass of components that are already available under the GPL terms. Extending the list of EUPL compatible licences (i.e. by adding the still missing GPLv3) will not change the issue resulting from the Spanish specifications, where the government requires the facility to distribute the received combined work under the single licence of its choice (the EUPL in this case).

    Solving problems related to the proliferation of copyleft licences requires setting up interoperability provisions between these main licences. It will create, and focus attention, on a kind of “circle of trust” where the original copyleft licensing condition of a software will never be changed, but where – just to take an example – a GPL component could be part of a combined work that the recipient (let’s say the French or the Spanish government) could distribute as a whole under the provision of the copyleft licence of its choice (i.e. CeCILL or the EUPL) provide these licences are allowed in the compatibility list.

    The Gnu GPLv3 includes (in its section 13) the exact provision corresponding to the above need, but it is directed to the AGPLv3 only. The intellectual effort to extend this provision in direction of a small list of interoperable licences seems easy to deliver. The reciprocal condition must be added: in the example above, the combined work could be also (as the need may be) distributed under the provisions of the GPLv3.

    This could be a way out to the deadlock where we are, due to the proliferation of incompatible licences.

    Looking for alternative? Pay a Black Duck patent licence!

    Patrice-Emmanuel Schmitz –

    [1] R.T. Nimmer, Legal issues in Open Source and Free Software distribution, The Law of Computer technology, Ch. 11 (1997, 2005 Supp.) (“Two different copyleft licenses are usually “incompatible”, which means it is illegal to merge the code using one license with the code using the other license; therefore it is good for the community if people use a single copyleft license (GPL)”).

    Dev8D – where collaboration happens and skills are learnt

    One of the most satisfying aspects of working for OSS Watch has been observing the outcomes from introducing people and then encouraging them to collaborate on new ideas. Outcomes such as personal development and new or improved projects. The recent Dev8D event was a real highpoint in several strands of OSS Watch activity and I was inspired to have been able to observe what happened and be part of the action. Dev8D is about developers getting together and seeding new project ideas, something that is an important aspect of open development, and so is important to OSS Watch. The Dev8D team, led by Mahendra Mahey, along with everyone who participated over the 4 days made Dev8D a fantastic event. Much developer happiness was expressed both during and after the activities, not to mention the many new project ideas that were hatched.

    Several coding challenges where made and our very own Sander won 2 of them with MuCoMaCo an interesting Google map mash-up of MLA library data. OSS Watch have provided continuing support services to the Apache Wookie (Incubating) project from the University of Bolton. Recently Sander recently organised a Wookie Training day at OSS Watch. This was led by Scott Wilson, Wookie project lead, and Ross, OSS Watch manager. Scott was with us at Dev8D, so it was no surprise that Sander’s winning entry was a Wookie served W3C widget. This enables it to be readily deployed in a range of contexts with little effort.

    The other winning entry that OSS Watch had close connections with was Wookie BaLTI. BaLTI is a sample Moodle course by Mark Johnson and Dan Hagon and features the newly developed Twirlymol and CollabMCE widgets. These 2 widgets use the widget loading and collaboration facilities of Wookie to provide a shared 3D interactive view of a molecule model and shared WYSIWYG editing. Several threads came together to make this happen. Those familiar with OSS Watch and open development will not be surprised at how open collaboration has made this possible.

    Since we spotted Mark’s announcement of his first small contribution to Moodle and subsequently invited him to present it at an OSS Watch workshop, he has gone from strength to strength. At the Dev8D awards dinner Mark deservedly one ‘Best newcomer’. An award that provided him with a small Lego car kit, which we ‘knocked up’ during the meal, and a rather handmade certificate. The point is that Mark’s enthusiasm for learning about and contributing to open development has been noticed and is sending ripples through the HE and FE communities. Mark attended the previously mentioned OSS Watch Wookie training day, which stood him in good stead for working on the challenge with Dan. Since the Dev8D fun, Mark has pledged on the Wookie lists to reimplement his editing widget without the current LGPL licensed code dependency and to contribute it to the project. Further evidence of Mark’s personal development and it’s far reaching affects comes from his recent blog post. He also supplied us with this comment.

    My manager appreciates the opportunities that open development provide to us as a department and an institution, both internally and externally. By letting me go to events like Dev8D even though the things I did there don’t relate directly to my current job, she gave me the opportunity to develop new skills which will allow me to provide new facilities to the staff and students in our institution. We’ve just started an ILT working group with our teaching staff, so hopefully I’ll be able to apply some of my new skills and knowledge to the ideas that come from that.

    Mark’s collaborator, Dan, was the first person I met at Dev8D, and as with Mark you can’t help but be affected by his friendly enthusiasm. When I discovered that he had created an interesting Google Wave widget I steered him towards Scott and Mark and watched the magic happen. Scott explain how trivial it was to port the widget to Wookie, and once they’d decided on the LTI challenge Scott added BasicLTI interfaces to Wookie in order to support the challenge work. These are now part of the Wookie project code. More great collaboration followed and someone gave them an Amazon cloud server to install Moodle on. I observed them working together, discussing ideas, fixing bugs and working on their ‘judgement’ presentation. I tried to muscle in on the action with another widget idea but got rather distracted and bogged down in practicalities. Still I do have some new ideas to add to Wookie at some point.

    I also attended their pitch to the judges as moral support and I’m glad I did. Chuck Severance of IMS and Sakia and Steve Vickers where in the judging team. A soon as the pitch was over, Chuck and Steve rushed off to add the widget to Sakai, Blackboard and WebCT. Chuck made the point that by using Wookie and LTI it is now possible to have something running in all these platforms, and in a matter of minutes. He then distributed screen shots around various lists.

    Dev8D saw many other collaborative ventures seeded or strengthened. New skills were learnt by motivated developers and new contacts made. For example I enjoyed watching Chuck and Tobias Schiebeck work on a tricky Sakai bug. Finally I’d like to mention GNOME who were running an overlapping event in London. After a bit of prodding for collaboration, Willie Walker and Brian Cameron came over and presented on accessibility and GNOME work. Both were very interested in the accessibility possibilities of “Mr gadget” Ben O’Sheen’s demonstration of software to talk to wiimotes. A representative from Dev8D also went over the to GNOME usability hackfest and I expect we will see fruitful cross pollination developing from this mutual interest. Not least is the opportunity for HE/FE developers to learn from a large, established and successful open source project.

    Opportunities for scientific research in open source projects

    There are many interesting open source projects that can be beneficial to academic research. As OSS Watch’s recent article on e-Research by Gabriel Hanganu shows there are social and organisational problems in adopting open source for e-Research, but there are many open source software projects there to be joined. Some projects are suited very well to be used in scientific research and I feel that this is especially true in the realm of big data databases.

    Google showed the way, really, with the MapReduce paper in 2004. They published their programming model for processing large amounts of data in parallel and although publishing it, they did not neglect to apply for a patent as well, which was recently granted. Hadoop, which originates from a project at Yahoo!, also implements the MapReduce pattern, but is completely open source being a project of the Apache Software Foundation. And now recently Apache Cassandra has joined the mix. Cassandra originates from Facebook, but has become open source in July 2008. It recently promoted from the Apache Incubator and is now an official top-level Apache project.
    Work has been initiated to facilitate integration between Cassandra and Hadoop, which simplified means the Hadoop database HBase is replaced with Cassandra. There has been discussion of this on the list and a feature has recently been implemented. So there’s Yahoo! working on Hadoop and Facebook working on Cassandra, and recently also Twitter has announced that it is working towards using Cassandra for their backend. Also worth mentioning is the open source implementation of Amazon’s Dynamo database which is named Voldemort. This project is used and actively developed by LinkedIn and is therefore another example of how you can benefit from the work this large company is investing by engaging with this project.

    To me, this all shows that there will be large investments in NoSQL databases from major companies in the coming years, and it will all be in open source software. This means that there is a lot of opportunity for anybody who has to deal with big data to profit from this investment. All you have to is try out the software and engage with these projects. Researchers also have to cope with more and more data, so I think they have good reason to follow these developments closely and step in to benefit.

    Can open source reduce costs?

    It is often said that open source software will reduce costs.

    Those with little or no experience of implementing computer systems assume these savings come from the fact the free and open source software does not carry a license fee. However, this is not usually the case.

    Anyone who has rolled out an software solution, even in a small organisation, will tell you that there are hidden costs. These include training, support,  customisation and maintenance.

    In 2005 BECTA published “A study of the spectrum of use and related ICT infrastructure costs” which concluded that training and support costs accounted for 60% of total cost for any software solution. The report also found that open source software reduced these costs by 40-50%.

    Further to reducing training and support costs, open source can reduce the cost of customisation for specific environments.

    It is extremely rare for a back-office software solution to be a perfect fit for any specific organisation straight out of the box. Consequently, the software needs to be customised to suit specific needs. In a closed source environment there is a single provider, or a limited set of approved providers, who can make these modifications. However, in an open source environment anyone with the appropriate skills can make these modifications, including internal staff.

    As a result of this competition, market forces can often result in a lower cost for a tailored product. Just how much can be saved here depends on the customisations you need to make.

    Finally, the open source culture of code sharing results in lower development costs for the software in the first instance. That is, once one user has commissioned a specific feature or configuration option the results of that work is available to all. As a result, the more a product is used and developed within any given domain, the more widely the development costs are shared. In addition to a reduction of costs open development can significantly increase the rate of innovation as it brings together great minds to collaborate on shared solutions.

    Where there is no pre-existing solution to match ones needs the open developent model can be an extremely cost effective way of reducing cost. This process is examined in more detail in our document “Meritocrats, cluebats and the open development method: an interview with Justin Erenkrantz.”