Rogō: an open source solution for high-stakes assessment

Rogo screenshots

If you’re looking for an open source library for adding some whizz-bang stuff to your website, or an open source text editor or graphics programme, then you’ve got plenty of options to choose from. But when it comes to solutions for core business functions, then your choices are usually a bit more restricted. So when I heard about Rogō I was immediately interested, as it tackles a business problem in education for which there are very few solutions – managing high stakes, summative assessment. So I caught up with Simon Wilkinson from the project at the ALT-C conference in Manchester to find out more.

There are plenty of projects out there that let you create and deliver formative, low-stakes assessments – self tests and diagnostics for example – but summative assessment is a whole different class of problem, where the critical features are things like security, reliability, performance and trust.

Rogō started out in 2003 as TouchStone, an in-house assessment management system used by schools within the Faculty of Medicine and Health Sciences at the University of Nottingham, before being selected by central IT services to be a core supported assessment platform at the University.

Since then the software has changed its name (Rogō is latin for “I ask”, in case you were wondering) and has been transitioning from an in-house system to a much more generic open source solution with the help of JISC (and OSS Watch of course).

Over the past year the Rogō team has been working with five other institutions, helping them install the software, integrate it with their systems and run assessments. This experience is then informing the community strategy for Rogō. Following on from a consultation with OSS Watch the team added a public issue tracker, selected a license, and started developing its community engagement processes.

However, as Simon Wilkinson at Rogō admits, the project is still very much at a “fledgling” stage as an open project. For Rogō, one key motivation for going the open source route is that “it mitigates against risk”; or as Simon puts it “if we [the core team at Nottingham] all get run over by a double decker the community can still pick it up.”

To support this objective,  the team are working to foster a developer community, and are working initially with their set of partner institutions to see how closely they are able to engage with the project  – can they move from being prospective users piloting the system, to active users contributing bug reports and requirements, to actual contributing code?

“Where we are now is a bit mixed; some of our partners struggled to get the software installed but they’re all there now, and Oxford are the most prolific in terms of posting tickets”. But how can they move from here to contributing to the code?

“One weakness identified of Rogō as a result of the review [performed by OSS Watch] was that we don’t have a clearly articulated design” says Wilkinson, “With a clearer articulation I think more people will come on board, and build using the same design ethos”. To this end, the team are focussing on improving documentation – in particular how they explain the overall structure and concepts of the platform to make it much easier for potential developers to understand how the system works, and what core terms and concepts mean; the term “course”, for example, means something different in Rogō than it does in Moodle.

For Rogō to go from a fledgling to a sustainable open project, the team need to look at a wide range of issues – how they can support and foster a more diverse user and developer community, how to improve documentation and installation processes, how to relate the software to services such as support, consultancy and hosting. As with any enterprise solution, software-as-a-service (SaaS) is also one of the options the team needs to consider as well as locally hosted and integrated systems.

A lot of work also needs to go into making core services such as authentication more pluggable, to support commonly used single-sign-on services such as CoSign (used by institutions including the University of Leicester) or WebAuth (used by Oxford). This is going to be a particularly good test for how well Rogō can move from a single-institution system: integrating with various kinds of  authentication services is a necessity for other organisations to adopt Rogō – but its also something which doesn’t provide any direct benefit to Nottingham, where the integration is already complete.

Looking ahead, I think Rogō has some key advantages over other “fledgling” open source projects. It has a long track record of use at Nottingham, something very important for establishing trust in a system that performs such a critical role. It is also in a niche which has very few other solutions – there is, for example, only one real closed-source competitor in UK HE – with the potential to gain a significant core of committed users and developers. I think we’re going to hear a lot more of Rogō in the future. And maybe even learn how to say it properly.

If you’d like to find out more about Rogō, you can visit the Rogō site at the University of Nottingham.

If you have a project in your institution and are looking to go open source, visit the OSS Watch site for more information and to find out how to get in touch with us.

Thanks to Simon Wilkinson for taking the time to meet up with me, and to ALT-C 2012.

1 thought on “Rogō: an open source solution for high-stakes assessment

  1. Pingback: Six Reasons Funded Projects Are Set Up To Fail at OSS Watch team blog

Comments are closed.