In case you hadn’t picked up on this already, 5AM is an Agile shop. Take a quick glance at our Glassbox Software Development Process
and you’ll see the unmistakable scrum methodology featured prominently on the top of our development process diagram. Although Agile-Scrum is now a well-established process in software development, those of us coming from a science/bioinformatics background are not as familiar with the concept. Another thing we believe in here at 5AM is that bringing good science and good engineering together is better than the sum of the parts – the two complement and enable one another. It’s this belief and our commitment to Agile development that led us to ask ourselves, “can our science projects benefit from following an Agile model?” That’s the experiment we’re pursuing now and this is a bit of our story so far.
The first step in the process has been to train almost all of us bioinformatics types in the basics of Scrum
. Since it’s not a package in R and has never appeared in an episode of Battlestar Galactica – most of us knew very little about it. I myself took Scrum Master training here in the Bay Area. As exotic as the title “Scrum Master” sounds, I was disappointed to find the training did not involve breaking boards or making a pilgrimage to an ancient monastery in the Himalayas. What it did involve was passing little red balls around as efficiently as possible, measuring the “points” value of everything on my desk and ranking countries by how “fun” they are. To the uninitiated, this might all seem a bit crazy, but the activities were showing us the core elements of Scrum – how to work effectively as a team, how to measure progress and stay agile. Thanks to Lyssa Adkins
for leading a great workshop.
Now back in the real world of juggling p-values rather than rubber balls, we have started to map the day to day efforts of our science projects to an Agile process. Some parts fit really well – in fact, they’re things we were doing already. Scrum’s short iterations with a focus on delivering “working software” at the end of each sprint is analogous to the iterative approach that our bioinformatics team takes towards data analysis and other projects in which we frequently share results and calibrate next steps with our clients. Scrum makes this more formal, but it’s easy to imagine tweaking what we do now to fit Scrum. The Scrum roles are also fairly familiar. Typically we have a primary investigator or business development professional (for us, this person is usually a client) who plays the role of Product Owner. Our teams also tend to be small, cross-functional and everyone pulls their own weight – values shared by Scrum.
Other aspects of Scrum are things we don’t do, but probably should. The Product and Sprint Backlogs are prioritized lists of tasks that need to be completed to produce the “product”. We usually have something like this in our science projects, but the prioritization piece is sometimes missing. I think back on all the times that each data analysis led to “just one more” interesting question to answer which drove us on a depth-first search to nowhere. A process that forces you to prioritize every new task and periodically review the list as a whole is a road map towards common sense. Scrum’s focus on measuring progress through points and visualizing progress through Burndown Charts is new, but again, not a bad idea. It might be a challenge to turn open-ended undertakings like “research oligo design tools” into quantifiable tasks, but I’m fairly confident it can be done – and it will be an improvement. And just as important as making tasks measurable is identifying when they’re “done”. This often goes unappreciated in our science projects, but Scrum forces us to address it.
So far, so good - maybe Agile science isn’t such a stretch after all! But before suffering from “premature exubaration” (as my sometimes off-color former boss liked to say), there are a few elements of Scrum that we still haven’t quite figured out. First, there’s the strict timeboxing of tasks. In Scrum, once the team has agreed to what they’re going to accomplish in an iteration, the team is expected to stick to this commitment and deliver. A typical iteration time is two weeks. How many science projects have I been on where priorities and tasks completely change in just days, let alone weeks? What’s the solution: shorter iterations, more discipline/more thoughtful tasks or get rid of the timeboxing altogether? Another issue we run into is the allocation of our team members. Often our team is made up of sub-contractors or 5AM employees who have only a fairly small commitment to the project – say at the 25% level. Can team members truly participate in Scrum with so little time available for the project?
So, our consensus opinion right now is Agile science has promise, but there are still some pieces we need to figure out. Like most things in life, proof will be in the execution. We’ll only really know how well Scrum works for us after using it on a few projects. We will periodically update you on our progress and hope that you’ll share your experiences and insights with us as well.