Non-technical sprints

"Sprints" come from the agile methodology of software development. However, at a stretch, the concept can be used for "management work" as well.

One of the most contentious topics in data science is the use of software engineering like “sprints”. I’m personally not a fan. I’m in the camp that believes that data science is fundamentally a creative profession, and putting in sprints either wastes time (when you allocate too much time for something) or puts teams under undue stress by committing to impossible timelines.

Rather, “sprint” is a good framework when you’re performing reasonably deterministic (low standard deviation of time for completion) tasks, and a lot of data science modelling doesn’t fall under this category.

Then again, I’m fundamentally a business guy who has stumbled into data science by way of analytics. A bulk of data scientists are far more technical than me, with many actually having worked as developers, and they are far more comfortable using sprints as part of their workflow. Moreover, in “industrial” data science, a significant part of the effort goes into putting models into production, rather than building the models themselves, and that is a task with low enough standard deviation in time taken that the use of sprints can actually be handy.

In any case, this is not a post about the merits and demerits of sprints in data science - that can be a debate for another day. I want to talk about the concept of sprints that can be extended to other fields as well.

In my opinion, a fundamental requirement for sprints to work is downtime between sprints. Keeping with the metaphor, the only way you can sprint consistently is if you can catch a breath after each one (and in a corporate context, no, it cannot be a weekend - it needs to be a low intensity period of business days). If you are not taking your breaks, consistently sprinting only suggests that you are overworked - or maybe running around like a headless chicken.

Anyway, I realise that in the 3-4 months since I resigned at Delhivery and started working on my current startup idea, I have actually got into the habit of doing “sprints”, though not in the software engineering sense.

In early October, I had a sketchy idea and lots of questions that needed to be answered. What market should I target? How should I go about building? What kind of cofounder to recruit? How do I structure equity in the company? How do I raise funds for this? And so on and so forth.

This meant I needed to talk to people, and so I started what I can now call a “initiation sprint”. In about 40 days, I met about one person on average a day (picked on a lot of Bangalore based people so I could do these meetings in person), and got tonnes of fundaes. After each meeting I would either rush home, or sit in a coffee shop or the back of an auto rickshaw, and furiously make notes, so I wouldn’t miss anything.

By the middle of November, I had some clarity in my head on a lot of these questions, including starting some fairly serious cofounder conversations. It was also time for my son to be born, and so, having finished a round of conversations (which now, in retrospect, I can call a “sprint”), I eased off.

After I had taken two weeks off (for “paternity”), I got back to work in the beginning of December, and then it was time to understand how people use dashboards and make decisions using data, in order to refine the hypothesis.

I put out a few posts - here, on LinkedIn and on Twitter - and that led to a slew of conversations. In a two week span, I had spoken to 20-25 people, on how they use dashboards and make decisions using data. Again with the full benefit of hindsight, this was yet another “sprint”, where I worked odd hours (to talk to people in other time zones), and did a Bayesian updation of the hypothesis after each conversation.

And then came the Christmas week, and I eased off, to synthesise my thoughts and get more clarity on the product. This, again, coincided with a lot of people (including my cofounder) taking their year end breaks.

This week (the first week of January) was again time for what turned out to be a “mini sprint”, as my cofounder got back and we sat down to get more clarity on our product. This was also the time when we got ourselves an office, and for two and a half days, we brainstormed incessantly, fleshing out the details on what we need to build and how we can potentially build it. On Friday, with us in a much much better place regarding the product, I went home early.

So you get the drift here - work so far has largely been in bursts (or “sprints”), punctuated by downtime. This has been helped in no small means by the fact I’ve been able to take things up one at a time. And now I feel like I might be hitting the limits of this “sprint strategy”, since there are multiple, possibly parallel, things to be done fairly soon now. Such as:

• fund raising

• “STP”, to figure out the precise customer segment to target and how we will position our product. This will again involve a LOT of conversations with potential customers, to gather data to guide this

• hiring

And these cannot be done serially. Trivially, we can’t hire until we have raised funds, and once we have raised funds, we cannot take an infinite amount of time putting together the team. So these need to happen in parallel. Simultaneously, the STP will precisely guide what we will build (and potentially who we might need to build it), and so we better do this in parallel as well.

I don’t know how well sprints work while doing things in parallel. Based on my experience so far, sprints work when there is a single main task to do at a time, and it can be compressed into a short period of time after which we can ease for a little bit. With so many things to do at once, I don’t know if sprints might be efficient.

And in this particular case, I might get away with “divide and conquer” (precise delineation of responsibilities with my cofounder), and each of us sprinting our respective responsibilities. However, as things begin to move and take off, the number of things to do will be so much that sprints won’t be a sustainable way of working. I’ll need to figure out a new paradigm then.

I guess there is a reason “sprints” aren’t too common in non-technical jobs.