Roman Imankulov

Roman Imankulov

Full-stack Python web developer from Porto

search results (esc to close)
31 Jan 2022

Tech debt hackathons is a waste of time

Rushing towards the backlog
Photo by Jess Zoerb

Repaying your tech debt with a code cleanup hackathon? Planning a bug fixing week to decrease the issues backlog? Likely, you’re wasting your time.

When I worked at Doist, we had a long-standing issue with Sentry. Sentry is an error tracking service, and our application recorded every exception there. Quickly, the list of issues grew so deep that our team learned to disregard its alerts.

Concerned about the backlog, we decided we needed a hackathon to clean it up. A few developers, including myself, spent two weeks fixing issues. It was a chore, but we did it! We went to the bottom of the backlog and started our life with a much cleaner system.

Fast forward a year, we got a wake-up call. A new developer who didn’t know about our adventure raised exactly the same concern as we did about a year ago, “Our Sentry is a graveyard of issues, and nobody cares. Let me clean it up!”

We were sure that Sentry issues were under our control, while they clearly were not. Our issue backlog was as large as it had been before the cleanup marathon. Our work was wasted, but we could not see it because the past victory lulled our attention.

I saw the issue repeating itself in different circumstances in different organizations to learn that any one-off attempts to fix a long-standing issue are doomed.

☝️ Most one-off attempts to fix systemic issues are doomed

While sometimes one-off events can get you to a better place, most of the time without a change in the system, the issue comes back. Sounds obvious in hindsight, but more than once I’ve seen teams trying to reach the “Sentry inbox zero” once and forever.

What can you do instead?

From hackathons to processes

Think about what is likely to happen in a few months after the hackathon. If your new steady state is the same as your current state, make a process instead of a one-time event.

Introducing a process requires changing the mindset. Instead of thinking, “we will jump in and do X to fix the issue,” you think, “to keep the issue from appearing, we will start doing X regularly.”

As with any regular activity, processes require time and space. Let’s make it very clear that this activity goes at the expense of something else that’s already on your plate. It helps if you talk with stakeholders and they understand the importance of the issue.

If you’re allergic to processes because they sound too bureaucratic, I would recommend reading The Null Process, a short and insightful essay on this topic.

Introducing the change comes at a price. Before the team gets to a new, better level, their morale will be tested. A short period of enthusiasm will be followed by a stage of disappointment from the lack of quick results. This is where the role of a passionate leader is crucial.

Kübler-Ross change curve

Kübler-Ross change curve

Building culture with passionate DRI

It’s enough to have one passionate person to change the status quo. Starting a new process is like building a new habit, but for the entire team. Building a habit is impossible without changing the mindset. You need a driver to run the change.

Find a preacher and give them resources (mostly time) and formal authority (right to take some decisions). If someone from the formal leadership team is ready to become a preacher, it’s perfect, but the grassroots movement is also a good starting point.

I find that the concept of DRI (directly responsible individuals) works well. Pretty much like, “Hey, I can see you are frustrated with that Sentry thing. Do you want to become the DRI of this and fix is for yourself and others.”

If they’re still unsure, try sharing this inspiring video on how movements get started.

The good part is that you don’t need to convince the person that the problem exists and needs to be solved; they are motivated enough already. Just let their passion drive them and step aside.

recap

Preachers -> Culture -> Processes -> Recurring actions

When one-off events still make sense

Not every one-off tech debt elimination is a waste of time, and not all improvements require a change in the process. Sometimes, it’s enough to switch to a better solution with a one-time investment. A few examples came to my mind.

  • Adding Sentry to the stack in the first place.
  • Migration from Python 2 to Python 3.
  • Migration of the payment system from Paypal to Stripe.
  • Fixing a long-standing bug that became a time sink for teams.
  • etc.

Sometimes, the new steady state is better than the old one without changing the processes. Don’t take it for granted, though.

Roman Imankulov

Hey, I'm Roman, a full-stack Python web developer.

If you like what you read and want to work with me, drop me a line to roman.imankulov@gmail.com.

About me