6 min read

Upgrading to R #2: Failure’s Coming, Get Ready!

This is definitely the most personal, and the most painful of my upgrading to R lessons. But first, here’s a basic outline of something that many first-time managers go through. I can guarantee something like this will happen if you’re managing a team upgrading to R.

In case you didn’t feel like sitting through that animation, here’s the bottom line: my team was doing great work upgrading to R, a mistake made it to a client, I WAY overreacted and burned a lot of goodwill that took a long time to get back.

Here’s a checklist to figure out if you’re in a similar situation [] - You’re upgrading your team to a new technology (R). [] - You have to keep doing real work while that transition happens.

That’s it. If you’re in that situation, mistakes are very likely to happen and likely to make it to someone who shouldn’t receive mistakes. You’ve got to prepare yourself and your team.

Prepare Technically

This is the obvious kind of preparation. Make sure that you’re doing everything you can to avoid mistakes.

Particularly if you’re in the business of producing data science insights (as opposed to making models for machines to use), the #1 best tool you have for catching errors is the data scientist or analyst’s brain. It’s sometimes possible to set up more formal error checking (e.g. unit or integration tests) in the style of software development. But more often, if you’re generating insights, the only way to know if what you’ve done is right is to examine the process. Here are a few ideas on how to make the process more ironclad.

Sanity Checks

It’s all too easy to churn out an analysis, finish up, breathe a huge sigh of relief, and send it off without thinking about how many 156 year-olds appeared in the data.1 Setting team norms (and putting together checklists or templates) for taking a close look through incoming data and examining outgoing analysis can make it way more likely that you’ll catch mistakes.

Share More Context

This is really only relevant if you’re the team lead…but share more context. Think you’ve shared enough and they don’t really need more? SHARE MORE CONTEXT.

I have never run into a situation where the amount of context I gave someone who was working with me was a hindrance, and have been bitten by not sharing enough.

Kinda vague? Here’s an example – I had a team member working on a project for a client that involved visualizing performance on several metrics to thresholds for the metrics. The plots looked something like:

The input data set had a field for performance (the color) that I assumed my team member was using. They assumed that the color could just be computed from the metric and the threshold. The context they didn’t have was that some of the thresholds changed over time, some referenced prior time periods, and some were whether the difference between the metric and the threshold was statistically significant. Because of my assumption that they would just use the column in the data, and my lack of sharing context, we produced incorrect plots that were spotted by the client (who knew the data well).

Code Review

Code review is great – but if you’re creating insight products, this is more-or-less your last resort. And even then, I think the primary purpose of code review is to trigger thinking by the person who wrote the code, not for them to be handed insight from on high.

Institute good code review practices as soon as you can. This can be a hard one, as it’s tempting to believe that code review that doesn’t involve leaving comments on pretty red and green diffs on github isn’t real. But any review, no matter how low-tech is 1,000x better than code review that doesn’t happen because you’re waiting for the team to “get up to speed with git”.2

In fact, one of my favorite ways to do code review is to print out the code, read through it with the person who wrote it, and mark it up like an essay. Decidedly low-tech, but really engages the internal compiler.

Low Tech

Low Tech

Prepare Emotionally

But even if you do all the things above, failure at some point is almost inevitable. It’s kinda cliche at this point to say that failure is inevitable, and what matters is how you handle that. But as someone who knows that but still handled it badly as a team leader, here are a few tips I’d share.

The number 1 most important thing you can do is avoid overreacting.3 No matter how important the thing that’s messed up is, the long term health of the team is more important. It’s definitely hard to feel this in the moment, but this surely isn’t the only thing the team is (or will be) working on. Burning bridges with your team by overreacting, assigning blame, and creating draconian new rules that will stifle people’s work ethic is unlikely to lead to the outcomes you want.

To be a little more concrete: it’s impossible to overrate the effectiveness of generating solutions as a team. Blameless postmortems on any major milestone or serious error are a great thing to start today, and they are also the right way to recover from a serious error or mistake. Sitting with the team and emphasizing solutions (rather than blame) are the way to recover. In particular, I’ve found several times that handing solutions down from on-high is ineffective. Meeting as a team to come up with solutions work great – even when the eventual team solution is the same as the initial one that didn’t work!

A Coda: Book Recs

Two books I couldn’t recommend more strongly in this vein are Rising Strong by Brene Brown and High Output Management by Andy Grove. Brown’s book is a great exploration of failure and recovery. Trying to embrace the mindset in this book, even with frequent failures, has helped me be a much more empathetic and effective leader. Andy Grove’s book is a much more traditional “management” book, but his explanation of managing to task-relevant maturity is a great way to think about avoiding failure points whenever possible.

  1. Or more likely 999 year-olds. Missing data should never have numeric codes, amirite?!?

  2. Talking to you, past Alex!

  3. Especially if you’re the team lead, but really no matter what.