R Packages for Experts, not Wizards
Write internal R packages for your team that disseminate expertise, not cleverness.
I, like so many other R aficionados, love writing R packages for my team. They’re a great way to get everyone on the same page and to ensure everyone has access to the same resources. But I’ve also (more than once) lovingly crafted an R package only to find that…no one used it.
I’d incorrectly thought a bunch of clever R code could solve my team’s problems. I’d assumed I, the team’s R wizard, could go write the perfect package, and return to great acclaim that I’d fixed everyone’s woes.
Unfortunately, my vision of how the package writing process was busted in at least three ways – ones that I’ve seen befall other teams and R aficionados. Read on for more.
The Wizard Package
When I wrote my team’s packages, I was the R wizard on the team – and if you’re reading this, you probably are too. I believed I could go off, write a package, and deliver it from on high a la Moses from Mount Sinai.1
I’ve come to call the result of this (not so great) pattern the Wizard Package.
Wizard Packages can work. I’ve seen Wizard Packages work when (1) the whole team is clear that a package will solve the problem, (2) the need is can be met by clever R code, and (3) the team is strongly motivated to adopt the package.
In my case, none of these conditions held. I wasn’t riding a wave of enthusiasm for R – I was trying to spark it. People weren’t super motivated to adopt the package, and I couldn’t really compel them to do so. Worst of all, my R code just wasn’t solving the keenest problems of my teammates.
Because they’re written by R wizards, Wizard Packages tend to solve R wizard problems. They have clever wrappers for other functions, or convenience functions for advanced R users.
Very often though, the team’s needs aren’t needs that are solved primarily by clever R code – the team needs to get their R Markdown report formatted with the right CSS, or calculate correct standard errors from a complicated estimator, or complete those weird database merges that only one person on the team knows how to do.2
Wizard packages convey expertise of a sort, but they often aren’t adopted because they’re not expert enough, or aren’t expert in the right ways.
Expert Packages
R packages are a tool to make your team’s expertise real. By incorporating knowledge from everyone on the team, you can create a package that disseminates that expertise to everyone on the team.
There’s tons of psych research (that I’m too lazy to actually cite) that one of the best ways to get someone excited about something is to get their help with it.
More importantly, there’s far more to package development than writing R code, so the process really can include everyone, regardless of their level of comfort with R.
A few ways for folks to help out who might not be the R wizards:
Designing the API Even folks who are novice R developers probably have a strong sense about what the important features and options are for things like plotting the data.
Designing the data model Someone who knows the problem really well probably has the best sense of what the important parts of the data are to keep track of.
Sharing expert knowledge An Expert Package shares expertise across the team. By collecting that knowledge directly, you can incorporate someone’s data vis, database access, or stats expertise without needing them to write R code.
Writing documentation, tests, or vignettes Internal packages often include documentation or vignettes that go beyond just explaining the functions themselves, but providing some context on why the function works as it does. People can also provide feedback on whether package tests are meaningful.
Make Your Packages Experts, not Wizards
There’s a reason that so many stories of Wizards include dark endings, evil omens, and fallen mages. Wizardry is ineherently disconnected from others, and it’s all about being clever.
In the land of R packages, being expert is so much more valuable than being clever – doubly so if you can provide expertise and value to others on your team, and the number one way to do that is to get your teammates involved in the package creation process the whole way along.