The Cohort Comparison graph shows how quickly a new developer is getting up to speed relative to past developer hires. The committer shown here scored the second highest ever Line Impact for a 6th week of tenure.
A cohort is a group of people who share common characteristics within a defined period of time. It’s a great method to compare an employee to others who are at the same stage in their career, or in a similar role. Comparing individuals to a group of their peers makes it easy to spot differences in performance or work style. It’s a good way to identify the highest (or lowest) performing developers and gain insight into the factors that influence their productivity.
But traditional cohort analysis can be a time-consuming and flawed process. First, you need to define the cohort you’ll be measuring – not an easy task if you manage multiple teams working on a disparate collection of coding assignments. What type of projects should receive the most emphasis? What data points will you use? Most often, engineer cohort analysis is based on data gleaned from ticketing systems like JIRA. But counting closed tickets is an overly simplistic way of evaluating engineer productivity. Once a manager makes all these decisions, the data still has to be imported into Excel, normalized and formatted before the first report can be created. No wonder so many managers loathe the task. It’s a perfect recipe for procrastination, followed by settling for recency bias.
GitClear uses cohort analysis to give development managers a powerful new code metrics tool built for reliability. Based on our experience, we’ve listed a few areas where code metrics can steer a new developer toward their path of maximum throughput and satisfaction:
Does your new hire excel in Python, but struggle with Ruby? What types of tasks do they excel in? Are they willing to learn others?
Is your new hire struggling to commit code? How little should you expect in weeks 1-4? Can better onboarding speed that up? How does that measure change by week ten?
Does the new hire possess the self-discipline needed to work remotely? How can you challenge them to work more efficiently, without sacrificing quality? How can you identify and award those that consistently improve their output?
Here are a few ways that GitClear uses cohort analysis and other visualizations to create a more accurate measure of engineer productivity.
The best chart we have to understand how a new developer compares to their teammates at the same point in their career is the Cohort Comparison chart. Compare the Line Impact over a configurable time range. Check up on how the first 2, 4, or 6 months of the Developer’s career compare to that of their colleagues at the same point of tenure. This chart can be filtered to only show “Junior Developers,” “Devops Team,” “Software Architects,” or whatever other groups your organization identifies.
Similar to the Cohort Chart, but not anchored to employee tenure. On this chart, you’ll learn how developers of all experience levels compare to one another over your selected time range. For developers past their second year of experience, it’s often most informative to use this chart, since it normalizes shared temporal conditions that may impact productivity (like the holiday season).