One of the most common questions posed by new GitClear users is some variation of:

How do I know that this "Diff Delta" metric has any relation to my own perception of which developers are making valuable contributions?

It's a great question! So much so that we have dedicated hundreds of hours of research to substantiating that Diff Delta corresponds to "developer effort" at scale.

But there might be a difference -- perhaps even a big difference -- between our metric "correlating at scale" vs "correlating on your team." That's why we built the Developer Calibration Calculator.

linkHow it works

The idea is simple:

open_in_new

Estimate the velocity of developers you work with

By dragging-and-dropping developers to the x-coordinate that corresponds with your perception, of their programming velocity we are able to calculate a Pearson correlation value that quantifies the extent to which your "velocity intuition" matches the last year worth of observed data.

linkWhat does it mean to have a "High Estimated Velocity"?

A developer with the highest estimated velocity (the very far right of the "Committer Drop Zone") would be akin to what Paul Graham calls "an animal." They "pass right through professional and cross over into obsessive." This is the first person on the team that the manager would think about if there was a critical bug they needed to get fixed FAST. Developers with the "Highest Estimated Velocity" don't just add a bunch of code -- they are expert in removing code, so that the repo can continue to be maintainable as the size of the team grows.

Conversely, a developer with "Lowest Estimated Velocity" is usually a junior developer, or a developer with ambivalence about their work. Their work is often behind schedule, and when it does finally arrive, it often needs multiple rounds of PR review to get it in a state where it can be deployed without introducing a new site defect.

link"But, isn't this just stack-ranking developers?"

Yes and no. Yes, you are using your subjective judgment to evaluate teammates, which might make some feel squeamish, given modern preferences to avoid picking "winners" and "losers."

But the difference between the Developer Calibration Tool's stack ranking, vs. the stack ranking that is widely loathed in the 2020s, is that this stack ranking is for your eyes only. Whether you are an admin, manager, or developer, GitClear will never divulge your subjective evaluation to your teammates or manager. Even the GitClear user in charge of paying for the company subscription can not get access to this data.

For more information about how GitClear stores & uses data from the Developer Calibration Calculator, see the "How calibration data is used" section, below.

linkReceiving your correlation report

After you submit your interpretation of your team, you will receive a report akin to the following:

open_in_new

Sample correlation data received after submitting velocity estimates

It will calculate the correlation between your "estimated velocity" and five of the most common developer metrics:

The latter two measurements are available in repos where there is sufficient data to calculate them. Some teams don't use pull requests, and some admins don't connect their issue tracker to GitClear. If you connect your issue tracker, and your team uses Story Points, then you'll receive a correlation value for how closely your "velocity intuition" corresponds to their actual "story points completed" among the resolved issues assigned to the developer.

linkHow calibration data is used

When you receive your correlation report (usually within an hour of submitting your Calibration data), you will be able to assess the extent to which the five most major developer metrics (Diff Delta, Commit Count, Code Lines, Merged PRs, Story Points) translate to your interpretation of "who's doing the heavy lifting on this team?"

You can use the data from GitClear's Calibration Report to build your intuition for which metric(s) deserve your attention. This can be very useful information when looking at the history of your team, to understand the circumstances where developers were maximizing the quantity that best correlates with "velocity."

Aside from your personal use (we do not show the calibration data anywhere on site -- the email you receive is the window to your result), GitClear will also periodically aggregate thousands of evaluations in order to contribute free published research to the community. We believe that a greater collective understanding can be generated by knowing which metrics are the ones that best align with what humans think of as "a valuable individual contributor."

When we publish such research, we will never identify individual developers -- all data reported would be only numeric "correlation measurements" that assess the data set as a whole.