Today marks the release of GitClear's latest AI research, "AI Coding Tools Attract Top Performers - But Do They Create Them?" As with past GitClear research, it is available for free download as a PDF.
linkResearch Abstract
The first quarter of data from GitClear's AI usage measurement is in, and it defies conventional wisdom. Developers who use AI throughout the day aren't just 10% faster—empirical data shows them authoring 4x to 10x more work than “AI non-users” during the weeks their AI use is highest.
However, our research suggests this isn't another story about the triumph of AI tools and agents. Moreso, this story is about who is using the tools and what company policies must prospective AI developers contend with?
Commit Count is significantly correlated with greater AI use
Constructed from AI provider APIs, the new data (from GitClear's AI Cohort Stats) lets us compare developers whose weekly AI use ranged from "Non-User" to "Power User." Contrary to our expectations, the difference in productivity data between the two sides amounts to a chasm.
Weekly “Commit Count” is far from an end-all productivity metric – but it mirrors the trend found across other proxies for developer output:
The difference in developer metrics among AI users is readily apparent
Finding such an extreme gap implies the existence of something akin to “dark matter” for developer productivity. There must be much more than “enlightened AI use” to explain a difference of this magnitude. In this research, we'll present and dissect data to understand “how can the highest engagement cohorts average 4x more progress across almost every metric tested?”
In other words, what’s the source of the “dark matter productivity” enjoyed by the “Power User” cohort? If factored out, how large of a difference remains?
That calculation can help executives hone their intuition for “what quantifiable improvement can we expect to observe, if our team succeeds at maximizing AI benefits while minimizing AI side effects?” As past AI research has demonstrated, the latter objective requires careful attention to fighting the code churn and code block duplication that accompanies contemporary AI use?
linkVenn Diagram of Developer Cohorts by AI Use
A breakdown of factors that contribute to a productivity chasm between the most avid AI users and those who use it primarily outside the context of their IDE.