When browsing through GitClear's expansive breadth of developer metrics offered, a careful observer will note that roughly a quarter of them involve some measurement of time. Based on customer notifications setup, five of the most popular time-based metrics include:

All of these metrics rely upon calculating how much time passed between an event "start point" and "end point." The naive way to calculate this time would be to take Time event ended - Time event started and return that difference as the "stat value." What makes this option pale in comparison to deriving the duration value based on hours that the team is actually working?

There are many answers. You can read the high level answer in the first section, or jump straight to a desired section, if one catches your eye:

linkProblems with "absolute time" calculations

There are several problems that emerge when one calculates event durations using "absolute" or "real" time -- time that doesn't factor in real-world working hours. Most of these problems are not obvious until you spend time mulling: what, exactly, do I want the team's developer metrics to capture?

It's an esoteric topic -- and one that our team has spent years considering.

When your team is first onboarding with GitClear, it's common to have someone ask "why don't they just measure this in "real" hours?" This page is dedicated to helping you answer their question with specific examples. It finishes by delving into technical specifics about how GitClear calculates accurate durations -- especially for the typical modern-day dev team, with members often scattered across many time zones.

link"Absolute time" problem #1: Most time is statistical noise

open_in_new

From a developer metrics standpoint, most of the hours in a week week are noise

The average work week is around 40 hours. The total number of hours in a week are 168. It does not take an advanced Mathematics degree to observe that only 23.8% of the week is made of time that we should expect progress to be made toward development goals.

Put another way, more than 75% of absolute time is irrelevant to what a normal development team wants to measure. Teams want to instrument how many hours passed in which a developer reasonably could have been expected to "review a pull request" or "start work to fix a Critical Defect"? When you calculate using absolute time, most of the hours in your calculated duration are "time spent their friends/family" or "time asleep."

Illustrative example: If your teammate posted a pull request at 4pm your time on Friday, and you reviewed it the following Monday at 10am, would you feel it more fair to say "the pull request awaited review for 2.5 business hours" or "it awaited review for 66 total hours"? If you prefer the former, you share the average developer's distaste toward being measured with a high-noise calculation method.

link"Absolute time" problem #2: Notifications triggered by hours passed

The single most common "team goal" chosen by GitClear customers is "Hours Awaiting Review" for a pull request, which is generally expected to be equal or less than one work day, aka "8 hours."

Without using business hours, if a developer submits their pull request for review on Friday afternoon, the system would generate an alert on Saturday afternoon that the pull request violated its target for "Hours Awaiting Review." It's not difficult to imagine your team's frosty reception if such a Slack notification is triggered on Saturday afternoon.

Likewise, many Critical Defect-based notifications (e.g., "Hours to Fix Activity Onset") are expected to be resolved within 1-2 business days. If the defect is detected at 6pm in the evening, the equivalent of two workdays (16 hours) might have passed by the time any developer is awake and available to work on the issue. A team is likely to have hostile feelings toward a manager that counts the hours they are sleeping against their developer performance.

link"Absolute time" problem #3: Any stat where the optimal time unit is "hours"

"Pull request time awaiting review," "pull request cycle time," and "time until defect resolved (aka 'Mean Time to Recovery')" are all metrics that teams often want to measure in "hours," because, ideally, a team should not let these issues linger for more than 5-10 work hours.

open_in_new

For urgent matters, it's preferable to measure by hours

Since more than 75% of all hours in a week are non-work hours, any measurement whose units are "hours" is going to be more than 75% noise if it's measured in "absolute time" instead of "business time."

link"Absolute time" problem #4: Developer experience

Setting aside the first three reasons for preferring "business hours" to "absolute hours," imagine that all things were equal, and you could measure stats in absolute or business hours. Which is going to work more toward your long-term benefit?

The ultimate reason that teams collect developer analytics is that they want to help their members understand how performance has varied over weeks, months or years. At the end of the day, these measurements are going to be discussed between developers, managers, and executives, as they evaluate where there should be changes in how the team is run.

In the world where you opt to measure by absolute hours, everything about the team's performance looks much worse. It will appear that developers regularly let 50+ hours pass before reviewing a pull request, if their teammate is "careless" enough to submit their pull request on a Friday afternoon (in which case, you have 48 weekend hours plus the time Friday night and early Monday morning). Of course, the cases where 50+ hours passed because "the weekend" or because "the team was actually too busy to look at it" will be indistinguishable without time to scrutinize the circumstances.

Do you want to put yourself in a position where you need to explain to a developers that their PR review time of "30 hours" is beyond the teams goal, and then have that developer respond that "I can't control that my teammates prefer to submit their PRs at the end of the day or end of the week?" Absolute time creates the impression that managers expect developers to be online and available 24/7. If that is not what you expect of your team, then measuring them as such is only going to cause unnecessary animosity.

Prefer the measurement unit that honors the humanity of your team. Prefer it because you respect your coworkers, or prefer it because you don't want your best developers to quit.

linkComplications of measuring by "business hours"

While we have covered several benefits to preferring "business hours" over "absolute hours," it is worthwhile to acknowledge that this measurement is not a panacea for extracting flawless estimations of how much work time passed between the "beginning" and "end" of the stat's definition. There are two main drawbacks of measuring by business hours: below is our advice on how to discuss these drawbacks with your team.

linkWork done on the weekend

Rare is the manager that would call it a "problem" if their team chooses to work over the weekend. However, it certainly does complicate the effort to assign a time measurement, if all of the time that elapsed while e.g., "waiting for a PR to receive a review" was outside of business hours.

If a developer submits a PR for review Saturday morning, and they receive a review on Saturday afternoon, GitClear will record this pull request as having taken "0 hours" to receive its review. This might seem odd, since in absolute time, there were in fact several hours that passed -- they were just not business hours.

GitClear's strong opinion is that smart managers think about measurements as "averages" and "medians," and a pull request that is handled outside of business hours should be treated as having a gravity that pulls the average time down. In a sense, when a developer elects to work on PRs off-hours, the employer receiving an unexpected gift from their team. Just like some teachers in school will offer "extra credit" as a means to supplement GPA for students willing to work extra hard, so too should a developer that is willing to work "evenings" or "weekends" be afforded an opportunity to reduce their "average PR turnaround" beyond what would be possible if they only worked during business hours.

When a PR is calculated to take "0 hours until initial review," the developer wins because their stats look better. The manager wins because they are getting labor that they should not expect to receive, at least over the long-term. We believe those are potent benefits that more-than-offset the peculiar appearance of PRs that were measured to have spent 0 minutes awaiting review.

linkSometimes absolute time matters

There are a couple metrics that teams sometimes. want to measure in absolute time. Chief among these are the "time to recovery" stats when a sufficiently Critical Defect has occurred that customers are completely unable to use the service. To instrument these critical metrics, GitClear offers

linkInitial time zone calculation

The other challenge of measuring stats as "business hours" is a challenge to the Developer Analytics provider: how to derive a developer's "working hours," when teams are often distributed across multiple time zones?

Fortunately for our customers, GitClear has been progressively improving its mechanism to match "developer" and "working time zone" since 2019. At first, the implementation was sketchy. All developers occasionally make commits or PR comments outside business hours. Some developers make a habit of working outside business hours. We have adapted to these edge case circumstances by building a robust framework to collect hundreds of data points for "developer activity," and to statistically resolve those data points into the single time zone that best clusters the developer's work within 4.5 hours of 1pm.

This still leaves the potential that a developer's "working hours" will occasionally shift, not least when Daylight Saving time in the US explicitly triggers developer working hour changes. We handle this eventuality by recognizing when a developer's assigned time zone has changed, and automatically queuing a recalculation of any time-based stats when this should occur.

Any developer metric that is calculated by time will use a per-developer time zone to make its calculation. So even if one developer's time zone changes, it will not impact the measurements from other developers on the team.

By a similar token, if 5pm in Developer A's time zone is 9am in Developer B's time zone, when Developer A submits a pull request for review at the end of their work day, Developer B is immediately "on the clock" for the hours until that pull request is reviewed (since it is the beginning of their day), whereas other developers in Developer A's time zone will not have their evaluation of "time until the PR was reviewed" begin until the following day at 8am. This means that business hours offer a stable source of truth for "how much work time has passed?" even if the team has members that work in a variety of different time zones.