Tracking Feature Usage with Instrumental Monitoring

We use Instrumental’s application monitoring to track all kinds of things. Many of them are about server and app performance, like latency, queue response time, and disk utilization. But we also have graphs for other kinds of monitoring. This series explores some of our more interesting graphs.

We acquired Gauges Web Analytics in January. Since then, we’ve reworked the server architecture, moved data centers, increased performance, updated the interface for retina displays, rethought our marketing…and that’s just what we’ve shipped so far! We’ve got plenty more in the works, of course. But we ran into a problem awhile ago as we prioritized some of our bigger feature ideas: we realized we had a gap in our understanding of how our users actually use Gauges. We’ve sent surveys and talked to many of our users directly, but that only goes so far. What people do and what they say are often different.

So, we decided to instrument feature usage. Here’s how we did it. The Gauges interface effectively has 3 layers of navigation. First is the gauge (site), then the gauge-specific sections of traffic data, then (in some cases) a subdivision of that data.

We added some JavaScript[1] code that tracks when the second two of those change. We use the word ‘feature’ to namespace them and dots between the second and third level, which gives us metrics like:


When we increment those in Instrumental, we can use the following query to see a graph of how much each feature is used as a percentage of total feature usage:

(feature.? * 100) / ts_sum(feature.?)

That gives us this awesome graph:

Now we can see exactly how much each feature gets used. Even better, when we add features, we can see how much they’re used compared to the existing features! We’ve already learned some interesting things in the short time we’ve been tracking feature usage in Gauges.

After we let this graph simmer for a couple of days, we learned something that surprised us: AirTraffic isn’t used as much as we thought.

Some people use it, but not many. That surprised us. AirTraffic shows pins dropping on a map of the world as new visitors reach your sites, in real time. It’s pretty cool to watch, and we use it to see the impact of big announcements. Turns out our customers don’t use it as much as we thought.

We’ve got some ideas about why our customers aren’t using AirTraffic, and what we can do about it. Now that we’re measuring feature usage this way, we can visualize how our changes affect how features are used in Gauges.

[1] The JavaScript actually hits our web server, which makes the request to Instrumental. That way, we use our existing setup using the Ruby Gem.

Instrumental Free Trial

Understanding what's happening with your software is only possible if you monitor it at the code layer. From agents to our metric-based pricing, we’re focused on making it easy to measure your code in real-time. Try Instrumental free for 30 days.