When looking for a new APM tool, you’ll want to closely consider the following options:
- Data Storage
Instrumental aggregates all data to a 60 second resolution. We store that data for at least 13 months. Let’s break that down:
Aggregation is what we do with data points sent to us more frequently than once per time period. For a gauge metric, we store the average (mean) of all data points received during that time period. For an increment metric, we store the sum of all the data points. Then you only have one data point to save in the database, making monitoring both cheaper and faster to query.
Some services do not aggregate more frequently than a second and instead only store the last data point, throwing all the other data away. We think this is horrible, forcing you to aggregate the data yourself or simply lose data. Instrumental can handle as much data as you can send. We’ll do all the aggregation for you on our end.
What does resolution mean?
All monitoring platforms have to choose how much data to store per time period. This unit of time is called a “resolution”. Smaller resolutions require more storage space, so it’s important to strike a balance between usefulness and cost.
Some services offer differing levels of resolution. Some let you chose the level, others automatically increase the resolution as time goes on. With Instrumental, all data is stored in 60-second periods.
Why 60 seconds?
In theory, 1-second resolution is great. In practice, resolutions smaller than 60 seconds are more misleading than helpful. It’s important that the resolution contain enough data to paint a realistic picture of what’s happening. Small resolutions cost more, show more jitter when graphed making them hard to interpret, and encourage false alerts.
In short, 60 seconds is almost always the smallest actionable time frame. The simplicity of a single resolution keeps our infrastructure lean, but still allows us to economically offer 13-month storage of all data. We think 60 seconds gives our customers the most bang for their buck.
Why 13 months?
So you can compare this month to the same month last year! Pretty handy! And we store up to 13 months at the 60 second resolution. Many competitors also store up to 13 months, but aggregate the data up to much larger resolutions.
We hope this helps you understand what happens deeper within the mechanics of your monitoring tool and what some of the terminology means! Hit us up at @Instrumental or firstname.lastname@example.org if you have any questions.