InstrumentalD Plugin Scripts
InstrumentalD provides essential metrics for common services, but you may have other pieces of architecture that require more custom monitoring. That's why InstrumentalD supports Plugin Scripts.
See below to learn how to write your own plugin scripts, and be sure to check out the examples on GitHub. We also work with customers to develop plugins on an as-needed basis. Please contact us if we can assist with plugin development.
Installing Plugin Scripts
To install scripts, place them in the directory $HOME/.instrumental_scripts. The InstrumentalD process will create this directory if it doesn't exist the first time you run the process with script functionality enabled (using the -e flag). You may also specify a specific directory to the InstrumentalD process with the -sflag.
Permission Requirements
The directory you use for plugin scripts and the scripts themselves must be readable and writable only by owner (0700), which must be the same user that the InstrumentalD process runs as. Otherwise, the process will exit with an error message alerting you to the fact that it cannot use the directory.
Writing Plugin Scripts
A script may be a binary or shell script that exists in the scripts directory ($HOME/.instrumental_scripts). Each time the InstrumentalD process collects system metrics, it will also execute your script with the following arguments:
| Argument 1 | The Unix timestamp of the last time this script had been executed, in seconds. If the process has not yet been successfully run by InstrumentalD, this value will be 0. |
|---|---|
| Argument 2 | The exit status of the process the last time this script had been executed. If the process has not yet been successfully run by InstrumentalD, this value will not be present. |
| STDIN | The STDIN pipe to your process will contain the output of your script the last time it had been executed. You may use this data to compute differences between the last time your script ran and the current execution. For example, the MySQL script uses this to compute rate metrics. |
| Environment | Any environment variables set for the InstrumentalD process will be available to your process. |
Your script is expected to output data in the following format on STDOUT in order to be sent back to Instrumental:
METRIC_NAME METRIC_VALUE
or
METRIC_NAME METRIC_VALUE UNIX_TIME_IN_SECONDS
For example, if a script named application_load were to report two metrics, memory and load, to the process, its output should be:
memory 1024.0 load 0.7
The InstrumentalD process will submit each metric to Instrumental in the following form:
HOST_NAME.SCRIPT_NAME.METRIC_NAME
Using the previous example, if the application_load script ran on a host named app-0001, its memory and load metrics would be submitted to Instrumental as
app-0001.application_load.memory
and
app-0001.application_load.load
The optional third parameter of the above formats, UNIX_TIME_IN_SECONDS, represents the time under which the submitted metric should be measured. Generally you do not need to provide this value, as InstrumentalD will default to recording the time when it receives the metric from your script as the time under which the measurement should be recorded.
Exit Codes
If you do not want the output of your script submitted to Instrumental, your process should exit with a non-zero exit code. Its STDOUT output will still be provided to your script on the next iteration.
Errors
You may output error information on STDERR of your process, and it will be output to the InstrumentalD log to aid in debugging script behavior.
Timeouts
Your script is responsible for managing timeouts. The InstrumentalD process will not attempt to terminate your process for you.