Customizing miteʼs stats

There are two components of the mite pipeline that are concerned with the reporting of test data: the stats component and the prometheus exporter. The prometheus exporter works in a fully automatic way, based on the behavior of the stats component. Therefore, when customizing the statistics that mite exports, it is sufficient to work on the stats component; the prometheus exporter will do the right thing.

Mite stats are registered as entry points in python. Therefore, any modules that you install that extend miteʼs stats will automatically be picked up when the stats process starts. It logs a message for each moduleʼs stats that it finds, so you will know exactly what is loaded.

Writing custom stats

Prometheus, miteʼs chosen data backend, supports three different kinds of metrics: a Gauge, a Counter, and a Histogram.

Note

Prometheus also supports a Summary metric type, but this is just (for our purposes) a less performant version of a Histogram, so it will nt be further discussed.

Each of these is exposed as a class in the mite.stats module. When instantiating one of these classes to create a custom metric, you will need to supply a name, a matcher, and an extractor. The name is directly translated into the prometheus name for the metric, so it is useful for it to start with mite_ so that related metrics will be grouped together (especially if the prometheus instance that collects your mite metrics also collects other data, like CPU and memory usage of application processes).

The stats component acts by processing a stream of messages generated by the runner and controller. The matcher is a function that filters these messages, determining which are of interest to a particular stat and which it can ignore. Because each message has a type field, it is often useful to match on the value of that field, which is the purpose of the included mite.stats.matcher_by_type function. (However, the matcher receives the entire message dict as an argument, and can perform arbitrary computation on its contents if it wishes.)

The extractor function operates on the messages that have been selected by the matcher. The extractor pulls out some labels from the message, which will be added as labels to the time series in prometheus. It is also, in most cases, responsible for extracting a numerical value from the message. (The exception is stats of the Counter type, which count the occurrences of a particular message type. There, no value dependent on the message is needed, as the value of the counter is always incremented by one). The built-in mite.stats.extractor function covers most use-cases. It takes two arguments. The first is a sequence of strings, which will be extracted as keys from the message dictionary. The values of these keys will be used as the labels on the prometheus metric. The second argument is a single string, which names the dictionary key containing the numeric value to be accumulated. For more advanced usages, it will be necessary to construct a mite.stats.Extractor class directly. See mite.stats.controller_report_extractor for an example of this.

Here is an example of a custom stat from our work on AMQP testing:

def _amqp_extract(msg):
    for key, value in msg["total_received"].items():
        yield (key,), value


_MITE_STATS = [
    Gauge(
        "mite_amqp_tx_stats",
        matcher_by_type("amqp_tx_stats"),
        extractor(["message_name"], "total_sent")
    ),
    Gauge(
        "mite_amqp_rx_stats",
        matcher_by_type("amqp_rx_stats"),
        Extractor(labels=["message_name"], extract=_amqp_extract)
    )
]

In our AMQP injection functions, we send two messages. The first, amqp_tx_stats, names a message_name and contains a total_sent value. That is extracted into a Gauge by the first stat. Because each message_name is represented by a separate message, this is a simple stat. The second is the mite_rx_stats. This message is send by the worker coroutine that drains the AMQP queues. Once a second, it reports on the total number of messages it has received – of all types. Therefore, we need to write a custom _amqp_extract function, which will yield a sequence of (key, value) tuples.

We also need to inform mite about our stats, using pythonʼs entry points mechanism. To do so, we add a section to the setup.cfg file for our package:

[options.entry_points]
mite_stats =
    mite_amqp = id_mite_nft.amqp:_MITE_STATS

Note

It is also possible to specify the equivalent information in setup.py if that is the configuration file your project uses.