pgpro-otel-collector is based on the OpenTelemetry Collector and consists of several components that work together in a pipeline (collection — processing — exporting):
Receivers — the primary components responsible for collecting data. pgpro-otel-collector includes the following receivers:
postgrespro — the central component of
the collector, which contains the main implementation
(built-in plugins) for collecting data from a database instance.
filelog — an auxiliary receiver
responsible for collecting information from database logs.
hostmetrics — an auxiliary receiver that
collects information from the operating system; may require root access.
journald — an auxiliary receiver that
collects logs from the systemd journal.
sqlquery — an auxiliary receiver that
collects metrics and/or logs from custom SQL queries.
The sqlquery receiver is currently
experimental and is not recommended for production use.
Processors receive data from receivers and process it. pgpro-otel-collector includes the following processors:
attributes processes specific attributes of log records.
resource adds extra log record attributes.
batch batches data and regulates the data
transmission mechanism.
transform modifies the data
using the OpenTelemetry Transformation Language (OTTL).
memory_limiter limits the
pgpro-otel-collector memory consumption.
filter allows filtering metrics and logs based on
specified conditions using OTTL.
metricstransform allows renaming
metrics and managing labels, including scaling and aggregation.
Exporters receive processed data from processors and deliver it to external consumers. pgpro-otel-collector supports the following exporters:
otlphttp exports and sends information over the
OTLP protocol (the standard OpenTelemetry protocol used by many
monitoring system vendors) with the help of the HTTP protocol.
prometheus exports metrics over the built-in HTTP
server and publishes them on the /metrics page in
the format of the Prometheus monitoring system.
prometheus is used by default because it does not
require additional configuration.
kafka sends metrics and logs to
Apache Kafka — an open-source
distributed event streaming platform.
zabbix sends metrics to
Zabbix — an open-source
monitoring solution.
Following the OpenTelemetry Collector architecture, the pgpro-otel-collector configuration allows using the same component multiple times. For example, you can use several copies of the same receiver to collect data from numerous DBMS instances. Similarly, the same exporter can be used for exporting data to various storages.
For detailed description of each component, see Chapter 7.