Set Up Fluentd Unified Logging

In Composer, you can use Fluentd as a logging layer to which you can direct the logs for various components of Composer. This allows you to customize the log output to meet the needs of your environment.

Composer leverages Fluentd’s unified logging layer to collect logs via a central API. Fluentd can be configured to aggregate logs to various data sources or outputs. For example, if you are directing all log files from your zoomdata-websocket.log and zoomdata-errors.log to a Fluentd server, you can add one of Fluentd’s plug-ins to write the log files to Elasticsearch to analyze web client errors for your environment.

Unified logging does not replace Composer's default logging architecture. It only augments that experience with an option for those wanting additional logging control.

By default, unified logging is disabled and needs to be enabled for each component you want captured in Fluentd.

At a high level, the steps to set up Fluentd are as follows:

  1. Set up a database to store the Fluentd log files. See Configure a Postgres Database to Store Fluentd Logs.

  2. Configure Fluentd logging for your Composer installatoin. See Configure Fluentd Logging. This includes configuring the Fluentd td-agent.conf file to identify the Composer microservice log data you want logged.

  3. Enable unified logging and configure the host and port information for your Fluentd server for each microservice log file from which you are extracting data for Fluentd logging. See Enable Fluentd Unified Logging in Composer v6

  4. Test the Fluentd logging setup. See Test Fluentd Logging.