Setting Up Fluentd Unified Logging

In Zoomdata, you can use Fluentd as a logging layer to which you can direct the logs for various components of Zoomdata. This enables you to customize the log output to meet the needs of your environment.

Zoomdata leverages Fluentd’s unified logging layer to collect logs via a central API. Fluentd can be configured to aggregate logs to various data sources or outputs. For example, if you are directing all log files from your zoomdata-websocket.log and zoomdata-errors.log to a Fluentd server, you can add one of Fluentd’s plug-ins to write the log files to Elasticsearch to analyze web client errors for your environment.

Unified logging does not replace Zoomdata's default logging architecture. It only augments that experience with an option for those wanting additional logging control.

By default, unified logging is disabled and needs to be enabled for each component you want captured in Fluentd.

Before You Begin

Zoomdata’s unified logging configuration relies on Fluentd to write logs from Fluentd to a particular database or source. Consequently, Fluentd should be configured appropriately. For a list of compatible Fluentd plug-ins, refer to Fluentd's website and documentation. If written to a data source compatible with Zoomdata’s connectors, Zoomdata can then be used to analyze and visualize your logs.


  • A server with Fluentd installed and configured. For information and steps, refer to Fluentd's documentation.
  • If Fluentd is not on the same server as Zoomdata, you need the host and port numbers for the server.


  • To output logs to another data source, that data source should be available and configured in conjunction with Fluentd to receive logs.
  • To visualize logs within Zoomdata, the data source referenced above should also be supported by Zoomdata. For more information, see the Data Connector Reference.


At a high level, the steps to set up Fluentd are as follows:

  • Decide what services (such as zoomdata, query-engine, etc.) you want to log to Fluentd
  • Enable unified logging for that particular service.
  • Configure the host and port information for your Fluentd server to the properties file.

To set up unified logging for your Zoomdata server:

  1. On your server, access the .properties file of the service you want to run unified logging on. For example, etc/zoomdata/ You can enable unified logging for these services:

    • Zoomdata Scheduler (
    • Web services (
    • Query engine (
    • Data source connectors (see Connector Properties for a list of property files)
    • Upload service (
    • Streamwriter framework (
  2. Define the following parameters the appropriate properties file:

    logging.unified.level = <log-level>
    logging.unified.tag = <service> = <yourFluentdServerIPaddress>
    logging.unified.port = <yourFluentdServerPort>

    Specify one of the following log levels for <log-level>: ERROR, WARN, INFO, DEBUG, TRACE, or OFF. If set to OFF, Fluentd unified logging is disabled.

    Specify the Zoomdata service name for <service> in the logging.unified.tag property. For example, logging.unified.tag = query-engine. This is important because the tag identifies which service the log message applies to. Valid values are query-engine, job-scheduler, zoomdata-server, stream-writer, upload-service, and edc-<connector-name> (where <connector-name> is one of the names listed in Connector Properties).

  3. Save your changes and exit the properties file.

  4. Restart the Zoomdata server.

On the Fluentd server, you can then direct your logs to a data source of your choice. For information and steps, see Fluentd's documentation on Output.