Composer Version 5 Summary

This is a summary of the changes made in version 5 of Composer. It is provided so you can quickly identify new and changed Composer features before upgrading from Zoomdata 4.9 to Composer. If you are upgrading from an earlier version of Zoomdata, be sure to read the release summary for Zoomdata 4.9 as well.

Be sure to back up your metadata store (see Backing Up and Restoring the Metadata Store) and read Installation and Upgrade Changes before you upgrade.

In addition to resolving problems and applying security fixes, Composer 5 introduced changes in the following areas. Select the area title for more detailed information.

Area New or Changed Feature Summary
Release Model Changes

Composer 5.9 is now the latest LTS release, joining Zoomdata versions 4.9 and 3.7. A new rapid release is in development.

Zoomdata 3.7 support will end on November 19, 2020.

Licensing Changes When a Zoomdata or Composer trial or standard license expires, only supervisors can log in (so they can update the license). Regular users will see a message stating that the license has expired.
Installation and Upgrade Changes Support for Java 8 (1.8.0_131) is deprecated and support for Java JDK 11.0.5 is introduced. The Composer Bootstrap installation script installs OpenJDK 11 on most supported platforms.
PostgreSQL 12 is now used for the Composer metadata store. In past versions, PostgreSQL 9.5 was used. Be sure to back up your metadata store before you upgrade and be sure to read about possible migration issues in Installation and Upgrade Changes.

Ubuntu 14 is deprecated as an officially supported platform. However support for Ubuntu 18 is added.

Support for CentOS 8 and Red Hat Linux 8 are added in this release.

CentOS 6 is no longer supported and the special installation Bootstrap procedure (bootstrap-zoomdata-centos6.run) is no longer provided with this product. CentOS 6 is no longer supported by Red Hat Linux (RHEL). Use CentOS 7 or 8 instead.

Migration Considerations The API privilege ROLE_SAVE_DASHBOARDS has been removed in this release. Use ROLE_CREATE_DASHBOARDS for all new API calls. All existing groups that had the ROLE_SAVE_DASHBOARDS privilege granted are automatically migrated to ROLE_CREATE_DASHBOARDS. However, if your organization has integrated Composer API calls into third-party software, you will need to manually change all references of ROLE_SAVE_DASHBOARDS to ROLE_CREATE_DASHBOARDS. See Group Privilege Reference.
Review the PostgreSQL 12 information in Installation and Upgrade Changes. It contains possible migration considerations regarding your metadata store. Be sure to back up your metadata store before you upgrade. See Backing Up and Restoring the Metadata Store.

Logi Analytics highly recommends that you vacuum the PostgreSQL database instance after every upgrade, but especially after upgrading to Composer 5.9. See Vacuuming Composer's Metadata Store.

The supplied palette names ZoomSequential and ZoomPalette were renamed DefaultSequential and DefaultQualitative, respectively. If you use the original names, be sure to change them.
Product Rebranding and Wording Changes

With this release, Logi Zoomdata has been rebranded as Logi Composer. A new logo, banner, and login page are introduced. In addition, many UI screens have been updated to support the name change. Logi Zoomdata 3 and Logi Zoomdata 4 retain the Logi Zoomdata brand. The original Logi Composer 3 and Logi Composer 4 documentation is no longer available on DevNet. Contact Customer Support if you need it.

The base URL for Composer changed from zoomdata to composer. However, the use of zoomdata in the base URL is still supported to provide backward API compatibility. Specifics are given in Product Rebranding and Wording Changes.

The supplied palette names ZoomSequential and ZoomPalette were renamed DefaultSequential and DefaultQualitative, respectively.

The Content-Type header vnd.composer.v2+json is now the default Content-Type used for API routes. The Content-Type header vnd.zoomdata.v2+json is still accepted for backward compatibility, but is deprecated and will be removed in a future release. API routes that respond with JSON that conforms to a schema, respond using the vnd.zoomdata.v2+json or vnd.zoomdata.v2+json content type, depending on the setting of the Accept header. If the Accept header specifies either vnd.composer.v2+json or vnd.zoomdata.v2+json, the API response is returned using the requested content type. However, if neither vnd.composer.v2+json or vnd.zoomdata.v2+json are specified in the Accept header, the API response uses the content type specified by the Content-Type header.

The term visual is now used to refer to chart, table, and map visuals that can be added to a dashboard. The term chart is reserved for specific visual styles, such as a pie chart or a bar chart. Tables and maps are considered visuals, but are not considered charts.

Query Engine Changes

The legacy query engine is no longer provided in this release. Only the new query engine (z-Engine) is available. It includes support for Data Sharpening.

A new calculations.detect.array.fields query engine property is now provided. This property allows you to disable the query engine validation of multivalue fields in a custom metric. See Managing the Composer Query Engine.
Field metadata is now stored in the query engine first-level cache instead of being stored in the Zoomdata metadata store. All requests for metadata (minimum or maximum values for numeric and time fields or distinct values for attributes) are now initially run through the query engine's first-level cache. In addition, the Zoomdata scheduler now refreshes the query engine's first-level cache, which stores the data in the Zoomdata query engine data base (zoomdata-qe) so the data is available if the query engine microservice is restarted. This use of the query engine cache for data queries can be controlled using the following new Zoomdata properties in the zoomdata.properties file: zoomdata.source.refresh.metadata.cache.timeout.minutes and zoomdata.source.refresh.values.maxDistinctValues.

A number of updates to support multivalued fields were performed:

  • Grouped multivalued time fields are now supported.

  • Complex queries containing aggregate filters are now supported in pivot tables with grouped multivalued fields.

  • Support for multivalued fields used as metrics when a query cannot be fully pushed down and handled by the query engine has been improved.

  • Sorting by multivalued fields has been improved. In an ascending sort, the data sorts by the minimum values of a multivalued field; in a descending sort, the data sorts by the maximum values of a multivalued field.

Authorization Changes Variables can now be inserted for any forced filters specified in a group definition.
Four new group management privileges have been added: Can Manage Visuals (ROLE_MANAGE_VISUALS), Can Manage Themes (ROLE_ADMINISTER_THEMES), Can Create Scheduled Reports (ROLE_CREATE_DASHBOARD_REPORTS), Can Manage Dashboard Permissions (ROLE_PERMISSION_DASHBOARDS).

The UI privilege Can Save Visuals & Dashboards has been renamed Can Create Dashboards.

A new API privilege associated with the Can Create Dashboards UI privilege has been added. It is called ROLE_CREATE_DASHBOARDS and it replaces the API privilege ROLE_SAVE_DASHBOARDS, which has been removed in this release.

A new UI privilege Can Generate Embed Code (API privilege ROLE_GENERATE_EMBED_CODE), is added in this release. Use this privilege to control whether users can generate the embeddable code for a dashboard snippet. See Group Privilege Reference.

The UI privilege Can Share Visuals & Dashboards has been renamed Can Generate Dashboard Public Link. However, the functionality controlled by this privilege (the ability to publicly share links from Composer) is officially deprecated and is replaced by the new dashboard authorization feature. In a future release, both the UI privilege and the API privilege will be removed from the Composer software.

Group permissions for data sources now work as follows:

  • Users in groups with read access to a data source can no longer select the data source on the Data Sources page or alter the data source in any way.
  • Users in groups with edit access to a data source can no longer delete the data source on the Data Sources page.
  • Users in groups with delete-only access to a data source can no longer edit the data source on the Data Sources page. They cannot review or edit the data source configuration.

For more information, see Restricting Data Source Configurations in a Group.

If a user does not have permission to use at least one of the data sources used by a visual or dashboard, they can only see the data from the data sources for which they have access. In past releases, if users did not have permission to use at least one of the data sources used by the visuals on a dashboard, they could not list or open the dashboard. I

The ability to use forced filters to filter data sources for a group has been moved from group definitions to data source definitions and is renamed row security. The Data Sources tab of a group definition in the UI no longer provides an option to add a filter for a data source. However, you can set row security filters for specific groups for a data source from the Sources page. See Restricting Group Access to Data Using Row Security for more information about row security for data sources. You can still restrict the overall use of data sources in a group definition. See Restricting Data Source Configurations in a Group.

The ability to restrict access to specific fields in a data source by group definition has been moved from group definitions to data source definitions and is renamed column security. The Data Sources tab of a group definition in the UI no longer provides an option to specify the data source fields available for the group. However, you can specify the data source fields available for the group using column security from the Sources page. See Restricting Group Access to Fields Using Column Security for more information.

A new OAuth 2.0 configuration property, oauth.token.always-create-new, was added in this release and is set to true by default. This property ensures that a valid access token is present all the time for Composer enhanced script-based dashboard embedding provided in this release.

If you submit a Composer API request using an invalid or expired OAuth 2.0 access token for the request, Composer now produces an error response. See Handling of OAuth 2.0 Invalid or Expired Access Tokens.

OAuth security is now enabled by default. However, if it is not enabled (by the supervisor in the supervisor UI), then all OAuth client and token-related API endpoints now return a 404 response code or a 401 unauthorized response code (when an existing token is used). In addition, the OAuth API endpoints are not visible in the Swagger documentation.

Row and Column Security This release introduces row and column security for data source configurations.
Connector Changes A new Couchbase connector was introduced that supports Couchbase Community Edition 6.0.0. In addition, support for using non-default Couchbase ports has been added. See Managing the Couchbase Connector.
Connector version information is now included in the connector logs. This will help when debugging connector issues with the Zoomdata support team.
Pushdown join support is now provided by the Amazon Redshift connector. See Managing the Amazon Redshift Connector.
Apache Solr data stores running versions 7.4 to 8.4.1 are now supported. In prior releases, the minimum supported version was 6.6 and the maximum supported version was version 8. User delegation is also now supported. Finally, two new properties have been added to the edc-apache-solr.properties file: solr.query.limit.grouped and solr.query.limit.ungrouped. See Connector Properties and Property Files and Managing the Apache Solr Connector.
The Cassandra over Presto connector has been officially renamed the Presto connector and Presto version support changed from version 0.132 to version 319. In addition, the property files, microservice name, connector service name, data source configuration name, and directory and folder names have been rebranded. See Managing the Presto Connector and Connector Changes.
Two new Cloudera Search properties have been added to the edc-cloudera-search.properties file: solr.query.limit.grouped and solr.query.limit.ungrouped. See Connector Properties and Property Files and Managing the Cloudera Search Connector.
The maximum version of Elasticsearch supported by the Zoomdata Elasticsearch connector is now 7.6. In addition, composite aggregations are now implemented by Elasticsearch 7 connectors. This support optimizes aggregations of Elasticsearch 7 data, except for queries with histograms, time groups with WEEK granularity, and multiple groups when group fields belong to different nested contexts. See Managing the Elasticsearch Connector and Connector Changes.
Support for Microsoft SQL Server 2019 versions was added. The minimum supported version is still 10.5, but the maximum is 15.0.2000. In prior releases, 2008 R2 versions were supported (10.5 - 14.0.1000). In addition, pushdown join support is now provided. See Managing the Microsoft SQL Server Connector.
Pushdown join support is now provided by MySQL connectors. See Managing the MySQL Connector.
Pushdown join support is now provided by Oracle connectors. See Managing the Oracle Connector.
Support for PostgreSQL data stores running versions up to 12.1 is now provided. In prior releases, the maximum was 11.1. In addition, pushdown join support is now provided by the PostgreSQL connector. See Managing the PostgreSQL Connector.
Kerberos support and pushdown join support are now provided by Spark SQL connectors. See Managing the Spark SQL Connector and Connecting to Spark SQL Sources on a Kerberized HDP Cluster.
Pushdown join support is now provided by Teradata connectors. See Managing the Teradata Connector.
Derived field and pushdown join support are now provided by Vertica connectors. See Managing the Vertica Connector.
Fusion Enhancements

In a fused data source, Zoomdata can now use information about column uniqueness when building a query plan that can be critical for a connector's resource usage and query time. Column uniqueness can be specified in the fused data source definition on the Editor tab by selecting the Unique checkbox. See Creating a Fusion Data Source.

Data Sharpening Changes

Data Sharpening is supported by the new query engine (z-Engine).

In addition, it is now enabled and configured within a data source configuration, if the data selected for the data source includes a time field. The Visuals tab of the data source configuration UI now includes new settings to support this. Data Sharpening is no longer configured using properties in the query engine properties file (query-engine.properties). See Enabling Data Sharpening and Configuring Its Defaults.

Connection Definition Changes

Variables can now be inserted for any connection parameter in a connection definition. The variables are passed to the connection string via custom attributes specified in the user definition or dynamically in the custom attributes specified in the SAML or LDAP configurations for your Zoomdata installation. You can insert variables for connection parameters using the following syntax:

${User.<custom-attribute-name>}

See Inserting Variables for Connection Parameters.

Dashboard Scheduling

This release introduces the ability to schedule the capture and delivery of dashboard reports using UI screens and API endpoints. See About Scheduled Dashboard Reports. If you will be scheduling many concurrent dashboard reports, you should consider altering the memory configuration settings for the Composer installation and Screenshot service pool configuration settings. See Setting Up the Screenshot Microservice.

Dashboard Authorization

This release introduces dashboard authorization, the ability to permit your account or specific users or groups in your account to read, write, or delete a dashboard. To manage dashboard authorization, you must be an administrator or your user definition must be a member of a group assigned the Can Manage Dashboard Permissions(ROLE_PERMISSION_DASHBOARDS) privilege. For complete information, see About Dashboard Authorization. API updates were made to support dashboard authorization. See REST API Changes.

Materialized Views Support for materialized views was added in this release. Materialized views cache the results of a query and can speed up query processing, especially when replicating a query with heavily aggregated data. Once the query results are cached, materialized views must be manually updated and the manual query updates make take as long to perform as the original query. See Using Materialized Views.
Data Manipulation Enhancements

Row-level functions and expressions can now be used in WHERE clauses in custom metrics. See Supported SQL-Like Expressions.

Dashboard Changes

The dashboard library no longer shows thumbnail view or a dashboard preview in the preview panel. This view and the preview panel are deprecated. See Using the Dashboard Library.

A number of changes were made to the dashboard library to support dashboard authorization and dashboard scheduling. See Dashboard Changes.

The Dashboard Filter sidebar has been moved from the left side of the dashboard canvas to the right side. See Applying Row-Level Filters.

This release enhances the dashboard embedding functionality available in past releases. To generate an embeddable dashboard snippet, you must be an administrator or your user definition must be a member of a group assigned the Can Generate Embed Code (ROLE_GENERATE_EMBED_CODE) privilege. In addition, OAuth authentication must be enabled and cross-origin sharing (CORS) must be enabled for your Composer instance. See Embed Composer v5 Dashboards.

Visual Changes

Visuals can now be managed independently of dashboards in a new Visual Gallery. They can be added, updated, and deleted in the Visual Gallery and placed on dashboards as needed.

Visual icons and the visual style sidebar appearance have changed. See Composer v5 Visual Metrics and Attributes Reference and Using the Visual Style Sidebar.

A new arc gauge visual type has been introduced in this release. See Arc Gauges.

Line trend attribute value and multiple metric charts now have a new visual style setting called Area Chart. The alternate and default option is Line Chart. The Area Chart setting turns on the fill option for the line chart (fills the chart with appropriate colors between the lines). The Line Chart setting turns off the fill option (so only lines appear). The Area Chart visual style setting is set on the Chart Style sidebar.

KPI chart color selections and default settings have been enhanced and the embedded color settings have been removed.

The color selection and management on the Color sidebar for many visuals was enhanced in this release. New color options are now available for many visuals.

A progress indicator now appears for a visual while it is determining the minimum, maximum, or distinct values requested for the visual.

You can now rotate the options on the radial menu clockwise (right) and counterclockwise (left) by holding down the shift key and using the left and right arrows on your keyboard. See Using the Radial Menu.

The Export sidebar for visuals has been replaced with Export pop-up dialogs. In past releases, the Export sidebar appeared to the left of the visual. See Exporting Visuals.

You can now control how users can interact with a visual using interactivity settings on the interactivity sidebar for the visual. Interactivity settings are useful in embedded use cases where the analytic on the parent application page needs to behave in a consistent manner with the rest of the application. See Controlling How Users Interact With a Visual.

A new sidebar menu now appears to the right of every visual edited from the Visual Gallery. It is also available when you select the new Edit Visual option on the visual drop-down menu when editing a visual in a dashboard. Selecting options on the sidebar menu opens and closes sidebars that allow you to control the style, settings, filters, time bar settings, sort and limit options, ruler and reference line settings, colors, interactivity settings and general information for the visual.

Data Store Connections Data store connections can now be reviewed and maintained on a new Connections page that is accessed from the new top-level navigation banner, from the Composer UI menu, and from the Home Page. You can still create data store connections when you create data source configurations, but you can no longer access the list of connections or maintain connections from the Sources page. See Managing Data Store Connections.
Data Source Configurations

The following changes have been made for data source configurations in this release.

  • They can now be reviewed, maintained, and imported on a new Sources page that is accessed from the new top-level navigation banner, from the Composer UI menu, and from the Home Page. See Manage Composer v5 Data Source Configurations.

  • After creating a new data source, the banner on the Sources page no longer provides a button that allows you to create a new dashboard and visual from the new data source. In addition, the icon to the right of a data source name that provided a shortcut to visual creation on the Sources page has been removed. Other methods of creating visuals and dashboards remain the same.

  • Default color settings for visuals on the Visuals tab of the data source configuration wizard have been removed. Color palettes are now defined for the Composer environment and are assigned to visuals using themes.

  • If you disable the Volume property in a data source configuration, it no longer appears in visual tooltips. The only exception to this is in histograms which plot the volume or number of records.

Themes

This release introduces themes for the UI. Three themes are provided with the product, composer, modern (light) and dark, but you can define and use your own themes. Themes are defined, controlled, and managed using a new api/customization/themes API endpoint and a themes JSON file.

At this time, you can only tailor theme colors. Other tailoring properties (such as fonts and font sizes) should not be changed. See Themes and Manage Composer v5 UI Themes.

User Interface (UI) Changes

The About dialog location and look changed in this release. It is no longer used to switch accounts (if your Zoomdata user name has access to more than one account). In addition, a new Logout button now appears on the About dialog. Select it to log out of the product. See About Dialog.

A new rebranded home page is used in the Composer UI when you are logged in as an administrator. Depending on your security profile, this home page displays the numbers of connection definitions, data source configuration definitions, visuals, and dashboards in the environment. It also provides easy access to lists of connection and data source configuration definitions, the Visual Gallery, and the dashboard library. See Home Page.
A new top-level navigation banner appears in the Composer UI when you are logged in as an administrator or a non-supervisor user. This menu provides easy access to connection definitions, data source configuration definitions, the visual gallery (visuals), and the dashboards in the UI. See The Top-Level Navigation Banner.
Almost all of the top-level navigation icons have been removed from both the regular user interface and the supervisor user interface. Only a hamburger icon () is available in both UIs. Each provides a menu from which you can select various options. The options available in previous versions using the former top-level navigation icons have been moved to these menus. See The Composer UI Menu and The Composer Supervisor Menu.
The Customize UI page in the supervisor UI has been rebranded with Logi Composer logos as well as rebranded support and documentation links.

A new Dashboard Permissions dialog now allows you to grant and revoke read, write, and delete permissions for specific dashboards to your entire account or to specific users or groups of users in your account. See About Dashboard Authorization.See About Dashboard Authorization.

A new option in the Actions column in the dashboard library allows you to generate and copy an embeddable snippet for the dashboard. When you select this icon, a new Embed Code dialog appears. See Embed Composer v5 Dashboards.

The table on the Sources page now includes columns that allow you to specify row and column security filters for a data source configuration. In addition, when you select the icons in these columns, new Row Security and Column Security dialogs appear. See Sources Page, Restricting Group Access to Data Using Row Security, and Restricting Group Access to Fields Using Column Security.

You can now specify more than one value for a single wild card filter. In past versions, you could only specify one value. .

The INCLUDE and EXCLUDE operators can now be used for numeric fields in filters. In past releases, they could only be used in filters for attribute fields. See Operators.

Custom attributes in user definitions now support multivalue fields (arrays).

Property Changes A number of properties in the zoomdata.properties file were added, renamed, or removed to support the feature changes in this release. See Property Changes.
REST API Changes A number of REST API endpoints were added, enhanced, deprecated and removed to support the feature changes in this release. In addition, specific properties and the output from some of the endpoint requests were enhanced or removed to support the feature changes in this release. See REST API Changes.
Features Deprecated in Version 5

A complete list of deprecated features for Composer 5 is given in Features Deprecated in Version 5.

Retired Releases

Zoomdata 2.6 is retired.

Zoomdata 3.7 support will end on November 19, 2020.

Email salesteam@logianalytics.com to purchase Composer.