Release notes

These notes describe the significant features and bug fixes that have been included with each Cantabular release.

Version 10.7.5

  • Fix a bug in calculation of HTTP client tracing log time field value.

Version 10.7.4

  • Changes to logging:

    • Add optional HTTP client tracing in logs. This can be enabled by setting environment variable CANTABULAR_HTTP_CLIENT_TRACE=on.

    • Add the time that each request spent queued to the HTTP event logs (timeQueued).

    • Improved log messages for dropped client connections.

    • Log an HTTP status of 0 on requests where writing did not begin prior to client disconnecting.

  • Public user interface:

    • Fixed a bug where under some circumstances a dropped client connection during XLSX download could result in a panic.

  • Admin user interface:

    • Fixed a bug where variable names were not URL encoded on the service tables checking page.

  • The software is built with Go 1.20.5.

Released: 16th June 2023

Version 10.7.3

Released: 5th June 2023

  • Allow configuration of maximum concurrent requests, and maximum queuing time using the CANTABULAR_MAX_CONCURRENT_REQUESTS and CANTABULAR_MAX_CONCURRENT_REQUESTS_WAIT_TIME environment variables.

  • The health checks now indicate unhealthy when the server is at its maximum concurrent requests. The time that the health check request will wait before reporting unhealthy is configurable with the environment variable CANTABULAR_MAX_CONCURRENT_REQUESTS_HEALTH_WAIT_TIME. This can be useful to direct a load balancer.

  • Log the time taken to process a request in milliseconds in the HTTP log records.

Version 10.7.2

Released: 30th May 2023

  • Health check endpoints:

    • The health check endpoint in cantabular-ui is no longer behind authentication.

    • Allowed configuration of a separate listener on a different IP address and port for health check endpoints in the user interface and extended API processes. For more information please see the information on health check endpoints in the Cantabular installation guide.

  • Public user interface:

    • Fixed accessibility issues identified in testing.

    • Improved handling and display of custom error messages returned from DRL.

    • Hid error messages related to internal network problems in the UI. A generic error message is displayed to the user and the underlying error is shown in the logs.

    • Improved page titles in metadata explorer.

    • Fixed a bug in cell height calculation for wrapped text in Excel downloads.

    • Fixed a bug with handling of redacted values in JSON table endpoint.

  • Removed some unnecessary clearing of the metadata cache in the Public UI and extended API. Improved logging about reasons for metadata cache clearing.

Version 10.7.1

Released: 5th May 2023

  • Fixed a bug where cantabular-inspect could not export microdata for flow datasets.

  • Public user interface:

    • Redesigned the landing page to move the metadata explorer into a new “Supporting information” sub-section.

    • Added the ability to customise the content of the Excel (XLSX) downloads. The CANTABULAR_UI_EXCEL_METADATA_TEMPLATE environment variable can be used to specify a XML file which defines what additional content should appear in the Excel downloads beyond the data itself. No cover sheet is included in Excel downloads if this environment variable is not specified. For more information, please see the Cantabular Installation Guide.

    • Improved the automatic sizing of cells in Excel downloads.

    • Added a feature to copy a permanent link for a table to a user’s clipboard. Only enabled when CANTABULAR_UI_PUBLIC_URL is set.

    • Dataset descriptions are now used on the metadata explorer index page.

    • Added a Build table … button to the variable metadata page.

    • Reversed the order of label and code columns in metadata tables such that the label column is now displayed on the left.

    • The category code for rule variables is now always shown in pivoted outputs (HTML, CSV, XLSX).

    • Changes to names for downloaded files:

      • A maximum length is now imposed on filenames.

      • A digest has also been added which is derived from the query used to create the table.

      • Filenames are now in lowercase.

      • Filenames can be customised by providing a filename prefix using the CANTABULAR_UI_DOWNLOAD_FILENAME_PREFIX environment variable.

  • The software is built with Go 1.20.4.

Version 10.7.0

Released: 13th April 2023

  • Added ability to configure cantabular-ui with a custom Content-Security-Policy header for HTML responses, using the CANTABULAR_UI_CONTENT_SECURITY_POLICY_HEADER environment variable.

  • Expanded the ability to customise certain sections of the UI at runtime by adding customisation points at the start and end of the HTML <head> and <body> sections. This provides more flexibility for users to add custom JavaScript content.

  • Added JSON download format to cantabular-ui. This machine-readable format makes it easier for users to build visualisations and integrate data dynamically into their own products and services.

  • Added a check when building datasets with cantabular-make-dataset to disallow non-printable characters in the dataset configuration, variable names, category codes and labels.

  • Executables built for x86-64 now require microarchitecture support level x86-64-v3. For further information about architecture levels and the implications of this change, please see https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels

  • The version of graphql-go/graphql used has changed from v0.8.0 to v0.8.1. For changes see https://github.com/graphql-go/graphql/releases .

    • Resolves CVE-2022-37315. The fork previously added in v10.2.1 which temporarily addressed this vulnerability has now been removed.

Version 10.6.0

Released: 5th April 2023

  • Added ability to make flow datasets (with more than one rule base variable)

    • Change configuration JSON format from RuleBaseVariable (type string) to RuleBaseVariables (type array of strings). Conventional datasets have only one name in the array whereas flow datasets have more than one. RuleBaseVariable is still supported for backwards compatibility.

    • Flow datasets are only for use with the new utility cantabular-flow-query and are ignored by cantabular-server and cantabular-admin.

    • Flow datasets do not have category keys because flow queries do not use zero perturbation.

    • Conventional (non-flow) datasets are compatible with the previous versions of Cantabular that use dataset file format version 75 (v10.0.0 onwards).

  • Added a batch command line utility cantabular-flow-query for performing flow queries and generating CSV output. For more detail see command line help which is displayed when running with no parameters. Flow query features include:

    • support for variables and mappings with any number of categories

    • support for rule variables which are incomplete mappings

    • filtering of CSV output by any query variable or a mapping thereof with consequently faster execution in the case of filtering of rule variables

    • display of output file size and uncompressed CSV size

    • option to omit lines with a zero count in CSV output

    • option to gzip CSV output

    • use of datasets with perturbation disabled

  • Added an example flow dataset which can be used to test cantabular-flow-query.

  • Public user interface

    • Changed dataset modal to only include non-rule base variables

    • Various bug and layout fixes

  • Added capability in cantabular-inspect to export microdata to CSV.

  • Changed incorrect uses of HTTP 400 response code to 404.

  • The software is built with Go 1.20.3.

Version 10.5.0

Released: 11th March 2023

  • Public user interface

    • Translation CSV files for English can now be used to override any piece of UI text.

    • Updated table download paths for consistency.

    • Table download links now have additional context explaining purpose of data format.

    • Added metadata explorer option to homepage.

  • The software is built with Go 1.20.2.

Version 10.4.0

Released: 6th March 2023

  • Contains all changes included in versions 10.2.1, 10.2.2 and 10.2.3.

  • Public user interface

    • Previously a theme was applied to the UI at build time. Now the UI uses Cantabular branding by default and a theme can be loaded at runtime using the CANTABULAR_UI_THEME_FOLDER environment variable.

    • Added the ability to customise certain sections of the UI at runtime. This allows user-defined metadata to be incorporated into the UI.

    • Added the ability to pivot output tables in the UI, CSV downloads and Excel downloads.

    • Enhanced Excel downloads to adhere to GSS guidance on accessibility.

    • Observation type formatting is now applied to CSV outputs to make the behaviour consistent with HTML and XLSX outputs.

    • Standard table metadata is now shown when a user builds a matching custom table. A standard table match is also preserved when a table is filtered or the geography variable is changed.

    • Removed all mentions of the total number of rows within a microdata dataset.

    • Added support for CSV on the Web (CSVW), a standard for describing the content of CSV tables. This machine-readable JSON output contains metadata about the dataset and variables used in an associated CSV output. To enable this feature, the CANTABULAR_UI_PUBLIC_URL environment variable must be set.

  • Added an endpoint at /health for every Cantabular service executable. An HTTP GET request can be made to these endpoints to perform health checks. They are intended for use with a load balancer to inform whether a service can accept traffic, and whether a service should be killed or restarted.

  • Fixed a bug which prevented the first dimension in tables from tabular datasets without a rule variable from being reordered.

  • The Metadata service has a -check-complete option which is used to check that metadata is supplied for every variable in every dataset accessible via a running instance of Cantabular Server. A new -check-datasets option has been added which allows a subset of datasets to be checked.

  • Enhanced detection of dropped connections through the addition of extra HTTP timeouts.

  • Removed Python API module (cantabular_client) from release.

  • The software is built with Go 1.20.1.

Version 10.2.3

Released: 16 February 2023

  • Added a new key to Cantabular logs to facilitate recording an instance name: default to hostname, but configurable and removable. See command line help for more information.

  • Fixed a bug whereby HTTP/2 support in the Go standard library was erroneously disabled. We are using the Mozilla SSL Configuration Generator but they had a bug which had the effect of disabling HTTP/2 for Go.

  • Fixed a bug on the DRL diagnostics page in Cantabular Admin when viewing diagnostics for buggy DRL code that produces an error when run.

  • Fixed a bug in the DRL where a fail statement without a string message / code in the top level “main” test would not set the message to “main” as per the specification. This could be misinterpreted as a pass by higher level code.

  • Fixed a bug with text editor state on the service tables page in Cantabular Admin when resetting service tables back to the default source, and when the default is invalid in some way.

  • Added an additional capability in Cantabular Server and Cantabular’s Extended API to indicate when a variable should be ignored by a user interface because all outputs for a variable are blocked by DRL code.

  • Added an additional isRule field on the Variable type in Cantabular’s Extended API to indicate if a variable is a rule variable or not.

  • The software is built with Go 1.20.1

Version 10.2.2

Released: 29 November 2022

  • Fixed a bug in cantabular-server caused by attempting to load datasets with duplicate names.

  • Fixed a bug with display of long variable labels in cantabular-ui.

  • The software is built with Go 1.19.3.

Version 10.2.1

Released: 1 September 2022

  • Fixed a data transformation bug that caused all counts between 256 to 299 inclusive in the Public UI and Extended API to have 256 subtracted.

  • Resolved CVE-2022-37315 in GraphQL library dependency by forking upstream repository and applying fix.

  • Added dataset language to some cantabular-metadata error logs.

Version 10.2.0

Released: 17 August 2022

  • A new codes parameter has been added to categories in the extended API allowing information on specific categories to be requested.

  • A new ignoreNotFound parameter has been added to categories and variables in the extended API. When set to true unknown category codes or variable names are ignored and no error is returned.

  • cantabular-server now blocks queries containing multiple variables which share the same base variable.

  • An issue has been fixed which required cantabular-ui to be restarted when cantabular-server was restarted with an updated version of a dataset.

  • The software is built with Go 1.19.

Version 10.1.1

Released: 22 July 2022

  • Fixed a bug that caused the Cantabular executables to not be statically linked.

  • Variable names are now case-insensitive matched between the cantabular-server API and the cantabular-metadata API in both cantabular-ui and cantabular-api-ext.

  • The software is built with Go 1.18.4.

Version 10.1.0

Released: 8 July 2022

  • Overall

    • The dataset file format is still at version 75. There is no need to rebuild existing datasets built with version 10.0.0 to work with this version of Cantabular, except to create a data only digest (see below).

    • The software is built with Go 1.18.3.

  • Building Cantabular datasets

    • Tabular (or pre-computed) datasets have been extended to allow decimal or fractional data to be loaded. Dataset configuration files now have an ObservationTypes field that can be used to describe the unit of measure in a Cantabular dataset. For more information and example please see the Cantabular Data Loading Guide.

    • Tabular (pre-computed) datasets can now also contain negative numbers.

    • Detailed base classifications present in microdata can now be replaced automatically with a less detailed classification specified in the codebook, via the ConvertToBaseVariable field in dataset configuration files.

    • Datasets are now built in a temporary file which is then renamed to the target filename when the build process is complete.

    • A data only digest field has been added to a built dataset to uniquely identify the dataset based only on those components in the dataset that affect the data and codebook. Notably it excludes disclosure rules and source file information. This is to facilitate verification that the data in a dataset is unchanged as disclosure rules are changed.

      This digest can be seen using cantabular-inspect, cantabular-admin or through cantabular-server’s REST API.

    • Added a new configuration option for microdata datasets to allow specific mappings present in a codebook to be ignored and not included in a built dataset.

    • Fixed a bug where cantabular-make-dataset would crash when building a tabular dataset if it contained more than one table with: no rule variables, the same number of variables, and where all variables are mappings of the same base variable with the same number of categories.

  • Public user interface

    • The UI is now usable end-to-end without JavaScript following a change to render disclosure control server-side on the data download page.

    • Various accessibility improvements have been made to the user interface, including:

      • All pages can be fully operated with the keyboard.

      • Focus state of all interactive elements is shown visually.

      • The accessibility of the slide-in modal has been improved.

      • Fieldset/legend elements are used where appropriate in HTML forms.

      • The font-sizing approach has been changed to allow reflow to a single column on text zoom.

    • Pre-computed tables can now be seen in the public user interface when they are specified in the metadata as pre-defined queries.

    • Columns containing variable category codes have been added to data downloads.

    • Users can now search for ready-made tables by name as well as by variable/dimension.

    • Some changes have been made to the structure of URLs in the user interface, including:

      • Geography filters are now specified as ~LA=W06000022+E07000087 instead of &g=LA&a=W06000022&a=E07000087 to make them consistent with other variable filters.

      • The disclosure control rules endpoint is now at /rules.json instead of /table/&f=summary-json.

    • New content has been added to the disclosure control feedback on the data download page to advise users what to do to improve their results when some areas are blocked.

    • The population selection page has been improved to make its design consistent with the rest of the user interface. A dataset metadata page has been added as a fallback for users without JavaScript.

    • The design of the data download page has been changed to make it more consistent with the design of the standard table download page.

  • cantabular-server

    • Various performance improvements have reduced query execution time by approximately 10%.

    • The API for cantabular-server has been updated to include decimal data and a new obsTypes field has been added to hold the observation type information supplied when a dataset is built.

  • Extended API

    • Observation type information for produced tables is available from within a Dataset and within a Table, including formatting information. For more information, please see the documentation built-in to the API using the GraphiQL interface.

    • The variables field on a Dataset has been extended with rule and base parameters to retrieve variables in a dataset based on their status as a rule or base variable.

    • The skip search parameter has been added to VariableCategoriesConnection to facilitate offset-based pagination of search results for categories within a variable. Due to performance drawbacks with this approach, we have added warnings to API documentation about its use in production. Cursor-based pagination is our recommendation for handling this capability, in-line with GraphQL standards.

    • The filterOnly field on a Variable is now of boolean type. It was previously mistakenly a string type and contained either "true" or "false".

Version 10.0.0

Released: 13 May 2022

  • Overall

    • The API version is now v10 due to a new capability to load pre-computed tables into a Cantabular dataset, which changes the way the query endpoint on the server API functions.

    • The dataset file format is now at version 75. Existing Cantabular datasets will need to be rebuilt with cantabular-make-dataset to work with this version of the software.

    • The software is built with Go 1.18.2

  • Loading and querying pre-computed tables

    • Cantabular datasets now come in two types. As well as microdata datasets, which allow cross-tabulations to be built directly from unit record data, we have added tabular datasets. Tabular datasets allow multiple pre-computed cross-tabulations to be compiled into a Cantabular dataset with a shared codebook.

    • In a microdata input file, a row represents a number of different observations about a statistical unit such as a person. In a tabular input file, a row represents a single observation such as a count of people with specific characteristics.

    • Dataset configuration files for cantabular-make-dataset have a new structure to allow the specification of these tabular datasets. The configuration file format has been modified to make it easier to see which fields are shared between the dataset types and which fields are specific to a particular dataset type.

      Backwards compatibility with existing configuration files has been maintained, so existing configuration files do not have to be updated. For full documentation of the configuration file and an example of configuration for both tabular and microdata datasets, please see the Cantabular Data Loading Guide.

    • The API provided by cantabular-server has been updated with two additional fields on a dataset: type, to indicate whether a dataset is microdata or tabular, and tables which is only present for a tabular dataset and represents the pre-computed tables available to be queried within it. The Cantabular API Specification and Open API .yaml file have been updated with further detail on these changes.

    • The query endpoint for cantabular-server behaves differently for tabular datasets: queries must be for variable combinations matching a loaded table. Queries for other combinations that do not correspond to a loaded table will receive a 403 response code along with a message of __NOT_PRESENT__.

    • The cantabular-api-ext GraphQL API schema has also been updated with a type field on a dataset and a tables field for tabular datasets. An example GraphQL query for tables has been added to the Cantabular Extended API Specification document.

    • The query page in cantabular-admin has been extended to allow selection of a table from within a tabular dataset and to filter, view and download the data within it. Variables other than the rule variable can also be reordered.

    • The Example dataset shipped with the software has been extended to include an example tabular dataset, with reference metadata, which can be loaded by the software. We hope this will allow users to easily see how this new functionality works in practice across the different software services.

  • LDAP authentication

    • The services cantabular-admin, cantabular-ui and cantabular-api-ext can now all be configured to use an LDAP server for authentication, when appropriate settings are provided via the environment. For more information on these settings, please see command line help for the individual executables.

    • Authentication is not available for cantabular-metadata which is intended only to host public metadata and delegates authentication to cantabular-ui and cantabular-api-ext. Subnet-based access control, however, is still available.

  • Administration user interface

    • The Cantabular admin UI has been extended with new functionality to allow service tables, also known as pre-defined queries, to be tested in bulk against disclosure rules.

    • These queries are automatically drawn from the metadata service when available and if the admin UI has been provided with a valid URL for the metadata service GraphQL endpoint via an environment variable.

  • cantabular-server

    • The zero perturbation algorithm has been updated to not perturb any zeros in output tables for areas that are exactly coincident with a structural zeros area.

    • Additional predefined data has been made available to the Disclosure Rules Language to allow table tests to take into account the specific rule variable category being evaluated to, for example, always pass or fail a sub-table for a particular geographic area.

    • Variable digests: A digest is computed for each variable by hashing its contents together with the digests of the variables from which it is derived. These have been added to the server API in hex format. The digests have been used to substantially reduce startup time and also reduce memory footprint in the server. They have also been used within our internal Go API client to reduce the memory footprint of both cantabular-ui and cantabular-api-ext by not storing multiple copies of variables that are identical between datasets.

  • Public user interface

    • Changes have been made to ensure that the UI works for users without JavaScript enabled.

    • The variable selection interaction in cantabular-ui has been split across two pages: the variables page now only shows base variables, with a link for each one to select a classification (mapping) to use in a query.

    • The variable search interaction has changed so that the search round-trips via the server.

    • Minor changes to the filtering interaction.

  • GraphQL APIs

    • Software version field: a new field has been added to the extended API and the metadata service API to report the semantic version string of the version of the software it is running.

Version 9.3.0

Released: 25 Mar 2022

  • Metadata service

    • Added a new tables field to the Service type of the cantabular-metadata GraphQL API to access a set of predefined queries (tables) that can be made to Cantabular. Tables are defined by their name, the variables they contain, and the Cantabular dataset to which they relate. They can be obtained by name, or as the entire set.

    • Definitions of each “table” (predefined query) and any associated metadata are loaded through a new input JSON file. The associated metadata can be provided in more than one language.

    • cantabular-metadata now expects to be told the paths of its input files via environment variables, rather than command-line arguments, to be consistent with other executables and twelve-factor app methodology. The environment variables are listed in command-line help (use -?).

    • For more information, see the separate documentation for the Cantabular metadata service.

  • Extended API

    • Added a new tables field to the Service type of the extended API which provides access to the above mentioned tables in the cantabular-metadata API. Each “table” is validated before it is made available to ensure that both the Cantabular dataset and the variables used in the table exist. Any invalid tables are dropped and reported in the cantabular-api-ext log. For more information including example queries, please see the documentation for the extended API.

    • Added a canBeFilteredBy field to the Variable type to simplify access to a list of variables that can be used as a filter for a specific query variable. This is particularly helpful for geography-related use cases.

    • Added a baseVariables field to the Dataset type to simplify access to a list of base variables in a Cantabular dataset. Base variables are typically those which are present in the underlying microdata but also include any multivariate mappings.

    • Added a skip parameter to various fields that support pagination. This facilitates offset pagination in addition to the conventional GraphQL model. See online GraphQL documentation in the GraphiQL IDE for details.

  • User interface

    • Updated Cantabular’s public UI to use the list of tables loaded into cantabular-metadata as its source of ready-made or standard tables, and removed the existing separate JSON-based configuration.

    • The standard tables loading guide document has been removed and replaced with additional information in the Cantabular installation guide on user interface configuration.

  • LDAP test utility

    • Added an executable that can be used to test integration with a LDAP server for authentication purposes. Configuration is provided through a set of environment variables which are listed by the command-line help. The same environment variables will be used in the future to add LDAP-based authentication to cantabular-admin and cantabular-ui.

  • Other changes

    • The software is built using Go 1.18.

Version 9.2.0

Released: 10 Feb 2022

  • User interface

    • Added multi-lingual support for all static text in the public UI.

    • Added a feature to show which areas in a query have been blocked by SDC rules, in the modal panel.

    • Added page to display metadata about a variable. This is initially intended for users without JavaScript who would otherwise be unable to view the variable modal panel.

    • Added support for gzip compression in responses by default, where appropriate.

    • Fix server failed to load when duplicate users exist in user authentication file. Recoverable errors in the file are logged. When the file format is bad, or no users are defined, the server will not start.

  • Extended API

    • Added multi-lingual support through lang parameter on dataset field.

    • Now integrates with the Cantabular metadata service to surface metadata content. The user defined metadata schema is spliced into the extended API schema at runtime.

    • Added support for gzip compression in responses by default, where appropriate.

    • Fix server failed to load when duplicate users exist in user authentication file. Recoverable errors in the file are logged. When the file format is bad, or no users are defined, the server will not start.

  • Metadata service

    • Added multi-lingual support with configurable default language.

    • Datasets now keyed on name and lang, previously just name. New lang key added in dataset JSON.

    • Use lang argument in GraphQL query for service and dataset fields to select the language variant.

    • Automatic fallback to default language, when requested language is not available.

    • Multiple datasets can be included while specifying language variants. The incl property now contains an array of objects with name and lang keys.

    • Added a digest field for the entire contents of the metadata service, to enable change detection by clients.

    • Added ETag headers to all responses from the metadata service.

    • Added descriptions to dataset and variable using a label text field. These are surfaced through the API and replace the cantabular-server codebook labels in the extended API and public UI.

    • Added support for category labels via catLabels field in metadata, mapping code strings to labels. These are used by the extended API and public UI to replace category labels sourced from cantabular-server, and are inherited through the incl feature.

    • Improve speed of metadata lookup and reduce runtime memory usage.

  • Other changes

    • The software is built using Go 1.17.6.

Version 9.1.3

Released: 22 Nov 2021

  • Allow editing of significantly longer rules text in the Admin UI (about 9x). The exact maximum size depends on the program text used because of compression.

  • Add an  else if  construct to the DRL. You can now write:

    if <cond>
  <statements>
else if <cond>
  <statements>
else
  <statements>
end

  • Make cantabular-metadata GraphQL dataset vars field have a mandatory names parameter. Rationale: metadata service is not the source of truth about which variables are in a dataset: this is the dataset itself (via cantabular-server). This mirrors the dataset field where you have to supply a name (there is no API facility to discover the datasets for which metadata is available).

  • Add logging of GraphQL API calls in cantabular-metadata and cantabular-api-ext.

  • Change URL paths in Public UI in preparation for multi-lingual capability. Examples:

    /level              => /en/custom/level
/standard/LC1117EW  => /en/standard/LC1117EW

Version 9.1.2

Released: 15 Nov 2021

  • Fixed a bug which limited the length of the DRL rules that could be stored in a dataset.

  • Added display of the fraction of the ptable associated with each perturbation amount into cantabular-inspect and the Admin UI.

  • The software is built with Go 1.17.3.

Version 9.1.1

Released: 11 Oct 2021

  • Overall

    • Reduced memory usage in the public UI and extended API.

    • Added note about configuration file to the introduction of the Cantabular Data Loading Guide.

    • The software is built with Go 1.17.2.

  • User interface

    • Enhancements to the view data page:

      • Removed hard-coded message about perturbation which doesn’t always apply.

      • Improved wording in message about the number of rows that are shown in data previews.

      • Added note to make it clear that suppressed rows are not included in the row/cell count. This is only shown when suppression has actually occurred.

    • Fixed a search issue on the variables page where mappings may be hidden if the base variable label matches the search string but the mapping label doesn’t.

    • Counts in HTML output tables now include thousands separators (commas).

  • Extended API

    • Substantially reduced the latency for extended API GraphQL queries returning counts for large tables.

  • Cantabular server

    • The server now returns a more specific error message when requests are received with the wrong API version.

Version 9.1.0

Released: 14 Sep 2021

  • Overall

    • Logs from all executables are now provided in one of key-value or JSON format to support machine readability. The format can be changed through the use of the CANTABULAR_LOG_FORMAT environment variable. For more information, please see command line help (-?) or the installation guide.

    • The specification of listen addresses for all services has been changed very slightly. If you wish to supply a port only then it must be preceded by a colon. IPv4 addresses can no longer use leading zeros on non-zero octets.

    • A new /metrics endpoint in the service executables can now be configured to run in order to provide Prometheus-compatible metrics on the health of the Go runtime. This is disabled by default but can be switched on by using a CANTABULAR_<NAME>_METRICS_LISTEN_ADDRESS environment variable. See command line help (-?) for details.

    • Dataset file format version is now 70 because of performance improvements. Existing datasets will need to be rebuilt to work with the new version.

    • The software is built with Go 1.17.1.

  • Cantabular server

    • The Cantabular Administration UI has been separated out from cantabular-server and is now provided in its own executable, cantabular-admin. This change removes the need to disable the Admin UI in production, increases confidence that cantabular-server cannot ever bypass SDC, reduces deployment executable size and facilitates use of external libraries in the Admin UI code in future without violating our “no-dependencies” policy for the server.

    • The performance of dataset loading at server startup has been improved and is now about twice as fast.

  • Dataset building

    • The performance of cantabular-make-dataset has been substantially improved and is now at least three times faster than in the previous version. Additionally, when using compressed microdata CSV files the process uses multiple cores which means that elapsed time is actually less when using compressed input even though there is more computation to do.

    • Added support for loading bzip2 compressed input files, including microdata, to go along with existing support for gzip compressed input files.

    • The number of file and data columns are now shown in the log output when building a dataset.

  • User interface

    • SDC feedback is now not shown in the public UI until one or more areas fail DRL checks or another error is encountered, in response to usability testing findings.

    • Support for IE11 has been dropped within JavaScript code to improve build process and speed, and to reduce bundle size. IE11 support can be achieved in future if necessary either through a small change to the build process or through full support for users with no JavaScript.

    • Fixed example configuration for standard tables supplied in documentation.

    • Fixed a bug on the view data page when query blocked by DRL table tests.

    • Various accessibility improvements in the public UI.

  • Metadata service

    • Logs for cantabular-metadata now show for which datasets the service is loading metadata.

    • Customisation of public UI landing page title and description can be achieved by providing websiteTitle and websiteDescription fields on the ServiceMetadata type.

    • Methodology and SDC metadata can be added to custom and standard table landing pages in the public UI using methodology and sdc fields on the DatasetMetadata type in cantabular-metadata.

    • Links to more information on variables can be made to appear in the public UI with variable metadata using the url field on the VariableMetadata type.

  • Extended API

    • The Cantabular API is no longer proxied through cantabular-api-ext.

    • The JSON-stat endpoint is now turned off by default but can be turned on with an environment variable: CANTABULAR_API_EXT_JSONSTAT_ENABLED=TRUE.

    • Improvements to JSON-stat error handling.

    • The version of graphql-go/graphql used has changed from v0.7.9 to v0.8.0. For changes see https://github.com/graphql-go/graphql/releases .

Version 9.0.1

  • Fixed a bug in the public UI where the SDC feedback widget failed to show the correct number of areas when a geography filter had been specified.

  • Fixed a bug with display of variable metadata in the public UI for variables with no metadata.

  • cantabular-metadata now returns a null value when metadata is requested for a variable that is not present in the specified dataset instead of an “unknown variable” GraphQL error.

  • Fixed an issue with GraphQL endpoints where they could accept unbounded HTTP request bodies. The default maximum allowed body size is now 1MB. This value is configurable via an environment variable.

  • By default, do not proxy the Cantabular API through the UI. It is still possible to proxy the API in the UI via a new environment variable on cantabular-ui documented in command line help available via cantabular-ui -h.

  • Improvements to error-handling in the public UI.

  • The software is built using Go 1.16.4.

Version 9.0.0

Released: 8 Apr 2021

  • The API version is now v9 due to modifications in how geographic variables are specified.

    • Any scripts or applications accessing the cantabular-server API or extended API may need to be modified.

    • Anyone using the Python client for the cantabular-server API will need to upgrade to the new version.

    • Anyone using the new version of cantabular-make-dataset will also need to make changes to their dataset configuration files.

  • The API changes have been made in order to support more complicated geographic hierarchies. Specifically, the following new concepts have been added:

    • Variables with multiple independent sources e.g. Local Authority could be independently mapped from one or more other variables such as LSOA and Parish. These are distinct from multivariate mappings where each category in a mapping is derived from a combination of categories in multiple source variables.

    • Incomplete mappings where some categories in a source variable are not mapped to any category in a mapping e.g. some OAs are not mapped to a Built-up Area.

    • Filter only variables where a variable cannot be used as a query variable, but can be used as a high-level filter on a source variable.

  • The API changes to support new geography functionality are as follows:

API Element
Old
New
Version used in URL for cantabular-server API and extended API JSON-stat queries
/v8
/v9
Variable mappings in the response from the /codebook endpoint in the cantabular-server API
Each mapping variable had mapFrom and mapFromCodes parameters. Together these described a single mapping from one or more (in the case of multivariate mappings) sources to the variable.
Each variable has a list of mapFrom objects, where each mapFrom object contains a sourceName and codes field. Each mapFrom object describes an independent mapping from a set of sources to the variable.
Source category codes in variable mapping
Each element in mapFromCodes was either the code of a category in the mapping variable or an empty string which indicated that the code was the same as the previous entry.
Some entries in the codes list of a mapFrom object can now be null. These identify source categories that are not mapped to any category in the mapping variable.
Variable mappings in GraphQL schema for the extended API
Each variable had a mapFrom field which was a single VariablesConnection object. It was a list of source variables, with one entry for univariate mappings and multiple entries for multivariate mappings.
mapFrom is now a list of VariablesConnection objects, with each entry listing the sources for an independent mapping.
Filter only variables
All variables could previously be used both as a query variable and as a filter variable to select categories in a lower level source.
Each variable in the cantabular-server API and the extended API GraphQL responses has an optional filterOnly boolean field. It is used to identify variables as filter only variables that cannot be used as query variables. It defaults to false.

  • Codebook configuration changes:

    • Variables can now be independently derived from different sources that share a common base variable. This allows branches of variable mappings from a common base to rejoin which can be useful when expressing more complicated geographic relationships.

    • Mappings that are rule variables can now also be incomplete mappings. These are mappings where some categories in a source variable are not mapped to any category in the mapping variable.

    • Variable categories must have a label specified where a variable file has a label column.

    • For more information on these changes please see the Cantabular Codebook Format documentation.

  • Dataset configuration changes:

    • Category keys for all variables (used in zero perturbation) must either all be read from a file or all calculated. The two approaches can no longer be combined. This behaviour can be specified in the dataset configuration file.

    • Variables can now specified in dataset configuration files as filter only variables. These are variables that can only be used as filter variables on lower level sources to select specific categories from the query variable. They cannot themselves be used as query variables.

    • For more information on these changes please see the Cantabular Data Loading Guide.

  • Other configuration changes:

    • GraphiQL IDE can be disabled in both cantabular-api-ext and cantabular-metadata using an environment variable.

  • The dataset file format has been changed to 68.

  • A new Cantabular Metadata application (cantabular-metadata) is supplied as a beta release to allow metadata to complement datasets in cantabular-server to be loaded and made available through a GraphQL API.

    • Loads from supplied JSON files validated against a user-supplied GraphQL schema specified and evaluated at runtime.

    • Optional validation of metadata against datasets within cantabular-server to check whether all datasets and variables loaded there have associated metadata.

    • Provides a GraphQL API for exploring and querying metadata at /graphql.

    • Provides interactive IDE, GraphiQL, for exploring API schema and testing queries.

    • For more information please read the Cantabular Metadata Service Specification.

  • Support caching of cantabular-server responses through addition of ETag headers:

    • ETag header included with query response headers to uniquely identify a response and allow downstream caches to be invalidated on dataset or software version changes.

    • Caching can be disabled with the HTTP_CACHING_OFF environment variable.

    • Further documentation is available in the Cantabular Installation Guide.

  • UI enhancements:

    • Restructured public UI to remove progress bar and allow a user to change their selections using change links from the final data page.

    • New design and functionality for geography selection page in public UI to allow user interaction with full capabilities of new geography model. Search for areas by category code or label, select all areas, or search within a higher level area.

    • Reworked initial dataset selection page to display dataset and service metadata, when available.

    • Added functionality to allow display of variable description metadata (when available) and a fuller list of categories within the variable.

    • Moved filtering functionality in public UI to an optional step after table creation.

    • Added support for Excel (Open Office XML; .xlsx) downloads in public and admin UIs using custom generation and streaming code for performance reasons. Excel files have an emboldened, frozen header row to allow easier review of contents and (in the public UI) a metadata sheet showing variable descriptions. Excel downloads are only available when the output table is within Excel file size limits.

    • Rules editor in admin UI now retains scroll position and highlights error location.

    • Accessibility improvements in public UI: radio button and checkbox code refactored; all pages now have descriptive titles; added skip to content link; improved keyboard navigation.

    • Added gzip compression of text responses from the admin UI.

  • DRL enhancements:

    • Optimise execution speed of more cases of multiple qin statements.

    • Better reporting of tabletest results in DRL diagnostics to take into account possible effect of qin operator.

    • Allow underscores in DRL integer literals for improved readability. E.g. 1_000_000 can be used instead of 1000000.

  • Changes to dataset build and inspection:

    • Error reporting for JSON files now includes line and column numbers.

    • cantabular-make-dataset now reports any unused files found in the codebook directory.

    • Dataset file version is now reported by cantabular-inspect, cantabular-make-dataset and cantabular-server.

    • cantabular-inspect can no longer be used to verify a set of codebook source files: use cantabular-make-dataset instead to check codebook files. This is because information from the dataset configuration file is now needed for codebook verification.

  • Addition of /scc-version endpoint to cantabular-api-ext and cantabular-metadata to facilitate service health-checking.

  • Performance improvements to tabulation and rule evaluation.

  • The software is built using Go 1.16.3.

Version 8.0.1

Released: 21 Dec 2020

  • Enhanced support for CORS requests on graphql endpoint in cantabular-api-ext.

  • The software is built using Go 1.15.6.

Version 8.0.0

  • The API version is now v8 due to the following modifications:

    • Each dataset object now has a ruleBaseVariable field which identifies the base rule variable from which all other rule variables are derived. This replaces the ruleRootVariable field which identified the highest level in the rule hierarchy.

    • Updated API parameter names for consistency purposes:

      • The query endpoint now uses v instead of dim to select query variables and f instead of incl to select filters.

      • The codebook endpoint now uses v instead of var to select variables that should be included in the response.

      • Errors now returned in JSON format rather than plain text.

  • Dataset file format version changed to 63.

  • A new Cantabular Extended API application (cantabular-api-ext) has been added, as a beta release. This application provides a powerful interface to publishable data hosted by a separate cantabular-server service. It uses two different data standards, GraphQL and JSON-stat, chosen to give an API user flexibility in requesting exactly what they need and increased usability in the output format. Includes:

    • GraphQL endpoint to allow flexible querying of codebook and cross-tabulation queries.

    • JSON-stat endpoint for pre-defined self-describing query responses that work with existing Python, R and JavaScript client libraries.

    • Interactive IDE, GraphiQL, for exploring API schema and testing queries.

    • Full documentation of endpoints and usage examples.

  • DRL enhancements:

    • Add feature to test query equivalence. This facilitates special handling of queries that use specific variables or a subset of those variables. For instance it can be used to bypass disclosure rules checks for a set of standard tables.

    • Fixed issue where querytest rules operated on filtered variables. This could result in queries with filters being permitted in cases where they should have been blocked because the output table would have too many cells.

    • Handle DRL compile panics.

    • Minor fixes to rules editor.

  • UI enhancements:

    • Add filtering page and switch order of geography pages in Public UI.

    • Fix Admin UI filtering bug in old versions of Edge.

    • Display perturbation table size in cantabular-inspect and Admin UI.

  • Configuration changes:

    • Enhanced error checking when loading perturbation tables.

    • Cell keys can now be specified in ranges where consecutive entries in the ptable file have the same perturbation value.

    • Fixed a bug where a mapping variable could also be specified as a column header in the microdata file. An error is now thrown when this is detected.

  • Support for alternate rule (geographic) hierarchies:

    • Allow multiple rule variables to be directly derived from a given lower level source variable. This facilitates multiple rule hierarchies.

    • Support for specifying multiple structural zeros variables. This facilitates zero perturbation involving variables in different rule hierarchies.

    • Remove sorting of rule variable categories. Rule variable categories are now displayed in the order they are specified in the variable definition file in the codebook.

    • AllowDuplicateCategoryLabels configuration parameter added to support variables with duplicate category labels.

    • Expose HTTP server configuration options via environment variables.

  • Perturbation performance improvements.

  • The software is built using Go 1.15.6.

Version 7.0.0

Released: 8 Oct 2020

  • The API version is now v7 because of changes due to introduction of the Disclosure Rules Language:

    • The error property of the query response has changed and is now partly controlled by the definition of the disclosure rules. See API Specification document and OpenAPI definition for details.

  • New datetime string added to datasets API response for each dataset being served. This is currently set to the dataset build date and time.

  • Dataset file format version changed to 61.

  • Addition of the Disclosure Rules Language (DRL). This removes the built in disclosure rules from Cantabular and replaces them with a language which enables the rules to be specified when building the dataset.

  • Faster computation of table statistics and margins which are used by disclosure rules.

  • Record metadata in the dataset about the software and machine that was used to build it.

  • Record in the dataset the date and time at which it was built.

  • Admin UI enhancements:

    • A new “Inspect” tab has been added to the Admin UI. This provides similar information to the cantabular_inspect command line program but via the UI. It enables view and export of the various metadata in the dataset.

    • New feature allowing drag-sort reordering of query variables. Rule variables are always displayed at the start of the variable list and ordered with the coarsest first.

    • New feature to show all the categories in a variable by clicking the “i” icon next to the name or label.

  • Security enhancements:

    • Added security headers to responses from Public UI, Admin UI and API in line with web security guidance from Mozilla. This includes setting the Content-Security-Policy header which protects against cross-site scripting. https://infosec.mozilla.org/guidelines/web_security.html

    • Set SameSite flag on all cookies to Lax.

    • Removed inline JavaScript from Public UI and Admin UI.

    • Fixed Open Redirection warning flagged by penetration testing.

    • Set Cache-Control header to no-store by default. Only allow caching of specific responses.

  • The software is built using Go 1.15.2.

Version 6.4.3

Released: 17 Jun 2020

  • Improvements to API specification document, including the addition of example requests and responses.

  • In the Public UI the variable tooltips now show the number of categories in the variable.

  • Logging enhancements to cantabular-server and cantabular-ui to explain the consequences of default subnets.

Version 6.4.2

Released: 3 Jun 2020

  • Support gzip encoding of API responses. Compression can be disabled by setting the environment variable CANTABULAR_API_HTTP_COMPRESS_OFF and the threshold for compression set using CANTABULAR_API_HTTP_COMPRESS_MIN_BYTES. The Public UI disables compression when making requests to the server.

  • Support gzip encoding of CSV downloads from the Public UI.

  • Deliver MacOS build for development use on Apple Mac PCs.

  • The software is built using Go 1.14.4.

Version 6.4.1

Released: 27 May 2020

  • All HTTP requests made to both cantabular-ui and cantabular-server are now logged to the standard output of each process. The log messages include source IP address, username (if present), HTTP request URI, HTTP status code of the response and the number of bytes returned.

  • Additional parameters have been added to the API codebook path that facilitates reducing the size of the response to exclude unwanted information.

    • If the cats query string parameter is set to false, then detailed category information is not included in the response. Category codes, labels and map_from_codes are omitted from the codebook JSON. The cats parameter defaults to true.

    • Information on specific variables can be requested using the var query string parameter. If no var parameters are supplied then information on all variables is returned.

    • A new len name is included in the JSON object for each variable. The value is the number of categories in the variable. This is redundant in the case where cats is true since the value will be the same as the length of the codes array but more useful when cats is false because codes is not included.

  • It is now possible to allow access to the proxied API in cantabular-ui on all client IP addresses, bypassing any subnet restrictions. This functionality is enabled by setting the CANTABULAR_UI_API_ALLOWED_ALL_SUBNETS environment variable. Access to the API will still be restricted by user authentication if that has been configured.

  • Web User Interface bug fixes:

    • In the Public UI on the variables page the expansion of mappings did not always display correctly.

    • The SDC diagnostics page failed to render in the case when no rule variable was included in the query.

    • Add links to progress bar items on the new layout of the data page which was added with the standard tables feature.

  • Python API module improvements:

    • In 6.4.0 cantabular-ui server was enhanced to proxy the server API through to cantabular-server after it has been authenticated. A new CantabularClient.login method has been added to the Python API module which allows authenticated access to the proxied API.

    • A new parameter has been added to CantabularClient that allows TLS certificate validation to be disabled on all API requests.

    • The CantabularClient.codebook method has been updated to take a new parameter that specifies whether codebook API calls set cats to true or false. Existing applications should be updated to include this parameter.

    • A new Codebook.populate method can be used to populate an existing Codebook with detailed information on variable categories. Calls to CantabularClient.query will automatically update the Codebook with category information on required variables.

    • A new CantabularClient.variables method has been added that allows a user to request codebook information on specified variables only.

    • Variable names used in CantabularClient.query are now validated before any API calls are made. Variable names must be in the same case as the names in the codebook.

    • A new CantabularClient.close method has been added. This closes the requests.Session object used to communicate with the API.

  • The software is built using Go 1.14.3.

Version 6.4.0

Released: 6 May 2020

  • New “Standard tables” feature (titled “Ready-made tables” in the UI):

    • A set of predefined queries that the user can search through and choose can be configured using a JSON file specified using the CANTABULAR_UI_STANDARD_TABLES_FILE environment variable passed to the cantabular-ui process.

    • The “Cantabular Standard Tables Loading Guide” document, which describes this feature in more detail, has been added.

  • cantabular-make-dataset includes improvements to memory handling by first counting the number of lines in the microdata file. This means that data is not repeatedly copied and heap fragmentation does not occur. Hence microdata loading does not get slower as dataset construction proceeds. It will also mean that cantabular-make-dataset uses less memory overall and thus is less likely to fail with an “out of memory” error.

  • cantabular-inspect has a new feature to export a codebook from a dataset. The codebook can be exported as a ZIP file using the -cbzip option or into a folder using the -cbdir option. Logging messages relating to export options have also been improved.

  • Error messages regarding codebook mapping files always include the filename.

  • Admin UI improvements:

    • The UI layout has been changed to move the navigation to the left.

    • User defined rules are now set separately for each dataset.

    • The status of whether rules are modified for a dataset is shown on the query page, the home page which lists all the datasets and in the rules editor page itself.

    • The SDC on/off status is set per dataset. As before, the SDC setting for all datasets is reset to “on” when a user logs out.

    • A new switch on the query page allows displaying variable names or labels.

    • Allow easier switching between query and rule pages by using tabs in the page layout.

    • Searching for a variable on the query page now also searches the variable label text.

    • The UI handles browser zoom out on query page better. The user gets to see more on the page.

    • The rules editor gives better feedback when rules are updated:

      • It displays confirmation when “Load defaults” is clicked.

      • Status messages disappear when the rules text is changed.

      • The complete current effective rules are shown immediately after “Save”.

      • The status message shows that the rules are set to default if the text is edited to match defaults.

  • The output table display has been improved using full browser width and wrapping to see more information without horizontal scrolling.

  • The layout for “Download” and “View data” in custom table journey in the Public UI has changed to use tabs to match the UI for the new “Standard Tables” feature.

  • An administrator can now disable the Admin UI completely in cantabular-server by setting the CANTABULAR_ADMIN_LISTEN_ADDRESS environment variable to blank or empty.

  • The Cantabular REST API is now available in the cantabular-ui server:

    • cantabular-ui server proxies each API request through to cantabular-server after it has been authenticated.

    • This enables a program to use a username / password logon to access the API rather than only by allowed IP address range as previously. Subsequently API calls are authenticated by a cookie containing a token.

  • Various security improvements after more in-house testing:

    • Some reflected erroneous user input has been removed.

    • Missing security headers have been added on some redirect responses.

    • The secure flag is set on all cookies when TLS is in use.

  • Various UI bug fixes, including:

    • The truncation message at end of view data table in Public UI has been added back after a long period of absence.

  • The software version has been added to PDF document filenames.

  • The software is built using Go 1.14.2.

Version 6.3.0

Released: 8 Apr 2020

  • New derived output variables and crosstab function in admin user interface:

    • A user can add related variables to the output table (e.g. “Local Authority” and “Output Area”) and Cantabular will work out the crosstab needed to generate the output for the selected variables. The crosstab that will be computed is displayed below the output variables. The crosstab is what is subject to the SDC process and determines the number of counts in the output table.

    • Variables in the generated crosstab are automatically added to the output variables list if they are not already present. This is to ensure that the categories for the output table counts are unambiguous and thus the table is self-descriptive.

  • New filtering function in admin user interface:

    • A user can add filters to any of the crosstab variables using the variables themselves or higher level mappings of those variables. This is achieved by clicking the filter icons next to the crosstab variables. Using the crosstab rather than output variables prevents the user from applying conflicting filters.

    • The current filter state is encoded in the page URL and is indicated in the user interface via a change in the filter icons next to the crosstab variables and via tooltips on those variables.

    • A user can search within the categories of a filter variable in order to quickly find categories.

    • A user is initially presented with the variable in the crosstab (rather than any of its mappings) when choosing a filter unless they have already set a filter on that crosstab variable in which case they are presented with the variable previously chosen. The exception is when the variable in the crosstab has too many categories (see below) in which case the mapping with the highest number of categories is initially presented.

    • Filtering on variables with more than 500 categories is disallowed by default. This limit is configurable using the CANTABULAR_ADMIN_FILTER_MAX_CATS variable. However setting this to much larger numbers will lead to very slow page rendering in the admin user interface when relevant variables are selected.

    • Applicable filter state is retained when adding or removing variables from a query.

  • In the admin user interface the download format of “codes”, “labels” or both is retained for each browser client via a cookie.

    • A user can also choose whether codes or labels are displayed first when using both for output.

  • Remove /download path (CSV output) from the API implementation.

    • We had already removed it from the specification, but the underlying implementation was still present.

    • The Public (demo) user interface now generates it’s own CSV download from counts returned by the API. Automated tests have confirmed that the new CSV download data is byte for byte identical with the old CSV download data from the server.

  • Deliver the OpenAPI specification for the Cantabular API with the documentation in the admin user interface and also in the installation directory within the /api directory.

  • Disable download button in admin user interface when no areas pass the rules.

    • The download button is now enabled if applicable only after the test query used for the disclosure control summary has returned.

  • Remove some responses in the API implementation and user interfaces when invalid HTTP requests are made (highlighted during internal security tests).

  • Update to Go 1.14.1

Version 6.2.2

Released: 16 Mar 2020

  • report multiple errors when reading microdata file using cantabular-make-dataset

    • number of errors reported can be controlled by CANTABULAR_MAX_MICRODATA_ERRORS environment variable (see data loading guide for details)

  • report multiple errors when reading codebook files using cantabular-inspect and cantabular-make-dataset.

    • number of errors reported can be controlled by CANTABULAR_MAX_CODEBOOK_FILE_ERRORS environment variable (see data loading guide for details)

  • update to Go 1.14 and associated code changes for new asynchronous preemption
    (see https://golang.org/doc/go1.14#runtime)

  • improved calculation of memory required for queries and associated improvements to query queuing

  • limit on tabulation concurrency which defaults to number of CPUs, configurable by CANTABULAR_CROSSTAB_MAXPROCS environment variable

  • new memory cache for cross-tabulation which can be disabled by environment variable: CANTABULAR_DISABLE_TABULATION_POOL=true

  • reduced GC pressure from re-use of memory during rules evaluation (e.g. for computing margin tables)

  • more named entities in OpenAPI definition to facilitate more readable generated OpenAPI client code

  • OpenAPI definition automatically tested in our test suite using a generated Go client

  • remove unimplemented excl parameter from API

  • use secure session cookie when using TLS or operating behind a TLS proxy

Version 6.2.1

Released: 7 Feb 2020

  • fixed FutureWarning in Python API example script ctb_client.py triggered by recently released version 1.0.0 of pandas module

Version 6.2.0

  • dataset file format changed to 56

    • facilitates perturbation performance improvement

  • perturbation performance improvements

  • zero perturbation consistency:

    • choose the same cells for zero to one perturbation regardless of the order that dimensions are specified in the query

    • query result is the same as in the previous version of the software when the variables are arranged in case-insensitive lexical order

  • start cantabular-server using all datasets in a directory:

    • allow directory names as well as filenames on the command line or in the CANTABULAR_DATASETS environment variable

    • log errant datasets rather than failing to start

    • display errant datasets on Admin UI

    • recurse flag (-r) flag to allow recursion into subdirectories of directories

    • test flag (-t) to check datasets and not start in server mode

  • Python API module improvements:

    • performance improvement in cantabular_client module when determining the blocked codes/labels following a query

    • all values returned by cantabular_client methods are now immutable

    • cantabular_client.Codebook.rules_hierarchy is now a tuple instead of a dict

    • resolved issues identified by pylint3

  • API specification document included

Version 6.1.0

Released: 26 Jan 2020

  • dataset file format changed to 55

    • perturbation table changes

  • dataset order listed on UI is the same as the order supplied to cantabular-server enabling the order to be chosen by an administrator

  • the number of the first row in the perturbation table that is reused for high cell values is now configurable in the dataset JSON file using: Perturbation.PTable.RepeatFromCellValue

  • change to definition of Rule H: now counts number of disclosures rather than totalling cell values and has no special behaviour for OA level and real cells.

  • Python API module (cantabular_client) added together with documentation and example code

  • rule parameters for dataset configuration are stored in separate file rather than in JSON

  • dataset hashes are recorded in logs on load

  • improvements to readability of output of cantabular-inspect

  • Cantabular documentation is embedded in Admin UI via “DOCS” link top right

  • admin UI enhancements:

    • SDC off function:

      • now uses slider style button

      • pre-query checks (rules) are not performed

      • automatic queries on adding each variable are not performed

      • new text for disclosure control widget

    • improved layout of SDC panel

    • SDC diagnostics view:

      • shows failing categories only

      • shows a maximum of 500 categories

      • has a link to diagnostics download

    • fixed bug where searching for variables did not show all matches

  • codebook format included in documentation

Version 6.0.0

Released: 06 Jan 2020

  • First release of Cantabular software.