Data Harmonization

UNCEFACT defines data harmonization as an iterative process of capturing, defining, analysing and reconciling government information requirements, and data standardization as the mapping of this simplified data to international standards (*).

Relevance to trade facilitation

Throughout a cross-border trade transaction, traders and other transport and logistics service providers have to provide information to many regulatory agencies and other partners along the supply chain. The information needed concerns goods, their packaging, weight and height, and means of transportation, and details of the importer and exporter. Hence, a multitude of different trade documents, still mainly paper format, are exchanged. Parties spend much time preparing the data, filling in the documents and submitting them. When data requirements are not harmonized and standardized, each agency and each document may have different requirements. Data is understood in various ways, and may not have the same representation. This increases the complexity of managing information requirements and increases the likelihood of errors. Differences in data sets also inhibit exchange across agencies, so that the same information often has to be submitted several times.

Benefits

Data harmonization and standardization facilitates the submission and processing of trade information (documents and data). It can help to:

reduce information requirements by eliminating redundancies and duplications, thus making the submission easier,
improve the quality of the data and therefore reduce errors,
facilitate receiving, processing and checking of information, and
facilitate exchange of data and improve automation as this ensures inter-operability.

Data harmonization is an important aspect of any automation project, in particular for a Single Window for Trade, for the migration to paperless trade, and for document alignment.

Data harmonization process

Data harmonization “involves a set of activities that improve the consistency in the use of data elements in terms of their meaning and representation format” (*). It is usually undertaken at the semantic level before considering document structures. Later the message syntax can be created from standard naming rules that may be part of standard Technical Specifications. This ensures the message syntax is also harmonized when derived from the semantics using a naming and design rule (NDR) as part of standards-based Technical Specifications.

There are currently two guides that explain the different activities and tools that can be used for data harmonization (see Solutions and Tools). The process usually starts with establishing an inventory of the current data requirements, definition of the data collected, analysis of the information requirements and data elements, and reconciliation of the data (i.e. consolidation of the defined and analysed trade data and alignment to international standards). The result of these steps is a simplified, standardized national data set so that e-documents can be developed.

Solutions and tools

A data harmonization process as summarized above ideally uses Business Process Analysis in the capture phase, to identify and map information and document requirements, and document and data flows. This analysis can reveal a clear picture of the current situation (AS IS situation) and enables identification of redundancies and the development of the future desired situation (TO BE situation).

The process of defining and analysing data requirements should integrate international standards, such as the UN/CEFACT Reference Data Models (RDMs) based on the UN/CEFACT Core Component Library (CCL), UN TED and recommended code lists. The RDMS provide a structured set of related semantics covering the data requirements of international trade. The UNTDED is a dictionary of trade data elements that provides a reference to clear and unambiguous identification of data elements (a data element name, description and four-digit number) and it can be considered as the data dictionary of the UN/CEFACT RDMs. Referencing the UNTDED data elements plus their path within the RDMs ensures consistency. In the reconciliation phase and for the development of e-documents, reference data models can be used to map the trade data and these can include the UN/CEFACT RDMs together with the WCO Data Model which covers detailed regulatory procedures. In cases where standards exist for messages within the project domain, the process should consider any harmonization that has already taken place.

Practical guides

UNNExT, UNCEFACT and WCO have developed practical guides that can guide and assist in data harmonization projects:
UNNExT Data Harmonization and Modelling Guide
UNECE/UNCEFACT Recommendation No. 34
WCO Data Model, SINGLE WINDOW DATA HARMONIZATION