The second tier, data integration, is primarily about aggregating and consolidating information from different sources together into a single common storage mechanism. Applications provide the data as exports, either with the content already mapped to the receiving system’s data model during export, or via an external transformation mechanism to then be loaded into the target system, a process of Export/Transform/Load (ETL). In this environment, the applications providing the data do not care, nor do they need to know, that the data integration (receiving) system exists.
A classic example of a data integration environment is document management. Documents, drawings, models, files and “containers” of all sorts are brought together and loaded into a common classification indexing or librarian system for storage and retrieval. Hexagon's solution for document management is SmartPlant Foundation.
Another more granular form of data integration is that of the engineering data warehouse (EDW), also supported by SmartPlant Foundation. “Content” from multiple disparate applications is brought together and harmonized to form a single uniform view of the “truth.” This more granular data integration also forms the foundation of the other tiers of integration. It supports the uni-directional movement of data between systems and requires the data to be mapped to the data model of the target system.
In point-to-point integrations, this is invariably a direct translation. But when multiple systems are required to share the same common data, pressures, temperatures, units of measure, etc., it is more advantageous to translate/map this data to a common intermediate application, agnostic and neutral in form, such as the SmartPlant Schema, thereby reducing the number of transformations required to support “enterprise integration.”
SmartPlant Foundation manages these two different levels of data granularity containers and contents simultaneously: documents (containers) define the boundary condition/scope for exchanges and provide the deliverable record, while the data (content) is extracted and aggregated together with that from other exchanges.
Clearly, if data are being brought together from multiple sources, it is possible that some duplications exist. If they don’t have information management capabilities, most tools importing data simply overwrite the existing data. Some may have revision management capabilities for this new data, but it is not common.
Therefore, as well as providing a common language for the exchange, the information management capability associated with data integration must also deal with this duplication – consider it a process of enforcing consistency on a project – correlation, aggregation, consolidation, etc.
Additionally, data integration should also deal with the provenance, status and security of the data. It is for these reasons that such capabilities are considered essential for the project data handover application of a data warehouse.
The benefits of data integration include the same benefits as presentation integration, plus the abilities to:
-
Aggregate/consolidate the data such that overlaps are removed, providing a cleansed, high-integrity exchange/store of data
-
Neutralize the data to a common form for ease of access, such that the source/originating applications (which could be many) are not required for the information consumer to install, learn or indeed pay for and support, and the data appears as a seamless whole.