Core Concepts
Last updated
Last updated
Integration platform as a Service (iPaaS) is a suite of multiple cloud services. The suite aims at integration and governance of various combinations of services - including On-Premise and On-Cloud applications, Service oriented architecture (SOA), cloud services, processes and data, within or across organizations. The iPaaS suite is an evolution of Integration as a Service and has been widely adopted for B2B and cloud services integration. It complements the application-development and hosting-oriented application Platform as a Service (PaaS).
Fundamentals Of Integration
The integration approach can be Middleware, iPaaS or EAI. All underlying concepts and principles around integrations are translated into Extraction, Transformation and Load (ETL).
ETL refers to the methods involved in accessing and manipulating source data and loading it into target database.
The ETL process comprises of three general steps:
Mapping the data between source and target database.
Cleansing source data in staging area.
Loading cleansed source data into the target system. It is worth noting that, the 2nd step may not be necessarily be the same for all the ETL tools. This is because a company's internal system architecture may vary from that of other companies.
Source System :
A database, application, file or other storage facility from which the data in a warehouse is derived.
Mapping:
The definition of the relationship and data flow between source and target objects.
Metadata:
Data that describes data and other structures - such as objects, business rules and processes. For example, the schema design of a data warehouse is typically stored in a repository as metadata. The metadata is then used to generate scripts to build and populate the data warehouse.
Staging Area:
A place where data is processed before entering the warehouse.
Cleansing:
The process of resolving inconsistencies and fixing anomalies in source data - a typical part of the ETL process.
Transformation:
The process of manipulating data . Any manipulation, beyond copying, is a transformation. Examples include cleansing, aggregating, and integrating data from multiple sources.
Transportation:
The process of moving copied, or transformed, data from a source to a data warehouse.
Target System:
A database, application, file, or other storage facility to which the "transformed source data " is loaded in a data warehouse.
