Simplifying Extract-Transform-Load for Ranked Hierarchical Trees via Mapping Specifications

TitleSimplifying Extract-Transform-Load for Ranked Hierarchical Trees via Mapping Specifications
Publication TypeBook Chapter
Year of Publication2015
AuthorsSoomro, S, Matsunaga, A, Fortes, JAB
EditorBouabana-Tebibel, T, Rubin, SH
Book TitleFormalisms for Reuse and Systems Integration
Series TitleAdvances in Intelligent Systems and Computing
Volume346
Pagination203-225
PublisherSpringer International Publishing
Keywordsdata integration, data transformation, EF-1115210, ETL, flat schema, hierarchical tree, mapping language, MSL, schema mapping, self-referential schema
AbstractA popular approach to deal with data integration of heterogeneous data sources is to Extract, Transform and Load (ETL) data from disparate sources into a consolidated data store while addressing integration challenges including, but not limited to: structural differences in the source and target schemas, semantic differences in their vocabularies, and data encoding. This work focuses on the integration of tree-like hierarchical data or information that when modeled as a relational schema can take the shape of a flat schema, a self-referential schema or a hybrid schema. Examples include evolutionary taxonomies, geological time scales, and organizational charts. Given the observed complexity in developing ETL processes for this particular but common type of data, our work focuses on reducing the time and effort required to map and transform this data. Our research automates and simplifies all possible transformations involving ranked self-referential and flat representations, by: (a) proposing MSL+, an extension to IBM’s Mapping Specification Language (MSL), to succinctly express the mapping between schemas while hiding the actual transformation implementation complexity from the user, and (b) implementing a transformation component for the Talend open-source ETL platform, called Tree Transformer (TT). We evaluated MSL+ and TT, in the context of biodiversity data integration, where this class of transformations is a recurring pattern. We demonstrate the effectiveness of MSL+ with respect to development time savings as well as a 2 to 25-fold performance improvement in transformation time achieved by TT when compared to existing implementations and to Talend built-in components.
URLhttp://dx.doi.org/10.1007/978-3-319-16577-6_9 http://www.springer.com/gp/book/9783319165769
DOI10.1007/978-3-319-16577-6_9