What Is Apatar Open Source Data Integration?

by Alex KhizhniakApril 27, 2013

Learn about a great open-source ETL project we supported from 2006 to 2011.

Table of Contents

Overview

The Gartner Group estimates that corporate developers spend 65% of their effort building bridges between applications. Altoros understands users’ need to complete their integration initiatives quickly and easily without the involvement of costly programming and coding efforts. That’s why in 2007 we delivered Apatar, an open-source data integration tool designed to help both business users and developers move their data in and out of numerous sources and formats.

No programming was required to accomplish even a complex integration when using Apatar—one needed to just “drag-and-drop” data between databases and apps by using a visual mapping interface. Community members could also benefit from reusing mapping schemas built by other developers and DBAs.

The tool provided connectivity to Oracle, MS SQL, MySQL, Sybase, DB2, MS Access, PostgreSQL, XML, CSV, MS Excel, Compiere ERP, Salesforce.com, SugarCRM, GoldMine, InstantDB, any JDBC data sources, and many more. Apatar was 100% Java-based and platform-independent, running on Windows, Linux, and MacOS. The product was developed using Eclipse.

Popular scenarios

A robust, scalable Apatar architecture enabled end-users and developers to successfully solve a broad spectrum of integration needs—from extracting, transforming, and loading (ETL) for data warehousing, to migration, synchronization, and app integration.

Apatar could be used in diverse data-driven systems where you have information that needs to be captured and transferred to a new app, data warehouse, or a single front end. Popular integration scenarios:

Move data from one location to another
Synchronize data between two or more data sources
Integrate data across an enterprise
Consolidate data from two or more data sources into a data warehouse
Populate a data warehouse or a data mart
Validate and correct key data
Schedule backups

Apatar had a user-friendly UI and, even for a nontechnical user, it would take just a couple of hours to get trained. Most users found it intuitive. Apatar had a number of built-in data quality tools, enabling data cleansing, de-duplication, etc.

We also promoted the concept of reusable DataMaps. Working with data sources, the tool generates XML metadata files (“DataMaps”) that store all the information captured by the Apatar designer and mapping graphical UIs. Apatar’s engine then manages, executes, and controls the total project execution plan described in the XML metadata files. In addition, for each executed job, Apatar generates a detailed report / log.

By adopting and following this metadata-centric approach, users can share the XML metadata files to reuse connectivity settings, transformation, and mapping. These XML metadata files can be sent to another user having the Apatar desktop app or to feed pre-built DataMaps to the Apatar server engine, saving time and money while integration.

Community and collaboration

Though Altoros was a major contributor, the project had an active community behind it, helping to improve the product and develop new connectors, as well as other functionality. The open-source code of Apatar allowed for building add-on connectors for any data integration applications, be it legacy, APIs, or semi-structured data sources / targets.

The Apatar manifesto stated:

Source Code. We believe that the quality of the core code base determines how easy or hard it is for all of us to work together and push Apatar forward.
Responsibility. We feel responsible for the success of the project, including the support of new contributors and facilitating communication across e-mail, mailing lists, and forums.
Presence. We strive—and encourage those who commit to the project—to be present in the mailing list and chat room so as to help fellow developers and new contributors feel less alone.
Helpfulness. Often, there just isn’t enough documentation available yet for new features and connectors. Sometimes there are questions to be answered. It is our goal to address every question in order to quickly solve the bottleneck standing between you and successful usage of Apatar.
Openness. We share our short- and long-term goals with you in order to leverage all benefits of a joint community effort to build a truly open, great piece of software.
Founders commitment. We strive to be dedicated to work on the project, investing time for the further development of the software and the community.
Patience. We understand that every community is different and lives by its own rules. We believe that all of us should be patient with each other and support new members so that they can become vibrant community leaders.

Licensing model

Apatar used a “dual-licensing” business model. Under this model, users could choose to use the product either under the free software/open-source GNU General Public License (known as the GPL) or under a commercial license. Apatar’s dual-licensing was focused on promoting freedom in two ways: first, it encouraged the growth of free software by licensing Apatar under the GPL; second, it made it possible to use our software in situations where the GPL was not applicable.

Apatar’s dual-license model supported the company’s mission: to make data integration affordable to all, including those users who traditionally could not afford expensive proprietary data integration tools. (This kind of software has traditionally been expensive, with a typical license costing $25,000 to $500,000.)

“The open-source community provides business users with an option to avoid vendor ‘lock-in.’ What users of open-source software really like is the total cost of ownership. When you don’t have to pay license fees for database software, data integration, and business intelligence software, this alone makes a healthy difference to your bottom line.” —Renat Khasanshyn, Founder, Apatar

The dual-licensing also created an opportunity for Apatar to give back to the community. The Apatar team believed that the open-source model of development and distribution is the most efficient way to produce high-quality software.

Renat Khasanshyn

“Open source is driven by real people like you and me, helping each other make our day-to-day work easier and more productive. Every contribution to the Apatar project extends the way companies manage their information, perhaps, one of the most valuable assets in the global economy. We value and appreciate each and every contribution you are making to the project.”
—Renat Khasanshyn, Founder, Apatar

The dual-licensing business model was becoming increasingly popular among open-source companies at that time, paving the way for long-term financial viability. Other companies that offered dual-licensing included Alfresco, MySQL, SugarCRM, Trolltech, Zimbra, and others.

Afterword

By 2012, the tool was mostly deprecated, only the latest stable release (v1.12.*) was supported for some time. It still can be downloaded from SourceForge. We do grateful to everyone who helped our core team to made this awesome project thrive.

Some of the most active Apatar contributors outside the core team

In addition to code contributions, many thanks to Robert Scoble, Jason Pratt, D.Lambert, Marc Russel, Brian Chee, Tim Fehlman, Krishnan Subramanian, and other awesome professionals who mentioned Apatar on their blogs, spreading the word across communities.

“Apatar is creating a shift in the way we look at data. With its on-demand data integration tools working together with complimentary RSS aggregation services Apatar will go a long way towards commoditizing information integration between applications, databases across the enterprise traditionally suffering from information integration issues among heterogeneous systems.” —Deepak Singh, Blogger-in-Residence, TechBizMedia.com
“Before Apatar, there were many proprietary vendors offering data integration solutions to the enterprise marketplace. In many cases they were not cost effective. Apatar entered this market as an open source vendor offering on-demand data integration coupled with product support and services at a much lower cost. Even though Apatar software is designed as a toolset for developers, it does not require manual coding and may receive adoption among business users.” —Krishnan Subramanian, Blogger-in-Residence, TechBizMedia.com

We do appreciate the contribution of everyone who was involved into Apatar’s ecosystem! Check out these video tutorials or read exhaustive media coverage to learn more about this great product.