Top 10 Trends in Data Integration (2007)

Let’s review what’s hot in data integration as the year begins.

 
1. Service-oriented data integration

As enterprises are moving toward Service-Oriented Architecture (SOA), the number of SOA elements is increasing, and data integration becomes an issue here. Reusable services and resources mashed up through a SOA platform, as well as SOA sharing with 3rd-party partners, require proper business process integration. Since IDC predicts that SOA-related services will grow to around $34 billion in 2010, data integration importance will only increase in the future years.

 
2. Rising business intelligence deployment and system integration

Consolidated corporate performance is the only way to maintain enterprise’s growth today. Business Intelligence is embracing internal and external infrastructure step by step. According to an Aberdeen survey, approximately 50% of enterprises aim at extending, integrating or implementing Customer Relation Management (CRM) system. Enterprise Resource Planning (ERP), Corporate Performance Management (CPM), Business Process Management (BPM), Customer Intelligence—i.e., Customer Data Integration from multiple sources and applications—all these corporate efforts represent the attempt to integrate data and processes into a single unified system. This trend is on the rise, definitely.

 
3. Metadata-driven interfaces will dominate in integration tools soon

Rich data integration and management interfaces as well as drag-and-drop features is a result of Web 2.0 revolution. There is a strong demand for managing and reporting tools that can be deployed by non-technical business users. Look at the latest data integration interfaces and engines. With pre-existing web services such as AJAX and mashups, data integration becomes truly intuitive.

 
4. Open-source software implementation became common

Open-source software (OSS) is climbing up to the highest levels of the data integration stack. According to recent surveys, enterprises tend to include OSS implementation in their IT plans. The reasons are well-known. Cutting the costs, scalability and, of course, reliability. A number of open-source data integration tools proved to be more reliable and cost-saving than proprietary ones. That’s why CEOs and CIOs shifted their attitude toward OSS from distrust to understanding open-source benefits.

 
5. Software as a service: data moves to the web

Software as a service (SaaS) provides enterprises with faster deployment of applications and lower costs, with guaranteed maintenance and upgrades. It is not only the way to solve a set of integration problems; it becomes a new data management philosophy that broadens the adoption of Business Intelligence. Gartner predicts that 25 percent of new business software will be delivered as on-demand services by 2011. Today we are watching as data integration processes are moving to the web.

 
6. The ‘structuring data’ challenge

The new trend shows that structured and unstructured data are converging across enterprise’s structure. Yes, data management systems provide us with structured data, but other information is still hidden away in e-mails, note books, contracts, Word or PDF files and other electronic documents. It is often estimated that up to 80 percent of business information is unstructured. Though it sounds challenging, the efforts to provide business users with more and more structured business data are on the rise.

 
7. Data quality is getting more important

According to 2006 TDWI report, 81% of IT and business professionals suffer from inaccurate reporting. Decision-making mistakes result in losses and delays. That’s why verification of data accuracy and completeness is so crucial. There is a strong need to standardize the types of customer information captured for analysis, as well as provide comprehensive customer data integration. Since not all enterprise information should be managed equally, data priority issue occurs, too. Cleansing data, eliminating duplications, and data filtration run through every data integration implementation.

 
8. Middleware becomes specialized

A dozen of buzzwords and acronyms—such as EAI, ESB, ETL, and EII—has appeared. There are tools for Enterprise Application Integration, Enterprise Service Bus, Extract Transform and Load actions, Enterprise Information Integration, and so on and so forth. Data integration adopters are penetrating new integration segments. As the number of messages, processes, and transactions flowing through an enterprise infrastructure continues to grow, the number of new acronyms is to increase during the next few years.

 
9. Security problems are extremely hot

While moving towards SaaS, open SOAs, Web 2.0 technologies, and mashup implementation, security becomes an issue for data-driven enterprises. Multiple external and internal data sources lack inherent security mechanisms too often. Data access is another problem while performing integration processes. All these challenges are forcing software vendors to look for more efficient ways to provide data safety.

 
10. Live data climbs the stack

Today the availability of timely and reliable data is seen as one of enterprise’s competitive advantages. The demand for visibility across customers, channels, distributors and dealers is leading software developers to improve data synchronization. With Enterprise Information Integration (EII) and other data integration technologies, the latest Business Intelligence suites tend to represent unified and live view on enterprise processes. Though data decentralization and increasing volume of enterprise’s information seem to be an obstacle here, data integration is struggling with it in the meantime. And guess who wins.

Let me know in the comments if there’s anything you want to add to this list.