News

ETL ain't dead, it's just taking a breather

• By Stephen Swoyer
• December 7, 2005

There's new research from market watcher Gartner that says the Extraction, Transformation and Loading (ETL) data tools market had a surprisingly successful 2004. According to Gartner's data, the worldwide ETL tools market grew by 6.7 percent last year. Gartner projects an ETL market CAGR of 6.3 percent through 2009, which should help generate half a billion dollars in new license revenue.

But this news isn't new. In a report dated April 25, 2005, Forrester Research predicts: ETL will pass $1 billion in 2005, almost 14 percent up from $949 million in 2004. The report, "Demand is Growing for Traditional and New Uses of ETL," explains that growth is driven by "expanding user practices, whether traditional (data warehousing) or new (general data integration outside warehousing)."

There's a further wrinkle, however: ETL as we know it is fast disappearing. "This market is rapidly evolving into a market of multipurpose data integration platforms suitable for use beyond the traditional domain of...BI and data warehousing," write Gartner analysts Colleen Graham and Ted Friedman. "As the market continues to evolve, larger vendors are building out their products toward this end, putting increased pressure on small pure-play ETL vendors and resulting in additional market consolidation. As a result, ETL, as a market, will not exist in the long term; instead, ETL will be one integration pattern among many that is supported by the new breed of data integration platforms."

"The trend with vendor tools is to include capabilities drawn from other types of integration tools," says Philip Russom, senior manager of research and services at TDWI. "For instance, EAI [Enterprise App Integration] tools support more data transformations, data quality tools can now integrate data, EII [Enterprise Information Integration] tools operate in real time like EAI and so on. ETL tools have seen a really dramatic evolution this decade, in that most now embed substantial capabilities for messaging, data quality, profiling and EII." Although ETL will always be at the heart of these tools, Russom says, we're quickly moving toward what he calls a "universal integration platform." The advantage of the UIP for users is that they can get best-of-breed ETL—plus closely related capabilities—in a single tool, deployed as a single server. He adds, "If ETL is to escape its data warehouse ghetto, it'll have to develop the broad appeal of the UIP."