New methods are emerging that allow data acquisition services to integrate information into real-time
data warehouse deployments. In many cases, these new methods involve innovative uses of technologies
that have existed for some time. The use of Change Data Capture (CDC) tools in combination with Extract,
Transform and Load (ETL) tools is an interesting development for continuous, near-real-time data
acquisition. The typical implementation with this technology combination involves grouping CDC into
"mini" batches, or trickle feeds, that are then transformed from raw staging tables into integrated
data warehouse content using the RDBMS platform as the transformation engine. Execution of the
metadata-driven SQL from the ETL tool may be as frequent as every few minutes to hourly or more
depending on the service levels for data freshness demanded by operational BI within the enterprise.
Organizations can leverage their existing data warehouse investments by tuning the current processes
to provide real time information to the business.
Twin Bridge Consulting Group can assist you in implementing a real-time data warehouse solution by
analyzing the needs of your organization and determining the proper course of action. We can
incorporate change data capture into an existing data warehouse or design and implement a new
warehousing solution that ensures your company is accessing and utilizing data at a refresh rate that
is appropriate.
