Change Data Capture — Convert your database into a stream with Debezium
Have you ever thought about creating a stream from database operations? In this story, you will learn what Change Data Capture is and how to use it while planning your system architecture. In the practical part, we will see the Debezium in action.
What is Change Data Capture?
Change Data Capture is a process of detecting changes made to the database. The changes can then be streamed and integrated with other databases and systems. In other words: we receive a stream of events from our database.
This allows us to make faster and more accurate decisions based on data (Stream Processing and Streaming ETL). It does not overload systems and networks as classical solutions do. Often it is the only reasonable solution to upgrade legacy systems.
Types of Change Data Capture
QUERY-BASED
What if I query the database every 5 seconds. Is this Change Data Capture? In a way, yes, but this method has its drawbacks:
- This method requires proper preparation of the table schema: a column with the time of record modification and the mechanism of its update;
- We do not detect all operations. In the case of record deletion…
