Terminology v1.3
This terminology is important to understand EDB Data Migration Service (DMS) functionality.
Change Data Capture (CDC)
CDC is a set of software design patterns used to determine and track changes in data sets by calculating deltas. EDB Data Migration Service uses CDC to stream changes from a database cluster to another.
EDB DMS Agent
The DMS Agent is a tool that can register databases for migration. In reader mode, it registers databases as a source for migrations. In writer mode, it registers them as a destination.
After you register the source and/or destination databases, the DMS Agent works with the Data Migration Service to perform data migrations.
Note
Starting from HM version 1.3 and later, the EDB Data Migration Service Agent, packaged as cdcagent
, replaces legacy DMS Reader and DMS Writer.
To ensure compatibility, use the DMS Agent on all 1.3 and later HM deployments. For Hybrid Manager deployments on version 1.2 and earlier, use the separately packaged EDB DMS Reader and EDB DMS Writer.
EDB DMS Agent in reader mode
When running in reader mode, the DMS Agent uses Debezium to perform CDC operations on the source database and Kafka/Object storage to produce messages containing the change events.
Install the DMS Agent on your source machine. Alternatively, if your source database runs on a machine managed by a cloud provider that you don't have admin access to, you can also install it on an auxiliary machine that has access to your source database.
Then configure and launch it in reader mode when performing any type of migrations, as it allows you to connect to and read the source.
EDB DMS Agent in writer mode
When running in writer mode, the DMS Agent consumes CDC records from Kafka/Object storage to replicate and apply changes on the destination database.
If the destination for your migration is a Hybrid Manager-managed (HM-managed) database, you don't need to install and launch the EDB DMS Agent in writer mode, because the DMS component that comes with the Hybrid Manager already includes a built-in writer.
If the destination is a self-managed database, install an additional DMS Agent on your destination machine. Then configure and launch it in writer mode as it allows you to connect to and migrate data to self-managed destinations.
HM Agent
The HM Agent, or Agent, packaged as beacon-agent
, is a tool that provides database observability, monitoring, usage reporting, and more. In the context of migrations, the HM Agent performs a migration assessment for each database you register with it. In the case of registered Oracle databases, it also performs schema extraction for subsequent handling with the Migration Portal integration.
Auxiliary machine
An auxiliary machine is a Linux-based server (bare metal, VM, or local) dedicated to running the EDB DMS Agent for database migrations. This machine acts as the migration engine, capable of connecting to various source and destination databases, whether local or remote.
While it can be the same host as where you have a source or destination database, as well as the same host where you have the HM running, its only requirement is a Linux operating system, since it is the only platform currently supporting the EDB DMS Agent.
Migration types
When creating migrations, you can choose whether to perform a Snapshot or Snapshot + Streaming migration.
Snapshot migration
When you select the Snapshot option, DMS captures an image of your database at a specific moment. This point-in-time copy is then used for the migration. Any new inserts and updates performed on the source database after the snapshot is taken won't be replicated to the destination database.
Snapshot + Streaming migration
When you select the Snapshot + Streaming option, DMS begins with a snapshot to establish a baseline copy, and then constantly employs CDC to apply subsequent changes, ensuring data consistency during migration. DMS continues to stream any data updates performed on the source database to the destination database until you mark the migration as completed.
Migration destination types
When creating migrations, you can choose whether to perform a migration to Managed Postgres or to External Postgres.
Managed Postgres
Managed Postgres destinations are databases created and managed through the HM. All database clusters created within the project where you are performing the migration can be selected as possible migration destinations.
External Postgres
External Postgres destinations are databases created outside of the HM and fully managed by you. To choose an external postgres database as a migration destination, you must first install and configure the DMS Agent and configure it in writer mode to allow the DMS to connect to the self-managed database and perform write operations on it.
Apache Kafka
Apache Kafka is an open-source, distributed-event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. EDB Data Migration Service uses Kafka to manage data streaming from the source to the destination database.
Debezium
Debezium is a Java-based, open-source platform for CDC that is supported by the Red Hat community. When running in reader mode, the EDB DMS Agent uses Debezium to perform reading operations and capture data changes on the source database.
Legacy terminology
EDB DMS Reader
In HM versions 1.2 and earlier, the EDB Data Migration Service Reader, packaged as cdcreader
, is a tool that uses Debezium to perform CDC operations on the source database and Kafka to produce messages containing the change events.
Install and configure the DMS Reader when performing any type of migrations, as it allows you to connect to and read the source. If the destination for your migration is an Hybrid Manager-managed (HM-managed) database, you don't need to install the DMS Writer. If the destination is another self-managed database, install the DMS Writer.
EDB DMS Writer
In HM versions 1.2 and earlier, the EDB Data Migration Service Writer, packaged as cdcwriter
, is a tool that uses CDC records to replicate and apply changes on the destination database.
Install and configure the DMS Writer on your destination machine when performing migrations to self-managed destination databases, as it allows you to connect to self-managed destinations. Additionally install the DMS Reader for any type of migrations.