Tunnel
| 功能 | 说明 |
|---|---|
增量实时同步 | 支持 INSERT, UPDATE, DELETE 常见 DML 同步 |
修改订阅 | 新增、删除、修改订阅表，支持历史数据迁移，文档：修改订阅 |
表名映射 | 支持 和源端保持一致, 转小写, 转大写, 以'_数字'后缀截取 |
元数据检索 | 从源端表查对端，查询设置过过滤条件的 |
高级功能
| 功能 | 说明 |
|---|---|
Removal of Target Data before Full Data Migration | Remove the existing data in the Target before running the Full Data Migration, applicable for DataJobs reruning and scheduled Full Data migrations. |
Recreating Target Table | Recreate target tables before running the Full Data Migration, applicable for DataJobs reruning and scheduled Full Data migrations. |
Incremental Data Write Conflict Resolution Rule | IGNORE: Ignore primary key conflicts (skip writing), REPLACE: Replace the entire row in case of primary key conflicts. |
Handling of Zero Value for Time | Allow setting zero value for time to different data types to prevent errors when writing to the Target. |
自定义代码 | 文档1：创建自定义代码任务 |
数据过滤条件 | 支持 WHERE 条件进行数据过滤，内容为 SQL 92 子集，文档：创建数据过滤任务 |
前置条件
| 条件 | 说明 |
|---|---|
Port Preparation | For CloudCanal (Docker version), modify the docker-compose.yml port mappings before installing/upgrading after extraction, and open relevant ECS security group ports for remote connection. |
任务参数
| 参数名称 | 说明 |
|---|---|
httpMaxContentSize | The maximum content size when Tunnel uses HTTP to transmit data. Currently it is used in Netty‘s HttpObjectAggregator. |
eventStoreSize | Size of the queue in the memory (it might be event count or memory size, in bytes). |
protocol | Protocol for data transfer in Tunnel. Support HTTP currently. |
uriPrefix | URI prefix for Tunnel data sources. |
contentCompressType | Data compression form for Tunnel data sources. Support DeInflater / GZIP currently. |
dbs | Definition of the data source schema. |
printDataInLog | Enable to print key data to logs for troubleshooting. Enabling this feature may impact performance and data security. |
schemaFormat | Type of Value Schema. |
printCustomCodeDebugLog | Print custom code DEBUG logs (Note that it will print data before and after processing, which may increase disk usage and impact sync performance). This parameter takes effect dynamically and DataJob restart is not required. |
Tips: 通用参数配置请参考 通用参数及功能
前置条件
| 条件 | 说明 |
|---|---|
Permissions for Account | |
Port Preparation | Allow the migration and sync node (Worker) to connect to the MySQL/MariaDB port (e.g., 3306). |
任务参数
| 参数名称 | 说明 |
|---|---|
keyConflictStrategy | Strategy for handling primary key conflicts during write in Incremental DataTask:
|
dstWholeReplace | Convert INSERT and UPDATE operations into full row replacement in the Target. |
deCycle | Enable filtering in bidirectional sync to filter DML/DDL with specific markers. |
specialSqlMode | Set a specific SQL mode when initializing the connection between databases. |
defaultGisSRID | Set the SRID for GIS data types. |
dstTimeZone | Target time zone, e.g., +08:00, Asia/Shanghai, America/New_York, etc. |
increParallelApplyStrategy | Parallel write strategy for relational databases in the Target:
|
Tips: 通用参数配置请参考 通用参数及功能