I am working on automating AWS DMS setup using AWS API / CLI / SDK (doesn’t matter, its actually makes the exact same thing under the hood)
Recently AWS released new functionality for DMS setup – using data providers, instance profiles, data migration projects
The setup looks like:
- You first add a data provider, holding db engine, host, port, dbname, etc, for source and target. In total 2 data providers for initial setup – source & target
- Then you have to add a subnet group, where your replication infrastructure will be launched
- Create an instance profile. Although the title is a bit confusing, instance profile is most about network settings, where infra will be launched – subnet group, vpc, vpc security group, kms key for DMS data encryption
- Username & password to access databases needs to be placed in secretsmanager, and appropriate IAM role needs to be created so DMS service can access these secrets. One more (or the same role, depends) is needed for DMS service so it is able to create infrastructure where replication will be running on (I am not pretty sure through, but believe so)
- Finally, you create a “database migration project”, where you specify instance profile, role to access secrets containing username & password to databases, source & target data provider descriptors, secrets itself (username & password – ARNs). In case of data migration is homogeneous, we are able to easily launch a datamigration by just creating “Data migration” in “Data migration project” – and specify service access IAM role, migration type (full or CDC, or both),CW logging configuration and # of jobs to be launched (defaults to 8)
That is a piece of AWS Console I am talking about:
That’s where data migration might be created in console:
There is also an official video on youtube describing the process: https://youtu.be/HOJfrR6lcuU
Finally!! Whats my problem?
Everything is fine when we are talking about AWS Console, however, considering AWS CLI / SDK / API – there is lack of documentation! It ends up with database migration project creation… I made everything, except the last step. I googled a lot, read a lot of AWS and third-party documentation, but none told me how to finally create a data migration. What should I do with migration project without a data migration then? It doesn’t make any sense..
I found how to create endpoints, replication tasks & configuration, but it is another part of DMS I won’t touch. I was trying to find smth using key words like “create data migration” + “migration project” + “aws cli” but, again, every documentation blog ends up with data migration project creation only
Any hint will be really appreciated!
Anything like aws dms create-data-migration --migration-project <project_name> … =(