A Comparison of AWS DMS with BryteFlow
AWS DMS (AWS Data Migration Service) or BryteFlow?
BryteFlow partners closely with AWS for data integration. BryteFlow is embedded in the modern cloud eco-system and uses various AWS services in its orchestration, for example EMR clusters on a pay-as-you-go basis, along with its own IP. We have customers who successfully use our solutions together and BryteFlow is sold primarily on the AWS Marketplace. This comparison has specifically been made to address queries we have received from customers recently and others who indicated a comparison between AWS DMS and BryteFlow would be helpful in making an informed decision on which software they should use before they start their project. Check out BryteFlow for AWS ETL
AWS DMS CDC vs BryteFlow CDC
Data Migration to Amazon S3
Data Migration to Snowflake
Data Migration to Amazon Redshift
What is AWS Database Migration Service?
AWS Database Migration Service or AWS DMS is a managed service designed to enable one-time and on-going migrations from multiple database sources to AWS data platforms. It is a low-cost service and you only pay for the compute resources used during the migration process and additional log storage if any. It supports several use cases, homogeneous migrations such as Oracle to Oracle, as well as heterogeneous migrations between different database platforms. It also supports movement of data between SQL, NoSQL, and text based targets.
Learn about AWS Glue as an option for AWS ETL
Using BryteFlow for data migration to AWS
BryteFlow specializes in data movement from various data platforms for Analytics, Machine Learning and reporting use cases. It specialises in moving data to Amazon S3, Redshift and Snowflake, and also supports other destinations like SQL Server, Vertica etc. It specialises in on-going replication or Change Data Capture including initial sync with the primary purpose of delivering the data for Analytics and related use cases. It is completely GUI driven and requires no coding whatsoever. More on BryteFlow for AWS ETL
Advantages and limitations of AWS DMS data migration
AWS DMS supports many more use cases than BryteFlow. It helps with migrations of multiple data sources to multiple AWS data platforms. It is a very cost-effective service and start-up costs are low. However, for Analytics or Reporting use cases for migration to AWS S3, Redshift and Snowflake, there may be significant programming involved, depending on the use case. The costs can add up for the initial coding and for the on-going maintenance of the entire solution. It can also take much longer for data to be ready for analytics or reporting due to the programming involved. AWS DMS Migration: What you need to know
A feature by feature comparison between
AWS DMS and BryteFlow
Data Migration to Amazon S3
AWS DMS data migration to Amazon S3 as compared to BryteFlow’s. Check out BryteFlow for AWS ETL
| Functionality | AWS DMS CDC | BryteFlow CDC |
|---|---|---|
| Deployment options | On Cloud | On-Prem, on Cloud, Hybrid. Flexibility for security or performance |
| Moving data to Amazon S3. | Full table and incremental data capture. | Full table and incremental data capture. Build a Data Lakehouse on Amazon S3 without Hudi or Delta Lake |
| Delivery of data to S3. | Delta files only | Delta files and fully merged data. The software orchestrates EMR to merge incremental data automatically with the previous data on S3. Build an S3 Data Lake in Minutes |
| Delivery of data to S3 with history of every change. | No | Since the software delivers merged data, it can be configured to automatically produce type 2 history of changes on S3 with zero coding. |
| Coding requirements when data is incrementally captured. | Extensive coding required. | No coding required; data is automatically merged on S3. |
| Automated Partitioning and Compression for replicated data on S3. | No | Yes. Partitioning can be done by configured columns automatically. File types and compression supported include – ORC (snappy), ORC(zlib), Parquet(snappy), gzip, bzip2. Bulk Loading Data to Cloud Data Warehouses |
| Automated Data reconciliation between source and S3 | No | Yes. With BryteFlow TruData you can automatically reconcile with counts and checksums to verify your data on S3 with your source systems. |
| Create an S3 Data Lake and a Redshift Data Lake in tandem. | No | Yes. BryteFlow supports this out-of-the-box with updating S3 and Redshift based on changes to the source with no coding required. Data is physically copied to Redshift. Alternatively data can be accessed using Redshift Spectrum. |
| S3 Data Lake can be queried with Amazon Athena automatically. | No | Yes. BryteFlow automatically creates tables in Athena and interfaces with Glue Data Catalog to interface with AWS Lake Formation and make the data available in the AWS Big Data eco system. |
| Large Volumes data extraction from transactional sources. | Limited Support. | Can ingest petabytes of data with high performance. BryteFlow XL Ingest has smart partitioning mechanisms to ingest data in parallel. |
| Serverless | Yes. No servers to manage. | No. The software has to be deployed on customer’s on-prem servers or EC2 and is sold as Private Offers on AWS Marketplace. |
| End to End Monitoring and Alerting. | No | Yes – since it manages the entire pipeline. |
| Data transformation capabilities. | No | Yes – Changed data can be transformed using a GUI and Apache Spark on EMR using BryteFlow Blend. |
| KMS, SSE support. | Yes | Yes. |
Data Migration to Snowflake
AWS DMS data migration to Snowflake as compared to BryteFlow’s.
| Functionality | AWS DMS CDC | BryteFlow CDC |
|---|---|---|
| Deployment options | On Cloud | On-Prem, on Cloud, Hybrid. Flexibility for security or performance |
| Moving data to Snowflake. | Full table and incremental data capture. | Full table and incremental data capture. |
| Delivery of data to Snowflake. | Delivery of delta files to S3 only. | Creates tables on Snowflake and then migrates existing data and then updates Snowflake based on changes to the source. |
| Delivery of data to Snowflake with history of every change. | No | Since the software delivers merged data, it can be configured to automatically produce type 2 history of changes on Snowflake with zero coding. |
| Coding requirements when data is incrementally captured. | Extensive coding required. | No coding required; data is automatically merged on Snowflake. |
| Automated Data reconciliation between source and Snowflake. | No | Yes. With BryteFlow TruData you can automatically reconcile with counts and checksums to verify your data on Snowflake with your source systems. |
| Large Volumes data extraction from transactional sources. | No | Can ingest petabytes of data with high performance to Snowflake. BryteFlow XL Ingest has smart partitioning mechanisms to ingest data in parallel. Bulk Loading Data to Cloud Data Warehouses |
| Serverless | Yes. No servers to manage. | No. The software has to be deployed on customer’s on-prem servers or EC2 and is sold as Private Offers on AWS Marketplace. |
| End to End Monitoring and Alerting. | No | Yes – since it manages the entire pipeline. |
| Data transformation capabilities. | No | Yes – Changed data can be transformed on S3 using a GUI and Apache Spark on EMR using BryteFlow Blend and then copied to Snowflake. |
| KMS, SSE support. | Yes | Yes |
Data Migration to Amazon Redshift
AWS DMS data migration to Amazon Redshift as compared to BryteFlow’s.
| Functionality | AWS DMS CDC | BryteFlow CDC |
|---|---|---|
| Deployment options | On Cloud | On-Prem, on Cloud, Hybrid. Flexibility for security or performance |
| Moving data to Redshift. | Full table and incremental data capture. | Full table and incremental data capture. |
| Delivery of data to Redshift . | Yes | Yes. Creates tables on Redshift , then migrates existing data and then updates Redshift based on changes to the source. |
| Delivery of data to Redshift with history of every change. | No | Software can be configured to automatically produce type 2 history of changes on Redshift with zero coding. |
| Automated Data reconciliation between source and Redshift. | No | Yes. With BryteFlow TruData you can automatically reconcile with counts and checksums to verify your data on Redshift with your source systems. |
| Large Volumes data extraction from transactional sources. | Limited support. | Can ingest petabytes of data with high performance to Redshift. BryteFlow XL Ingest has smart partitioning mechanisms to ingest data in parallel. Bulk Loading Data to Cloud Data Warehouses |
| Serverless | Yes. No servers to manage. | No. The software has to be deployed on customer’s on-prem servers or EC2 and is sold as Private Offers on AWS Marketplace. |
| End to End Monitoring and Alerting. | Yes | Yes |
| Create an S3 data Lake and Redshift Data Lake . | No | Yes. Both can be created in tandem. An EMR cluster is orchestrated by the BryteFlow software to create the S3 data lake and then copy the data to Redshift. The data can be viewed using Redshift Spectrum as well. Build an S3 Data Lake in Minutes |
| Data transformation capabilities. | No | Yes – Changed data can be transformed on S3 using a GUI and Apache Spark on EMR using BryteFlow Blend and then copied to Redshift. |
| KMS, SSE support. | Yes | Yes |