SCC

Brasil

os cloud gurus

Software Cloud Consulting

Your software development, cloud, consulting & shoring company

Copy data from a DynamoDB table to another existing DynamoDB table


7

By Daniel Nascimento


AWS gives some simple options to copy data from one dynamodb to a new dynamodb table,
like using export a csv file and importing it to a new table,
but unfortunatly the situation is not so simple if your use case needs to put the data into an existing DynamoDB Table,
to overcome this problem, we created a simple lambda that copy data from one dynamodb table to another in the same account.

This lambda normally is used as a Adhoc process
so we create an Cloudformation template to deploy this lambda when we need it, and destroy the resources easily.

The lambda uses aws sdk, to scan the source table and write the result in batches of 20 items to the destination table.
The table names are passed to the Lambda using Environment Variables, that are set when you deploy the Cloudformation template.

  • The Lambda and Cloudformation code are in the Yaml file in this repo:

    The repo for the Copy DynamoDB Lambda :
    Git Copy DynamoDB Lambda


  • To deploy the Cloudfomation template, access your aws account and go to Cloudformation > Create Stack, and upload the dynamo-copy-lambda.yaml file from the repo.

  • SMM

  • Create a new Stack and Fill the parameters

  • SMM

  • After created you can go to the lambda console and test it manually with and empty test. This will copy all data from the Source table to the Destination Table.

  • SMM

  • PS: If you need to change the SourceTable or DestinationTable you need to Update the stack with the new table names in the parameters.


  • This is a simple solution to have on hand while aws doesn't provide a out of the box solution.
    I hope this was helpful. Thank you.

  • Back to Blog Overview
  • Autor


    ...

    Daniel do Nascimento

    Cloud Engineer

    3 x AWS Certified

    Terraform Certified