This repository contains the code used by the ReplaceRedshiftData-qa and ReplaceRedshiftData-production AWS lambda functions. It copies data into a table while ensuring some key remains unique. For now, that key is hardcoded to be the patron_id, but it would be easy to generalize if necessary. Given a staging table and a main table, this function does the following:
- Checks that all rows in the staging table have a unique value for the specified column. If not and all the duplicate rows are the same, deletes all but one of these rows from the staging table. If the duplicate rows are different, throws an error.
- Deletes all rows in the main table that match rows in the staging table for a specified column
- Copies over all the rows from the staging table into the main table
- Deletes all rows in the staging table
This repo uses the Main-QA-Production git workflow.
main has the latest and greatest commits, qa has what's in our QA environment, and production has what's in our production environment.
CI/CD is not enabled. To deploy a new version of this function, first modify the code in the git repo and open a pull request to the appropriate environment branch. Then run source deployment_script.sh and upload the resulting zip. Note that if any files are added or deleted, this script must be modified. For more information, see the directions here.