Add mParticle Data Subject Request (DSR) API Integration

[andresilva-guardian]

What does this change?

Collecting workspace informationWhat does this change?

This PR introduces a new AWS Lambda function, written in TypeScript, that integrates with the mParticle Data Subject Request (DSR) API. The Lambda automates the submission, tracking, and processing of data subject requests (DSRs) to mParticle, supporting The Guardian’s compliance with privacy regulations such as GDPR and CCPA. This new development was made in the context of this Trello ticket. Key features include:

• Submitting DSRs (access, portability, erasure) to mParticle in OpenDSR format.
• Querying the status of DSRs by request ID.
• Processing status callbacks from mParticle and emitting events as required.
• Setting user attributes to control event forwarding and audience membership during erasure waiting periods.
• Strong input validation using Zod.
• Comprehensive unit tests for all endpoints and logic.

How to test

• Install dependencies:
  Run pnpm install from the root of the repo.

• Build:
  Run pnpm --filter mparticle-api build to build the Lambda.

• Run locally:
  Use npm run run-local -- --file=path/to/request.json to invoke the Lambda locally with a test event.

• Unit tests:
  Run pnpm --filter mparticle-api test to execute all tests for this Lambda.

• Environment variables:
  Ensure mParticle API credentials and other secrets are available via AWS Secrets Manager in deployed environments, or via a .env file for local development.

• Manual testing:
  Use example payloads from the runs/ directory and verify API responses for all endpoints.

How can we measure success?

• All endpoints respond as documented and pass validation.
• DSRs are successfully submitted to mParticle and can be tracked by request ID.
• Status callbacks from mParticle are processed and logged correctly.
• User attributes are set as expected for event forwarding control.
• All unit and integration tests pass.
• No validation or runtime errors are observed in logs.
• The Lambda can be deployed and invoked in both local and AWS environments.

Have we considered potential risks?

• Incorrect handling of DSRs could result in non-compliance with privacy regulations.
• Failure to validate input or process callbacks securely could introduce security vulnerabilities.
• Missing or misconfigured secrets could cause runtime failures.
• Changes to the mParticle API or callback payloads may require updates to validation logic.
• Incomplete test coverage could allow regressions to slip through.
• Errors in setting user attributes could impact downstream event forwarding or audience membership.

All these risks are mitigated by strong input validation, comprehensive testing, and clear error logging as described in the README.

Images

N/A - This is a backend API integration with no visual components.

Accessibility

N/A - This change implements a backend API handler with no user-facing interface elements.

[johnduffell]

this seems to have failed to deploy to PROD
https://riffraff.gutools.co.uk/deployment/view/d8f335a9-093d-4bab-acb2-dd5759628637

[johnduffell]

the new lambda code needed to be copied to S3 before the CDK could be deployed. I did that and now it's all passed https://riffraff.gutools.co.uk/deployment/view/feffc250-fd05-49a7-a7aa-0a338e18aa59