Skip to content

✨ Source dynamodb: Allow role based access #37530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 30, 2024
Merged

✨ Source dynamodb: Allow role based access #37530

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ccb1879
Support role based access for DynamoDb source connector
keiththompson Apr 24, 2024
7c339af
update casing
keiththompson Apr 30, 2024
c0061d5
update config
keiththompson Apr 30, 2024
c4cf7b5
update docs for role based access
keiththompson Apr 30, 2024
502a795
Merge branch 'master' into keith/dynamodb_role_based_access
keiththompson Apr 30, 2024
155e00d
code formatting
keiththompson Apr 30, 2024
45fc2bb
Merge branch 'master' into keith/dynamodb_role_based_access
keiththompson Apr 30, 2024
00e354e
Merge branch 'master' into keith/dynamodb_role_based_access
keiththompson Apr 30, 2024
b7446ab
update spec.json per comments
keiththompson Apr 30, 2024
a08730e
Merge branch 'master' into keith/dynamodb_role_based_access
keiththompson Apr 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-dynamodb/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: 50401137-8871-4c5a-abb7-1f5fda35545a
dockerImageTag: 0.2.3
dockerImageTag: 0.3.0
dockerRepository: airbyte/source-dynamodb
documentationUrl: https://docs.airbyte.com/integrations/sources/dynamodb
githubIssueLabel: source-dynamodb
Expand Down
6 changes: 4 additions & 2 deletions ...source-dynamodb/src/main/java/io/airbyte/integrations/source/dynamodb/DynamodbConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,16 @@ public record DynamodbConfig(
) {

public static DynamodbConfig createDynamodbConfig(JsonNode jsonNode) {
JsonNode access_key_id = jsonNode.get("access_key_id");
JsonNode secret_access_key = jsonNode.get("secret_access_key");
JsonNode endpoint = jsonNode.get("endpoint");
JsonNode region = jsonNode.get("region");
JsonNode attributeNames = jsonNode.get("reserved_attribute_names");
return new DynamodbConfig(
endpoint != null && !endpoint.asText().isBlank() ? URI.create(endpoint.asText()) : null,
region != null && !region.asText().isBlank() ? Region.of(region.asText()) : null,
jsonNode.get("access_key_id").asText(),
jsonNode.get("secret_access_key").asText(),
access_key_id != null && !access_key_id.asText().isBlank() ? access_key_id.asText() : null,
secret_access_key != null && !secret_access_key.asText().isBlank() ? secret_access_key.asText() : null,
attributeNames != null ? Arrays.asList(attributeNames.asText().split("\\s*,\\s*")) : List.of());
}

Expand Down
15 changes: 13 additions & 2 deletions .../source-dynamodb/src/main/java/io/airbyte/integrations/source/dynamodb/DynamodbUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,14 @@
import java.time.Instant;
import java.util.List;
import java.util.Optional;

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import org.apache.commons.lang3.StringUtils;

public class DynamodbUtils {

Expand All @@ -27,10 +32,16 @@ private DynamodbUtils() {

public static DynamoDbClient createDynamoDbClient(final DynamodbConfig dynamodbConfig) {
final var dynamoDbClientBuilder = DynamoDbClient.builder();
AwsCredentialsProvider awsCredentialsProvider;
if (!StringUtils.isBlank(dynamodbConfig.accessKey()) && !StringUtils.isBlank(dynamodbConfig.secretKey())) {
AwsCredentials awsCreds = AwsBasicCredentials.create(dynamodbConfig.accessKey(), dynamodbConfig.secretKey());
awsCredentialsProvider = StaticCredentialsProvider.create(awsCreds);
} else {
awsCredentialsProvider = DefaultCredentialsProvider.create();
}

// configure access credentials
dynamoDbClientBuilder.credentialsProvider(StaticCredentialsProvider.create(
AwsBasicCredentials.create(dynamodbConfig.accessKey(), dynamodbConfig.secretKey())));
dynamoDbClientBuilder.credentialsProvider(awsCredentialsProvider);

if (dynamodbConfig.region() != null) {
dynamoDbClientBuilder.region(dynamodbConfig.region());
Expand Down
1 change: 0 additions & 1 deletion airbyte-integrations/connectors/source-dynamodb/src/main/resources/spec.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Dynamodb Source Spec",
"type": "object",
"required": ["access_key_id", "secret_access_key"],
"additionalProperties": false,
"properties": {
"endpoint": {
Expand Down
19 changes: 10 additions & 9 deletions docs/integrations/sources/dynamodb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,13 @@ This guide describes in details how you can configure the connector to connect w

## Changelog

| Version | Date | Pull Request | Subject |
|:--------| :--------- | :-------------------------------------------------------- |:---------------------------------------------------------------------|
| 0.2.3 | 2024-02-13 | [35232](https://github.com/airbytehq/airbyte/pull/35232) | Adopt CDK 0.20.4 |
| 0.2.2 | 2024-01-24 | [34453](https://github.com/airbytehq/airbyte/pull/34453) | bump CDK version |
| 0.2.1 | 2024-01-03 | [#33924](https://github.com/airbytehq/airbyte/pull/33924) | Add new ap-southeast-3 AWS region |
| 0.2.0 | 18-12-2023 | https://github.com/airbytehq/airbyte/pull/33485 | Remove LEGACY state |
| 0.1.2 | 01-19-2023 | https://github.com/airbytehq/airbyte/pull/20172 | Fix reserved words in projection expression & make them configurable |
| 0.1.1 | 02-09-2023 | https://github.com/airbytehq/airbyte/pull/22682 | Fix build |
| 0.1.0 | 11-14-2022 | https://github.com/airbytehq/airbyte/pull/18750 | Initial version |
| Version | Date | Pull Request | Subject |
|:--------| :--------- | :-------------------------------------------------------- |:-----------------------------------------------------------------------|
| 0.3.0 | 2024-04-24 | [37530](https://github.com/airbytehq/airbyte/pull/37530) | Allow role based access |
| 0.2.3 | 2024-02-13 | [35232](https://github.com/airbytehq/airbyte/pull/35232) | Adopt CDK 0.20.4 |
| 0.2.2 | 2024-01-24 | [34453](https://github.com/airbytehq/airbyte/pull/34453) | bump CDK version |
| 0.2.1 | 2024-01-03 | [#33924](https://github.com/airbytehq/airbyte/pull/33924) | Add new ap-southeast-3 AWS region |
| 0.2.0 | 18-12-2023 | https://github.com/airbytehq/airbyte/pull/33485 | Remove LEGACY state |
| 0.1.2 | 01-19-2023 | https://github.com/airbytehq/airbyte/pull/20172 | Fix reserved words in projection expression & make them configurable |
| 0.1.1 | 02-09-2023 | https://github.com/airbytehq/airbyte/pull/22682 | Fix build |
| 0.1.0 | 11-14-2022 | https://github.com/airbytehq/airbyte/pull/18750 | Initial version |
Loading