-
Notifications
You must be signed in to change notification settings - Fork 4.5k
🎉 New Source: Appsflyer #6497
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🎉 New Source: Appsflyer #6497
Conversation
Unit test result:
|
Unit test result:
|
Acceptance test result:
|
Hey @asyarif93 thanks for your contribution 🔥 . We are currently focusing cloud launch we will be able to get back and review early next week. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Main point is to use CDK transform class instead of custom transformation code. It super easy to config just with one line. Here is details : https://docs.airbyte.io/connector-development/cdk-python/schemas#default-type-transformation
airbyte-integrations/connectors/source-appsflyer/integration_tests/integration_test.py
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-appsflyer/integration_tests/catalog.json
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-appsflyer/source_appsflyer/source.py
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-appsflyer/source_appsflyer/source.py
Show resolved
Hide resolved
airbyte-integrations/connectors/source-appsflyer/source_appsflyer/source.py
Show resolved
Hide resolved
airbyte-integrations/connectors/source-appsflyer/source_appsflyer/source.py
Show resolved
Hide resolved
@asyarif93, thanks for commits. Please write you consideration on using CDK transform class. If it fits to your needs to do transformation? If no, how we can modify Transformer class to use it in you connector? If it would take a lot of time and you want this connector asap we can merge it but create an issue to address this in some near future. |
I still wondering how to transform some data. for date-time and decimal type seems provided transform class can get the job done. for some other cases, e.g. i had to replace Also, seems I have realize that i'm using USD currencies while write fieldname with IDR suffix which is a bug. I have yet to decide, should I force everything in USD or allow user to chose their preference, If the latter, I might need override (edit) seems my airbyte-cdk was before |
022e671
to
0aeae20
Compare
This feature is new, AFAIK o connector using it so far. You can find examples of custom transformation in docs: https://docs.airbyte.io/connector-development/cdk-python/schemas#custom-schema-type-transformation class MyStream(Stream):
...
transformer = Transformer(TransformConfig.DefaultSchemaNormalization | TransformConfig.CustomSchemaNormalization)
...
@transformer.registerCustomTransform
def transform_function(orginal_value: Any, field_schema: Dict[str, Any]) -> Any:
if orginal_value == 'na':
return None
return transformed_value Note that transform_function should not take self argument.
You have to set CDK version in dependencies and add transformer = Transformer(TransformConfig.DefaultSchemaNormalization) to your class member variables (details) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice one, thanks for the commitment and special thanks for great unit tests.
I still need to add Documentation, |
@avida we can merge this ? |
also change currency code to USD
add newline for fields init file
6d38cc2
to
acc9a31
Compare
What
Adding new connector Appsflyer using CDK
Resolves #5718
How
Describe the solution
Recommended reading order
spec.json
source.py
Pre-merge Checklist
Expand the relevant checklist and delete the others.
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/SUMMARY.md
docs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing./publish
command described here