✨ Source Surveymonkey: Stream survey_responses add field metadata.respondent.language

[leo-schick]

What

Sychronizing field metadata.respondent.language inside stream survey_responses

Example values returned from API request https://api.surveymonkey.com/v3/surveys/{survey_id}/responses/{response_id}

How

Expanding the schema definitino for stream survey_responses

🚨 User Impact 🚨

Extension of schema fields, no breaking changes.

Pre-merge Actions

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• If new credentials are required for use in CI, add them to GSM. Instructions.

[github-actions]

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

• PR name follows PR naming conventions
• Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan and you've followed all steps in the Breaking Changes Checklist
• Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
• Secrets in the connector's spec are annotated with airbyte_secret
• All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
• Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
• The connector tests are passing in CI
• You've updated the connector's metadata.yaml file (new!)
• If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

1. Check for hidden checklists in your PR description

2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

[marcosmarxm]

@leo-schick please move from draft and let us know when this is ready to review! thanks for the contribution

[leo-schick]

Hello @marcosmarxm ,

I figured out that the response from Survey Monkey differs in the metadata section dependent on the request send to the API:
When you request a single response (GET /surveys/{survey_id}/responses/{response_id}), you get the metadata as shown above. The connector uses the bulk response (GET /surveys/{id}/responses/bulk), one does not get the metadata.

It is a pitty that the API from SurveyMonkey does not work equal here. When we would use the single response request, this would create a huge load on the API and many Airbyters would be required to book the enterprise plan which is quite expensive compared to the other plans. I am not sure how to get forward with this issue.

Three options I have in mind:

1. do not change it (update connector documentation that getting metadata extended fields is not supported)
2. change the used API call to a single call instead of bulk call
3. add a config for the connector if metadata should be downloaded with a hint that high number of API requests will be send

[marcosmarxm]

@leo-schick is this ready to review?

[leo-schick]

@marcosmarxm Actually not. This needs architectual decision:

What I did in this PR is just extending the schema of the stream. But the used API endpoint currently does not delivere the data. It would be necessary to switch the endpoint (see above), what would result in many more API requests - which probably will break existing installations.

I had the idea that it could be part of the configuration which endpoint to use. Or alternatively to claim at SurveyMonkey that these API endpoints should work the same.

Nevertheless for myself I found an alternative solution and I do not require the extra configured metadata. This turned my priority away from this issue.

Maybe someone else could take up this issue when they have a need for it.

[marcosmarxm]

@leo-schick I'll close because I'm trying to clean the community contribution backlog. I'll move the discussion first to an issue.