Skip to content

Generate spec-first FastAPI Server #18886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 113 commits into from
Nov 10, 2022
Merged

Generate spec-first FastAPI Server #18886

merged 113 commits into from
Nov 10, 2022

Conversation

sherifnada
Copy link
Contributor

What

Closes #17813

Generates a FastAPI server from our connector builder OpenAPI spec.

How

Uses openAPI custom templates to generate a working spec-first FastAPI server.

connector_builder/generated/ contains everything autogenerated
connector_builder/generated/models/ contains pydantic types
connector_builder/generated/apis contains API contracts
connector_builder/impl contains API implementations
openapi contains templates and configuration needed for the OpenAPI generator

Recommended reading order

  1. connector_builder/generated/apis/default_api_interface.py (this is the API contract generated by the template at openapi/templates/api_interfaces.mustache)
  2. connector_builder/impl/default_api.py <-- API implementation
  3. entrypoint.py is where the server is configured
  4. build.gradle
  5. README.md
  6. take a quick look at connector_builder/generated/models/ just to get an idea for what the types look like

Next steps

  • Wire up codegen diff checks in CI (will do in a different PR)

@sherifnada sherifnada temporarily deployed to more-secrets November 6, 2022 22:20 Inactive
Copy link
Contributor

@brianjlai brianjlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny formatting things but looks good to me!

# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny nit: extra line


from {{modelPackage}}.extra_models import TokenModel # noqa: F401


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra line i n between these imports, i also saw it in the generated file, but presumably the template will be rerun before checkin

Base automatically changed from alex/connector_builder_fastapi to master November 9, 2022 21:07
@girarda girarda requested a review from a team as a code owner November 9, 2022 21:07
@@ -0,0 +1,3 @@
.coverage
.venv
state_*.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want to add connector_builder/generated to this .gitignore, right? Since I don't think we want to be committing generated files to github, as that would require people to always commit the generator output to github

@github-actions github-actions bot added area/platform issues related to the platform kubernetes labels Nov 10, 2022
@sherifnada sherifnada temporarily deployed to more-secrets November 10, 2022 00:46 Inactive
@github-actions
Copy link
Contributor

github-actions bot commented Nov 10, 2022

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector version
  • Add changelog
  • Publish the new version

❌ Sources (63)

Connector Version Changelog Publish
source-airtable 0.1.3
source-amazon-ads 0.1.25
source-amazon-seller-partner 0.2.28
source-amazon-sqs 0.1.0
source-amplitude 0.1.17
source-appsflyer 0.1.0
(doc not found)

(not in seed)
source-asana 0.1.4
source-azure-table 0.1.3
source-braintree 0.1.3
source-cart 0.2.0
(changelog missing)
source-chargebee 0.1.16
(diff seed version)
source-commercetools 0.1.0
source-confluence 0.1.1
source-datadog 0.1.0
source-delighted 0.1.4
source-drift 0.2.5
source-facebook-marketing 0.2.72
source-facebook-pages 0.1.6
source-freshcaller 0.1.0
source-freshsales 0.1.2
source-freshservice 0.1.1
source-github 0.3.7
source-gitlab 0.1.6
source-google-ads 0.2.3
source-google-search-console 0.1.18
source-greenhouse 0.3.0
source-harvest 0.1.11
source-instagram 1.0.0
source-iterable 0.1.21
source-klaviyo 0.1.10
source-lemlist 0.1.1
(changelog missing)

(diff seed version)
source-lever-hiring 0.1.3
source-linnworks 0.1.5
(changelog missing)
source-mailchimp 0.2.15
source-mailgun 0.1.0
(doc not found)
source-monday 0.1.4
source-notion 0.1.10
source-okta 0.1.13
source-onesignal 0.1.2
source-openweather 0.1.6
source-outreach 0.1.2
source-pardot 0.1.0
(doc not found)

(not in seed)
source-paystack 0.1.1
source-pinterest 0.1.9
source-pipedrive 0.1.13
source-plaid 0.3.2
(changelog missing)
source-posthog 0.1.7
source-prestashop 0.2.0
(doc not found)
source-quickbooks-singer 0.1.5
(doc not found)
source-recharge 0.2.4
(changelog missing)
source-retently 0.1.2
source-salesforce 1.0.24
source-salesloft 0.1.3
source-sendgrid 0.2.16
source-sentry 0.1.7
source-strava 0.1.2
source-surveymonkey 0.1.11
source-tplcentral 0.1.1
(not in seed)
source-twilio 0.1.13
source-youtube-analytics 0.1.3
source-zendesk-sunshine 0.1.1
source-zendesk-talk 0.1.5
source-zenloop 0.1.3
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (1)

Connector Version Changelog Publish
destination-snowflake 0.4.39
  • See "Actionable Items" below for how to resolve warnings and errors.

Actionable Items

(click to expand)

Category Status Actionable Item
Version
mismatch
The version of the connector is different from its normal variant. Please bump the version of the connector.

doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.

changelog missing
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish
not in seed
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.

diff seed version
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@sherifnada sherifnada temporarily deployed to more-secrets November 10, 2022 00:58 Inactive
@github-actions github-actions bot removed kubernetes area/platform issues related to the platform labels Nov 10, 2022
@sherifnada sherifnada temporarily deployed to more-secrets November 10, 2022 01:05 Inactive
@sherifnada sherifnada merged commit 125f35f into master Nov 10, 2022
@sherifnada sherifnada deleted the sherif/gen-fastapi branch November 10, 2022 02:14
akashkulk pushed a commit that referenced this pull request Dec 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generate python connector builder API server code
5 participants