Source mailersend: Migrate to manifest-only (#44766)

Author: ChristoGrab

airbyte-integrations/connectors/source-mailersend/README.md

@@ -1,89 +1,63 @@
 # Mailersend source connector
 
+This directory contains the manifest-only connector for `source-mailersend`.
+This _manifest-only_ connector is not a Python package on its own, as it runs inside of the base `source-declarative-manifest` image.
 
-This is the repository for the Mailersend source connector, written in Python.
-For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.com/integrations/sources/mailersend).
+For information about how to configure and use this connector within Airbyte, see [the connector's full documentation](https://docs.airbyte.com/integrations/sources/mailersend).
 
 ## Local development
 
-### Prerequisites
-* Python (~=3.9)
-* Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
+We recommend using the Connector Builder to edit this connector.
+Using either Airbyte Cloud or your local Airbyte OSS instance, navigate to the **Builder** tab and select **Import a YAML**.
+Then select the connector's `manifest.yaml` file to load the connector into the Builder. You're now ready to make changes to the connector!
 
+If you prefer to develop locally, you can follow the instructions below.
 
-### Installing the connector
-From this connector directory, run:
-```bash
-poetry install --with dev
-```
-
-
-### Create credentials
-**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/mailersend)
-to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_mailersend/spec.yaml` file.
-Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
-See `sample_files/sample_config.json` for a sample config file.
-
-
-### Locally running the connector
-```
-poetry run source-mailersend spec
-poetry run source-mailersend check --config secrets/config.json
-poetry run source-mailersend discover --config secrets/config.json
-poetry run source-mailersend read --config secrets/config.json --catalog sample_files/configured_catalog.json
-```
+### Building the docker image
 
-### Running unit tests
-To run unit tests locally, from the connector directory run:
-```
-poetry run pytest unit_tests
-```
+You can build any manifest-only connector with `airbyte-ci`:
 
-### Building the docker image
 1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
 2. Run the following command to build the docker image:
+
 ```bash
 airbyte-ci connectors --name=source-mailersend build
 ```
 
 An image will be available on your host with the tag `airbyte/source-mailersend:dev`.
 
+### Creating credentials
+
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/mailersend)
+to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `spec` object in the connector's `manifest.yaml` file.
+Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
 
 ### Running as a docker container
-Then run any of the connector commands as follows:
-```
+
+Then run any of the standard source connector commands:
+
+```bash
 docker run --rm airbyte/source-mailersend:dev spec
 docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-mailersend:dev check --config /secrets/config.json
 docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-mailersend:dev discover --config /secrets/config.json
 docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-mailersend:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
 ```
 
-### Running our CI test suite
-You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
-```bash
-airbyte-ci connectors --name=source-mailersend test
-```
+### Running the CI test suite
 
-### Customizing acceptance Tests
-Customize `acceptance-test-config.yml` file to configure acceptance tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
-If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
 
-### Dependency Management
-All of your dependencies should be managed via Poetry. 
-To add a new dependency, run:
 ```bash
-poetry add <package-name>
+airbyte-ci connectors --name=source-mailersend test
 ```
 
-Please commit the changes to `pyproject.toml` and `poetry.lock` files.
-
 ## Publishing a new version of the connector
-You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
-1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-mailersend test`
-2. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)): 
+
+If you want to contribute changes to `source-mailersend`, here's how you can do that:
+1. Make your changes locally, or load the connector's manifest into Connector Builder and make changes there.
+2. Make sure your changes are passing our test suite with `airbyte-ci connectors --name=source-mailersend test`
+3. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)):
     - bump the `dockerImageTag` value in in `metadata.yaml`
-    - bump the `version` value in `pyproject.toml`
-3. Make sure the `metadata.yaml` content is up to date.
 4. Make sure the connector documentation and its changelog is up to date (`docs/integrations/sources/mailersend.md`).
 5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
 6. Pat yourself on the back for being an awesome contributor.

airbyte-integrations/connectors/source-mailersend/__init__.py

@@ -4,7 +4,7 @@ connector_image: airbyte/source-mailersend:dev
 acceptance_tests:
   spec:
     tests:
-      - spec_path: "source_mailersend/spec.yaml"
+      - spec_path: "manifest.yaml"
   connection:
     tests:
       - config_path: "secrets/config.json"

airbyte-integrations/connectors/source-mailersend/acceptance-test-config.yml

@@ -0,0 +1,192 @@
+version: 4.5.4
+
+type: DeclarativeSource
+
+check:
+  type: CheckStream
+  stream_names:
+    - activity
+
+definitions:
+  streams:
+    activity:
+      type: DeclarativeStream
+      name: activity
+      primary_key:
+        - id
+      retriever:
+        type: SimpleRetriever
+        requester:
+          $ref: "#/definitions/base_requester"
+          path: /activity/{{ config['domain_id'] }}
+          http_method: GET
+          request_parameters:
+            limit: "100"
+        record_selector:
+          type: RecordSelector
+          extractor:
+            type: DpathExtractor
+            field_path:
+              - data
+        paginator:
+          type: DefaultPaginator
+          page_token_option:
+            type: RequestOption
+            inject_into: request_parameter
+            field_name: page
+          page_size_option:
+            type: RequestOption
+            inject_into: request_parameter
+            field_name: page_size
+          pagination_strategy:
+            type: PageIncrement
+            page_size: 100
+      incremental_sync:
+        type: DatetimeBasedCursor
+        cursor_field: created_at
+        cursor_datetime_formats:
+          - "%s"
+        datetime_format: "%s"
+        start_datetime:
+          type: MinMaxDatetime
+          datetime: "{{ config['start_date'] }}"
+          datetime_format: "%s"
+        start_time_option:
+          type: RequestOption
+          field_name: date_from
+          inject_into: request_parameter
+        end_datetime:
+          type: MinMaxDatetime
+          datetime: "{{ now_utc().strftime('%s') }}"
+          datetime_format: "%s"
+        step: P1D
+        cursor_granularity: PT1S
+      schema_loader:
+        type: InlineSchemaLoader
+        schema:
+          $ref: "#/schemas/activity"
+  base_requester:
+    type: HttpRequester
+    url_base: https://api.mailersend.com/v1
+    authenticator:
+      type: BearerAuthenticator
+      api_token: '{{ config["api_token"] }}'
+
+streams:
+  - $ref: "#/definitions/streams/activity"
+
+spec:
+  type: Spec
+  connection_specification:
+    type: object
+    $schema: http://json-schema.org/draft-07/schema#
+    required:
+      - api_token
+      - domain_id
+    properties:
+      api_token:
+        type: string
+        description: >-
+          Your API Token. See <a
+          href="https://www.mailersend.com/help/managing-api-tokens">here</a>.
+        airbyte_secret: true
+        order: 0
+      domain_id:
+        type: string
+        description: The domain entity in mailersend
+        examples:
+          - airbyte.com
+          - linkana.com
+        order: 1
+      start_date:
+        type: number
+        description: Timestamp is assumed to be UTC.
+        examples:
+          - 123131321
+        title: "1668311800"
+        order: 2
+    additionalProperties: true
+
+metadata:
+  autoImportSchema:
+    activity: false
+  testedStreams: {}
+  assist: {}
+
+schemas:
+  activity:
+    type: object
+    $schema: http://json-schema.org/draft-07/schema#
+    additionalProperties: true
+    properties:
+      type:
+        type:
+          - string
+      created_at:
+        type:
+          - string
+        format: date-time
+      email:
+        type:
+          - object
+        properties:
+          created_at:
+            type:
+              - string
+            format: date-time
+          from:
+            type:
+              - string
+          html:
+            type:
+              - "null"
+              - string
+          id:
+            type:
+              - string
+          recipient:
+            type:
+              - object
+            properties:
+              created_at:
+                type:
+                  - string
+                format: date-time
+              deleted_at:
+                type:
+                  - string
+              email:
+                type:
+                  - string
+              id:
+                type:
+                  - string
+              updated_at:
+                type:
+                  - string
+                format: date-time
+          status:
+            type:
+              - string
+          subject:
+            type:
+              - string
+          tags:
+            type:
+              - "null"
+              - string
+          text:
+            type:
+              - "null"
+              - string
+          updated_at:
+            type:
+              - string
+            format: date-time
+      id:
+        type:
+          - string
+      updated_at:
+        type:
+          - string
+        format: date-time

airbyte-integrations/connectors/source-mailersend/main.py

@@ -2,15 +2,15 @@ data:
   connectorSubtype: api
   connectorType: source
   definitionId: 2707d529-3c04-46eb-9c7e-40d4038df6f7
-  dockerImageTag: 0.1.15
+  dockerImageTag: 0.2.0
   dockerRepository: airbyte/source-mailersend
   githubIssueLabel: source-mailersend
   icon: mailersend.svg
   license: MIT
   name: MailerSend
   remoteRegistries:
     pypi:
-      enabled: true
+      enabled: false
       packageName: airbyte-source-mailersend
   registryOverrides:
     cloud:
@@ -20,8 +20,8 @@ data:
   releaseStage: alpha
   documentationUrl: https://docs.airbyte.com/integrations/sources/mailersend
   tags:
-    - language:python
     - cdk:low-code
+    - language:manifest-only
   ab_internal:
     sl: 100
     ql: 100
@@ -38,5 +38,5 @@ data:
   #           type: GSM
   #           alias: airbyte-connector-testing-secret-store
   connectorBuildOptions:
-    baseImage: docker.io/airbyte/python-connector-base:2.0.0@sha256:c44839ba84406116e8ba68722a0f30e8f6e7056c726f447681bb9e9ece8bd916
+    baseImage: docker.io/airbyte/source-declarative-manifest:4.5.4@sha256:b07a521add11f987c63c0db68c1b57e90bec0c985f1cb6f3c5a1940cde628a70
 metadataSpecVersion: "1.0"