Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source-buzzsprout contribution from btkcodedev #45608

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions airbyte-integrations/connectors/source-buzzsprout/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Buzzsprout
This directory contains the manifest-only connector for `source-buzzsprout`.

## Documentation reference:
Visit `https://github.com/buzzsprout/buzzsprout-api/tree/master/sections` for API documentation

## Authentication setup
`Source-algolia` uses API keys and podcast id for its authentication,
Visit `https://www.buzzsprout.com/my/profile/api` for getting api key and podcast id
Visit `https://github.com/buzzsprout/buzzsprout-api/tree/master?tab=readme-ov-file#authentication` for knowing more about authentication.

## Usage
There are multiple ways to use this connector:
- You can use this connector as any other connector in Airbyte Marketplace.
- You can load this connector in `pyairbyte` using `get_source`!
- You can open this connector in Connector Builder, edit it, and publish to your workspaces.

Please refer to the manifest-only connector documentation for more details.

## Local Development
We recommend you use the Connector Builder to edit this connector.

But, if you want to develop this connector locally, you can use the following steps.

### Environment Setup
You will need `airbyte-ci` installed. You can find the documentation [here](airbyte-ci).

### Build
This will create a dev image (`source-buzzsprout:dev`) that you can use to test the connector locally.
```bash
airbyte-ci connectors --name=source-buzzsprout build
```

### Test
This will run the acceptance tests for the connector.
```bash
airbyte-ci connectors --name=source-buzzsprout test
```

Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-buzzsprout:dev
acceptance_tests:
spec:
tests:
- spec_path: "manifest.yaml"
connection:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
discovery:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
basic_read:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
incremental:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
full_refresh:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
28 changes: 28 additions & 0 deletions airbyte-integrations/connectors/source-buzzsprout/icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
183 changes: 183 additions & 0 deletions airbyte-integrations/connectors/source-buzzsprout/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
version: 4.6.2

type: DeclarativeSource

description: >-
Website: https://www.buzzsprout.com/

API Docs: https://github.com/buzzsprout/buzzsprout-api/tree/master/sections

Auth Docs:
https://github.com/buzzsprout/buzzsprout-api/tree/master?tab=readme-ov-file#authentication

API Keys page: https://www.buzzsprout.com/my/profile/api

check:
type: CheckStream
stream_names:
- episodes

definitions:
streams:
episodes:
type: DeclarativeStream
name: episodes
primary_key:
- id
retriever:
type: SimpleRetriever
requester:
$ref: "#/definitions/base_requester"
path: "{{ config['podcast_id'] }}/episodes.json"
http_method: GET
request_headers:
Authorization: Token token={{ config["api_key"] }}
Content-Type: application/json; charset=utf-8
User-Agent: >-
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36
error_handler:
type: CompositeErrorHandler
error_handlers:
- type: DefaultErrorHandler
max_retries: 3
backoff_strategies:
- type: ExponentialBackoffStrategy
factor: 2
response_filters:
- type: HttpResponseFilter
action: RATE_LIMITED
http_codes:
- 429
error_message: Rate limits hit
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: []
incremental_sync:
type: DatetimeBasedCursor
cursor_field: published_at
cursor_datetime_formats:
- "%Y-%m-%dT%H:%M:%S.%f%z"
datetime_format: "%Y-%m-%dT%H:%M:%S.%f%z"
start_datetime:
type: MinMaxDatetime
datetime: '{{ config["start_date"] }}'
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
end_datetime:
type: MinMaxDatetime
datetime: "{{ now_utc().strftime('%Y-%m-%dT%H:%M:%SZ') }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/episodes"
podcasts:
type: DeclarativeStream
name: podcasts
primary_key:
- id
retriever:
type: SimpleRetriever
requester:
$ref: "#/definitions/base_requester"
path: podcasts.json
http_method: GET
request_headers:
Authorization: Token token={{ config["api_key"] }}
Content-Type: application/json; charset=utf-8
User-Agent: >-
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36
error_handler:
type: CompositeErrorHandler
error_handlers:
- type: DefaultErrorHandler
max_retries: 3
backoff_strategies:
- type: ExponentialBackoffStrategy
factor: 2
response_filters:
- type: HttpResponseFilter
action: RATE_LIMITED
http_codes:
- 429
error_message: Rate limit hit
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: []
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/podcasts"
base_requester:
type: HttpRequester
url_base: https://www.buzzsprout.com/api/

streams:
- $ref: "#/definitions/streams/episodes"
- $ref: "#/definitions/streams/podcasts"

spec:
type: Spec
connection_specification:
type: object
$schema: http://json-schema.org/draft-07/schema#
required:
- api_key
- podcast_id
- start_date
properties:
api_key:
type: string
order: 0
title: API Key
airbyte_secret: true
podcast_id:
type: string
description: Podcast ID found in `https://www.buzzsprout.com/my/profile/api`
title: Podcast ID
order: 1
start_date:
type: string
title: Start date
format: date-time
pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$
order: 2
additionalProperties: true

metadata:
autoImportSchema:
episodes: false
podcasts: false
testedStreams:
episodes:
streamHash: 2cbf7e4baa763d48b3afec49b302f097e82fa6b7
hasResponse: true
responsesAreSuccessful: true
hasRecords: true
primaryKeysArePresent: true
primaryKeysAreUnique: true
podcasts:
streamHash: 220c9469645385daaeea05a4d1821c1b003c4ca4
hasResponse: true
responsesAreSuccessful: true
hasRecords: true
primaryKeysArePresent: true
primaryKeysAreUnique: true
assist: {}

schemas:
episodes:
type: object
$schema: http://json-schema.org/draft-07/schema#
additionalProperties: true
properties: {}
podcasts:
type: object
$schema: http://json-schema.org/draft-07/schema#
additionalProperties: true
properties: {}
35 changes: 35 additions & 0 deletions airbyte-integrations/connectors/source-buzzsprout/metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
metadataSpecVersion: "1.0"
data:
allowedHosts:
hosts:
- "www.buzzsprout.com"
registryOverrides:
oss:
enabled: true
cloud:
enabled: true
remoteRegistries:
pypi:
enabled: false
packageName: airbyte-source-buzzsprout
connectorBuildOptions:
baseImage: docker.io/airbyte/source-declarative-manifest:4.6.2@sha256:f5fcd3d4703b7590b6166a7853c5ed1686731607cd30a159a8c24e2fe2c1ee98
connectorSubtype: api
connectorType: source
definitionId: 6ad23bfc-cb11-4faa-a243-f9ccdb0145cc
dockerImageTag: 0.0.1
dockerRepository: airbyte/source-buzzsprout
githubIssueLabel: source-buzzsprout
icon: icon.svg
license: MIT
name: Buzzsprout
releaseDate: 2024-09-16
releaseStage: alpha
supportLevel: community
documentationUrl: https://docs.airbyte.com/integrations/sources/buzzsprout
tags:
- language:manifest-only
- cdk:low-code
ab_internal:
ql: 100
sl: 100
35 changes: 35 additions & 0 deletions docs/integrations/sources/buzzsprout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Buzzsprout
This page contains the setup guide and reference information for the [Buzzsprout](https://www.buzzsprout.com/) source connector.

## Documentation reference:
Visit `https://github.com/buzzsprout/buzzsprout-api/tree/master/sections` for API documentation

## Authentication setup
`Source-buzzsprout` uses API keys and podcast id for its authentication,
Visit `https://www.buzzsprout.com/my/profile/api` for getting api key and podcast id
Visit `https://github.com/buzzsprout/buzzsprout-api/tree/master?tab=readme-ov-file#authentication` for knowing more about authentication.

## Configuration

| Input | Type | Description | Default Value |
|-------|------|-------------|---------------|
| `api_key` | `string` | API Key. | |
| `podcast_id` | `string` | Podcast ID. Podcast ID found in `my/profile/api` | |
| `start_date` | `string` | Start date. | |

## Streams
| Stream Name | Primary Key | Pagination | Supports Full Sync | Supports Incremental |
|-------------|-------------|------------|---------------------|----------------------|
| episodes | id | No pagination | ✅ | ✅ |
| podcasts | id | No pagination | ✅ | ❌ |

## Changelog

<details>
<summary>Expand to review</summary>

| Version | Date |PR| Subject |
|------------------|------------|---|----------------|
| 0.0.1 | 2024-09-16 |[45608](https://github.com/airbytehq/airbyte/pull/45608)| Initial release by [@btkcodedev](https://github.com/btkcodedev) via Connector Builder|

</details>
Loading