Skip to content

fix(connectors docs): updated connectors to follow standard template #39328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Aug 14, 2024
Merged

fix(connectors docs): updated connectors to follow standard template #39328

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4832af3
updated connectors to follow standard template
darynaishchenko Jun 6, 2024
2941499
Merge branch 'master' into daryna/update-connectors-docs-to-follow-te…
darynaishchenko Jun 11, 2024
7c65c56
added valid link to google-sheets.md
darynaishchenko Jun 11, 2024
cb36bba
fix For Airbyte Cloud/Open Source sections
darynaishchenko Jun 25, 2024
c19788c
Merge branch 'master' into daryna/update-connectors-docs-to-follow-te…
darynaishchenko Jun 25, 2024
3a0ca91
Merge branch 'master' into daryna/update-connectors-docs-to-follow-te…
darynaishchenko Aug 12, 2024
c97da7d
updated docs for new connectors with sl=300
darynaishchenko Aug 12, 2024
34e67e3
Merge branch 'master' into daryna/update-connectors-docs-to-follow-te…
darynaishchenko Aug 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions docs/integrations/sources/airtable.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# Airtable

<HideInUI>

This page contains the setup guide and reference information for the [Airtable](https://airtable.com/api) source connector.

</HideInUI>

## Prerequisites

- An active Airtable account
Expand Down Expand Up @@ -40,24 +44,25 @@ This page contains the setup guide and reference information for the [Airtable](
### For Airbyte Cloud:

1. [Log into your Airbyte Cloud](https://cloud.airbyte.com/workspaces) account.
2. In the left navigation bar, click **Sources**. In the top-right corner, click **+new source**.
3. On the Set up the source page, enter the name for the Airtable connector and select **Airtable** from the Source type dropdown.
4. You can use OAuth or a Personal Access Token to authenticate your Airtable account. We recommend using OAuth for Airbyte Cloud.
2. Click Sources and then click + New source.
3. On the Set up the source page, select Airtable from the Source type dropdown.
4. Enter a name for the Airtable connector.
5. You can use OAuth or a Personal Access Token to authenticate your Airtable account. We recommend using OAuth for Airbyte Cloud.

- To authenticate using OAuth, select **OAuth2.0** from the Authentication dropdown click **Authenticate your Airtable account** to sign in with Airtable, select required workspaces you want to sync and authorize your account.
- To authenticate using a Personal Access Token, select **Personal Access Token** from the Authentication dropdown and enter the Access Token for your Airtable account.
:::info
When using OAuth, you may see a `400` or `401` error causing a failed sync. You can re-authenticate your Airtable connector to solve the issue temporarily. We are working on a permanent fix that you can follow [here](https://github.com/airbytehq/airbyte/issues/25278).
:::

5. Click **Set up source**.
6. Click **Set up source**.
<!-- /env:cloud -->

<!-- env:oss -->

### For Airbyte OSS:
### For Airbyte Open Source:

1. Navigate to the Airbyte Open Source dashboard
1. Navigate to the Airbyte Open Source dashboard.
2. In the left navigation bar, click **Sources**. In the top-right corner, click **+new source**.
3. On the Set up the source page, enter the name for the Airtable connector and select **Airtable** from the Source type dropdown.
4. Select **Personal Access Token** from the Authentication dropdown and enter the Access Token for your Airtable account.
Expand All @@ -70,12 +75,12 @@ Please keep in mind that if you start syncing a table via Airbyte, then rename i

## Supported sync modes

The airtable source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts#connection-sync-modes):
The Airtable source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts/#connection-sync-modes):

- [Full Refresh - Overwrite](https://docs.airbyte.com/understanding-airbyte/glossary#full-refresh-sync)
- [Full Refresh - Append](https://docs.airbyte.com/understanding-airbyte/connections/full-refresh-append)

## Supported Tables
## Supported Streams

This source allows you to pull all available tables and bases using `Metadata API` for a given authenticated user. In case you rename or add a column to any existing table, you will need to recreate the source to update the Airbyte catalog.

Expand Down Expand Up @@ -127,6 +132,9 @@ See information about rate limits [here](https://airtable.com/developers/web/api

## Changelog

<details>
<summary>Expand to review</summary>

| Version | Date | Pull Request | Subject |
|:--------|:-----------| :------------------------------------------------------- |:---------------------------------------------------------------------------------------|
| 4.4.0 | 2024-07-16 | [41160](https://github.com/airbytehq/airbyte/pull/41160) | Update CDK version to v3.5.2 |
Expand Down Expand Up @@ -160,3 +168,5 @@ See information about rate limits [here](https://airtable.com/developers/web/api
| 0.1.3 | 2022-10-26 | [18491](https://github.com/airbytehq/airbyte/pull/18491) | Improve schema discovery logic |
| 0.1.2 | 2022-04-30 | [12500](https://github.com/airbytehq/airbyte/pull/12500) | Improve input configuration copy |
| 0.1.1 | 2021-12-06 | [8425](https://github.com/airbytehq/airbyte/pull/8425) | Update title, description fields in spec |

</details>
39 changes: 30 additions & 9 deletions docs/integrations/sources/amazon-ads.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Amazon Ads

This page contains the setup guide and reference information for the Amazon Ads source connector.
<HideInUI>

This page contains the setup guide and reference information for the [Amazon Ads](https://advertising.amazon.com) source connector.

</HideInUI>

## Prerequisites

Expand Down Expand Up @@ -29,13 +33,30 @@ To use the [Amazon Ads API](https://advertising.amazon.com/API/docs/en-us), you

<!-- env:cloud -->

**For Airbyte Cloud:**
#### For Airbyte Cloud:

1. [Log into your Airbyte Cloud](https://cloud.airbyte.com/workspaces) account.
2. In the left navigation bar, click **Sources**. In the top-right corner, click **+ new source**.
3. On the source setup page, select **Amazon Ads** from the Source type dropdown and enter a name for this connector.
4. Click **Authenticate your Amazon Ads account**.
5. Log in and Authorize to the Amazon account.
2. Click Sources and then click + New source.
3. On the Set up the source page, select Amazon Ads from the Source type dropdown.
4. Enter a name for the Amazon Ads connector.
5. Click **Authenticate your Amazon Ads account**.
6. Log in and Authorize to the Amazon account.
7. Select **Region** to pull data from **North America (NA)**, **Europe (EU)**, **Far East (FE)**. See [docs](https://advertising.amazon.com/API/docs/en-us/info/api-overview#api-endpoints) for more details.
8. **Start Date (Optional)** is used for generating reports starting from the specified start date. This should be in YYYY-MM-DD format and not more than 60 days in the past. If a date is not specified, today's date is used. The date is treated in the timezone of the processed profile.
9. **Profile IDs (Optional)** you want to fetch data for. The Amazon Ads source connector supports only profiles with seller and vendor type, profiles with agency type will be ignored. See [docs](https://advertising.amazon.com/API/docs/en-us/concepts/authorization/profiles) for more details.
10. **Marketplace IDs (Optional)** you want to fetch data for. _Note: If Profile IDs are also selected, profiles will be selected if they match the Profile ID **OR** the Marketplace ID._
11. Click **Set up source**.
<!-- /env:cloud -->

<!-- env:oss -->

#### For Airbyte Open Source:

1. Navigate to the Airbyte Open Source dashboard.
2. Set the name for your source.
3. **Client ID** of your Amazon Ads developer application. See [onboarding process](https://advertising.amazon.com/API/docs/en-us/setting-up/overview) for more details.
4. **Client Secret** of your Amazon Ads developer application. See [onboarding process](https://advertising.amazon.com/API/docs/en-us/setting-up/overview) for more details.
5. **Refresh Token**. See [onboarding process](https://advertising.amazon.com/API/docs/en-us/setting-up/overview) for more details.
6. Select **Region** to pull data from **North America (NA)**, **Europe (EU)**, **Far East (FE)**. See [docs](https://advertising.amazon.com/API/docs/en-us/info/api-overview#api-endpoints) for more details.
7. **Start Date (Optional)** is used for generating reports starting from the specified start date. This should be in YYYY-MM-DD format and not more than 60 days in the past. If a date is not specified, today's date is used. The date is treated in the timezone of the processed profile.
8. **Profile IDs (Optional)** you want to fetch data for. The Amazon Ads source connector supports only profiles with seller and vendor type, profiles with agency type will be ignored. See [docs](https://advertising.amazon.com/API/docs/en-us/concepts/authorization/profiles) for more details.
Expand Down Expand Up @@ -63,7 +84,7 @@ If you have only agency profile, please use accounts associated with the profile

## Supported sync modes

The Amazon Ads source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts/#connection-sync-mode):
The Amazon Ads source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts/#connection-sync-modes):

- Full Refresh
- Incremental
Expand Down Expand Up @@ -114,7 +135,7 @@ Report data synchronization only covers the last 60 days - [details](https://adv

Information about expected report generation waiting time can be found [here](https://advertising.amazon.com/API/docs/en-us/get-started/developer-notes).

### Data type mapping
### Data type map

| Integration Type | Airbyte Type |
| :----------------------- | :----------- |
Expand Down Expand Up @@ -157,7 +178,7 @@ Information about expected report generation waiting time can be found [here](ht
| 3.2.0 | 2023-09-18 | [30517](https://github.com/airbytehq/airbyte/pull/30517) | Add suggested streams; fix unexpected column issue |
| 3.1.2 | 2023-08-16 | [29233](https://github.com/airbytehq/airbyte/pull/29233) | Add filter for Marketplace IDs |
| 3.1.1 | 2023-08-28 | [29900](https://github.com/airbytehq/airbyte/pull/29900) | Add 404 handling for no associated with bid ad groups |
| 3.1.0 | 2023-08-08 | [00000](https://github.com/airbytehq/airbyte/pull/00000) | Add `T00030` tactic support for `sponsored_display_report_stream` |
| 3.1.0 | 2023-08-08 | [29212](https://github.com/airbytehq/airbyte/pull/29212) | Add `T00030` tactic support for `sponsored_display_report_stream` |
| 3.0.0 | 2023-07-24 | [27868](https://github.com/airbytehq/airbyte/pull/27868) | Fix attribution report stream schemas |
| 2.3.1 | 2023-07-11 | [28155](https://github.com/airbytehq/airbyte/pull/28155) | Bugfix: validation error when record values are missing |
| 2.3.0 | 2023-07-06 | [28002](https://github.com/airbytehq/airbyte/pull/28002) | Add sponsored_product_ad_group_suggested_keywords, sponsored_product_ad_group_bid_recommendations streams |
Expand Down
59 changes: 46 additions & 13 deletions docs/integrations/sources/azure-blob-storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Azure Blob Storage

This page contains the setup guide and reference information for the Azure Blob Storage source connector.
<HideInUI>

This page contains the setup guide and reference information for the [Azure Blob Storage](https://learn.microsoft.com/en-us/azure/?product=popular) source connector.

</HideInUI>

:::info
Cloud storage may incur egress costs. Egress refers to data that is transferred out of the cloud storage system, such as when you download files or access them from a different location. For more information, see the [Azure Blob Storage pricing guide](https://azure.microsoft.com/en-us/pricing/details/storage/blobs/).
Expand Down Expand Up @@ -60,25 +64,52 @@ Follow these steps to set up an IAM role:

### Step 2: Set up the Azure Blob Storage connector in Airbyte

1. [Log in to your Airbyte Cloud](https://cloud.airbyte.com/workspaces) account, or navigate to your Airbyte Open Source dashboard.
2. In the left navigation bar, click **Sources**. In the top-right corner, click **+ New source**.
3. Find and select **Azure Blob Storage** from the list of available sources.
4. Enter the name of your Azure **Account**.
5. Enter your Tenant ID and Click **Authenticate your Azure Blob Storage account**.
6. Log in and authorize the Azure Blob Storage account.
7. Enter the name of the **Container** containing your files to replicate.
8. Add a stream
<!-- env:cloud -->

### For Airbyte Cloud:

1. [Log into your Airbyte Cloud](https://cloud.airbyte.com/workspaces) account.
2. Click Sources and then click + New source.
3. On the Set up the source page, select Azure Blob Storage from the Source type dropdown.
4. Enter a name for the Azure Blob Storage connector.
5. Enter the name of your Azure **Account**.
6. Enter your Tenant ID and Click **Authenticate your Azure Blob Storage account**.
7. Log in and authorize the Azure Blob Storage account.
8. Enter the name of the **Container** containing your files to replicate.
9. Add a stream
1. Write the **File Type**
2. In the **Format** box, use the dropdown menu to select the format of the files you'd like to replicate. The supported formats are **CSV**, **Parquet**, **Avro** and **JSONL**. Toggling the **Optional fields** button within the **Format** box will allow you to enter additional configurations based on the selected format. For a detailed breakdown of these settings, refer to the [File Format section](#file-format-settings) below.
3. Give a **Name** to the stream
4. (Optional)—If you want to enforce a specific schema, you can enter a **Input schema**. By default, this value is set to `{}` and will automatically infer the schema from the file\(s\) you are replicating. For details on providing a custom schema, refer to the [User Schema section](#user-schema).
5. Optionally, enter the **Globs** which dictates which files to be synced. This is a regular expression that allows Airbyte to pattern match the specific files to replicate. If you are replicating all the files within your bucket, use `**` as the pattern. For more precise pattern matching options, refer to the [Path Patterns section](#path-patterns) below.
10. (Optional) Enter the endpoint to use for the data replication.
11. (Optional) Enter the desired start date from which to begin replicating data.
<!-- /env:cloud -->

<!-- env:oss -->
### For Airbyte Open Source:

1. Navigate to the Airbyte Open Source dashboard.
2. Click Sources and then click + New source.
3. On the Set up the source page, select Azure Blob Storage from the Source type dropdown.
4. Enter a name for the Azure Blob Storage connector.
5. Enter the name of your Azure **Account**.
6. Enter your Tenant ID and Click **Authenticate your Azure Blob Storage account**.
7. Log in and authorize the Azure Blob Storage account.
8. Enter the name of the **Container** containing your files to replicate.
9. Add a stream
1. Write the **File Type**
2. In the **Format** box, use the dropdown menu to select the format of the files you'd like to replicate. The supported formats are **CSV**, **Parquet**, **Avro** and **JSONL**. Toggling the **Optional fields** button within the **Format** box will allow you to enter additional configurations based on the selected format. For a detailed breakdown of these settings, refer to the [File Format section](#file-format-settings) below.
3. Give a **Name** to the stream
4. (Optional)—If you want to enforce a specific schema, you can enter a **Input schema**. By default, this value is set to `{}` and will automatically infer the schema from the file\(s\) you are replicating. For details on providing a custom schema, refer to the [User Schema section](#user-schema).
5. Optionally, enter the **Globs** which dictates which files to be synced. This is a regular expression that allows Airbyte to pattern match the specific files to replicate. If you are replicating all the files within your bucket, use `**` as the pattern. For more precise pattern matching options, refer to the [Path Patterns section](#path-patterns) below.
9. (Optional) Enter the endpoint to use for the data replication.
10. (Optional) Enter the desired start date from which to begin replicating data.
10. (Optional) Enter the endpoint to use for the data replication.
11. (Optional) Enter the desired start date from which to begin replicating data.
<!-- /env:oss -->

## Supported Streams
## Supported sync modes

The Azure Blob Storage source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts#connection-sync-modes):
The Azure Blob Storage source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts/#connection-sync-modes):

| Feature | Supported? |
| :--------------------------------------------- | :--------- |
Expand All @@ -89,6 +120,8 @@ The Azure Blob Storage source connector supports the following [sync modes](http
| Replicate Multiple Streams \(distinct tables\) | Yes |
| Namespaces | No |

## Supported Streams

### File Compressions

| Compression | Supported? |
Expand Down
Loading
Loading