Skip to content

Commit 37af2f0

Browse files
source-google-tasks contribution from btkcodedev (#45427)
Co-authored-by: Octavia Squidington III <[email protected]>
1 parent a85de6f commit 37af2f0

File tree

6 files changed

+449
-0
lines changed

6 files changed

+449
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Google Tasks
2+
This directory contains the manifest-only connector for `source-google-tasks`.
3+
4+
## Documentation reference:
5+
Visit `https://developers.google.com/tasks/reference/rest` for API documentation
6+
7+
## Authentication setup
8+
`Source-productive` uses bearer token authentication,
9+
Visit `https://support.google.com/googleapi/answer/6158849?hl=en&amp;ref_topic=7013279` for getting bearer token via OAuth2.0
10+
11+
## Setting postman for getting bearer token
12+
Currently Code granted OAuth 2.0 is not directly supported by airbyte, thus you could setup postman for getting the bearer token which could be used as `api_key`,
13+
Steps:
14+
- Visit google cloud `https://console.cloud.google.com/apis/api/tasks.googleapis.com/metrics` and enable the tasks api service
15+
- Go to the consent screen `https://console.cloud.google.com/apis/credentials/consent` and add your email for enabling postman testing access
16+
- Visit `https://console.cloud.google.com/apis/credentials` and create new credentails for OAuth 2.0 and copy client id and client secret
17+
- Add callback url `https://oauth.pstmn.io/v1/callback` while credential creation
18+
- Goto postman client and select new tab for setting authorization to OAuth 2.0
19+
- Set scope as `https://www.googleapis.com/auth/tasks https://www.googleapis.com/auth/tasks.readonly`
20+
- Set access token URL as `https://accounts.google.com/o/oauth2/token`
21+
- Set auth URL as `https://accounts.google.com/o/oauth2/v2/auth`
22+
- Click `Get New Access Token` and authorize via your google account
23+
- Copy the resulted bearer token and use it as credential for the connector
24+
25+
## Usage
26+
There are multiple ways to use this connector:
27+
- You can use this connector as any other connector in Airbyte Marketplace.
28+
- You can load this connector in `pyairbyte` using `get_source`!
29+
- You can open this connector in Connector Builder, edit it, and publish to your workspaces.
30+
31+
Please refer to the manifest-only connector documentation for more details.
32+
33+
## Local Development
34+
We recommend you use the Connector Builder to edit this connector.
35+
36+
But, if you want to develop this connector locally, you can use the following steps.
37+
38+
### Environment Setup
39+
You will need `airbyte-ci` installed. You can find the documentation [here](airbyte-ci).
40+
41+
### Build
42+
This will create a dev image (`source-google-tasks:dev`) that you can use to test the connector locally.
43+
```bash
44+
airbyte-ci connectors --name=source-google-tasks build
45+
```
46+
47+
### Test
48+
This will run the acceptance tests for the connector.
49+
```bash
50+
airbyte-ci connectors --name=source-google-tasks test
51+
```
52+
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
2+
# for more information about how to configure these tests
3+
connector_image: airbyte/source-google-tasks:dev
4+
acceptance_tests:
5+
spec:
6+
tests:
7+
- spec_path: "manifest.yaml"
8+
connection:
9+
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
10+
discovery:
11+
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
12+
basic_read:
13+
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
14+
incremental:
15+
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
16+
full_refresh:
17+
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
Loading
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
version: 4.6.2
2+
3+
type: DeclarativeSource
4+
5+
description: >-
6+
Website: https://tasksboard.com/app
7+
8+
API Docs: https://developers.google.com/tasks/reference/rest
9+
10+
Auth Docs:
11+
https://support.google.com/googleapi/answer/6158849?hl=en&ref_topic=7013279
12+
13+
check:
14+
type: CheckStream
15+
stream_names:
16+
- tasks
17+
18+
definitions:
19+
streams:
20+
tasks:
21+
type: DeclarativeStream
22+
name: tasks
23+
retriever:
24+
type: SimpleRetriever
25+
paginator:
26+
type: DefaultPaginator
27+
page_token_option:
28+
type: RequestOption
29+
field_name: pageToken
30+
inject_into: request_parameter
31+
pagination_strategy:
32+
type: CursorPagination
33+
cursor_value: "{{ response.get('nextPageToken') }}"
34+
stop_condition: "{{ not response.get('nextPageToken') }}"
35+
requester:
36+
$ref: "#/definitions/base_requester"
37+
path: users/@me/lists
38+
http_method: GET
39+
error_handler:
40+
type: CompositeErrorHandler
41+
error_handlers:
42+
- type: DefaultErrorHandler
43+
max_retries: 5
44+
response_filters:
45+
- type: HttpResponseFilter
46+
action: RATE_LIMITED
47+
http_codes:
48+
- 429
49+
error_message: Rate limit hit
50+
backoff_strategies:
51+
- type: ExponentialBackoffStrategy
52+
factor: 2
53+
request_parameters:
54+
maxResults: "{{ config['records_limit'] }}"
55+
record_selector:
56+
type: RecordSelector
57+
extractor:
58+
type: DpathExtractor
59+
field_path:
60+
- items
61+
primary_key:
62+
- id
63+
schema_loader:
64+
type: InlineSchemaLoader
65+
schema:
66+
$ref: "#/schemas/tasks"
67+
incremental_sync:
68+
type: DatetimeBasedCursor
69+
cursor_field: updated
70+
end_datetime:
71+
type: MinMaxDatetime
72+
datetime: "{{ now_utc().strftime('%Y-%m-%dT%H:%M:%SZ') }}"
73+
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
74+
start_datetime:
75+
type: MinMaxDatetime
76+
datetime: '{{ config["start_date"] }}'
77+
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
78+
datetime_format: "%Y-%m-%dT%H:%M:%S.%fZ"
79+
cursor_datetime_formats:
80+
- "%Y-%m-%dT%H:%M:%S.%fZ"
81+
lists_tasks:
82+
type: DeclarativeStream
83+
name: lists_tasks
84+
retriever:
85+
type: SimpleRetriever
86+
paginator:
87+
type: DefaultPaginator
88+
page_token_option:
89+
type: RequestOption
90+
field_name: pageToken
91+
inject_into: request_parameter
92+
pagination_strategy:
93+
type: CursorPagination
94+
cursor_value: "{{ response.get('nextPageToken') }}"
95+
stop_condition: "{{ not response.get('nextPageToken') }}"
96+
requester:
97+
$ref: "#/definitions/base_requester"
98+
path: lists/{{ stream_partition.tasklist_id }}/tasks
99+
http_method: GET
100+
error_handler:
101+
type: CompositeErrorHandler
102+
error_handlers:
103+
- type: DefaultErrorHandler
104+
max_retries: 5
105+
response_filters:
106+
- type: HttpResponseFilter
107+
action: RATE_LIMITED
108+
http_codes:
109+
- 429
110+
error_message: Rate limit hit
111+
backoff_strategies:
112+
- type: ExponentialBackoffStrategy
113+
factor: 2
114+
request_parameters:
115+
maxResults: "{{ config['records_limit'] }}"
116+
record_selector:
117+
type: RecordSelector
118+
extractor:
119+
type: DpathExtractor
120+
field_path:
121+
- items
122+
partition_router:
123+
type: SubstreamPartitionRouter
124+
parent_stream_configs:
125+
- type: ParentStreamConfig
126+
stream:
127+
$ref: "#/definitions/streams/tasks"
128+
parent_key: id
129+
partition_field: tasklist_id
130+
primary_key:
131+
- id
132+
schema_loader:
133+
type: InlineSchemaLoader
134+
schema:
135+
$ref: "#/schemas/lists_tasks"
136+
incremental_sync:
137+
type: DatetimeBasedCursor
138+
cursor_field: updated
139+
end_datetime:
140+
type: MinMaxDatetime
141+
datetime: "{{ now_utc().strftime('%Y-%m-%dT%H:%M:%SZ') }}"
142+
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
143+
start_datetime:
144+
type: MinMaxDatetime
145+
datetime: '{{ config["start_date"] }}'
146+
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
147+
datetime_format: "%Y-%m-%dT%H:%M:%S.%fZ"
148+
cursor_datetime_formats:
149+
- "%Y-%m-%dT%H:%M:%S.%fZ"
150+
base_requester:
151+
type: HttpRequester
152+
url_base: https://tasks.googleapis.com/tasks/v1/
153+
authenticator:
154+
type: BearerAuthenticator
155+
api_token: '{{ config["api_key"] }}'
156+
157+
streams:
158+
- $ref: "#/definitions/streams/tasks"
159+
- $ref: "#/definitions/streams/lists_tasks"
160+
161+
spec:
162+
type: Spec
163+
connection_specification:
164+
type: object
165+
$schema: http://json-schema.org/draft-07/schema#
166+
required:
167+
- api_key
168+
- start_date
169+
properties:
170+
api_key:
171+
type: string
172+
order: 1
173+
title: API Key
174+
airbyte_secret: true
175+
start_date:
176+
type: string
177+
order: 2
178+
title: Start date
179+
format: date-time
180+
pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$
181+
records_limit:
182+
type: string
183+
description: The maximum number of records to be returned per request
184+
order: 0
185+
title: Records Limit
186+
default: "50"
187+
additionalProperties: true
188+
189+
metadata:
190+
assist:
191+
docsUrl: https://developers.google.com/tasks/reference/rest
192+
testedStreams:
193+
tasks:
194+
hasRecords: true
195+
streamHash: 06d4c0949ab53aa3ea7e089781332fea60caf2f1
196+
hasResponse: true
197+
primaryKeysAreUnique: true
198+
primaryKeysArePresent: true
199+
responsesAreSuccessful: true
200+
lists_tasks:
201+
hasRecords: true
202+
streamHash: 6d6137d9a88020f99badd5a98968cb1a12e07cdd
203+
hasResponse: true
204+
primaryKeysAreUnique: true
205+
primaryKeysArePresent: true
206+
responsesAreSuccessful: true
207+
autoImportSchema:
208+
tasks: true
209+
lists_tasks: true
210+
211+
schemas:
212+
tasks:
213+
type: object
214+
$schema: http://json-schema.org/schema#
215+
required:
216+
- id
217+
- updated
218+
properties:
219+
id:
220+
type: string
221+
etag:
222+
type:
223+
- string
224+
- "null"
225+
kind:
226+
type:
227+
- string
228+
- "null"
229+
title:
230+
type:
231+
- string
232+
- "null"
233+
updated:
234+
type: string
235+
selfLink:
236+
type:
237+
- string
238+
- "null"
239+
additionalProperties: true
240+
lists_tasks:
241+
type: object
242+
$schema: http://json-schema.org/schema#
243+
required:
244+
- id
245+
- updated
246+
properties:
247+
id:
248+
type: string
249+
due:
250+
type:
251+
- string
252+
- "null"
253+
etag:
254+
type:
255+
- string
256+
- "null"
257+
kind:
258+
type:
259+
- string
260+
- "null"
261+
links:
262+
type:
263+
- array
264+
- "null"
265+
title:
266+
type:
267+
- string
268+
- "null"
269+
status:
270+
type:
271+
- string
272+
- "null"
273+
updated:
274+
type: string
275+
position:
276+
type:
277+
- string
278+
- "null"
279+
selfLink:
280+
type:
281+
- string
282+
- "null"
283+
completed:
284+
type:
285+
- string
286+
- "null"
287+
webViewLink:
288+
type:
289+
- string
290+
- "null"
291+
additionalProperties: true

0 commit comments

Comments
 (0)