Skip to content

Commit d616279

Browse files
vladimir-remarmarcosmarxm
vladimir-remar
and
marcosmarxm
authored
Source hubspot: add Feedback Submissions stream and update Ticket Pipelines stream (#9081)
* update: on api.py update url for ticket_pipelines * update: on client.py add new entries for feedback_submissions streams and ticket_pipelines * update: add to shchemas feedback_submissions and ticket_pipelines * update: to integration_tests_abnormal_state add new value for feedback_submissions * update: in sample_files add new entries for feedback_submissions and ticket_pipelines * update: bump connector version * fix: in feedback submission replace timestamp to updatedAt * update version in source_definitons and Dockerfile * run format * config seed * bump connector version in config file Co-authored-by: Marcos Marx <[email protected]>
1 parent f5beea1 commit d616279

File tree

13 files changed

+227
-10
lines changed

13 files changed

+227
-10
lines changed

‎airbyte-config/init/src/main/resources/config/STANDARD_SOURCE_DEFINITION/36c891d9-4bd9-43ac-bad2-10e12756272c.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"sourceDefinitionId": "36c891d9-4bd9-43ac-bad2-10e12756272c",
33
"name": "HubSpot",
44
"dockerRepository": "airbyte/source-hubspot",
5-
"dockerImageTag": "0.1.33",
5+
"dockerImageTag": "0.1.35",
66
"documentationUrl": "https://docs.airbyte.io/integrations/sources/hubspot",
77
"icon": "hubspot.svg"
88
}

‎airbyte-config/init/src/main/resources/seed/source_definitions.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -307,7 +307,7 @@
307307
- name: HubSpot
308308
sourceDefinitionId: 36c891d9-4bd9-43ac-bad2-10e12756272c
309309
dockerRepository: airbyte/source-hubspot
310-
dockerImageTag: 0.1.34
310+
dockerImageTag: 0.1.35
311311
documentationUrl: https://docs.airbyte.io/integrations/sources/hubspot
312312
icon: hubspot.svg
313313
sourceType: api

‎airbyte-config/init/src/main/resources/seed/source_specs.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3057,7 +3057,7 @@
30573057
supportsNormalization: false
30583058
supportsDBT: false
30593059
supported_destination_sync_modes: []
3060-
- dockerImage: "airbyte/source-hubspot:0.1.34"
3060+
- dockerImage: "airbyte/source-hubspot:0.1.35"
30613061
spec:
30623062
documentationUrl: "https://docs.airbyte.io/integrations/sources/hubspot"
30633063
connectionSpecification:

‎airbyte-integrations/connectors/source-hubspot/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,5 +34,5 @@ COPY source_hubspot ./source_hubspot
3434
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
3535
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
3636

37-
LABEL io.airbyte.version=0.1.34
37+
LABEL io.airbyte.version=0.1.35
3838
LABEL io.airbyte.name=airbyte/source-hubspot

‎airbyte-integrations/connectors/source-hubspot/acceptance-test-config.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,14 +18,14 @@ tests:
1818
basic_read:
1919
- config_path: "secrets/config.json"
2020
configured_catalog_path: "sample_files/full_refresh_catalog.json"
21-
empty_streams: ["workflows", "form_submissions"]
21+
empty_streams: ["workflows", "form_submissions", "ticket_pipelines"]
2222
- config_path: "secrets/config_oauth.json"
2323
configured_catalog_path: "sample_files/configured_catalog_for_oauth_config.json"
2424
# The `campaigns` stream is empty in this case, because we use a catalog with
2525
# incremental streams: subscription_changes and email_events (it takes a long time to read)
2626
# and therefore the start date is set at 2021-10-10 for `config_oauth.json`,
2727
# but the campaign was created on 2021-01-11
28-
empty_streams: ["campaigns", "workflows", "contacts_list_memberships", "form_submissions"]
28+
empty_streams: ["campaigns", "workflows", "contacts_list_memberships", "form_submissions", "ticket_pipelines"]
2929
incremental:
3030
- config_path: "secrets/config.json"
3131
configured_catalog_path: "sample_files/configured_catalog.json"

‎airbyte-integrations/connectors/source-hubspot/integration_tests/abnormal_state.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@
1414
"email_events": {
1515
"timestamp": "2221-10-12T13:37:56.412000+00:00"
1616
},
17+
"feedback_submissions": {
18+
"updatedAt": "2221-10-12T13:37:56.412000+00:00"
19+
},
1720
"engagements": {
1821
"lastUpdated": 7945393076412
1922
},

‎airbyte-integrations/connectors/source-hubspot/sample_files/configured_catalog.json

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,18 @@
9090
"cursor_field": ["lastUpdated"],
9191
"destination_sync_mode": "append"
9292
},
93+
{
94+
"stream": {
95+
"name": "feedback_submissions",
96+
"json_schema": {},
97+
"supported_sync_modes": ["full_refresh", "incremental"],
98+
"source_defined_cursor": true,
99+
"default_cursor_field": ["updatedAt"]
100+
},
101+
"sync_mode": "incremental",
102+
"cursor_field": ["updatedAt"],
103+
"destination_sync_mode": "append"
104+
},
93105
{
94106
"stream": {
95107
"name": "forms",
@@ -189,6 +201,15 @@
189201
"cursor_field": ["updatedAt"],
190202
"destination_sync_mode": "append"
191203
},
204+
{
205+
"stream": {
206+
"name": "ticket_pipelines",
207+
"json_schema": {},
208+
"supported_sync_modes": ["full_refresh"]
209+
},
210+
"sync_mode": "full_refresh",
211+
"destination_sync_mode": "overwrite"
212+
},
192213
{
193214
"stream": {
194215
"name": "workflows",

‎airbyte-integrations/connectors/source-hubspot/sample_files/full_refresh_catalog.json

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,15 @@
135135
"sync_mode": "full_refresh",
136136
"destination_sync_mode": "overwrite"
137137
},
138+
{
139+
"stream": {
140+
"name": "ticket_pipelines",
141+
"json_schema": {},
142+
"supported_sync_modes": ["full_refresh"]
143+
},
144+
"sync_mode": "full_refresh",
145+
"destination_sync_mode": "overwrite"
146+
},
138147
{
139148
"stream": {
140149
"name": "workflows",

‎airbyte-integrations/connectors/source-hubspot/source_hubspot/api.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -781,10 +781,10 @@ class DealPipelineStream(Stream):
781781
class TicketPipelineStream(Stream):
782782
"""Ticket pipelines, API v1
783783
This endpoint requires the tickets scope.
784-
Docs: https://legacydocs.hubspot.com/docs/methods/pipelines/get_pipelines_for_object_type
784+
Docs: https://developers.hubspot.com/docs/api/crm/pipelines
785785
"""
786786

787-
url = "/crm-pipelines/v1/pipelines/tickets"
787+
url = "/crm/v3/pipelines/tickets"
788788
updated_at_field = "updatedAt"
789789
created_at_field = "createdAt"
790790

‎airbyte-integrations/connectors/source-hubspot/source_hubspot/client.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
MarketingEmailStream,
2525
OwnerStream,
2626
SubscriptionChangeStream,
27+
TicketPipelineStream,
2728
WorkflowStream,
2829
)
2930

@@ -50,6 +51,7 @@ def __init__(self, start_date, credentials, **kwargs):
5051
"deals": DealStream(associations=["contacts"], **common_params),
5152
"email_events": EmailEventStream(**common_params),
5253
"engagements": EngagementStream(**common_params),
54+
"feedback_submissions": CRMObjectIncrementalStream(entity="feedback_submissions", associations=["contacts"], **common_params),
5355
"forms": FormStream(**common_params),
5456
"form_submissions": FormSubmissionStream(**common_params),
5557
"line_items": CRMObjectIncrementalStream(entity="line_item", **common_params),
@@ -58,6 +60,7 @@ def __init__(self, start_date, credentials, **kwargs):
5860
"products": CRMObjectIncrementalStream(entity="product", **common_params),
5961
"subscription_changes": SubscriptionChangeStream(**common_params),
6062
"tickets": CRMObjectIncrementalStream(entity="ticket", **common_params),
63+
"ticket_pipelines": TicketPipelineStream(**common_params),
6164
"workflows": WorkflowStream(**common_params),
6265
}
6366

‎airbyte-integrations/connectors/source-hubspot/source_hubspot/schemas/feedback_submissions.json

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"type": ["null", "object"],
4+
"properties": {
5+
"id": {
6+
"type": ["null", "string"]
7+
},
8+
"properties": {
9+
"type": ["null", "object"],
10+
"properties": {
11+
"hs_all_accessible_team_ids": {
12+
"type": ["null", "string"]
13+
},
14+
"hs_all_assigned_business_unit_ids": {
15+
"type": ["null", "string"]
16+
},
17+
"hs_contact_email_rollup": {
18+
"type": ["null", "string"]
19+
},
20+
"hs_contact_id": {
21+
"type": ["null", "string"]
22+
},
23+
"hs_content": {
24+
"type": ["null", "string"]
25+
},
26+
"hs_created_by_user_id": {
27+
"type": ["null", "string"]
28+
},
29+
"hs_createdate": {
30+
"type": ["null", "string"]
31+
},
32+
"hs_engagement_id": {
33+
"type": ["null", "string"]
34+
},
35+
"hs_form_guid": {
36+
"type": ["null", "string"]
37+
},
38+
"hs_ingestion_id": {
39+
"type": ["null", "string"]
40+
},
41+
"hs_knowledge_article_id": {
42+
"type": ["null", "string"]
43+
},
44+
"hs_lastmodifieddate": {
45+
"type": ["null", "string"]
46+
},
47+
"hs_merged_object_ids": {
48+
"type": ["null", "string"]
49+
},
50+
"hs_object_id": {
51+
"type": ["null", "string"]
52+
},
53+
"hs_response_group": {
54+
"type": ["null", "string"]
55+
},
56+
"hs_submission_name": {
57+
"type": ["null", "string"]
58+
},
59+
"hs_submission_timestamp": {
60+
"type": ["null", "string"]
61+
},
62+
"hs_submission_url": {
63+
"type": ["null", "string"]
64+
},
65+
"hs_survey_channel": {
66+
"type": ["null", "string"]
67+
},
68+
"hs_survey_id": {
69+
"type": ["null", "string"]
70+
},
71+
"hs_survey_name": {
72+
"type": ["null", "string"]
73+
},
74+
"hs_survey_type": {
75+
"type": ["null", "string"]
76+
},
77+
"hs_unique_creation_key": {
78+
"type": ["null", "string"]
79+
},
80+
"hs_updated_by_user_id": {
81+
"type": ["null", "string"]
82+
},
83+
"hs_user_ids_of_all_notification_followers": {
84+
"type": ["null", "string"]
85+
},
86+
"hs_user_ids_of_all_notification_unfollowers": {
87+
"type": ["null", "string"]
88+
},
89+
"hs_user_ids_of_all_owners": {
90+
"type": ["null", "string"]
91+
},
92+
"hs_value": {
93+
"type": ["null", "string"]
94+
},
95+
"hs_visitor_id": {
96+
"type": ["null", "string"]
97+
}
98+
}
99+
},
100+
"createdAt": {
101+
"type": ["null", "string"],
102+
"format": "date-time"
103+
},
104+
"updatedAt": {
105+
"type": ["null", "string"],
106+
"format": "date-time"
107+
},
108+
"archived": {
109+
"type": ["null", "boolean"]
110+
},
111+
"archivedAt": {
112+
"type": ["null", "string"],
113+
"format": "date-time"
114+
},
115+
"contacts": {
116+
"type": ["null", "array"],
117+
"items": {
118+
"type": ["null", "string"]
119+
}
120+
}
121+
}
122+
}

‎airbyte-integrations/connectors/source-hubspot/source_hubspot/schemas/ticket_pipelines.json

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"type": ["null", "object"],
4+
"properties": {
5+
"label": {
6+
"type": ["null", "string"]
7+
},
8+
"id": {
9+
"type": ["null", "string"]
10+
},
11+
"archived": {
12+
"type": ["null", "boolean"]
13+
},
14+
"stages": {
15+
"type": ["null", "array"],
16+
"items": {
17+
"type": ["null", "object"],
18+
"properties": {
19+
"label": {
20+
"type": ["null", "string"]
21+
},
22+
"displayOrder": {
23+
"type": ["null", "integer"]
24+
},
25+
"metadata": {
26+
"type": ["null", "object"],
27+
"properties": {
28+
"ticketState": {
29+
"type": ["null", "string"]
30+
},
31+
"isClosed": {
32+
"type": ["null", "string"]
33+
}
34+
}
35+
},
36+
"id": {
37+
"type": ["null", "string"]
38+
},
39+
"createdAt": {
40+
"type": ["null", "integer"]
41+
},
42+
"updatedAt": {
43+
"type": ["null", "integer"]
44+
},
45+
"active": {
46+
"type": ["null", "boolean"]
47+
}
48+
}
49+
}
50+
},
51+
"createdAt": {
52+
"type": ["null", "integer"]
53+
},
54+
"updatedAt": {
55+
"type": ["null", "integer"]
56+
}
57+
}
58+
}

‎docs/integrations/sources/hubspot.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -110,8 +110,9 @@ If you are using Oauth, most of the streams require the appropriate [scopes](htt
110110

111111
| Version | Date | Pull Request | Subject |
112112
|:--------|:-----------| :--- |:-----------------------------------------------------------------------------------------------------------------------------------------------|
113-
| 0.1.34 | 2022-01-20 | [9641](https://github.com/airbytehq/airbyte/pull/9641) | Add more fields for `email_events` stream |
114-
| 0.1.33 | 2022-01-14 | [8887](https://github.com/airbytehq/airbyte/pull/8887) | More efficient support for incremental updates on Companies, Contact, Deals and Engagement streams |
113+
| 0.1.35 | 2021-12-24 | [9081](https://github.com/airbytehq/airbyte/pull/9081) | Add Feedback Submissions stream and update Ticket Pipelines stream |
114+
| 0.1.34 | 2022-01-20 | [9641](https://github.com/airbytehq/airbyte/pull/9641) | Add more fields for `email_events` stream |
115+
| 0.1.33 | 2022-01-14 | [8887](https://github.com/airbytehq/airbyte/pull/8887) | More efficient support for incremental updates on Companies, Contact, Deals and Engagement streams |
115116
| 0.1.32 | 2022-01-13 | [8011](https://github.com/airbytehq/airbyte/pull/8011) | Add new stream form_submissions |
116117
| 0.1.31 | 2022-01-11 | [9385](https://github.com/airbytehq/airbyte/pull/9385) | Remove auto-generated `properties` from `Engagements` stream |
117118
| 0.1.30 | 2021-01-10 | [9129](https://github.com/airbytehq/airbyte/pull/9129) | Created Contacts list memberships streams |

0 commit comments

Comments
 (0)