Skip to content

feat(source-hubspot): Migrate deals_archived, forms, form_submissions, owners, owners_archived to low-code #58105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 28 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f889b0c
Migrate deal_archived stream to low code
tolik0 Apr 16, 2025
b15ad85
Fix unit tests
tolik0 Apr 16, 2025
ff847e6
Add NewtoLegacyFieldTransformation
tolik0 Apr 16, 2025
e355d64
Fix docs for FlattenFieldWithPrefix
tolik0 Apr 16, 2025
ea285f7
chore: auto-fix lint and format issues
octavia-squidington-iii Apr 16, 2025
14b898d
Add scopes for deals_archived
tolik0 Apr 16, 2025
8878724
Change the property chunking limit to 14000
tolik0 Apr 17, 2025
7779399
Change property limit for property chunking to 10000
tolik0 Apr 17, 2025
7342330
Add missing cursor format
tolik0 Apr 17, 2025
622d9b9
Migrate forms and form_submissions
tolik0 Apr 18, 2025
b8d2383
Fix errors from tests
tolik0 Apr 18, 2025
8b29f3e
Add owners and owners_archived streams
tolik0 Apr 18, 2025
decf49c
Add scopes for owners and owners_archived
tolik0 Apr 18, 2025
d84c5a9
Delete migrated python streams
tolik0 Apr 22, 2025
b7e65e1
Fix unit tests
tolik0 Apr 23, 2025
e370a64
Merge branch 'master' into tolik0/source-hubspot/migrate-deals-archived
tolik0 May 8, 2025
4e9234c
Add dynamic schema loader and state migration
tolik0 May 8, 2025
08499aa
chore: auto-fix lint and format issues
octavia-squidington-iii May 8, 2025
b3576a2
Update CDK version
tolik0 May 8, 2025
6105df6
updated unit tests
darynaishchenko May 8, 2025
143fada
Fix unit tests
tolik0 May 9, 2025
d59fa01
Merge branch 'master' into tolik0/source-hubspot/migrate-deals-archived
tolik0 May 9, 2025
c243ff4
Add missing cursor formats in state migration
tolik0 May 9, 2025
9f6834b
Update version
tolik0 May 9, 2025
d360e9f
chore: auto-fix lint and format issues
octavia-squidington-iii May 9, 2025
390abc2
Fix unit tests
tolik0 May 9, 2025
79a334b
chore: auto-fix lint and format issues
octavia-squidington-iii May 9, 2025
a125a2c
Add missing primary key for ticket_pipelines
tolik0 May 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions airbyte-integrations/connectors/source-hubspot/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,17 @@
# Copyright (c) 2024 Airbyte, Inc., all rights reserved.
#

from typing import Any, Dict, Optional
from dataclasses import InitVar, dataclass
from typing import Any, Dict, List, Mapping, Optional, Union

import dpath

from airbyte_cdk.sources.declarative.interpolation.interpolated_string import InterpolatedString
from airbyte_cdk.sources.declarative.transformations import RecordTransformation
from airbyte_cdk.sources.types import Config, StreamSlice, StreamState


@dataclass
class NewtoLegacyFieldTransformation(RecordTransformation):
"""
Implements a custom transformation which adds the legacy field equivalent of v2 fields for streams which contain Deals and Contacts entities.
Expand All @@ -18,8 +23,7 @@ class NewtoLegacyFieldTransformation(RecordTransformation):
hs_v2_date_exited_{stage_id} -> hs_date_exited_{stage_id} where {stage_id} is a user-generated value
"""

def __init__(self, field_mapping: Dict[str, str]) -> None:
self._field_mapping = field_mapping
field_mapping: Mapping[str, str]

def transform(
self,
Expand All @@ -36,7 +40,7 @@ def transform(
is_record = record_or_schema.get("properties") is not None

for field, value in list(record_or_schema.get("properties", record_or_schema).items()):
for legacy_field, new_field in self._field_mapping.items():
for legacy_field, new_field in self.field_mapping.items():
if new_field in field:
transformed_field = field.replace(new_field, legacy_field)

Expand Down
Loading
Loading