[source-postgres] State counter on postgres#34724

Merged

xiaohansong merged 252 commits intomasterfrom

xiaohan/ctiditerator

Mar 7, 2024

This pull request is big! We're only showing the most recent 250 commits

Commits on Dec 11, 2023

Commits on Dec 12, 2023

Commits on Jan 30, 2024

Commits on Jan 31, 2024

xmin tested
xiaohansong
committed

Commits on Feb 5, 2024

add/edit test for xmin
xiaohansong
committed

Commits on Feb 13, 2024

Source Faker: Declare primary keys (#34644)

aaronsteers
authored and
xiaohansong
committed
Destination Redshift - Bump CDK version to 0.16.3 (#34680)

jbfbell
authored and
xiaohansong
committed
Destination Teradata: make connector avaialble on Airbyte Cloud (#28667)

authored and
xiaohansong
committed
Support resuming initial snapshot when id type is String, Int, Long (#34641)

rodireich
authored and
xiaohansong
committed
Publish to pypi batch4 (#34666)

Joe Reuter
authored and
xiaohansong
committed
airbyte-ci: Test pypi published properly (#34689)

Joe Reuter
authored and
xiaohansong
committed
Publish to pypi batch5 (#34668)

Joe Reuter
authored and
xiaohansong
committed
Publish to pypi batch6 (#34672)

Joe Reuter
authored and
xiaohansong
committed
Publish to pypi batch7 (#34673)

Joe Reuter
authored and
xiaohansong
committed
Kubernetes docs: update instructions to use external database (#34604)

marcosmarxm
authored and
xiaohansong
committed
Update file-based connectors for compatibility with concurrent CDK (#34681)

clnoll
authored and
xiaohansong
committed
🚨🚨🐛Source Hubspot: update marketing_emails stream schema (#34492)

Anton Karpets
authored and
xiaohansong
committed
Publish to pypi batch8 (#34690)

Joe Reuter
authored and
xiaohansong
committed
Source Azure Table Storage: CDK Update (#34576)

ChristoGrab
authored and
xiaohansong
committed
Publish to pypi batch9 (#34691)

Joe Reuter
authored and
xiaohansong
committed
Build a resume token with a pipeline consisting of selected streams (#34594)

authored and
xiaohansong
committed
Publish to pypi batch10 (#34692)

Joe Reuter
authored and
xiaohansong
committed
Destination Postgres: Upgrade CDK with fixed dependency and unpin cloud (#34683)

gisripa
authored and
xiaohansong
committed
CAT: fix NoAdditionalPropertiesValidator (#34709)

artem1205
authored and
xiaohansong
committed
airbyte-lib: Use connector metadata (#34697)

authored and
xiaohansong
committed
remove alloydb once again (#34642)

erohmensing
authored and
xiaohansong
committed
Publish to pypi batch11 (#34694)

Joe Reuter
authored and
xiaohansong
committed
Publish to pypi batch12 (#34664)

Joe Reuter
authored and
xiaohansong
committed
:bug: Source Bing Ads: fix duplicates in report-based streams (#34712)

darynaishchenko
authored and
xiaohansong
committed
🐛Source Amazon Seller Partner: fix date format in state message for streams with custom dates formatting (#34654)

Anton Karpets
authored and
xiaohansong
committed
airbyte-lib: Hidden documentation (#34702)

authored and
xiaohansong
committed
Publish to pypi batch13 (#34658)

Joe Reuter
authored and
xiaohansong
committed
Publish to pypi batch14 (#34662)

Joe Reuter
authored and
xiaohansong
committed
🐛 Source Recharge: Added the UI toggle `Use 'Orders' Deprecated API` to switch between `deprecated` and `modern` api versions for `Orders` stream (#34707)

Baz
authored and
xiaohansong
committed
Bump source-mysql and source-postgres to cdk version 0.16.3 (#34723)

akashkulk
authored and
xiaohansong
committed
Source S3: pin CDK and moto (#34665)

clnoll
authored and
xiaohansong
committed
Source Kyriba: Updates CDK, Increases Testing Coverage, Fixes Acceptance Test Config, Adds Expected Records (#34545)

pnilan
authored and
xiaohansong
committed
CDK destinations: Future based output reader for T+D test (#34727)

gisripa
authored and
xiaohansong
committed
Destination BigQuery: CDK updates for latest changes (#34728)

gisripa
authored and
xiaohansong
committed
🐛 Source Hubspot: Fix pagination for companies_properties_history stream (#34635)

lazebnyi
authored and
xiaohansong
committed
airbyte-ci: enable connectors tests in draft prs (#34756)

Marius Posta
authored and
xiaohansong
committed
🐛 bump cdk versions for java sources (#34453)

chandlerprall
authored and
xiaohansong
committed
Bump Airbyte version from 0.50.45 to 0.50.46

pmossman
authored and
xiaohansong
committed
:bug: Source Google Analytics Data API: Add rounding integer values that may be float (#34708)

tolik0
authored and
xiaohansong
committed
🐛 Source GitHub: Continue Sync on Stream failure (#34700)

artem1205
authored and
xiaohansong
committed
Source Recharge: Fix airbyte-lib integration (#34772)

Joe Reuter
authored and
xiaohansong
committed
add log4jConfig to docs (#34777)

colesnodgrass
authored and
xiaohansong
committed
AirbyteLib: fix examples in getting started (#34762)

aaronsteers
authored and
xiaohansong
committed
AirbyteLib: Add len() support on SQL datasets and Mapping behaviors for ReadResult (#34763)

aaronsteers
authored and
xiaohansong
committed
🎉 New Source: Microsoft SharePoint (#33537)

authored and
xiaohansong
committed
AirbyteLib: Treat error trace as logs (#34771)

Joe Reuter
authored and
xiaohansong
committed
Docs: Make sure reference section is always added (#34770)

Joe Reuter
authored and
xiaohansong
committed
AirbyteLib: Ignore unused Airbyte Protocol message types (#34779)

aaronsteers
authored and
xiaohansong
committed
📘Docs: Update source Linnworks page (#34788)

ChristoGrab
authored and
xiaohansong
committed
Bump Airbyte version from 0.50.46 to 0.50.47

bgroff
authored and
xiaohansong
committed
AirbyteLib: Fix column count mismatch bug (#34783)

aaronsteers
authored and
xiaohansong
committed
AirbyteLib: friendly install and post-install messaging (#34816)

aaronsteers
authored and
xiaohansong
committed
AirbyteLib: Add pip_url helpers to streamline connector debugging (#34827)

aaronsteers
authored and
xiaohansong
committed
fix bullet style

aaronsteers
authored and
xiaohansong
committed
🐛 Source Coin API: Fix catalog typos (#34826)

aaronsteers
authored and
xiaohansong
committed
Source Salesforce: concurrent incremental syncs (#33522)

clnoll
authored and
xiaohansong
committed
Source S3: updates for compatibility with the concurrent CDK (#34591)

clnoll
authored and
xiaohansong
committed
:bug: Source Google Ads: Add ignore fields for multiple fields (#34844)

tolik0
authored and
xiaohansong
committed
Source Zoom: Disable pypi (#34848)

Joe Reuter
authored and
xiaohansong
committed
Source Gong: Adjust schemas (#34847)

Joe Reuter
authored and
xiaohansong
committed
Kubernetes docs: external logs with S3 (#34621)

marcosmarxm
authored and
xiaohansong
committed
[source-mongodb-v2] : Fail sync if initial snapshot for any stream fails (#34759)

akashkulk
authored and
xiaohansong
committed
Docs: update pg13 requirement for external db (#34858)

marcosmarxm
authored and
xiaohansong
committed
Improve error messages for concurrent CDK (#34754)

maxi297
authored and
xiaohansong
committed
Update http-streams.md typo (#34861)

rwask
authored and
xiaohansong
committed
🤖 Bump patch version of Python CDK

maxi297
authored and
xiaohansong
committed
rollback source-github to 1.5.7 (#34870)

pedroslopez
authored and
xiaohansong
committed
AirbyteLib: Support write strategies: 'merge' and 'auto' (#34592)

authored and
xiaohansong
committed
AirbyteLib: detect REPL and disable Rich.Live if so (#34782)

aaronsteers
authored and
xiaohansong
committed
AirbyteLib: Add basic secrets management (#34822)

aaronsteers
authored and
xiaohansong
committed
airbyte-lib: Add testing to connectors (#34044)

Joe Reuter
authored and
xiaohansong
committed
✨ Source S3: Add region to S3 source (#34842)

authored and
xiaohansong
committed
airbyte-lib: Fix processed records counter (#34857)

Joe Reuter
authored and
xiaohansong
committed
airbyte-lib: Improve source factory (#34849)

Joe Reuter
authored and
xiaohansong
committed
airbyte-lib: Use proper segment key (#34863)

Joe Reuter
authored and
xiaohansong
committed
airbyte-ci: Make pypi publish enabled for certified connectors (#34836)

Joe Reuter
authored and
xiaohansong
committed
airbyte-lib: Stream state (#34778)

authored and
xiaohansong
committed
Source Zendesk Support: integration tests for empty streams (#34840)

roman-yermilov-gl
authored and
xiaohansong
committed
✨ Source Stripe: Events stream concurrent on incremental syncs (#34619)

maxi297
authored and
xiaohansong
committed
S3 and Google Analytics v4: Enable pypi publishing (#34903)

Joe Reuter
authored and
xiaohansong
committed
✨ Source Google Analytics Data API: Add spec parameter to convert `conversions:purchase` to float (#34907)

tolik0
authored and
xiaohansong
committed
Connector templates: Publish by default (#34766)

Joe Reuter
authored and
xiaohansong
committed
File-based CDK: log warning on no sync mode instead of raising exception (#34924)

clnoll
authored and
xiaohansong
committed
Remove connector ops team (#34867)

authored and
xiaohansong
committed
🤖 Bump minor version of Python CDK

clnoll
authored and
xiaohansong
committed
Destination Postgres: Remove varchar limit of 64k, defaults to 10MiB limit (#34891)

gisripa
authored and
xiaohansong
committed
source-s3: pin to version 4.4.1 (#34926)

pedroslopez
authored and
xiaohansong
committed
AirbyteLib: Case insensitive missing column checks, deterministic column ordering in duckdb inserts (#34824)

aaronsteers
authored and
xiaohansong
committed
CDK: fix flaky scenario-based tests by sorting on k & v (#34912)

clnoll
authored and
xiaohansong
committed
File based sources fix sync mode bug (#34936)

clnoll
authored and
xiaohansong
committed
🐛Source Amazon Seller Partner: fix date formatting for ledger reports with aggregation by month (#34914)

Anton Karpets
authored and
xiaohansong
committed
:bug: Source Facebook Marketing: Add missing fields (#34845)

authored and
xiaohansong
committed
Source Hubspot: add notes about property history stream (#34915)

roman-yermilov-gl
authored and
xiaohansong
committed
Source S3: bump CDK version to fix issue when SyncMode is missing from catalog (#34930)

clnoll
authored and
xiaohansong
committed
internal poetry packages: declare poe tasks and airbyte-ci sections in pyproject.toml (#34735)

alafanechere
authored and
xiaohansong
committed
Update reset.md (#34855)

evantahler
authored and
xiaohansong
committed
airbyte-ci: run poe tasks declared in pyproject.toml file of internal poetry packages (#34736)

alafanechere
authored and
xiaohansong
committed
✨ Source Google Analytics Data API: Replace convert_conversions_purchase with convert_conversions_event (#34951)

tolik0
authored and
xiaohansong
committed
fix version of airbyte-cdk in Dockerfile pip install (#34941)

lmossman
authored and
xiaohansong
committed
🤖 Bump patch version of Python CDK

lmossman
authored and
xiaohansong
committed
Emit multiple error trace messages and continue syncs by default (#34636)

brianjlai
authored and
xiaohansong
committed
AirbyteLib: Improved progress print, especially in the terminal (#34973)

aaronsteers
authored and
xiaohansong
committed
airbyte-ci: embed junit xml reports into user-facing html report (#34923)

Marius Posta
authored and
xiaohansong
committed
db-sources-java-cdk: fix logic to calculate the state stats count in db sources iterator (#34948)

subodh1810
authored and
xiaohansong
committed
[ISSUE #34755] do not propagate parameters on InlineSchemaLoader (#34853)

maxi297
authored and
xiaohansong
committed
🤖 Bump patch version of Python CDK

maxi297
authored and
xiaohansong
committed
✨ Source Hubspot: Add contacts form submissions stream (#34829)

lazebnyi
authored and
xiaohansong
committed
make exclusive containers first class citizens (#34892)

authored and
xiaohansong
committed
remove useLocalCdk for source-postgres (#34981)

stephane-airbyte
authored and
xiaohansong
committed
source-postgres/mysql: fix wrong tags in metadata (#34980)

alafanechere
authored and
xiaohansong
committed
AirbyteLib: Require stream selection (#34979)

aaronsteers
authored and
xiaohansong
committed
AirbyteLib: Add 'get_available_connectors()' option (#34982)

aaronsteers
authored and
xiaohansong
committed
Revert "Emit multiple error trace messages and continue syncs by default" (#34990)

brianjlai
authored and
xiaohansong
committed
File-based CDK: make incremental syncs concurrent (#34540)

clnoll
authored and
xiaohansong
committed
AirbyteLib: Use case-insensitive method of finding column objects (#34985)

aaronsteers
authored and
xiaohansong
committed
🤖 Bump minor version of Python CDK

clnoll
authored and
xiaohansong
committed
improve startup performance of airbyte-ci (#34430)

stephane-airbyte
authored and
xiaohansong
committed
airbyte-ci: only install main dependencies when calling poetry install (#34945)

alafanechere
authored and
xiaohansong
committed
source-google-sheets: use poetry for dependency management (#34944)

alafanechere
authored and
xiaohansong
committed
Revert "source-google-sheets: use poetry for dependency management (#… (#35007)

alafanechere
authored and
xiaohansong
committed
airbyte-ci: poetry install --no-root in builder (#35010)

alafanechere
authored and
xiaohansong
committed
publish-workflow: expose airbyte-ci-binary-url input (#35011)

alafanechere
authored and
xiaohansong
committed
source-google-sheets: use poetry for dependency management [2] (#35008)

alafanechere
authored and
xiaohansong
committed
✨ Source Amazon Ads: Add missing field to `sponsored_display_budget_rules` stream schema (#35013)

tolik0
authored and
xiaohansong
committed
CAT: Validate connector documentation (#34380)

darynaishchenko
authored and
xiaohansong
committed
CDK: allow ConnectorStateManager stream_instance_map to take ConfiguredAirbyteStream or Stream (#35000)

clnoll
authored and
xiaohansong
committed
🐛 Source Freshservice - add backoff policy to requested_items stream and bump changelog (#34633)

sajarin
authored and
xiaohansong
committed
Add Information about Snowflake Column Case change to V2 Doc (#34989)

jbfbell
authored and
xiaohansong
committed
fix low-code tags on source-sendgrid/sentry/intercom (#35028)

alafanechere
authored and
xiaohansong
committed
Source Klaviyo: Add missing fields to stream schemas (#34998)

ChristoGrab
authored and
xiaohansong
committed
Source Linnworks: CDK update (#34717)

ChristoGrab
authored and
xiaohansong
committed
Remove sources with LEGACY STATE from registry (#35038)

evantahler
authored and
xiaohansong
committed
🐛 Destination snowflake: use 200MB batches (#34502)

edgao
authored and
xiaohansong
committed
[source-postgres] : Provide option to advance LSN (#34781)

akashkulk
authored and
xiaohansong
committed
airbyte-ci: fix missing test reports (#35039)

Marius Posta
authored and
xiaohansong
committed
CAT: add validation for stream statuses (#34675)

artem1205
authored and
xiaohansong
committed
async-destination-framework: make GlobalAsyncStateManager entirely thread safe (#35027)

subodh1810
authored and
xiaohansong
committed
Checkpointing source-mssql (#34182)

committed
AirbyteLib: Show list of actually available connectors (#35018)

Joe Reuter
authored and
xiaohansong
committed
airbyte-lib: Clean up test schema in Snowflake (#35015)

Joe Reuter
authored and
xiaohansong
committed
airbyte-lib: Fix telemetry for streaming (#34955)

Joe Reuter
authored and
xiaohansong
committed
Docs reference: Fix bug with hidden array item properties (#34946)

Joe Reuter
authored and
xiaohansong
committed
Bump Airbyte version from 0.50.47 to 0.50.48

bgroff
authored and
xiaohansong
committed
java CDK: clean up dependencies, refactor modules (#34745)

Marius Posta
authored and
xiaohansong
committed
undo useLocalCdk = true from #34745 (#35054)

Marius Posta
authored and
xiaohansong
committed
[Docs] January 2024 release notes (#34753)

nataliekwong
authored and
xiaohansong
committed
Update documentation for aws secret manager configs (#35057)

terencecho
authored and
xiaohansong
committed
✨ Source Bing Ads: Add ignore fields for ads stream (#35019)

tolik0
authored and
xiaohansong
committed
Delete resources/example/airflow (#35056)

evantahler
authored and
xiaohansong
committed
Source Monday: add integration tests (#35016)

roman-yermilov-gl
authored and
xiaohansong
committed
✨ Source Intercom: Add missing fields (#35063)

tolik0
authored and
xiaohansong
committed
:bug: Source Amazon Seller Partner: Fix check command to check access to correct streams (#35062)

tolik0
authored and
xiaohansong
committed
[ISSUE #34910] add headers to HttpResponse for test framework (#35105)

maxi297
authored and
xiaohansong
committed
🤖 Bump patch version of Python CDK

maxi297
authored and
xiaohansong
committed
✨ source-github: migrate to poetry (#35087)

alafanechere
authored and
xiaohansong
committed
✨ source-greenhouse: migrate to poetry (#35077)

alafanechere
authored and
xiaohansong
committed
✨ source-stripe: migrate to poetry (#35068)

alafanechere
authored and
xiaohansong
committed
[Docs] Create homepage for Sources/Destinations (#34391)

authored and
xiaohansong
committed
Destination redshift: Switch back to jooq execution; add e2e special characters test (#34562)

edgao
authored and
xiaohansong
committed
destination-snowflake: adopt cleaned-up cdk (#34747)

Marius Posta
authored and
xiaohansong
committed
source-mongodb-v2: adopt cleaned-up cdk (#34748)

Marius Posta
authored and
xiaohansong
committed
source-postgres: adopt cleaned-up cdk (#34751)

Marius Posta
authored and
xiaohansong
committed
source-mssql: adopt cleaned-up cdk (#34749)

Marius Posta
authored and
xiaohansong
committed
source-mysql: adopt cleaned-up cdk (#34750)

Marius Posta
authored and
xiaohansong
committed
✨ source-linkedin-ads: migrate to poetry (#35086)

alafanechere
authored and
xiaohansong
committed
Destination bigquery: update test fixtures (#34575)

edgao
authored and
xiaohansong
committed
Source Google Ads: add test for empty streams (#34983)

artem1205
authored and
xiaohansong
committed
[Source-mysql] Add soft link in mysql test db (#35045)
xiaohansong
committed
Destination snowflake: update test fixtures (#34574)

edgao
authored and
xiaohansong
committed
airbyte-lib: Escape column names (#34969)

authored and
xiaohansong
committed
Update docs reference from Java 17 to 21 (#34418)

ambirdsall
authored and
xiaohansong
committed
migrate everything from java 17 to java 21 (#35103)

Marius Posta
authored and
xiaohansong
committed
Update airbyte-protocol.md (#35123)

evantahler
authored and
xiaohansong
committed
[docs] On final table re-creation (#35124)

evantahler
authored and
xiaohansong
committed
✨ source-google-analytics-v4: migrate to poetry (#35101)

alafanechere
authored and
xiaohansong
committed
✨ source-mailchimp: migrate to poetry (#35092)

alafanechere
authored and
xiaohansong
committed
✨ source-klaviyo: migrate to poetry (#35088)

alafanechere
authored and
xiaohansong
committed
✨ source-zendesk-support: migrate to poetry (#35083)

alafanechere
authored and
xiaohansong
committed
✨ source-marketo: migrate to poetry (#35078)

alafanechere
authored and
xiaohansong
committed
✨ source-paypal-transaction: migrate to poetry (#35075)

alafanechere
authored and
xiaohansong
committed
✨ source-google-analytics-data-api: migrate to poetry (#35073)

alafanechere
authored and
xiaohansong
committed
destination-async-framework: use the value from stats counter for global state conversion (#35111)

subodh1810
authored and
xiaohansong
committed
source-stripe: fix license in `pyproject.toml` (#35137)

alafanechere
authored and
xiaohansong
committed
source-google-sheets: fix license in pyproject.toml (#35136)

alafanechere
authored and
xiaohansong
committed
✨ Source Amazon Seller Partner: Add logs for the failed check command (#35106)

tolik0
authored and
xiaohansong
committed
✨ source-surveymonkey: migrate to poetry (#35168)

alafanechere
authored and
xiaohansong
committed
✨ source-monday: migrate to poetry (#35146)

alafanechere
authored and
xiaohansong
committed
✨ source-salesforce: migrate to poetry (#35147)

alafanechere
authored and
xiaohansong
committed
✨ source-intercom: migrate to poetry (#35148)

alafanechere
authored and
xiaohansong
committed
✨ source-iterable: migrate to poetry (#35150)

alafanechere
authored and
xiaohansong
committed
✨ source-mixpanel: migrate to poetry (#35151)

alafanechere
authored and
xiaohansong
committed
✨ source-typeform: migrate to poetry (#35152)

alafanechere
authored and
xiaohansong
committed
✨ source-twilio: migrate to poetry (#35153)

alafanechere
authored and
xiaohansong
committed
✨ source-notion: migrate to poetry (#35155)

alafanechere
authored and
xiaohansong
committed
✨ source-zendesk-talk: migrate to poetry (#35156)

alafanechere
authored and
xiaohansong
committed
✨ source-amplitude: migrate to poetry (#35162)

alafanechere
authored and
xiaohansong
committed
✨ source-jira: migrate to poetry (#35160)

alafanechere
authored and
xiaohansong
committed
✨ source-google-ads: migrate to poetry (#35158)

alafanechere
authored and
xiaohansong
committed
🐛 Source Slack: Join to the channels while `read` instead of `discovery` (#35131)

Baz
authored and
xiaohansong
committed
✨ source-hubspot: migrate to poetry (#35165)

alafanechere
authored and
xiaohansong
committed
✨ source-pinterest: migrate to poetry (#35159)

alafanechere
authored and
xiaohansong
committed
✨ source-sentry: migrate to poetry (#35145)

alafanechere
authored and
xiaohansong
committed
✨ source-chargebee: migrate to poetry (#35169)

alafanechere
authored and
xiaohansong
committed
source-snapchat-marketing: adopt our base image (#35170)

alafanechere
authored and
xiaohansong
committed
✨ source-snapchat-marketing: migrate to poetry (#35171)

alafanechere
authored and
xiaohansong
committed
source-faker: adopt our base image (#35172)

alafanechere
authored and
xiaohansong
committed
✨ source-faker: migrate to poetry (#35174)

alafanechere
authored and
xiaohansong
committed
✨ source-amazon-ads: migrate to poetry (#35180)

alafanechere
authored and
xiaohansong
committed
Source Github: add integration tests (#34933)

artem1205
authored and
xiaohansong
committed
✨ source-bing-ads: migrate to poetry (#35179)

alafanechere
authored and
xiaohansong
committed
✨ source-instagram: migrate to poetry (#35177)

alafanechere
authored and
xiaohansong
committed
rename
xiaohansong
committed
✨ source-facebook-marketing: migrate to poetry (#35178)

alafanechere
authored and
xiaohansong
committed
destination-async-framework: make emission of state from FlushWorkers synchronized (#35144)

subodh1810
authored and
xiaohansong
committed
Merge remote-tracking branch 'origin/master' into xiaohan/ctiditerator
xiaohansong
committed
save work
xiaohansong
committed

Commits on Feb 14, 2024

Commits on Feb 27, 2024

Commits on Feb 28, 2024

postgres use stream in iterator
xiaohansong
committed

Commits on Feb 29, 2024

Commits on Mar 5, 2024

Commits on Mar 7, 2024