-
Notifications
You must be signed in to change notification settings - Fork 4.6k
mysql-source:fix tinyint unsigned handling #18619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
NOTE
|
/test connector=connectors/source-mysql
Build PassedTest summary info:
|
Great find! |
@subodh1810 awesome work! send it 🚀 |
/test connector=connectors/source-mysql
Build PassedTest summary info:
|
NOTE
|
NOTE Sources (3)
Destinations (0) |
/publish connector=connectors/source-mysql
if you have connectors that successfully published but failed definition generation, follow step 4 here |
/publish connector=connectors/source-mysql-strict-encrypt
if you have connectors that successfully published but failed definition generation, follow step 4 here |
* mysql-source:fix tinyint unsigned handling * update doc * format * upgrade version * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]>
* mysql-source:fix tinyint unsigned handling * update doc * format * upgrade version * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]>
* solve conflicts * solve conflict in json schema * bump to version 0.1.8 for the changes of this pr * change ad account id in the schemas * query to include data plane attributes (#18531) * query to include data plane attributes * rename functions * fix java build * more renaming fix * Fix unit tests in source relational db (#18789) * Fix unit tests * Add extra test case for record count > 1 * Store record count in variable * ci: use custom test-reporter action to upload job results (#18004) * ci: use custom action to upload job results * Correct coinmarket spec (#18790) * correct coinmarket spec * remove duplicate support normalization from source spec * rollback coinmarketcap version in source def seed * update connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]> * Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197) * Remove lines that always add Postgres to list of destinations * Parameterize all tests in test_ephemeral by destination * 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734) * 🐛 Source Facebook Marketing: reduce request limit after specific error * 🐛 Source Facebook Marketing: bump version; update docs * 🐛 Source Facebook Marketing: add test * 🐛 Source Facebook Marketing: increase timeout * [charts/airbyte-cron] Cleanup env vars (#18787) * [charts/airbyte-cron] Cleanup env vars * Remove unused env var * Use equalsIgnoreCase (#18810) * Bump helm chart version reference to 0.40.40 (#18815) Co-authored-by: perangel <[email protected]> Co-authored-by: Kyryl Skobylko <[email protected]> * 🐛Destination Google Sheets: Fix empty headers list (#18729) * Fix empty headers list * Updated PR number * Bumped version * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]> * 🐛Source Exchange Rates: Fix handling error during check connection (#18726) * Fix handling error during check connection * Updated PR number * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]> * Add normalization changelog and bump normalization version in platform (#18813) * Remove ConfigPersistence usage from SecretsMigrator (#18747) * remove config persistence from seeding logic (#18749) * Remove the bulk actions from ConfigPersistence (#18800) * hide ConfigPersistence inside ConfigRepository to discourage use (#18803) * ci: add job and run id to test reports (#18832) * Bump Airbyte version from 0.40.17 to 0.40.18 (#18827) Co-authored-by: grishick <[email protected]> * 🪟🔧 Remove styled components (round 1) (#18766) * refactor EditorHeader (untested) * refactor BaseClearView * delete unused Subtitle * refactor ConfirmationModal * refactor Arrow * refactor BulkHeader * refactor CatalogTreeSearch * refactor StreamFieldTable * refactor StreamHeader * refactor ConnectorIcon * refactor TreeRowWrapper * refactor DeleteBlock * refactor EmptyResourceBlock * revert unintended element change * fixed acceptance tests (#18699) * 🪟🔧 Reactor Breadcrumbs component to use anchors (#18764) * refactor breadcrumbs to use actual links * PR comments on styles * increase test timeout for some webapp tests to prevent flakes (#18807) * Remove "Filters and Segments" from Google Analytics v4 (#18508) Filters and Segments info was incorrectly added to the Google Analytics v4 connector instead of the Google Analytics (Universal Analytics) Connector. * Add notes about EU OAUth (#18835) EU OAuth is not fully tested so adding a note to account for that. * 🪟🐛 Fix: visual regression in ConnectorIcon (#18849) * fix visual regression * remove unused prop * Add links to demo page (#18828) * mysql-source:fix tinyint unsigned handling (#18619) * mysql-source:fix tinyint unsigned handling * update doc * format * upgrade version * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]> * 🪟 🎉 Allow environment specific sections in docs (#18829) * Allow environment specific sections in docs * Change syntax to lower case * ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables (#18809) * ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables * ci: replace github set-ouput with new syntax in ./tools/bin/ * Add connection ID to span (#18852) * edited connector docs (#18855) * 🪟 🔧 Upgrade husky to 8.0.1 (#18719) * Upgrade Husky * Upgrade Husky * Upgrade Husky * Upgrade Husky * Upgrade Husky * ci: replace GITHUB_OUTPUT with GITHUB_ENV for multiline variables (#18857) * Avoid NPE when adding connection ID to trace (#18856) * Filter exit errors by operation name (#18850) * add label * auto-bump connector version Co-authored-by: marcosmarxm <[email protected]> Co-authored-by: Xiaohan Song <[email protected]> Co-authored-by: Liren Tu <[email protected]> Co-authored-by: Conor <[email protected]> Co-authored-by: Marcos Marx <[email protected]> Co-authored-by: Octavia Squidington III <[email protected]> Co-authored-by: Greg Solovyev <[email protected]> Co-authored-by: Artem Inzhyyants <[email protected]> Co-authored-by: perangel <[email protected]> Co-authored-by: Jonathan Pearlin <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: perangel <[email protected]> Co-authored-by: Kyryl Skobylko <[email protected]> Co-authored-by: Serhii Lazebnyi <[email protected]> Co-authored-by: Charles <[email protected]> Co-authored-by: Octavia Squidington III <[email protected]> Co-authored-by: Joey Marshment-Howell <[email protected]> Co-authored-by: darynaishchenko <[email protected]> Co-authored-by: Michael Siega <[email protected]> Co-authored-by: Tyler B <[email protected]> Co-authored-by: Yowan Ramchoreeter <[email protected]> Co-authored-by: Tim Roes <[email protected]> Co-authored-by: Subodh Kant Chaturvedi <[email protected]> Co-authored-by: Volodymyr Pochtar <[email protected]> Co-authored-by: Amruta Ranade <[email protected]>
Issue : #17510
When we upgraded the debezium version from 1.4.2 to 1.9, the default
TinyIntOneToBooleanConverter
that is provided by debezium started converting the values for TINYINT unsigned to boolean values as well. The old version (1.4) only converted the values for TINYINT(1).We recognise TINYINT UNSIGNED as INTEGER in the catalog and so while trying to map the boolean values
true
/false
to INTEGER type, normalisation started failing.We can not identify TINYINT UNSIGNED as boolean is because it lacks the length information. MySQL8 doesn't show the length of tinyint unsigned type.
For instance take a look at the following table structure.
I used the following query to create the table
create table blah_3 (id int primary key, col_1 tinyint(1), col_2 tinyint(1) unsigned, col_3 tinyint(3), col_4 tinyint(3) unsigned);
.If you look at the information schema details of the column types listed below and compare
col_1
which istinyint(1)
andcol_2
which istinyint(1) unsigned
, you will see that theCOLUMN_TYPE
info of both the columns is different. One clarifies its a tinyint(1) while the other one doesnt.This PR brings back the old behaviour and only converts tinyint column types to boolean. This also updates the document.