Skip to content

WideRowHandling: spanner to source db IT #2252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

akashthawaitcc
Copy link
Contributor

This PR introduces validation checks to enforce various constraints on table and column creation, ensuring compliance with database limitations.
Changes Implemented

Table Name Length Constraint (1 to 128 characters)

  • Successfully creates tables with names up to 128 characters.
  • Fails for names exceeding 128 characters.

Maximum Columns per Table (1,024)

  • Successfully creates tables with up to 1,024 columns.
  • Fails for tables exceeding 1,024 columns.

Column Name Length Constraint (1 to 128 characters)

  • Successfully creates columns with names up to 128 characters.
  • Fails for column names exceeding 128 characters.

String Cell Size Constraint (Max 2,621,440 Unicode characters)

  • Successfully creates a table with a string column of 2,621,440 characters.
  • Fails when exceeding the limit.

Copy link

google-cla bot commented Mar 17, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Copy link

codecov bot commented Mar 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 49.18%. Comparing base (6d8c445) to head (90d0bbc).
Report is 4 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2252      +/-   ##
============================================
- Coverage     49.19%   49.18%   -0.01%     
- Complexity     4620     4951     +331     
============================================
  Files           924      924              
  Lines         55852    55852              
  Branches       5998     5998              
============================================
- Hits          27475    27472       -3     
  Misses        26403    26403              
- Partials       1974     1977       +3     
Components Coverage Δ
spanner-templates 70.22% <ø> (-0.02%) ⬇️
spanner-import-export 68.02% <ø> (-0.04%) ⬇️
spanner-live-forward-migration 78.42% <ø> (ø)
spanner-live-reverse-replication 80.40% <ø> (ø)
spanner-bulk-migration 87.77% <ø> (ø)

see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

asthamohta
asthamohta previously approved these changes Apr 11, 2025
@taherkl taherkl requested review from a team as code owners April 15, 2025 18:46
@taherkl
Copy link
Contributor

taherkl commented Apr 16, 2025

Hi @VardhanThigle PR comments have been resolved. Please review and rerun the pipeline.

@pawankashyapollion pawankashyapollion force-pushed the wide_row/spanner_to_sourcedb_it branch from 7131b4a to 757988d Compare April 17, 2025 10:39
akashthawaitcc and others added 13 commits April 17, 2025 16:41
* Added allowes Packet Size

* Added and remove unwanted Boundry Check
* Added allowes Packet Size

* Added and remove unwanted Boundry Check

* Added Foxes for Max in Size IT

* Added Fixes for max cols

* Added 10mb

* Added Fixes

* Added Fixes

* Spotless fixes

* Added Fixes

* Added FIxes

* removed unwanted
@taherkl taherkl force-pushed the wide_row/spanner_to_sourcedb_it branch from 6bb351d to 90d0bbc Compare April 17, 2025 11:15
@asthamohta asthamohta merged commit 76d99b9 into GoogleCloudPlatform:main Apr 22, 2025
32 of 35 checks passed
ron-gal pushed a commit to ron-gal/DataflowTemplates that referenced this pull request Apr 25, 2025
* Wide row rr handling (GoogleCloudPlatform#148)

* Resolve Merge Conflict

* resolve merge conflict

* Wide row rr fixes (GoogleCloudPlatform#155)

* Added allowes Packet Size

* Added and remove unwanted Boundry Check

* IT Resource Exhausted Fixes

* Added allowes Packet Size

* Added and remove unwanted Boundry Check

* Added Foxes for Max in Size IT

* Added Fixes for max cols

* Added 10mb

* Added Fixes

* Added Fixes

* Spotless fixes

* Added Fixes

* Added FIxes

* removed unwanted

* updated the Comments

* PR Review comment fixes

* rename function

* update comments

* Added CopyRight Fixes

* Removed and Cleanup code (GoogleCloudPlatform#194)

* fix checkstyle

* fix schema creation for N columns

* fix changestream issue

* widerow: add missing changes

---------

Co-authored-by: pawankashyapollion <[email protected]>
Co-authored-by: Taher Lakdawala <[email protected]>
Co-authored-by: taherkl <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ignore-for-release integration testing Migration of integration tests to github size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants