Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Improve Defaults for Index Backfill Performance #26676

Open
1 task done
mrajcevic01 opened this issue Apr 3, 2025 · 0 comments
Open
1 task done

[YSQL] Improve Defaults for Index Backfill Performance #26676

mrajcevic01 opened this issue Apr 3, 2025 · 0 comments
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage

Comments

@mrajcevic01
Copy link
Contributor

mrajcevic01 commented Apr 3, 2025

Jira Link: DB-16055

Description

I recently ran a test with tweaks to our default index backfill performance flags and found it to finish 27% faster:

Flags changed:

backfill_index_timeout_grace_margin_ms: 60000
backfill_index_write_batch_size: 2000

Results:

--without backfill gflag changes
test-database=> create index test on transactions (user_id);
CREATE INDEX
Time: 5114479.282 ms (01:25:14.479)

--with backfill gflag changes (27% improvement)
test-database=> create index test2 on transactions (user_id);
CREATE INDEX
Time: 3735466.539 ms (01:02:15.467)

This table is 170GB large and the index ends up being ~70GB. We should do some further testing to validate these improvements (and cons) before changing the defaults, but seems promising.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@mrajcevic01 mrajcevic01 added area/ysql Yugabyte SQL (YSQL) status/awaiting-triage Issue awaiting triage labels Apr 3, 2025
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage
Projects
None yet
Development

No branches or pull requests

2 participants