Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix cagg_migrate_to_time_bucket during update #7763

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix cagg_migrate_to_time_bucket during update #7763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 0 additions & 20 deletions sql/updates/2.14.2--2.15.0.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,23 +427,3 @@ DROP FUNCTION IF EXISTS _timescaledb_functions.policy_job_error_retention_check(
--
-- END bgw_job_stat_history
--

-- Migrate existing CAggs using time_bucket_ng to time_bucket
CREATE PROCEDURE _timescaledb_functions.cagg_migrate_to_time_bucket(cagg REGCLASS)
AS '@MODULE_PATHNAME@', 'ts_continuous_agg_migrate_to_time_bucket' LANGUAGE C;

DO $$
DECLARE
cagg_name regclass;
BEGIN
FOR cagg_name IN
SELECT pg_catalog.format('%I.%I', user_view_schema, user_view_name)::regclass
FROM _timescaledb_catalog.continuous_agg cagg
JOIN _timescaledb_catalog.continuous_aggs_bucket_function AS bf ON (cagg.mat_hypertable_id = bf.mat_hypertable_id)
WHERE
bf.bucket_func::text LIKE '%time_bucket_ng%'
LOOP
CALL _timescaledb_functions.cagg_migrate_to_time_bucket(cagg_name);
END LOOP;
END
$$;
19 changes: 19 additions & 0 deletions sql/updates/post-update.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,3 +171,22 @@ $$;
-- Repair relations that have relacl entries for users that do not
-- exist in pg_authid
CALL _timescaledb_functions.repair_relation_acls();

-- Migrate existing CAggs using time_bucket_ng to time_bucket
DO $$
DECLARE
cagg_name regclass;
BEGIN
FOR cagg_name IN
SELECT
pg_catalog.format('%I.%I', user_view_schema, user_view_name)::regclass
FROM
_timescaledb_catalog.continuous_agg
JOIN _timescaledb_catalog.continuous_aggs_bucket_function USING (mat_hypertable_id)
WHERE
bucket_func::text LIKE '%time_bucket_ng%'
LOOP
CALL _timescaledb_functions.cagg_migrate_to_time_bucket(cagg_name);
END LOOP;
END
$$;
Comment on lines +175 to +192
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We apparently made a decision to do this inside an update script some time in the past. However, I am unsure how expensive the operation is to update the continuous aggregates. This could potentially block an upgrade for a long time if it has to move a lot of data around, or create weird errors if the script is not properly written.

Are we sure this script is not going to block updates for production cases?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We apparently made a decision to do this inside an update script some time in the past. However, I am unsure how expensive the operation is to update the continuous aggregates. This could potentially block an upgrade for a long time if it has to move a lot of data around, or create weird errors if the script is not properly written.

This function replace the time_bucket_ng by time_bucket in the cagg query tree... it is very cheap.

Are we sure this script is not going to block updates for production cases?

This change is exactly to don't block updates cause now it is blocking with the following error:

TimescaleDB: 2025-02-17 10:54:00 UTC [425]: [67b3153a.1a9-3] 2313713420 postgres@tsdb,app=deployer [XX000] STATEMENT:  ALTER EXTENSION "timescaledb" UPDATE TO "2.18.1";
TimescaleDB: 2025-02-17 10:54:00 UTC [425]: [67b3153a.1a9-1] 2313713420 postgres@tsdb,app=deployer [XX000] ERROR:  tried calling catalog_get when extension isn't loaded
TimescaleDB: 2025-02-17 10:54:00 UTC [425]: [67b3153a.1a9-2] 2313713420 postgres@tsdb,app=deployer [XX000] CONTEXT:  SQL statement "CALL _timescaledb_functions.cagg_migrate_to_time_bucket(cagg_name)"

Loading