Skip to content

Remove Asset.previous field #2008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 24, 2024
Merged

Remove Asset.previous field #2008

merged 2 commits into from
Oct 24, 2024

Conversation

mvandenburgh
Copy link
Member

@mvandenburgh mvandenburgh commented Aug 20, 2024

Now that audit is implemented, we can remove the previous field from the Asset model. This field was used to store the previous version of an asset, but it is no longer needed since we can track changes using the audit log.

This also unblocks garbage collection, which will follow this PR.

(GC docs -
https://github.com/dandi/dandi-archive/blob/master/doc/design/garbage-collection-1.md
https://github.com/dandi/dandi-archive/blob/master/doc/design/garbage-collection-uploads-asset-blobs.md
)

Keeping this as draft until it's ready to merge, since we'll want to perform a DB backup before doing so.

@mvandenburgh mvandenburgh force-pushed the remove-asset-previous-link branch from 0a1dd28 to 68749e3 Compare August 20, 2024 17:15
@mvandenburgh
Copy link
Member Author

Keeping this as draft until it's ready to merge, since we'll want to perform a DB backup before doing so.

@waxlamp
Copy link
Member

waxlamp commented Aug 28, 2024

Keeping this as draft until it's ready to merge, since we'll want to perform a DB backup before doing so.

I think we talked about it already, but can you remind me why we need a DB backup before rolling this out? We have the DB snapshot just before audit rolled out, and since that time everything we want to track should be in the audit table. We could take another DB backup out of an abundance of caution, but it's not strictly necessary, is that right?

(The tradeoff here is just a downtime if we want to take the backup for safety purposes.)

@satra
Copy link
Member

satra commented Aug 28, 2024

@waxlamp - was the previous link captured in the audit log? i.e. did audit backfill old entries based on the database?

@waxlamp
Copy link
Member

waxlamp commented Sep 20, 2024

was the previous link captured in the audit log? i.e. did audit backfill old entries based on the database?

No, instead of backfilling the audit records table with that information, we captured a database backup that preserves whatever historical information exists, giving us the option of performing the backfill at any time. The major problem will be that most of the information that we currently capture in audit records is not recoverable from the previous links (see the audit design doc for details).

If we decide we want to do this backfill, we should open an issue and discuss it there. Let me know what you think.

@mvandenburgh mvandenburgh force-pushed the remove-asset-previous-link branch from 68749e3 to e3574ed Compare October 23, 2024 15:28
Now that audit is implemented, we can remove the `previous` field from
the `Asset` model. This field was used to store the previous version of
an asset, but it is no longer needed since we can track changes using
the audit log.
@mvandenburgh mvandenburgh force-pushed the remove-asset-previous-link branch from e3574ed to a3c3288 Compare October 23, 2024 15:54
@mvandenburgh mvandenburgh marked this pull request as ready for review October 24, 2024 16:10
@mvandenburgh mvandenburgh merged commit be16949 into master Oct 24, 2024
11 checks passed
@mvandenburgh mvandenburgh deleted the remove-asset-previous-link branch October 24, 2024 16:33
@dandibot
Copy link
Member

🚀 PR was released in v0.3.108 🚀

@dandibot dandibot added the released This issue/pull request has been released. label Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released This issue/pull request has been released.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants