Skip to content
This repository was archived by the owner on Nov 1, 2023. It is now read-only.

Add events retention policy #3186

Merged
merged 16 commits into from
Jul 7, 2023

Conversation

tevoinea
Copy link
Member

@tevoinea tevoinea commented Jun 13, 2023

Summary of the Pull Request

What is this about?

Closes #3099

Adds a retention policy system for blobs

Info on Pull Request

What does this include?

  1. Blobs can be tagged with an Expiry tag
  2. Event blobs will now be tagged with an expiry date
  3. Event Messages now contain their expiration date
  4. New Timer function that runs daily which deletes expired blobs
  5. New Feature Flag EnableBlobRetentionPolicy
  6. New Feature Flag EnableDryRunBlobRetention
  7. Tests

Validation Steps Performed

How does someone test & validate?

Made sure there's 3 types of blobs in the events container:

  1. Blobs with no Expiry tag
  2. Blobs with Expiry tag but it's in the future
  3. Blobs with Expiry tag that is in the past

Verification steps:

  1. When DryRun feature flag was enabled, no modifications were made
  2. When DryRun feature flag was disabled:
    • Blobs with no Expiry tag were untouched
    • Blobs with Expiry tag in the future were untouched
    • Blobs with Expiry tag in the past were deleted and appropriate metrics were logged
  3. Ran check-pr

@tevoinea tevoinea marked this pull request as ready for review June 22, 2023 14:48
@tevoinea tevoinea requested review from Porges and chkeita June 22, 2023 14:52
@codecov-commenter
Copy link

codecov-commenter commented Jun 22, 2023

Codecov Report

Merging #3186 (f330b57) into main (f503097) will increase coverage by 0.30%.
The diff coverage is 41.66%.

@@            Coverage Diff             @@
##             main    #3186      +/-   ##
==========================================
+ Coverage   29.65%   29.96%   +0.30%     
==========================================
  Files         332      302      -30     
  Lines       39302    33744    -5558     
==========================================
- Hits        11654    10110    -1544     
+ Misses      27648    23634    -4014     
Impacted Files Coverage Δ
.../ApiService/ApiService/Functions/TimerRetention.cs 0.00% <0.00%> (ø)
src/ApiService/Tests/OrmModelsTest.cs 0.00% <0.00%> (ø)
src/ApiService/ApiService/onefuzzlib/Containers.cs 62.12% <37.09%> (-12.03%) ⬇️
...iService/ApiService/onefuzzlib/RententionPolicy.cs 80.00% <80.00%> (ø)
src/ApiService/ApiService/onefuzzlib/Events.cs 70.68% <83.33%> (-0.15%) ⬇️
src/ApiService/ApiService/OneFuzzTypes/Events.cs 73.75% <100.00%> (+0.78%) ⬆️
src/ApiService/ApiService/OneFuzzTypes/Webhooks.cs 100.00% <100.00%> (ø)

... and 53 files with indirect coverage changes

@tevoinea tevoinea enabled auto-merge (squash) July 7, 2023 14:33
@tevoinea tevoinea merged commit da967ac into microsoft:main Jul 7, 2023
@AdamL-Microsoft AdamL-Microsoft mentioned this pull request Jul 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add retention policy to events container
3 participants