Usage parsers refactoring

[erikbocks]

Description

Currently, all Usage parsers implement a static method with an identical signature, which is invoked by the UsageManagerImpl#parseHelperTables() method. This PR refactors these parser classes by standardizing them to extend the UsageParser class and requiring an override of the parse method, improving code consistency, maintainability, and extensibility.

Additionally, by leveraging Spring Dependency Injection, it was possible to provide a major cleanup in the UsageManagerImpl#parseHelperTables() method, by injecting all UsageParser implementations into a list, then iterating over it and calling the parse method. The parse log was also improved to provide better information about the parsing result, the object being parsed and the target account.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Screenshots (if appropriate):

How Has This Been Tested?

1. First, I triggered the Usage job execution without the changes and validated that the current logs contained minimal information about the process.
2. Then, installed the packages with the changes in my local env.
3. After the update, I triggered the Usage job again and accompanied the process via debug, to ensure that all the implementations were being injected correctly, and the enhanced log was being displayed after each parser execution.

Update log example ¹:

2025-06-03 13:55:51,183 DEBUG [usage.parser.VMSnapshotUsageParser] (Usage-Job-1:[]) (logid:) Parsing all VM Snapshot usage events for account: [null]
2025-06-03 13:55:51,186 DEBUG [usage.parser.VMSnapshotUsageParser] (Usage-Job-1:[]) (logid:) No VM snapshot usage events for this period
2025-06-03 13:55:51,186 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_fe18a690] (Usage-Job-1:[]) (logid:) VM Snapshot usage was successfully parsed for [Account [{"accountName":"system","id":1,"uuid":null}]].

Footnotes

1. During the tests, I validated that the UUID column of cloud_usage.accounts is set to NULL after the accounts import from the cloud database. An issue was mapped to fix this behaviour (NULL value in uuid column of cloud_usage.account table #11077) ↩

[DaanHoogland]

thanks for this initiative @erikbocks , I’ll look at this (including doing some testing)

[Copilot]

Pull Request Overview

This PR refactors individual Usage parser classes to inherit from a new abstract UsageParser, replaces static methods/fields with instance methods and injected DAOs, unifies logging behavior, and simplifies UsageManagerImpl.parseHelperTables() by injecting and iterating a list of parser beans.

• All parser classes now extend UsageParser, remove static parsing logic, and use an injected UsageDao
• Introduced common logging (beforeParse) and standardized debug messages
• Updated UsageManagerImpl to inject List<UsageParser> and loop through parsers instead of calling each statically

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

File	Description
UsageParser.java	New base class with doParsing(), beforeParse() and common usageDao
UsageManagerImpl.java	Injects List<UsageParser> and iterates in parseHelperTables()
usage/src/main/java/com/cloud/usage/parser/*.java	Each parser now extends UsageParser, removes static methods, replaces DAO calls, and switches to instance logger

Comments suppressed due to low confidence (1)

usage/src/main/java/com/cloud/usage/parser/NetworksUsageParser.java:40

• [nitpick] The parser name is pluralized here, while most other parsers use a singular form. Consider using "Network" for consistency.

        return "Networks";

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 177 lines in your changes missing coverage. Please review.

Project coverage is 16.58%. Comparing base (7632814) to head (a8b07b4).
Report is 41 commits behind head on main.

Files with missing lines	Patch %	Lines
.../cloud/usage/parser/VMSnapshotOnPrimaryParser.java	0.00%	12 Missing ⚠️
.../com/cloud/usage/parser/VMSnapshotUsageParser.java	0.00%	12 Missing ⚠️
...om/cloud/usage/parser/LoadBalancerUsageParser.java	0.00%	11 Missing ⚠️
.../cloud/usage/parser/PortForwardingUsageParser.java	0.00%	11 Missing ⚠️
...m/cloud/usage/parser/SecurityGroupUsageParser.java	0.00%	11 Missing ⚠️
...ava/com/cloud/usage/parser/VPNUserUsageParser.java	0.00%	11 Missing ⚠️
...a/com/cloud/usage/parser/IPAddressUsageParser.java	0.00%	10 Missing ⚠️
...cloud/usage/parser/NetworkOfferingUsageParser.java	0.00%	10 Missing ⚠️
...ava/com/cloud/usage/parser/StorageUsageParser.java	0.00%	10 Missing ⚠️
...java/com/cloud/usage/parser/VolumeUsageParser.java	0.00%	10 Missing ⚠️
... and 9 more

Additional details and impacted files

@@             Coverage Diff             @@
##               main   #11097     +/-   ##
===========================================
  Coverage     16.57%   16.58%             
- Complexity    13868    13968    +100     
===========================================
  Files          5719     5743     +24     
  Lines        507178   510354   +3176     
  Branches      61571    62026    +455     
===========================================
+ Hits          84085    84617    +532     
- Misses       413674   416275   +2601     
- Partials       9419     9462     +43     

Flag	Coverage Δ
uitests	3.90% <ø> (-0.07%)	⬇️
unittests	17.47% <0.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13935

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[winterhazel]

The code looks good. I think we can still do some more refactoring inside the parse methods in the future, but this is already a very good improvement :)

I will do some testing too.