Skip to content

i.evapo.mh: add test file #5519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

jayneel-shah18
Copy link
Contributor

@jayneel-shah18 jayneel-shah18 commented Apr 8, 2025

This PR introduces a comprehensive regression test suite for the i.evapo.mh module in GRASS GIS. These tests are designed to detect unintended changes in computational results and ensure consistent evapotranspiration output under various input scenarios and module flag settings.

Changes Made

Implemented tests for:

  • Output clamping with -z flag (test_z_flag_negative_values)
    Ensures that negative ET values are set to zero as expected.

  • Precipitation exclusion with -h flag (test_h_flag_precipitation_ignored)
    Confirms that results remain unchanged when precipitation is omitted under the -h flag.

  • Alternative scaling behavior with -s flag (test_s_flag_different_output)
    Verifies that the -s flag produces different output from the default mode.

  • Hargreaves formula consistency (test_hargreaves_formula)
    Cross-validates module output against the original Hargreaves reference implementation using known inputs.

Performance

Platform Runtime
Windows 1.712s
Ubuntu 1.633s
macOS 1.028s

This PR significantly improves the test coverage for i.evapo.mh. Looking forward to feedback or suggestions for further refinement!

I have found a bug in the module implementation. The attached issue is #5520

@github-actions github-actions bot added Python Related code is in Python module imagery tests Related to Test Suite labels Apr 8, 2025
@echoix
Copy link
Member

echoix commented Apr 17, 2025

Why isn't any of these new tests failing since the issue #5520 you reported isn't addressed yet?

@echoix
Copy link
Member

echoix commented Apr 17, 2025

If you'd want to prove that the issue was "fixed" correctly, you could have the test added here with an expected failure, and remove it with the PR that would fix the issue (x1000 scale)

@jayneel-shah18
Copy link
Contributor Author

Thanks for the feedback!
In this case, the reason none of the tests are failing is because I used the calculate_et() function to simulate the expected output based on the current behavior — mainly to ensure the test setup works correctly.
Also, for the other tests (e.g., those using r.univar), since the bug hasn’t been fixed yet, I don’t have the correct reference stats. So for now, I have used the outputs from the current implementation to populate the expected values.
To better demonstrate the bug, I can remove the calculate_et() function and let the test fail. Then, once the bug is fixed, that test will pass — effectively verifying the fix.
I will add that change in the next commit. Appreciate the suggestion!

@echoix
Copy link
Member

echoix commented Apr 18, 2025

Your more mathematical and physical approach of creating your tests remind me of the spirit of test driven development (TDD), where you write the tests first, see them fail, then implement it and see the tests pass, having confidence in those tests. In your latest PRs, you are effectively going back to the papers and define the specification, a bit like spec.js files in JavaScript/node testing.

The univar tests we often use are more related to a snapshot-based regression testing. It is useful to start with, to make sure nothing else changes. Instead of a CRC or a checksum, we reduce the output, of a raster for example, to a list of univariate statistics, hoping they'll let us catch changes in the distribution of data, but allow architecture-specific and math optimisation variations without failing; thus avoiding to store complete expected data in the repo.

So, if the univar results are the only blocking part, I would be comfortable to having it either 1) "broken" and skipped, then fixed in the PR fixing the bug, or 2) just added in the second PR when it is known.
You found a math error SI unit handling, so snapshotting bad results isn't that helpful in my opinion.

@jayneel-shah18
Copy link
Contributor Author

I have updated the expected statistics in the test_s_flag_different_output test to reflect the corrected radiation conversion. As expected, all values are now lower by a factor of 1000. Specifically:

  • max: reduced from 2276.226764 to 2.276227
  • mean: reduced from 985.087234 to 0.985087

These changes confirm that the fix is correctly applied and the output now aligns with the expected SI unit handling.

@echoix
Copy link
Member

echoix commented May 9, 2025

Great work!

@echoix
Copy link
Member

echoix commented May 9, 2025

And we saw the tests fail before, everything seems right now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
imagery module Python Related code is in Python tests Related to Test Suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants