Skip to content

Fix RWS with score mode MAX_SUBTASK #1111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 18, 2019

Conversation

lucach
Copy link
Contributor

@lucach lucach commented Feb 12, 2019

Tasks with a score type that was not group (i.e., without subtasks)
were not handled correctly.

We consider them as having one subtask (as already done elsewhere in
CMS).

Important note
I think that the proposed change can work in the event of an unscored submission (which should have 0 as score), but please double-check what is being done inside those lines. Feel free to modify them as needed.
They were disputed since their introduction and have led to a number of subtle bugs in these last months. 😂


This change is Reviewable

Tasks with a score type that was not group (i.e., without subtasks)
were not handled correctly.

We consider them as having one subtask (as already done elsewhere in
CMS).
@wil93
Copy link
Member

wil93 commented Feb 12, 2019

Mmm I feel like a better solution would be to avoid altogether having tasks with score_mode = 'max_subtask' and no subtasks.

A DB constraint maybe?

@codecov
Copy link

codecov bot commented Feb 12, 2019

Codecov Report

Merging #1111 into master will increase coverage by 0.11%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1111      +/-   ##
==========================================
+ Coverage   62.02%   62.14%   +0.11%     
==========================================
  Files         230      230              
  Lines       16588    16588              
==========================================
+ Hits        10289    10308      +19     
+ Misses       6299     6280      -19
Flag Coverage Δ
#functionaltests 45.97% <0%> (+0.2%) ⬆️
#unittests 43.27% <0%> (ø) ⬆️
Impacted Files Coverage Δ
cmsranking/Scoring.py 57.84% <0%> (ø) ⬆️
cms/db/filecacher.py 78.36% <0%> (-0.99%) ⬇️
cms/service/ResourceService.py 57.99% <0%> (-0.92%) ⬇️
cms/db/fsobject.py 80.46% <0%> (-0.79%) ⬇️
cms/io/service.py 68.9% <0%> (-0.61%) ⬇️
cms/grading/Sandbox.py 68.23% <0%> (-0.37%) ⬇️
cms/grading/Job.py 89.57% <0%> (+0.47%) ⬆️
cms/db/util.py 52.59% <0%> (+0.74%) ⬆️
cms/service/EvaluationService.py 67.55% <0%> (+1.06%) ⬆️
cms/service/ProxyService.py 58.51% <0%> (+1.06%) ⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bfb359b...1fed420. Read the comment docs.

@lucach
Copy link
Contributor Author

lucach commented Feb 12, 2019

That is another option, it depends on how you watch the issue.

I chose this path because it is how CMS treats this situation elsewhere in the codebase (see https://github.com/cms-dev/cms/blob/master/cms/grading/scoring.py#L247-L249 ).

@stefano-maggiolo
Copy link
Member

Yeah, I see. Seems to make sense, but I'll take another look for the corner case you pointed out. But why does this bother you? Are you using this combination of configurations?

@stefano-maggiolo stefano-maggiolo self-assigned this Feb 18, 2019
@stefano-maggiolo
Copy link
Member

Yes, I think it makes sense, since in reset_history the score is set to 0 by default (then changes are applied).

@stefano-maggiolo stefano-maggiolo merged commit 1fed420 into cms-dev:master Feb 18, 2019
@lucach
Copy link
Contributor Author

lucach commented Feb 19, 2019

But why does this bother you? Are you using this combination of configurations?

Not intentionally (but it happened by mistake during the final round of the Italian Olympiad in Teams 😆 )

Thanks for merging the fix, at least now CMS behaves properly. For the future some more profound changes, as those suggested by @wil93, can be implemented.

@lucach lucach deleted the fix_rws_maxsubtask branch February 19, 2019 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants