fix: Ensure problem response reports include all descendant problems regardless of nesting or randomization #36677
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR ensures that the instructor "Problem Responses" report in Open edX includes all student responses to all problems under any selected block, including those that are nested or randomized (such as those from legacy library_content blocks). Previously, the report could miss responses to problems that were not directly visible to the instructor or admin user generating the report, especially in courses using randomized content blocks or deep nesting.
In courses that use randomized content (e.g., legacy
library_content
blocks) or have deeply nested structures, the instructor dashboard’s problem response report was incomplete. It only included responses to problems visible in the block tree for the user generating the report (typically the admin or instructor). As a result, responses to problems served randomly to students, or problems nested in containers, were omitted from the CSV export. This led to inaccurate reporting and made it difficult for instructors to audit all student answers.Technical Approach
The backend now recursively expands any block selected for reporting (not just
library_content
blocks) to collect all descendant blocks of typeproblem
. This is done regardless of the nesting level or block type.The logic is encapsulated in a static method (
resolve_problem_descendants
) within theProblemResponses
class, ensuring clear code organization.When generating the report, the backend uses this method to build the list of all relevant problem usage_keys, guaranteeing that all student responses are included in the export, even for randomized or deeply nested problems.
The code also improves how problem titles are resolved, falling back to the modulestore if the display name is not available in the course block structure.
Impact
Reports now accurately reflect all student responses, regardless of how problems are served or structured in the course.
The change only affects backend report generation; there is no impact on the student experience, grading, or other LMS features.
In courses with very large or deeply nested structures, report generation may take slightly longer, but this is necessary to ensure completeness.
How to reproduce:
Select the block that you want to use to generate the report.
For this scenario I created 99 users to solve the exam, each user must answer 5 questions, the csv output is supposed to have 495 + 1(labeling row) rows.
You will receive much less rows than 496 because the report will only include the responses visible to the user generating the report:
Testing
After apply the changes and repeating the process in the how to test section I received:
While the data is accurate, showing 496 of 496 expected rows, the "title" column (B) incorrectly displays "problem" across all rows. This happens because the title itself remains hidden if the question is not visible to the user who is generating the report.
That is why I propose the fallback in
_build_problem_list
, it will allow the CSV task to get the problem title from themodulestore
without any problem, and the report will looks like:So: