Future Tasks

Overview

This page documents a number of tasks that have been previously identified as either "current bugs" or "desired enhancements", but which have never yet been entered in GitHub Issues. Many of these tasks are suitable for new developers to tackle, while others are quite complex and probably require extensive experience with PC2v9 system internals before attempting to work on them.

In some cases, a fair amount of work has already been done on the task, but that work has not been merged into any branch in the PC2v9 GitHub repo (such cases are documented in the task description).

In any case, each separate task should be entered as a GitHub Issue (and should at the same time be removed from this page) before being worked on.

Tasks

• Add upgrades and fixes to the WTI web team interface project.

  • See the separate Web Team Interface Upgrades page for details

• Add a way to submit CDP judge's submissions

  • This issue is described in considerable detail in the old PC2 "Bugzilla" system, specifically at https://pc2.ecs.csus.edu/bugzilla/show_bug.cgi?id=1334.
  • Considerable development work on this has already been done under the above Bugzilla bug, and (probably) already had code incorporated into the current (GitHub) system -- just not made visible to any user.

• Implement the CLICS Contest API

  • The CLICS Contest API is described at this CLICS page.
  • This issue has already had considerable work done, as described at https://pc2.ecs.csus.edu/bugzilla/show_bug.cgi?id=1310. In fact, it is likely that most of the CLICS API is already implemented; what remains is to
    • Verify the correct operation of the endpoints which have been implemented, and fix them if necessary;
    • Identify what endpoints are missing, and implement them.

• Add CLICS "Multi-contest" support

  • In addition to supporting each individual API endpoint (as described by the preceding task), PC2 needs to support CLICS API "multi-contest awareness". This means that beyond just supporting each of the CLICS endpoints (e.g. /teams, /problems, etc.) the API needs to support contests/<contestid> as the starting point for each separate contest.

• Provide support for server-side Test Run feature

  • The ability to have team clients (whether the "fat client" installed on every team machine, or the "web client" which uses a browser to access PC2) execute a run on the server-side would be a significant benefit.
    • For browser-based systems, it would provide a facility which currently doesn't exist in PC2.
    • For fat-client installations, it would eliminate the need to install all the compiler tools on the team machines (although it is arguable that perhaps they would want to do this installation anyway).
  • See bug 853 and bug 1091 additional information
  • See branch b_1208_TestRun, specifically commit fa03548e180aff2a080dbbe154122cbdd940e7fa, in the PC2v9 ECS repo, on which a subtantial amount of work to support this task has already been done (but not ported to GitHub).

• Show team stderr to human judges

  • A recent fix to PC2, described in bug 1657, appropriately removed the concatenation of a team's stderr output in the output forwarded to the validator (so that the validator only evaluates stdout, not also stderr). Unfortunately, this now means there is no place in the system where a human judge can see the stderr output of a team's execution.
  • There should be a way for the human judge to see stderr, probably by adding another column to the Test Case output table/screen.
  • See Bug 990 for additional information about this task.

• Shadow Controller needs a user-selectable control for whether to convert judgement acronym to CLICS "Big5"

  • See bug 1642 for a detailed description of this task.
  • This task should include both implementing a GUI-based setting and implementing a YAML setting.

• Verify since-id parameter in Shadow Controller (GUI and YAML) works correctly

  • Bug 1602 provided support for connecting to a Remote CCS in Shadow Mode, and this was tested during at least one major contest (NAC 2020) -- except that the since-id parameter support was not tested during that contest.
  • Test that since-id works, both using the Shadow Controller GUI and the YAML shadow configurations.
  • Note that in order to test this it requires having a CCS which correctly implements the since-id feature in its event feed. One solution is to use PC2 to test this -- but need to verify that the PC2 Event Feed does indeed correctly support it.

• Add a tool to test judge's solutions

  • The CLICS CCS specification (in general at this page, and specifically at this page) documents a structure for judges to provide sample solutions (both correct and incorrect). The data structure under which these solutions are held is commonly referred to as a Problem Package referenced by a Contest Data Package (CDP).
  • PC2 should have a mechanism by which the Contest Administrator can run all of the judge's solutions found in a CDP (Problem Package) and verify/document that they produce the expected results.
  • Substantial work on this task has already been done and can be found on the b_1600_cdp_test_submissions_tool branch in the PC2v9 ECS repository. This task is described and discussed further at this bug.

• Add support for generating account passwords

  • Currently when PC2 accounts are generated they have "joe" passwords (the password for account "joe" is "joe"). Passwords can be changed, either manually via the Admin>Accounts interface or by loading password files. However, there should be a capability which allows for generation of different kinds of passwords (e.g. "set from a list", "generate randomly", "generate according to a specified pattern", etc.) at the time accounts are generated.
  • Some proposals for what this feature should look like are documented in Bug 1509.
  • The following bugs are related to this task and may contain useful information: bug 291, bug 577, bug 728, bug 854, bug 1458, bug 1522.

• Clean up handling of Java "packages"

  • There are currently problems with the way PC2 handles team submissions written in Java (and probably in Kotlin as well) if the submission uses the package statement. The infrastructure for supporting this needs to be cleaned up.
  • See in particular bug 1671 for details.

• Provide support for setting/enforcing compilation and validation time limits

  • The CLICS CCS specification indicates (at this page) that a CCS should enforce time limits on both compilation and validation. There needs to be a way to specify these limits in PC2 (both via a GUI and via YAML settings), and the Execution pipeline (specifically, the Executable class) needs to be updated to enforce the specified limits.
  • See bug 1669 for additional information on this task.

• Create a Run Submitter tool for both CLICS Event Feed and PC2 Extract Replay runs

  • A common need in developing/testing PC2 is to be able to "simulate" a contest by fetching runs from a previously-executed contest and submitting them to a PC2 system as if there were actually teams doing the submission.
  • There are two common formats through which submissions can be obtained: the CLICS API event feed and submissions endpoints, and PC2 Extract Replay Runs Reports. It is desirable to be able to fetch submissions using either of these formats and to then submit them to a running PC2 system.
  • Substantial work toward this task has already been accomplished (but not yet merged into the PC2v9 GitHub code stream). See bug 1646 and bug 1667 for details.

• Output size limit should be per-problem

  • Currently PC2 supports a single "output size limit" value (via a setting on the Admin Settings pane or by specifying max-output-size-K: setting in YAML). The CLICS CCS spec says that output size limits should be specified in the "problem.yaml" file -- meaning, they are specified "per problem" instead of globally (one limit for all problems) as PC2 currently does.
  • PC2 should be updated to recognize the max-output-size value in problem.yaml and should maintain max output size settings for each problem.

• Provide support for en masse rejudging of submissions

  • Currently the PC2 Admin or Judge can select any individual submission and "rejudge" it. Sometimes in a contest it is desired to select a SET of submissions (e.g., "all submissions for Problem A", "all runs from Team 3", "all Java submissions", "these 12 specific submissions", etc.) and rejudge them all. At the moment this is laborious; each submission has to be individually selected and rejudged manually. This task would support selecting a set of submissions and rejudging them all.
  • The implementation should include the ability to use "computer judging" (i.e., a PC2 AutoJudge) to do the rejudging, so that a person doesn't have to process each rejudged submission by hand.
  • The task should take advantage of (or extend as necessary) the existing run filtering (submission filtering) capability in the system -- allow the selection of the set of submissions which are to be rejudged by using the filter.
  • See Bug 1616 for the original entry of this task, along with additional information about it.