fix: commerce and enrollment apis return 400 when enrollment is not allowed

[efortish]

Summary

This PR improves both backend and frontend handling of enrollment errors, ensuring users receive clear feedback and the system returns appropriate HTTP responses when enrollment is not allowed.

---

Objective

• Return HTTP 400 (Bad request) instead of HTTP 500 when enrollment is not allowed.
• Prevent infinite loading spinners on the enrollment page.
• Display clear error messages to users before redirecting them.
• Ensure consistent and meaningful responses from enrollment-related API endpoints.

---

Technical Details

Backend (views.py in enrollments and commerce/api/v0)

• Exception Handling:
  • Updated logic to catch InvalidEnrollmentAttribute.
  • API now returns HTTP 400 with a descriptive error message instead of HTTP 500.
• API Consistency:
  • Both /api/enrollment/v1/enrollment and /api/commerce/v0/baskets/ endpoints now provide coherent and user-friendly error responses.

Frontend (enrollment.js)

• Spinner Fix:
  • Fixed the issue where the spinner would remain indefinitely if enrollment failed.
• User Feedback:
  • When enrollment is not allowed, the error message is shown in a pop-up for 6 seconds before redirecting the user to the learner dashboard.
  • Users are now informed of the specific reason for the failure.

---

Test cases and instructions:

• To replicate the issue I used this filter org.openedx.learning.course.enrollment.started.v1 with a custom pipeline to restrict the enrollment by domain.
• The pipeline has this:

def run_filter(self, user, course_key, mode):   # pylint: disable=unused-argument, arguments-differ
        """Filter."""

        other_course_settings = get_other_course_settings(course_key)
        domains_allowed = (
            other_course_settings.get("value", {}).get("filter_enrollment_by_domain_list") or
            other_course_settings.get("value", {}).get("filterEnrollmentByDomainList")
        )

        if domains_allowed:
            if not user.is_active:
                platform_name = configuration_helpers.get_value("platform_name", settings.PLATFORM_NAME)
                exception_msg = _(
                    "You need to activate your account before you can enroll in the course. "
                    "Check your {email} inbox for an account activation link from {platform_name}."
                ).format(email=user.email, platform_name=platform_name)
                raise CourseEnrollmentStarted.PreventEnrollment(exception_msg)

            cea = get_student_course_enrollment_allowed(user, course_key)
            # if the student is allowed to enroll, skip checking the email domain
            # validate the email domain if user is not already enrolled
            if not cea and not get_enrollment(user, course_key):
                if not FilterEnrollmentByDomain._is_user_email_allowed(user, domains_allowed):
                    custom_message = other_course_settings.get("value", {}).get(
                        "filter_enrollment_by_domain_custom_exception_message",
                        _("If you think this is an error, contact the course support."))
                    exception_msg = _("You can't enroll on this course because your email domain is not allowed. "
                                      "%(custom_message)s") % {
                        'custom_message': custom_message}
                    raise CourseEnrollmentStarted.PreventEnrollment(exception_msg)

        return {}

    @staticmethod
    def _is_user_email_allowed(user, domains_allowed):
        """
        Check if the user email is a domain or sub-domain of the allowed domains.
        """
        user_domain = user.email.split("@")[1].lower()
        for domain in domains_allowed:
            if user_domain == domain or fnmatch(user_domain, f"*.{domain}"):
                return True
        return False

• Also yo need to allow the other_course_settings with:

hooks.Filters.ENV_PATCHES.add_item(
    (
        "openedx-common-settings",
"""
FEATURES["ENABLE_OTHER_COURSE_SETTINGS"] = True
"""
    )
)

• And set the allowed domains in the course advanced_settings such as:

• And use an user who doesn't have an allowed email

1. The 500 error occurred when making a POST request to these endpoints when enrollment is not allowed: /api/enrollment/v1/enrollment and /api/commerce/v0/baskets/ , now it drops 400 error instead

2. When using: http://apps.local.edly.io:1999/authn/login?course_id=<course_name>&enrollment_action=enroll, an spinner and a message are rendered and remains 6 seconds before redirect when attempting to enroll when it is not allowed.

2025-05-09.14-29-48.mp4

Impact

• Users receive immediate and clear feedback when enrollment is not possible.
• The frontend no longer hangs on failed enrollments.
• API responses are more accurate and easier to debug.

---

[openedx-webhooks]

Thanks for the pull request, @efortish!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[MaferMazu]

Hello @efortish, thanks for this contribution 🙌

I think adding that catch of exception is valuable since the enrollment function may now fail thanks to filters.

My comments in general are minor improvements, and I would like to know if there is a better way to show the message to users.

I still need to test this. Can you clarify the how-to test phase? What steps should I take, and what is the expected result?

[MaferMazu]

I know that the code was status == 403, but again, I think it is better if you define:

const HttpStatus = {
  BAD_REQUEST: 400,
  FORBIDDEN: 403,
  // ... others if you needed it
};

And use jqXHR.status === HttpStatus.FORBIDDEN to avoid magic numbers.

[MaferMazu]

@dcoa, is this a good practice? Because it is common in the backend, but I was trying to find examples in other .js files in the edx platform and didn't find a good one.

[deborahgu]

Whether or not it is commonly used in frontends doesn't change the fact that it is best practice as a general rule. For the most part the legacy JavaScript in edx-platform is quite old, and our modern conceptions of best practice would be in the micro frontends, where we do exactly this:

https://github.com/edx/frontend-app-exams-dashboard/blob/29baea2e429a818d0cd4f472111e9aed0d222c7e/src/data/constants.js#L39-L46

export const ErrorStatuses = {
  badRequest: 400,
  unauthorized: 401,
  forbidden: 403,
  notFound: 404,
  conflict: 409,
  serverError: 500,
};