Skip to content

Fixes and completes install_addon() tests on firefox #1609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 19, 2024
Merged

Fixes and completes install_addon() tests on firefox #1609

merged 2 commits into from
Mar 19, 2024

Conversation

jkbzh
Copy link
Contributor

@jkbzh jkbzh commented Mar 14, 2024

User description

test_install_addon_directory() was being called with an xpi file fixture instead of a directory fixture.

I fixed this by making a new fixture for
test_install_addon_directory() that points to the test extension directory. See details here below.

I also added an additional test and fixture so that the tests take into account whether a path to an extension directory ends with a trailing slash. See details here below.

This required renaming the existing addon_path fixture to avoid confusion among the three related fixtures:

  • addon_path_xpi

    renamed fixture gives a path to the xpi file (was addon_path)

    updated test_install_addon() and test_unistall_addon()

  • addon_path_dir

    new fixture that gives a path to the extension directory, ending without a '/'

    updated test_install_unsigned_addon_directory() to use it

  • addon_path_dir_slash (new fixture, and test)

    new fixture that gives a path to the extension directory, ending with a '/'

    added new test: test_install_unsigned_addon_directory_slash()

Note that I don't have the appropriate environment to run these tests before pushing them.

Thanks for contributing to the Selenium site and documentation!
A PR well described will help maintainers to review and merge it quickly

Before submitting your PR, please check our contributing guidelines.
Avoid large PRs, and help reviewers by making them as simple and short as possible.

Description

test_install_unsigned_addon_directory() was using the same fixture for test_install_addon(), which ended duplicating
the test, but not testing what was intended. I fixed this by declaring a new fixture that points to the intended
extension directory.

I also added a new test to check that install_addon() handles correctly directory paths that end with a '/'. This is
a recent bug I found in the firefox's webdriver (issue and PR already submitted).

I added suffixes to the the three addon_path fixtures to be able to distinguish between them.

Motivation and Context

It fixes test_install_unsigned_addon_directory() which was using the wrong fixture.

It add a new test that allows to detect if install_addon() is handling correctly directory paths that end with a '/'.
Note that until a fix (my PR or some other one) is applied to firefox's webdrive, this test will fail.

Also note that I couldn't install the test environment on my box so these changes are proposed without having
been actually tested, hoping they work or that they can give hints to someone else on how to fix / improve
the install_addon() related tests.

Types of changes

  • Change to the site (I have double-checked the Netlify deployment, and my changes look good)
  • Code example added (and I also added the example to all translated languages)
  • Improved translation
  • Added new translation (and I also added a notice to each document missing translation)

Checklist

  • I have read the contributing document.
  • I have used hugo to render the site/docs locally and I am sure it works.

Type

enhancement, tests


Description

  • Renamed the addon_path fixture to addon_path_xpi and updated relevant tests to use this fixture.
  • Introduced two new fixtures, addon_path_dir and addon_path_dir_slash, to differentiate between directory paths with and without a trailing slash.
  • Added a new test test_install_unsigned_addon_directory_slash to specifically test addon installation from a directory path ending with a slash.
  • Updated test_install_unsigned_addon_directory to include an assertion for verifying the injected content, ensuring the addon is installed correctly.

Changes walkthrough

Relevant files
Tests
test_firefox.py
Enhance Firefox Addon Installation Tests and Add New Test for
Directory Paths Ending with Slash

examples/python/tests/browsers/test_firefox.py

  • Updated test_install_addon and test_uninstall_addon to use the
    addon_path_xpi fixture.
  • Modified test_install_unsigned_addon_directory to use the
    addon_path_dir fixture and added an assertion to check the injected
    content.
  • Added a new test test_install_unsigned_addon_directory_slash using the
    addon_path_dir_slash fixture to handle directory paths ending with a
    slash.
  • +17/-6   
    conftest.py
    Update Fixtures for Firefox Addon Installation Tests         

    examples/python/tests/conftest.py

  • Renamed addon_path fixture to addon_path_xpi to clarify its purpose.
  • Added new fixtures addon_path_dir and addon_path_dir_slash for testing
    directory paths with and without a trailing slash.
  • +21/-1   

    PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    test_install_addon_directory() was being called with an xpi file
    fixture instead of a directory fixture.
    
    I fixed this by making a new fixture for
    test_install_addon_directory() that points to the test extension
    directory. See details here below.
    
    I also added an additional test and fixture so that the tests
    take into account whether a path to an extension directory ends
    with a trailing slash. See details here below.
    
    This required renaming the existing addon_path fixture to avoid
    confusion among the three related fixtures:
    
    - addon_path_xpi
    
      renamed fixture gives a path to the xpi file (was addon_path)
    
      updated test_install_add() and test_unistall_addon()
    
    - addon_path_dir
    
      new fixture that gives a path to the extension directory,
      ending without a '/'
    
      updated test_install_unsigned_addon_directory() to use it
    
    - addon_path_dir_slash (new fixture, and test)
    
      new fixture that gives a path to the extension directory,
      ending with a '/'
    
      added new test: test_install_unsigned_addon_directory_slash()
    
    Note that I don't have the appropriate environment to run these tests
    before pushing them.
    Copy link

    netlify bot commented Mar 14, 2024

    👷 Deploy request for selenium-dev pending review.

    Visit the deploys page to approve it

    Name Link
    🔨 Latest commit 892a9b5

    @CLAassistant
    Copy link

    CLAassistant commented Mar 14, 2024

    CLA assistant check
    All committers have signed the CLA.

    @qodo-merge-pro qodo-merge-pro bot added enhancement New feature or request tests labels Mar 14, 2024
    Copy link
    Contributor

    PR Description updated to latest commit (82f5998)

    Copy link
    Contributor

    PR Review

    ⏱️ Estimated effort to review [1-5]

    2, because the changes are straightforward and well-described, but require understanding of the Selenium testing framework and the specific use case of testing Firefox addons. The PR introduces new fixtures and updates existing tests, which are generally easy to follow.

    🧪 Relevant tests

    Yes

    🔍 Possible issues

    Test Environment Assumption: The note by the contributor that they haven't run the tests due to a lack of an appropriate environment raises a concern. It's crucial to ensure that tests are executed in a relevant environment before merging.

    Fixture Path Assumption: The logic to determine the path of the addon based on the current directory might not be robust in all environments or setups. Consider verifying or documenting the expected working directory for these tests.

    🔒 Security concerns

    No


    ✨ Review tool usage guide:

    Overview:
    The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
    When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

    /review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...
    

    With a configuration file, use the following template:

    [pr_reviewer]
    some_config1=...
    some_config2=...
    
    Utilizing extra instructions

    The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.

    Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.

    Examples for extra instructions:

    [pr_reviewer] # /review #
    extra_instructions="""
    In the 'possible issues' section, emphasize the following:
    - Does the code logic cover relevant edge cases?
    - Is the code logic clear and easy to understand?
    - Is the code logic efficient?
    ...
    """
    

    Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

    How to enable\disable automation
    • When you first install PR-Agent app, the default mode for the review tool is:
    pr_commands = ["/review", ...]
    

    meaning the review tool will run automatically on every PR, with the default configuration.
    Edit this field to enable/disable the tool, or to change the used configurations

    Auto-labels

    The review tool can auto-generate two specific types of labels for a PR:

    • a possible security issue label, that detects possible security issues (enable_review_labels_security flag)
    • a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)
    Extra sub-tools

    The review tool provides a collection of possible feedbacks about a PR.
    It is recommended to review the possible options, and choose the ones relevant for your use case.
    Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example:
    require_score_review, require_soc2_ticket, and more.

    Auto-approve PRs

    By invoking:

    /review auto_approve
    

    The tool will automatically approve the PR, and add a comment with the approval.

    To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following:

    [pr_reviewer]
    enable_auto_approval = true
    

    (this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository)

    You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag:

    [pr_reviewer]
    maximal_review_effort = 5
    
    More PR-Agent commands

    To invoke the PR-Agent, add a comment using one of the following commands:

    • /review: Request a review of your Pull Request.
    • /describe: Update the PR title and description based on the contents of the PR.
    • /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
    • /ask <QUESTION>: Ask a question about the PR.
    • /update_changelog: Update the changelog based on the PR's contents.
    • /add_docs 💎: Generate docstring for new components introduced in the PR.
    • /generate_labels 💎: Generate labels for the PR based on the PR's contents.
    • /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

    See the tools guide for more details.
    To list the possible configuration parameters, add a /config comment.

    See the review usage page for a comprehensive guide on using this tool.

    Copy link
    Contributor

    qodo-merge-pro bot commented Mar 14, 2024

    PR Code Suggestions

    CategorySuggestions                                                                                                                                                       
    Enhancement
    Add exception handling for addon installation and uninstallation.

    Consider handling exceptions for driver.install_addon and driver.uninstall_addon to ensure
    the test suite continues to run even if addon installation or uninstallation fails. This
    can be done using a try-except block.

    examples/python/tests/browsers/test_firefox.py [94-106]

    -driver.install_addon(addon_path_xpi)
    -driver.uninstall_addon(id)
    +try:
    +    driver.install_addon(addon_path_xpi)
    +except Exception as e:
    +    print(f"Failed to install addon: {e}")
    +try:
    +    driver.uninstall_addon(id)
    +except Exception as e:
    +    print(f"Failed to uninstall addon: {e}")
     
    Reliability
    Add explicit waits to ensure elements are loaded before interaction.

    To improve test reliability, consider adding explicit waits after navigating to a page to
    ensure that the elements you are trying to interact with are fully loaded. This can be
    achieved using WebDriverWait along with expected conditions.

    examples/python/tests/browsers/test_firefox.py [96-108]

     driver.get("https://www.selenium.dev/selenium/web/blank.html")
    +WebDriverWait(driver, 10).until(EC.presence_of_element_located((webdriver.common.by.By.ID, "webextensions-selenium-example")))
     injected = driver.find_element(webdriver.common.by.By.ID, "webextensions-selenium-example")
     
    Maintainability
    Use a constant for the URL instead of hardcoding it.

    Instead of using a hardcoded URL "https://www.selenium.dev/selenium/web/blank.html",
    consider defining it as a constant at the beginning of your test file. This makes the code
    cleaner and easier to maintain, especially if the URL needs to be updated or reused.

    examples/python/tests/browsers/test_firefox.py [96-128]

    -driver.get("https://www.selenium.dev/selenium/web/blank.html")
    +SELENIUM_BLANK_PAGE = "https://www.selenium.dev/selenium/web/blank.html"
    +driver.get(SELENIUM_BLANK_PAGE)
     
    Refactor repeated code into a helper function for better maintainability.

    For better readability and maintainability, consider refactoring the repeated code for
    navigating to a URL and checking for the presence of an element into a separate helper
    function.

    examples/python/tests/browsers/test_firefox.py [96-128]

    -driver.get("https://www.selenium.dev/selenium/web/blank.html")
    -injected = driver.find_element(webdriver.common.by.By.ID, "webextensions-selenium-example")
    +def navigate_and_check_element(driver, url, element_id):
    +    driver.get(url)
    +    return driver.find_element(webdriver.common.by.By.ID, element_id)
     
    +injected = navigate_and_check_element(driver, "https://www.selenium.dev/selenium/web/blank.html", "webextensions-selenium-example")
    +
    Use a helper function to generate addon paths and reduce code duplication.

    To avoid code duplication in defining the paths for addon_path_xpi, addon_path_dir, and
    addon_path_dir_slash, consider creating a helper function that generates these paths based
    on input parameters. This will make the code cleaner and easier to maintain.

    examples/python/tests/conftest.py [97-121]

    -if os.path.abspath("").endswith("tests"):
    -    path = os.path.abspath("extensions/webextensions-selenium-example.xpi")
    -else:
    -    path = os.path.abspath("tests/extensions/webextensions-selenium-example.xpi")
    +def generate_addon_path(extension):
    +    if os.path.abspath("").endswith("tests"):
    +        return os.path.abspath(f"extensions/{extension}")
    +    else:
    +        return os.path.abspath(f"tests/extensions/{extension}")
     
    +addon_path_xpi = generate_addon_path("webextensions-selenium-example.xpi")
    +addon_path_dir = generate_addon_path("webextensions-selenium-example")
    +addon_path_dir_slash = generate_addon_path("webextensions-selenium-example/")
    +

    ✨ Improve tool usage guide:

    Overview:
    The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
    When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

    /improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...
    

    With a configuration file, use the following template:

    [pr_code_suggestions]
    some_config1=...
    some_config2=...
    
    Enabling\disabling automation

    When you first install the app, the default mode for the improve tool is:

    pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...]
    

    meaning the improve tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.

    Utilizing extra instructions

    Extra instructions are very important for the improve tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project.

    Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.

    Examples for extra instructions:

    [pr_code_suggestions] # /improve #
    extra_instructions="""
    Emphasize the following aspects:
    - Does the code logic cover relevant edge cases?
    - Is the code logic clear and easy to understand?
    - Is the code logic efficient?
    ...
    """
    

    Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

    A note on code suggestions quality
    • While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically.
    • Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base.
    • Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions 💎 tool
    • With large PRs, best quality will be obtained by using 'improve --extended' mode.
    More PR-Agent commands

    To invoke the PR-Agent, add a comment using one of the following commands:

    • /review: Request a review of your Pull Request.
    • /describe: Update the PR title and description based on the contents of the PR.
    • /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
    • /ask <QUESTION>: Ask a question about the PR.
    • /update_changelog: Update the changelog based on the PR's contents.
    • /add_docs 💎: Generate docstring for new components introduced in the PR.
    • /generate_labels 💎: Generate labels for the PR based on the PR's contents.
    • /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

    See the tools guide for more details.
    To list the possible configuration parameters, add a /config comment.

    See the improve usage page for a more comprehensive guide on using this tool.

    Copy link
    Member

    @diemol diemol left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Could you please sign the CLA?

    @jkbzh
    Copy link
    Contributor Author

    jkbzh commented Mar 18, 2024

    Hi @diemol ,

    Could you please sign the CLA?

    I already signed it the day I opened the first PR. If open the CLA link, it shows the signed agreement. It may be something has issues with my having done the commit with a different user name or email address than those on my github account? But the commit and PR were sent directly from my gh account and I'm the same person.

    cla

    Copy link
    Member

    @diemol diemol left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I think the issue is the commit does not have your email associated:
    image

    @jkbzh
    Copy link
    Contributor Author

    jkbzh commented Mar 19, 2024

    I think the issue is the commit does not have your email associated
    @diemol Thanks. I think I fixed it.

    Copy link
    Member

    @diemol diemol left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Thank you, @jkbzh!

    @diemol diemol merged commit 14a74c9 into SeleniumHQ:trunk Mar 19, 2024
    6 checks passed
    selenium-ci added a commit that referenced this pull request Mar 19, 2024
    test_install_addon_directory() was being called with an xpi file
    fixture instead of a directory fixture.
    
    I fixed this by making a new fixture for
    test_install_addon_directory() that points to the test extension
    directory. See details here below.
    
    I also added an additional test and fixture so that the tests
    take into account whether a path to an extension directory ends
    with a trailing slash. See details here below.
    
    This required renaming the existing addon_path fixture to avoid
    confusion among the three related fixtures:
    
    - addon_path_xpi
    
      renamed fixture gives a path to the xpi file (was addon_path)
    
      updated test_install_add() and test_unistall_addon()
    
    - addon_path_dir
    
      new fixture that gives a path to the extension directory,
      ending without a '/'
    
      updated test_install_unsigned_addon_directory() to use it
    
    - addon_path_dir_slash (new fixture, and test)
    
      new fixture that gives a path to the extension directory,
      ending with a '/'
    
      added new test: test_install_unsigned_addon_directory_slash()
    
    Note that I don't have the appropriate environment to run these tests
    before pushing them.
    
    [deploy site] 14a74c9
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    3 participants