Skip to content

Speed up Go tests by fixing the slow fleetctl tests #27927

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
7 tasks
getvictor opened this issue Apr 6, 2025 · 1 comment
Open
7 tasks

Speed up Go tests by fixing the slow fleetctl tests #27927

getvictor opened this issue Apr 6, 2025 · 1 comment
Assignees
@getvictor
Labels
~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. #g-mdm MDM product group :product Product Design department (shows up on 🦢 Drafting board) story A user story defining an entire feature

Comments

@getvictor
Copy link
Member

getvictor commented Apr 6, 2025 •
edited by lukeheath
Loading

Goal

User story
As a contributor,
I want to I want to see the fleetctl CI test results as soon as possible
so that I can so that I can merge my changes and increase dev velocity.

Testing the fleetctl package is currently the long pole in our CI tests. The package needs to be refactored so that tests can run in parallel. The issue is that fleetctl is a main package -- it needs to be split into main, fleetctl, and integrationtest packages.

image.png

Key result

Original requests

Context

Changes

Engineering

  • Test plan is finalized
  • Contributor API changes: TODO
  • Feature guide changes: TODO
  • Database schema migrations: TODO
  • Load testing: TODO

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

  • Requires load testing: TODO
  • Risk level: Low / High TODO
  • Risk description: TODO

Test plan

Make sure to go through the list and consider all events that might be related to this story, so we catch edge cases earlier.

  1. Step 1
  2. Step 2
  3. Step 3

Testing notes

Confirmation

  1. Engineer: Added comment to user story confirming successful completion of test plan.
  2. QA: Added comment to user story confirming successful completion of test plan.
@getvictor getvictor added story A user story defining an entire feature ~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. labels Apr 6, 2025
@getvictor getvictor mentioned this issue Apr 6, 2025
Merged
getvictor added a commit that referenced this issue Apr 7, 2025
@getvictor
Improvements to test-go CI job (#27916)
7558d5a 
This PR makes several improvements to test-go CI job
- remove ZSH dependency from all test suites except for a new `scripts`
suite
- add a `fast` suite that does not have out-of-process dependencies,
which completes in ~1.5 minutes
- contributors can add their fast tests to this suite so they can see
the results in CI faster
- Rename `core` to `main` test suite to be consistent with Makefile. It
is the default bucket for tests.
- Cleaned up Makefile so that it is more straightforward to add new test
suites or move Go packages between suites
- Do not stop the test suites on a fail.
- We do not want to be blocked by a test fail that another product team
introduced
- Sometimes, we want to see all test failures so we can fix them all at
once.
- Removed `test-schema` and `mock` prerequisites for `test-go` since
they are not needed and just take up time.
- But also added `test-schema` run to one of the test suites just in
case.

Unfortunately, `fleetctl` is still the bottleneck and needs to be
refactored. New issue filed:
#27927

## Before
<img width="248" alt="image"
src="https://github.com/user-attachments/assets/110ffc1d-f090-4d3e-be77-0419b9577d20"
/>

## After
<img width="320" alt="image"
src="https://github.com/user-attachments/assets/8d01ea11-408f-4eb6-81d8-9c25410b8830"
/>
@getvictor getvictor mentioned this issue Apr 11, 2025
Open
@lukeheath lukeheath added the :product Product Design department (shows up on 🦢 Drafting board) label Apr 14, 2025
@lukeheath
Copy link
Member

@getvictor Thanks for filing. Improving our CI run time pays dividends and is a good investment. Prioritizing to the drafting board for you to complete the drafting process.

@noahtalerman noahtalerman added the #g-mdm MDM product group label Apr 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@getvictor getvictor

Labels
~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. #g-mdm MDM product group :product Product Design department (shows up on 🦢 Drafting board) story A user story defining an entire feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lukeheath @getvictor @noahtalerman