Skip to content

add message checking to integration tests #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 4, 2024

Conversation

Arthurm1
Copy link
Contributor

@Arthurm1 Arthurm1 commented Apr 4, 2024

Currently the integration tests only handle direct replies to BSP messages e.g. workspace/buildTargets returns WorkspaceBuildTargetsResult

This PR allows monitoring of async messages e.g. build/taskProgress and adds tests for these.

It's the first step in testing progress reports per build target and eventually diagnostic reports.

It also gets rid of the global Launcher.client which I think is a good thing.

Also fixed: the onBuildTaskProgress notification has a compile-task data kind so data must contain a CompileTask

@jdneo
Copy link
Member

jdneo commented Apr 24, 2024

Had a quick look and the idea looks good to me.

Could you please resolve the conflict?

@Arthurm1 Arthurm1 force-pushed the improve_integ_tests branch from 36e73b5 to 21aef0c Compare April 24, 2024 10:06
@Arthurm1
Copy link
Contributor Author

@jdneo Done

@Arthurm1 Arthurm1 force-pushed the improve_integ_tests branch from 21aef0c to 593001e Compare April 25, 2024 10:29
@Arthurm1
Copy link
Contributor Author

rebased

@jdneo jdneo self-requested a review April 25, 2024 16:45
@Arthurm1 Arthurm1 force-pushed the improve_integ_tests branch from 593001e to 4566a7b Compare April 28, 2024 16:15
Copy link
Member

@jdneo jdneo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jdneo jdneo merged commit 59b80ec into microsoft:develop May 4, 2024
@jdneo
Copy link
Member

jdneo commented May 4, 2024

Thank you @Arthurm1!

@Arthurm1 Arthurm1 deleted the improve_integ_tests branch May 4, 2024 11:07
CompileResult compileResult = gradleBuildServer.buildTargetCompile(compileParams).join();
assertEquals(StatusCode.OK, compileResult.getStatusCode());
client.waitOnMessages(2, 2, 1);
BuildTargetIdentifier testBt = getBt(btIds, "/junit5-jupiter-starter-gradle/", "test");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Arthurm1 Does this case pass on your side? Due to the current implementation at:

// Note: this might be anti-sepc, because the spec limits that one compile task report
// can only have one build target id. While we aggregate all compile related tasks into one
// Gradle call for the perf consideration. So, the build target id passed into the reporter
// is not accurate.
TaskProgressReporter reporter = new TaskProgressReporter(
new CompileProgressReporter(client, btIds.iterator().next()));
, it should fail.

It fails on my machine, but I have no idea why it passes in GitHub Actions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdneo Yes it passes on mine. What line does it fail on - 324? The messages I have at that point are...
image
It's possible you have additional messages. I noticed that the CompletableFuture.runAsync(this::reloadWorkspace); triggered after a compile will cause more messages to be sent to the client but that's normally too slow to register. I have fix for this and can submit a PR, but it's only ever happened to me when I've sat on a breakpoint for a while.

Copy link
Member

@jdneo jdneo May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this limitation causes the failure:

Target id for both main and test are sent to be compiled, and the task classes and testClasses are executed in one invocation. But because of

only main target id is set to the compile reporter, which fails line 326?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdneo This makes sense - it could be failing on your machine due to message ordering differences. I've finished off #116 which should fix this if you'd like to review it. It reports on progress-per-target rather than just reporting everything on the first target.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! We can eliminate a protocol violation then. I'll review it these days. Thank you!

@jdneo jdneo added this to the 0.2.0 milestone Jun 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants