Adds headless tests for example apps #2478

infomiho · 2025-01-31T18:10:18Z

Martinsos · 2025-03-14T16:56:23Z

What are next steps here?

sodic

Excellent work!

But I can't decide whether we should remove most of the tests. They're pretty repetitive and unlikely to fail if waspc/todoApp works. Then again, we run them manually every time...

Another big repetition is the playwright config, so we should decide whether to do something about that too.

sodic · 2025-03-17T18:36:30Z

examples/hackathon-submissions/.gitignore

+# Headless tests
+!.env.*.headless
+test-results/


Hmm, I may have changed my mind on this grouping 😅

It makes sense to kepp all the headless stuff together. But netlify should definitely be pushed up.

This app is now gone, but the headless ignore patterns are grouped together in all other apps as well 👍

examples/hackathon-submissions/headless-tests/playwright.config.ts

examples/hackathon-submissions/src/projects.ts

sodic · 2025-03-17T18:38:29Z

examples/streaming/headless-tests/tests/simple.spec.ts

Since this app is supposed to test streaming, can we somehow write the test to check for that (not just the final content)?

I've migrated streaming to todoApp with a test that you mention, you'd be pleased 👍

sodic · 2025-03-17T18:40:23Z

examples/tutorials/TodoApp/headless-tests/tests/helpers.ts

+  await page.locator("button").click();
+}
+
+export async function performLogin(


We could probably skip on testing the signup and the login in all apps. It's a lot of repetitive code that brings little benefit I'd say.

However, if and where we test for it (e.g., in waspc/todoApp), we should also test for unsuccessful logins and signups.

Jumping in a bit, but I see this is in helper.ts and I guess login is just needed to test the rest of the app, so probably no point in trying to do less of it?

Yep, testing login and signup is something we do along the way because we need to authenticate anyways to test the apps.

sodic · 2025-03-17T18:41:32Z

examples/tutorials/TodoApp/headless-tests/tests/simple.spec.ts

+    // Fill input[name="description"] with random task
+    await page.locator("input[name='description']").fill(randomTask);
+    // Click input[type="submit"] to submit the form
+    await page.locator("input[type='submit']").click();
+    // Expect to see the task on the page
+    await expect(page.locator("body")).toContainText(randomTask);
+    // Check the task as done input[type="checkbox"]
+    await page.locator("input[type='checkbox']").click();
+    // Reload the page
+    await page.reload();
+    // Expect the task to be checked
+    await expect(page.locator("input[type='checkbox']")).toBeChecked();
+  });


The comments are probably gpt leftovers, we can remove them :)

Yep, probably me telling Copilot what to write (or I generated this with Playwright's extension, not sure).

examples/tutorials/TodoAppTs/headless-tests/tests/simple.spec.ts

Signed-off-by: Mihovil Ilakovac <[email protected]>

infomiho · 2025-05-19T13:18:36Z

examples/waspello/src/auth/LoginPage.tsx

@@ -8,16 +8,16 @@ import addWaspSourceHeader from '../common/addWaspSourceHeader'

 import mainLogo from '../common/waspello-logo.svg'


Move to Typescript to cleanup a bit, there was a type error hiding in the Signup page, so I wanted to convert the Login as well.

infomiho · 2025-05-19T13:19:38Z

examples/waspello/src/auth/SignupPage.tsx

@@ -7,18 +7,16 @@ import addWaspSourceHeader from '../common/addWaspSourceHeader'



There was a type error (which I didn't notice) when using this code with the latest released Wasp, but this code works with main Wasp CLI. Cleaned up and converted to Typescript just in case.

infomiho · 2025-05-19T13:19:54Z

examples/waspello/tsconfig.json

@@ -11,38 +11,18 @@
 {


Migrated to latest tsconfig.json

infomiho · 2025-05-21T10:58:28Z

examples/websockets-realtime-voting/src/pages/MainPage.tsx

@@ -1,13 +1,21 @@
-import { type ServerToClientPayload, useSocket, useSocketListener } from "wasp/client/webSocket";


Most of this file is formatting, but I added some classes for easier headless testing

sodic

Went through it again.

Everything looks ok functionally, but I'm still uneasy about the duplication:

playwright.config.ts files seem to be identical, at least we intend them to be. But they're already drifting apart through small unintended differences (e.g., quotes and variables, check screenshot below).
I believe the same applies for simple.spec.ts files.

Can we do anything about this? It doesn't need to be fancy. For example, we could symlink them all to the same file.

sodic · 2025-05-22T09:12:36Z

examples/waspello/src/auth/LoginPage.tsx

-    } catch (err) {
-      if (err.statusCode == 422) {
+    } catch (err: any) {
+      // TODO: Update this to check against WaspHttpError https://github.com/wasp-lang/wasp/issues/2767


Can we mention this in the issue? We'll otherwise forget to update it.

sodic · 2025-05-22T09:13:15Z

examples/waspello/src/auth/SignupPage.tsx

-    } catch (err) {
-      if (err.statusCode == 422) {
+    } catch (err: any) {
+      // TODO: Update this to check against WaspHttpError https://github.com/wasp-lang/wasp/issues/2767


Also please mention it in the issue.

infomiho · 2025-05-22T10:20:56Z

Everything looks ok functionally, but I'm still uneasy about the duplication:

I get you, I agree with you on the config duplication, but in the tests I wouldn't try to be fancy since most of the work is just replicating what users would do manually while testing the app. t might get quite messy if we try to abstract that away.

Can we do anything about this? It doesn't need to be fancy. For example, we could symlink them all to the same file.

I guess we are trying to solve the issue of sharing JS code between multiple different Wasp apps. I'm not sure symlinking is the best way to go (it feels OS specific to me, correct if I'm wrong). This is one argument for publishing the wasp-app-runner as an npm package which would enable use to centralise our common headless apps setup, but then I'd rename it into wasp-headless-tests-utils or smth like that, so it reflects it's wider scope.

Maybe we can locally install the wasp-app-runner package? But then it's not accessible to open-saas for example.

sodic

Talked out of band, we'll handle the duplication separately. @infomiho Please link the issue when you create it.

Good to go!

infomiho · 2025-05-22T12:34:50Z

Great, thanks! I'll merge this then and here's the issue for deduplication effort: #2790

infomiho force-pushed the miho-merge-todo-apps branch from e1ee1c0 to 4a9c181 Compare March 5, 2025 11:51

infomiho changed the base branch from miho-merge-todo-apps to miho-wasp-app-runner March 5, 2025 12:07

infomiho force-pushed the miho-example-apps-headless-tests branch from 3ea3f9d to 494a31d Compare March 5, 2025 12:57

sodic reviewed Mar 17, 2025

View reviewed changes

infomiho force-pushed the miho-wasp-app-runner branch from 119a4b4 to f7cb11b Compare April 4, 2025 14:49

Base automatically changed from miho-wasp-app-runner to main April 9, 2025 10:12

infomiho added 2 commits April 9, 2025 12:26

Add headless tests for example apps

ab1d91a

Signed-off-by: Mihovil Ilakovac <[email protected]>

Fix hackathon-submisisons tests

a8ea42c

infomiho force-pushed the miho-example-apps-headless-tests branch from f0e42a0 to a8ea42c Compare April 9, 2025 10:32

Updated example apps package*.json files

af79320

Signed-off-by: Mihovil Ilakovac <[email protected]>

FranjoMindek assigned infomiho Apr 18, 2025

infomiho added 2 commits May 19, 2025 14:22

Merge branch 'main' into miho-example-apps-headless-tests

81f09f8

Update tests to work with latest changes

9dc07b4

infomiho commented May 19, 2025

View reviewed changes

infomiho requested a review from sodic May 19, 2025 13:24

infomiho added 2 commits May 19, 2025 15:26

Cleanup

5acf4e4

Missing newline

7378cc1

infomiho commented May 21, 2025

View reviewed changes

sodic reviewed May 22, 2025

View reviewed changes

infomiho mentioned this pull request May 22, 2025

Export WaspHttpError from wasp/client/api #2767

Open

sodic approved these changes May 22, 2025

View reviewed changes

infomiho mentioned this pull request May 22, 2025

Reduce duplication between headless tests #2790

Open

infomiho merged commit abdd636 into main May 22, 2025

infomiho deleted the miho-example-apps-headless-tests branch May 22, 2025 12:35

		@@ -8,16 +8,16 @@ import addWaspSourceHeader from '../common/addWaspSourceHeader'

		import mainLogo from '../common/waspello-logo.svg'

		@@ -7,18 +7,16 @@ import addWaspSourceHeader from '../common/addWaspSourceHeader'

		@@ -1,13 +1,21 @@
		import { type ServerToClientPayload, useSocket, useSocketListener } from "wasp/client/webSocket";

Uh oh!

Adds headless tests for example apps #2478

Adds headless tests for example apps #2478

Uh oh!

Conversation

infomiho commented Jan 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Martinsos commented Mar 14, 2025

Uh oh!

sodic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sodic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

infomiho commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sodic left a comment

Choose a reason for hiding this comment

Uh oh!

infomiho commented May 22, 2025

Uh oh!

Uh oh!

infomiho commented Jan 31, 2025 •

edited

Loading

infomiho commented May 22, 2025 •

edited

Loading