Full-stack compile-time type safety #1090

sodic · 2023-03-30T17:08:21Z

Fixes #976.

To keep TypeScript from lying to the user, we had to add a data serialization/transformation layer. I've decided to use superjson for the time being (thanks @Zeko369!).

In the future, I hope we can make this transformer configurable (as tRPC does).

Motivating example

To understand why we had to use Superjson, let's see what would happen if we implemented #976 without it.

Imagine you had the following query:

export const getPerson: GetPerson<{ id: number }, { name: string, dob: Date }> = ({ id }, context) => {
  // You do some processing and return a value of type { name: string, dob: Date }.
  // All is well with the type checker.
}

Then, you do this on the client:

import getPerson from '@wasp/queries/getPerson'

const person = getPerson({ id: 12 })

TypeScript will tell you that person has type { name: string, dob: Date }. This is probably the type you wanted, but not the one you actually have:

Your query returns a value of type { name: string, dob: Date }, so far so good
Experss middleware serializes it into a JSON, but JSON serializes Dates into strings
The payload is sent over the network.
Axios deserializes the payload back into an object of type { name: string, dob: string }, but the type checker things you have an object of type { name: string, dob: Date }.
Chaos

Todo list:

Ensure proper serialization/deserialization to ensure types match runtime types
Clean up the Haskell code and reorganize it
Implement full-stack type safety for actions
Discuss whether server imports are a problem or not (could be creating technical debt)
Require operation Input and Output types to extend superjson's serializable type

…k-type-safety

waspc/src/Wasp/Generator/WebAppGenerator/OperationsGenerator.hs

sodic · 2023-04-05T14:31:39Z

waspc/data/Generator/templates/react-app/src/auth/useAuth.ts

+// todo(filip): turn into a proper import
+import { type SanitizedUser } from '../../../server/src/_types/' 


@infomiho Is this something that we should be handling differently after you've implemented the new importing system?

You don't really need to use the new import style, it only helps you to define a JS import statement in Haskell and then print it in a JS file.

If want to ensure that this import is importing from the proper place (server/src) you can Strong Path it and define a new JSImport and then output it in the template. This would move the information about this weird import in Haskell, which would be a win for us IMHO.

Nit: remove trailing/

sodic · 2023-04-06T10:20:39Z

waspc/data/Generator/templates/react-app/src/queries/_query.ts

+const query = createQuery<{= operationTypeName =}>(
+  '{= queryRoute =}',
+  {=& entitiesArray =},
+)
+
+export default query


Although we decided to abandon default exports (see #546 for details), we're still postponing it. It requires many breaking changes in our API (and the docs).

In the meantime, I separated the export from its definition because it helps with VS Code's "on hover" feature.

When exporting the value directly:

When exporting the variable after the assignment:

sodic · 2023-04-06T11:42:43Z

waspc/data/Generator/templates/universal/types.ts

+// TypeScript's native Awaited type exhibits strange behavior in VS Code (see
+// todo for details). Until it's fixed, we're using our own type for this.
+export type _Awaited<T extends Promise<unknown>> = T extends Promise<infer V>
+  ? V
+  : never


Take a look at this playground and try the same snippet in VS Code. This strange behavior causes a problem for operations with unspecified inputs.

Example

Let's say your backend query definition looks like this (i.e., input and output types are omitted):

export const getTasks: GetTasks = async (_args, context) => { // ... }

This is what we want TypeScript to infer on the client:

Unfortunately, if we use the built-in Awaited type, this happens:

sodic · 2023-04-06T11:44:43Z

waspc/data/Generator/templates/react-app/src/test/vitest/helpers.tsx

@@ -26,10 +26,11 @@ export function renderInContext(ui: ReactElement): RenderResult {
 }

 type QueryRoute = Query<any, any>['route']
+type MockQuery = <Input, Output, MockData extends Output>(query: Query<Input, Output>, resJson: MockData) => void


@shayneczyzewski I've changed this signature to improve type safety. Check this playground for the demo.

Sounds good to me, thanks!

sodic · 2023-04-06T11:50:40Z

waspc/data/Generator/templates/react-app/src/queries/index.d.ts

 }

-export function useQuery<Input, Output, Error = unknown>(
+export function useQuery<Input, Output>(


We always throw something that resembles a general Error, so let's keep it simple for now.

sodic · 2023-04-06T11:55:28Z

waspc/data/Generator/templates/react-app/src/queries/core.d.ts

+type QueryFor<BackendQuery extends GenericBackendQuery> = Expand<
+  Query<Parameters<BackendQuery>[0], _Awaited<ReturnType<BackendQuery>>>
+>


Applying Expand to this type results in a better DX.

On-hover with Expand:

On-hover without Expand:

sodic · 2023-04-06T12:27:53Z

waspc/data/Generator/templates/react-app/src/actions/index.ts

+export type Action<Input, Output> = 
+  [Input] extends [never] ? 
+  (args?: unknown) => Promise<Output> :
+  (args: Input) => Promise<Output>


To understand why we need a conditional type here and why it looks the way it looks, we need to establish a couple of things.

Default input and output types for Operations

Users can specify their operation's Input and Output types like this:

export const updateTaskIsDone: UpdateTask<Pick<Task, "id">, Task> = async ({ id }, context) => { // ... }

We also want to allow them to omit input and output types. Sure, they won't get full-stack type safety, but it helps rapid prototyping:

export const updateTaskIsDone: UpdateTask = async (_args, context) => { // ... }

The main question is: What should be the default input and output types?

Assuming we don't want to use any unless absolutely necessary, the correct answer is: never for the input, and unknown for the output. Read #989 (comment) to understand why.

Frontend behavior with omitted input types

Assuming the user decides not to specify input and output types:

export const updateTaskIsDone: UpdateTask = async (_args, context) => { // ... }

We want the client-side RPC call to accept either nothing or anything. More precisely, all following calls should pass the type checker:

updateTaskIsDone() updateTaskIsDone(12) updateTaskIsDone("asdf")

To implement such behavior, we need to detect whether a backend action's Input type is never and type the frontend action's argument appropriately:

If the Input type is never (i.e. the user hasn't specified an input type), the action should take a single optional argument of type unknown.

If the Input type isn't never (i.e., the user has specified an input type), the action should simply use the user-specified input type.

Disabling the distribution of never

All that's left to answer is Why do we wrap the types inside an array?

The following type wouldn't work (playground link):

export type Action<Input, Output> = Input extends never ? (args?: unknown) => Promise<Output> : (args: Input) => Promise<Output> type X = Action<never, number> // X is 'never', not (args?: unknown) => Promise<number>

This is because conditional types (in our case, Action) distribute over unions (in our case, the empty union never). We need a way to prevent the distribution, and that's what the array wrapper is for.

Read microsoft/TypeScript#31751 (comment) for more details.

I appreciate the depth of this comment. The TS is still a bit of magic to me that enables it, but I appreciate how much thought went into this! :D

sodic · 2023-04-06T12:33:17Z

waspc/data/Generator/templates/react-app/src/actions/_action.ts

Mostly the same as former _action.js, but Git didn't detect it as a rename.

sodic · 2023-04-06T12:41:29Z

waspc/data/Generator/templates/react-app/src/auth/useAuth.ts

@@ -1,12 +1,14 @@
 import { useQuery } from '../queries'
 import api, { handleApiError } from '../api'
 import { HttpMethod } from '../types'
+// todo(filip): turn into a proper import
+import { type SanitizedUser } from '../../../server/src/_types/' 

 export default function useAuth(queryFnArgs, config) {


I've decided not to go with the module boundaries principle (wasp-lang/vscode-wasp#6 (comment)) to avoid unnecessary verbosity.

This short function that's basically a wrapper to two other (explicitly typed) public functions: useQuery and getMe. If that changes, we'll make the type signature explicit.

sodic · 2023-04-06T12:44:03Z

waspc/data/Generator/templates/react-app/src/operations/index.ts

-    const response = await api.post(operationRoute.path, args)
-    return response.data
+    const superjsonArgs = superjsonSerialize(args)
+    const response = await api.post(operationRoute.path, superjsonArgs)


We're still relying on axios to fully serialize a "semi-serialized" superjson object under the hood. This happens for both the request and the response.

More precisely, Axios will turn { json: "...", meta: "..." } (object) into '{"json": "...", "meta": "..." }' (string).

The same thing happens on the server. I describe the process in more detail there: #1090 (comment)

sodic · 2023-04-06T12:46:36Z

waspc/data/Generator/templates/server/src/actions/_action.ts

@@ -7,7 +7,7 @@ import prisma from '../dbClient.js'
          consider in the future if it is worth removing this duplication. =}

 export default async function (args, context) {
-  return {= jsFn.importIdentifier =}(args, {
+  return {= jsFn.importIdentifier =}(args as never, {


This is pretty ridiculous, but is the quickest possible way to fix a type issue caused by the requirements described here: #1090 (comment).

We'll come up with a more long-term solution eventually.

Todo: make an issue

sodic · 2023-04-06T12:52:53Z

waspc/data/Generator/templates/server/src/routes/operations/_action.js

 {=& operationImportStmt =}

 export default handleRejection(async (req, res) => {
-  const args = req.body || {}
+  const args = (req.body && superjsonDeserialize(req.body)) || {}


TODO: I should probably be catching something here.

Maybe write a helper function called deserializeRequestBody or similar so it's more readable.

sodic · 2023-04-06T12:59:49Z

waspc/data/Generator/templates/server/src/routes/operations/_action.js

 {=& operationImportStmt =}

 export default handleRejection(async (req, res) => {
-  const args = req.body || {}
+  const args = (req.body && superjsonDeserialize(req.body)) || {}


Our deserialization process on the server is analogous to the one on the client: #1090 (comment)

More precisely:

When deserializing, Express will turn '{"json": "...", "meta": "..." }' (string) to { json: "<serialized_data>", meta: ".<metadata>" } (object) , and we'll do the rest with superjsonDeserialize. The end result is data.

When serializing, we will turn data into { json: "<serialized_data>", meta: ".<metadata>" } (object) using superjsonSerialize, and Express will do the rest. The end result is '{"json": "<serialized_data>", "meta": "<metadata>" }' (string).

Martinsos · 2023-04-06T14:26:41Z

waspc/src/Wasp/Generator/WebAppGenerator/OperationsGenerator.hs

+import Wasp.Generator.ServerGenerator.Common (serverSrcDirInServerRootDir)
+import Wasp.Generator.ServerGenerator.OperationsG (operationFileInSrcDir)


Maybe import these qualified in a way that makes it clear this is coming from Server. For serverSrcDirInServerRootDir it is clear, but for operationFileInSrcDir it is not.

infomiho · 2023-04-06T14:42:47Z

waspc/data/Generator/templates/react-app/src/operations/resources.js

@@ -11,7 +11,7 @@ const updateHandlers = makeUpdateHandlersMap(hashQueryKey)
 /**


Nit: maybe convert this to Typescript, it doesn't seem too complex

I'm afraid

No time unfortunately.

infomiho · 2023-04-06T14:49:30Z

waspc/data/Generator/templates/react-app/src/test/vitest/helpers.tsx

@@ -40,14 +41,15 @@ export function mockServer(): {
  })
  afterAll(() => server.close())

-  function mockQuery({ route }: { route: QueryRoute }, resJson: any): void {
+  const mockQuery : MockQuery = (query, mockData) => {


Nit

Suggested change

const mockQuery : MockQuery = (query, mockData) => {

const mockQuery: MockQuery = (query, mockData) => {

infomiho · 2023-04-06T14:50:04Z

waspc/data/Generator/templates/react-app/src/test/vitest/helpers.tsx

@@ -65,3 +67,9 @@ export function mockServer(): {

  return { server, mockQuery }
 }
+
+type InternalQuery<Input, Output> = {


We are hiding the route from user?

Yes, for the time being, reasoning here: #1055 (comment) (spoiler: not much of a reasoning)

infomiho · 2023-04-06T14:50:21Z

waspc/data/Generator/templates/server/src/_types/index.ts

@@ -72,20 +73,11 @@ type Context<Entities extends _Entity[]> = Expand<{
 }>

 {=# isAuthEnabled =}
-type ContextWithUser<Entities extends _Entity[]> = Expand<Context<Entities> & UserInContext>
+type ContextWithUser<Entities extends _Entity[]> = Expand<Context<Entities> & { user: SanitizedUser}>


Nit

Suggested change

type ContextWithUser<Entities extends _Entity[]> = Expand<Context<Entities> & { user: SanitizedUser}>

type ContextWithUser<Entities extends _Entity[]> = Expand<Context<Entities> & { user: SanitizedUser }>

shayneczyzewski · 2023-04-06T14:52:40Z

waspc/data/Generator/templates/server/src/routes/apis/index.ts

+router.{= routeMethod =}(
+  '{= routePath =}',
+  handleRejection(


This change loses the auth middleware (old line 18), which will put the user into the context. Without it, when you go to the dashboard you get the following error:

Server(stderr): TypeError: Cannot read properties of undefined (reading 'username') Server(stderr): at fooBar (file:///home/shayne/dev/wasp/waspc/examples/todoApp/.wasp/out/server/dist/ext-src/apis.js:3:44) Server(stderr): at file:///home/shayne/dev/wasp/waspc/examples/todoApp/.wasp/out/server/dist/routes/apis/index.js:14:12 Server(stderr): at file:///home/shayne/dev/wasp/waspc/examples/todoApp/.wasp/out/server/dist/utils.js:15:15

infomiho · 2023-04-06T14:54:07Z

waspc/data/Generator/templates/universal/types.ts

+// compiler, but expands the type's representatoin in IDEs (i.e., inlines all
+// type constructors) to make it more readable for the user.
+//
+// It expands this SO answer to funcitons: https://stackoverflow.com/a/57683652


Suggested change

// It expands this SO answer to funcitons: https://stackoverflow.com/a/57683652

// It expands this SO answer to functions: https://stackoverflow.com/a/57683652

infomiho · 2023-04-06T14:55:33Z

waspc/examples/todoApp/src/client/App.tsx


  return (
    <div className="app border-spacing-2 p-4">
      <header className="flex justify-between">
        <h1 className="font-bold text-3xl mb-5">
          <Link to="/">ToDo App</Link>
        </h1>
+        <h2>
+          Your site was loaded at: {date?.toLocaleString()}


Like in the good old PHP websites.

infomiho · 2023-04-06T14:57:23Z

waspc/examples/todoApp/src/client/pages/Task.tsx

@@ -13,9 +13,9 @@ type TaskPayload = Pick<Task, "id" | "isDone">
 const Todo = (props: any) => {
  const taskId = parseInt(props.match.params.id)

-  const { data: task, isFetching, error } = useQuery<Pick<Task, "id">, Task>(getTask, { id: taskId })


infomiho · 2023-04-06T14:58:29Z

waspc/examples/todoApp/todoApp.wasp

@@ -29,7 +29,7 @@ app todoApp {
    setupFn: import setup from "@server/serverSetup.js"
  },
  client: {
-    rootComponent: import { App } from "@client/App.jsx",
+    rootComponent: import { App } from "@client/App",


We wanted to keep the extensions in the main.wasp so it's not confusing that we MUST include them in backend but they are OPTIONAL in the client. 🤷‍♂️

Changing it back... and the worst thing is: I think it was my suggestion 😄

sodic · 2023-04-06T16:51:18Z

waspc/data/Generator/templates/react-app/src/test/vitest/helpers.tsx

Most of the changes in the file are prettier formatting.

sodic · 2023-04-06T16:52:09Z

waspc/data/Generator/templates/react-app/src/test/vitest/helpers.tsx

+export type Route = { method: HttpMethod; path: string }
+
+export type MockQuery = <Input, Output, MockOutput extends Output>(
+  query: Query<Input, Output>,
+  resJson: MockOutput
+) => void
+
+export type MockApi = (route: Route, resJson: unknown) => void


I've decided to export these guys because the rule of thumb for exporting types is: If users can dig it out themselves (e.g., using utility types like Parameters), make it public.

sodic · 2023-04-06T16:56:18Z

waspc/data/Generator/templates/react-app/src/auth/useAuth.ts


-export default function useAuth(queryFnArgs, config) {
+export default function useAuth(queryFnArgs?: unknown, config?: any) {


It seems like useQuery will send queryFnArgs to getMe, but getMe doesn't expect them so they end up ignored.

Todo: make an issue for this or investigate.

…-safety

shayneczyzewski

Functionally looks good to me! Great job @sodic! Much of the TS is still over my head, but I can see the benefits of this already. :D 🥳 🚀

infomiho

LGTM, it's really cool to get type in FE just like that 🤯

…-safety

sodic · 2023-04-06T18:50:16Z

I'm merging to get RC out asap, but will take a look at what else to do tomorrow.

sodic added 2 commits March 30, 2023 19:04

Add first draft of typesafe query RPCs

de229dd

Merge branch 'main' of github.com:wasp-lang/wasp into filip-full-stac…

83e546a

…k-type-safety

sodic commented Mar 31, 2023

View reviewed changes

waspc/src/Wasp/Generator/WebAppGenerator/OperationsGenerator.hs Outdated Show resolved Hide resolved

Solve type errors in tests

358abb1

infomiho self-requested a review April 3, 2023 12:19

sodic added 4 commits April 3, 2023 18:05

Improve types in frontend tests

56be970

Remove redundant import

3beb15b

Make useAuth hook type safe

534c0e7

Full stack type safety for actions

b00633c

sodic commented Apr 4, 2023

View reviewed changes

waspc/src/Wasp/Generator/WebAppGenerator/OperationsGenerator.hs Outdated Show resolved Hide resolved

sodic added 2 commits April 5, 2023 00:59

Clean up haskell code

48d2276

Add superjson to operations

d365de6

sodic commented Apr 5, 2023

View reviewed changes

Make Expand type universal

8c6ed5b

sodic commented Apr 6, 2023

View reviewed changes

Fix Awaited type issue

518e6ec

sodic commented Apr 6, 2023

View reviewed changes

Fix Action type condition

f526bcb

sodic commented Apr 6, 2023

View reviewed changes

Fix useAuth arguments

060efcc

sodic commented Apr 6, 2023

View reviewed changes

Fix type error for unspecified queries

7670381

sodic commented Apr 6, 2023

View reviewed changes

sodic requested review from Martinsos and removed request for Martinsos April 6, 2023 13:53

sodic marked this pull request as ready for review April 6, 2023 14:12

Martinsos approved these changes Apr 6, 2023

View reviewed changes

infomiho reviewed Apr 6, 2023

View reviewed changes

shayneczyzewski reviewed Apr 6, 2023

View reviewed changes

infomiho reviewed Apr 6, 2023

View reviewed changes

sodic added 2 commits April 6, 2023 17:45

Fix small issues

e9baffb

Fix testing issues

0d83b17

sodic commented Apr 6, 2023

View reviewed changes

waspc/data/Generator/templates/react-app/src/test/vitest/helpers.tsx

Copy link

Contributor Author

sodic Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the changes in the file are prettier formatting.

sodic commented Apr 6, 2023

View reviewed changes

sodic added 3 commits April 6, 2023 19:10

Fix small typos

6ebea83

Serialize auth/me on the backend for consistency

90ea331

Merge remote-tracking branch 'origin/main' into filip-full-stack-type…

d174e81

…-safety

shayneczyzewski approved these changes Apr 6, 2023

View reviewed changes

infomiho approved these changes Apr 6, 2023

View reviewed changes

Merge remote-tracking branch 'origin/main' into filip-full-stack-type…

d2af83c

…-safety

Update e2e tests

833d5f0

sodic merged commit 945d777 into main Apr 6, 2023

sodic deleted the filip-full-stack-type-safety branch April 6, 2023 19:12

sodic mentioned this pull request Apr 8, 2023

Ensure users can't send unserializable payloads #1119

Closed

sodic mentioned this pull request May 10, 2024

Replace superjson with a generated serialization layer #2027

Open

		// todo(filip): turn into a proper import
		import { type SanitizedUser } from '../../../server/src/_types/'

		import Wasp.Generator.ServerGenerator.Common (serverSrcDirInServerRootDir)
		import Wasp.Generator.ServerGenerator.OperationsG (operationFileInSrcDir)

		@@ -11,7 +11,7 @@ const updateHandlers = makeUpdateHandlersMap(hashQueryKey)
		/**

	const mockQuery : MockQuery = (query, mockData) => {
	const mockQuery: MockQuery = (query, mockData) => {

	type ContextWithUser<Entities extends _Entity[]> = Expand<Context<Entities> & { user: SanitizedUser}>
	type ContextWithUser<Entities extends _Entity[]> = Expand<Context<Entities> & { user: SanitizedUser }>

	// It expands this SO answer to funcitons: https://stackoverflow.com/a/57683652
	// It expands this SO answer to functions: https://stackoverflow.com/a/57683652


		export default function useAuth(queryFnArgs, config) {
		export default function useAuth(queryFnArgs?: unknown, config?: any) {

Full-stack compile-time type safety #1090

Full-stack compile-time type safety #1090

Conversation

sodic commented Mar 30, 2023 • edited by infomiho Loading

Motivating example

Todo list:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Example

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Default input and output types for Operations

Frontend behavior with omitted input types

Disabling the distribution of never

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

infomiho Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sodic Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shayneczyzewski left a comment

Choose a reason for hiding this comment

infomiho left a comment

Choose a reason for hiding this comment

sodic commented Apr 6, 2023

sodic commented Mar 30, 2023 •

edited by infomiho

Loading

sodic Apr 6, 2023 •

edited

Loading

sodic Apr 6, 2023 •

edited

Loading

sodic Apr 6, 2023 •

edited

Loading

Disabling the distribution of `never`

sodic Apr 6, 2023 •

edited

Loading

sodic Apr 6, 2023 •

edited

Loading

infomiho Apr 6, 2023 •

edited

Loading

sodic Apr 6, 2023 •

edited

Loading

sodic Apr 6, 2023 •

edited

Loading

sodic Apr 6, 2023 •

edited

Loading