schema-stream does not seems to validate the schema

[Aschen]

Bug report

I'm trying to use the library for JSON streaming + schema validation.

I tried to stream this JSON string { "foobar": "foobar" } with this Zod schema:

const schema = z.object({
  someString: z.string(),
  someNumber: z.number(),
})

I don't get any error from the parser but since the Zod schema is not respected, I should get one

Is it the normal behavior?

Full working example:

import { SchemaStream } from 'schema-stream'
import { z } from 'zod'

const schema = z.object({
  someString: z.string(),
  someNumber: z.number(),
})

async function getSomeStreamOfJson(
  jsonString: string
): Promise<{ body: ReadableStream }> {
  const stream = new ReadableStream({
    start(controller) {
      const encoder = new TextEncoder()
      const jsonBytes = encoder.encode(jsonString)

      for (let i = 0; i < jsonBytes.length; ) {
        const chunkSize = Math.floor(Math.random() * 5) + 2
        const chunk = jsonBytes.slice(i, i + chunkSize)
        controller.enqueue(chunk)
        i += chunkSize
      }
      controller.close()
    },
  })

  return { body: stream }
}

async function main() {
  const response = await getSomeStreamOfJson(`{"foobar": "foobar"}`)

  let result = {}

  const parser = new SchemaStream(schema)

  const streamParser = parser.parse({})
  response.body?.pipeThrough(streamParser)

  const reader = streamParser.readable.getReader()
  const decoder = new TextDecoder()

  while (true) {
    const { value, done } = await reader.read()

    if (done) {
      console.log('Final result:   ', result)
      break
    }

    const chunkValue = decoder.decode(value)

    result = JSON.parse(chunkValue)
  }
}

main()

[roodboi]

I tried to compose these packages in a few different ways before they all ended up in the current form but the high level on how things are organized/responsibilities of each are this:

schema stream: pure streaming json parser - the schema input allows us to stub the final object which enables you to start reading from the response asap - value here is that you can define what the expected output will be so as it streams through you have a stub that is readable testObject.nestedObject.someKey for instance will be in the structure from the start with default value for the type or a provided default ( https://github.com/hack-dance/island-ai/blob/main/public-packages/schemaStream/src/utils/streaming-json-parser.ts#L100 )

zod stream: given a zod schema pass through to schema stream for streaming json over the wire - we don't do the fullzod schema.parse since that throws but we safe parse and update the _meta object on the response to show u in realtime what paths in the object have completed and are/are not valid (
https://github.com/hack-dance/island-ai/blob/main/public-packages/zod-stream/src/structured-stream.client.ts#L79 )

instructor: uses both under the hood and implements retry logic/actual schema validation/parsing and extends the openai sdk to make it as close as possible to what people are already using which means to adopt you only need to change an import ( https://github.com/instructor-ai/instructor-js )

[roodboi]

I've updated all of the documentation in this repo which might help clear some things up - but also just tldr.

the final output should have a finalResult._meta.isValid key that is the result of zod safeParse upon completion of the stream.