AICI -> llguidance?

[mmoskal]

I noticed you guys forked a bunch of controller code from AICI for your constraints. I think you might be interested in https://github.com/microsoft/llguidance - it implements a more general constraint parser and is significantly faster. It can be also easily consumed as a Rust crate.

Grammars for llguidance can be generated from the Python Guidance https://github.com/guidance-ai/guidance (in particular there is code there that translates a JSON schema into Guidance and then llguidance; it shouldn't be too hard to port that into Rust but I have not gotten around to do that yet)

I also did a quick PoC in TypeScript at https://github.com/mmoskal/guidance-ts

I expect more client libraries to emerge.

Of course it's easy to just write a grammar with a single regex if this is what you're after (llguidance is using https://github.com/microsoft/derivre for regexes/lexers).

LLguidance is deployed as preview in Azure for Phi-3.5-mini

[mmoskal]

See PR #899