-
Notifications
You must be signed in to change notification settings - Fork 12
add is-my-json-valid 2.0.2 to the benchmark #74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Fast work, you guys. I updated json-schema-tests to use IMJV 2.0.2 in its examples, and it confirms you're passing almost all of the draft 4 tests. I'm seeing optional/bignum and optional/format(hostnames) failures still, but I can't assert that's not my problem. Let me know if you have any objections to the way I'm testing your lib. |
Oh it seems I forgot to add the optional test cases. Give me a couple of minutes. |
Those won't pose performance problems anyway, @mafintosh. |
@automatthew no you're right. 2.0.3 supports the optional tests now though for completeness (except for zeroTerminatedFloats) |
To answer your question, yeah your example looks fine 👍 |
Here's my plan, @mafintosh. Giles's talk of "fastest JSON Schema validator" was based on our benchmarking tool at the time we ran it. Implicit in our statement was also "supporting mostly all of draft X". We ignored some competing packages on that basis. IMJV now passes my threshold for inclusion in our benchmarks, so I'm almost certainly going to accept the PR (after I pull and play with it). Then, we need to blog about it. Key theme: the power of Open Source and Open Boasting to improve the world. Possibly also about the comparative values of code generation versus JSCK's approach. But then there's some goalpost-moving that I'll do … honorably.
So any further benchmarks we write will take into account these two factors, at least. I would also value any contributions or suggestions you have to make about the particular schemas and documents we're using, as well as the benchmarking setup in general. |
👍
All for it! There should be a json-schema-tests module on npm that contains a good set of test cases
Completely agree. The first iteration of IMJV bailed on first error but later versions now always do a full pass to give a complete error report (available on |
Except for DoS situations. |
True. The the cost is still (more or less) the same as validating a valid document |
I love that package. It's the best software I've ever seen. Whoever wrote it should get a medal. And a bottle of Scotch. Lagavulin, preferably. That package relies on the same official test suite you've added to IMJV, which tests individual JSON Schema keywords in limited and isolated circumstances. Complex interactions are not tested. It may well all add up to success, but I'm still skeptical. Incidentally, JSCK has recently added some meta-tests (to assert whether a schema is valid or invalid) using a similar data-driven approach that should eventually be usable by any project in any language. This flushed out several problems in my code. |
wow, that was fast. |
also fyi @mafintosh I believe @automatthew wrote the json-schema-tests package on npm. :-) |
add is-my-json-valid 2.0.2 to the benchmark
\o/ |
well-deserved high five ^_^ |
(I noticed #73 added the error handler wrongly in the benchmark so I decided to fix that in a new pr)
Since is-my-json-valid 2.0.2 now passes the JSONSchema v4 test suite (except for remoteRef, unicode surrogate pairs in maxLength/minLength) this PR adds it to the benchmark.
On my macbook air the benchmark yields the following result (is-my-json-valid 2.0.2 is between 5x-10x faster than the 2nd fastest)