Q: performance benchmarks

[JCash]

Hi!

I was wondering if you had any available benchmark numbers comparing with libraqm/harfbuzz?

In my current due diligence, for libraqm I'm using this example (by modifying their unit test to do the same):

static void DoTest(const char* name, raqm_t* rq, FT_Face face, const uint32_t* text, uint32_t len)
{
    raqm_clear_contents(rq);

    uint64_t tstart = GetTime();

    assert (raqm_set_text(rq, text, len));
    assert (raqm_set_freetype_face (rq, face));

    if (len < 600)
      raqm_set_par_direction (rq, RAQM_DIRECTION_LTR);
    else
      raqm_set_par_direction (rq, RAQM_DIRECTION_RTL);

    assert (raqm_layout (rq));

    size_t count;
    raqm_glyph_t *glyphs = raqm_get_glyphs (rq, &count);
    uint64_t tend = GetTime();

    // printf("glyph count: %zu\n", count);
    // for (size_t i = 0; i < count; i++)
    // {
    //     printf ("gid#%d off: (%d, %d) adv: (%d, %d) idx: %d\n",
    //             glyphs[i].index,
    //             glyphs[i].x_offset,
    //             glyphs[i].y_offset,
    //             glyphs[i].x_advance,
    //             glyphs[i].y_advance,
    //             glyphs[i].cluster);
    // }

    float t = (tend - tstart)/1000.0f;
    printf("RAQM: %s: Segmented %u codepoints into %zu glyphs in %.3fms (%.4f ms / glyph)\n", name, len, count, (tend - tstart)/1000.0f, t/len);
}

And I'm using your suggested SegmentText from the Readme.md.
(I've attached the test app in the zip)

The numbers I get are a fair bit slower than the raqm/harfbuzz test, so I'm curious if I'm perhaps missing something obvious in my setup?
Are the numbers what you'd expect compared to your benchmarks?

Thanks for any info!
/Mathias

• The test text is the first paragraph of Lorem Ipsum, 309 codepoints
• Results are milliseconds (and milliseconds/glyph)
• Font is NotoSansArabic-Regulat.ttf

EDIT:
By using NotoSans-Regulat.ttf the timing becomes a lot faster:

Segmented 309 codepoints into 309 glyphs in 0.293ms

Still, about 2x the time from raqm/harfbuzz.

The KB test app (contains both fonts for easy testing)

bench-kb.zip

[JimmyLefevre]

Your'e opening the font once to shape a single string. This is the worst possible case for kbts.
The whole point of this library is that opening fonts is slow and shaping is fast.
Also, the example ShapeText function is written in a convenient way, but, as a result, it allocates all over the place. The API is set up so you only have to allocate once.

Comparing only shaping time against Harfbuzz on their own test suite (+ a few of my own), this is what I get on my AMD Ryzen 9 5900HX:

We are ~4.5x faster on average.

[JimmyLefevre]

I have explained this here: https://github.com/JimmyLefevre/kb/blob/970609b3f7c6913c3e10929aa816dcf78deda690/kb_text_shape.h#L289

Although these are just my numbers. I would love for more people to benchmark the library and share their results.

[JCash]

Your'e opening the font once to shape a single string. This is the worst possible case for kbts.

But, I don't do the timing until after loading the font?
I only do the timing of the text shaping? Or am I still missing something? (quite possible...)

static void TimeSegmentation(const char* name, uint32_t* codepoints, uint32_t num_codepoints)
{
    uint64_t tstart = GetTime();

    uint32_t num_glyphs = SegmentText(codepoints, num_codepoints);

    uint64_t tend = GetTime();
    float t = (tend - tstart)/1000.0f;
    printf("%s: Segmented %u codepoints into %u glyphs in %.3fms (%.3f ms / glyph)\n", name, num_codepoints, num_glyphs, (tend - tstart)/1000.0f, t/num_codepoints);
}

Also, the example ShapeText function is written in a convenient way, but, as a result, it allocates all over the place

Apart from the global glyph buffer, which are the parts I should refactor to make use of "best practices" (as opposed to "convenient example code")?
Do you have another piece of code one could look at for inspiration?

[JCash]

Regarding the note you linked to, this line does stand out, given my different results between NotoSans-Regular.ttf and NotoSansArabic-Regulat.ttf:

Font files can be arbitrarily complex, making it difficult to predict system behavior at runtime.

However, I am still curious why the timings of the actual shaping differs so much when using the Arabic version (as seen in the OP). Does your benchmarks behave the same (4-5x faster) compared to Harfbuzz, even when using NotoSansArabic-Regulat.ttf?

[JimmyLefevre]

You're right, sorry. I'm being scatterbrained right now.

We weren't doing binary searches for glyph tables yet, which indeed becomes a problem with large fonts.

This is fixed in 5d397d8

Here is what I get from your test script with the new version:

Using font NotoSans-Regular.ttf
hello: Segmented 11 codepoints into 11 glyphs in 0.056ms (0.005 ms / glyph)
lorem: Segmented 309 codepoints into 309 glyphs in 0.157ms (0.001 ms / glyph)
lorem arabic: Segmented 671 codepoints into 671 glyphs in 0.193ms (0.000 ms / glyph)

Using font NotoSansArabic-Regular.ttf
hello: Segmented 11 codepoints into 11 glyphs in 0.034ms (0.003 ms / glyph)
lorem: Segmented 309 codepoints into 309 glyphs in 0.130ms (0.000 ms / glyph)
lorem arabic: Segmented 671 codepoints into 893 glyphs in 0.276ms (0.000 ms / glyph)

[JimmyLefevre]

Your'e opening the font once to shape a single string. This is the worst possible case for kbts. The whole point of this library is that opening fonts is slow and shaping is fast. Also, the example ShapeText function is written in a convenient way, but, as a result, it allocates all over the place. The API is set up so you only have to allocate once.

Comparing only shaping time against Harfbuzz on their own test suite (+ a few of my own), this is what I get on my AMD Ryzen 9 5900HX:

We are ~4.5x faster on average.

We are now ~4.6x faster on average..!

[JimmyLefevre]

Hi!

I was wondering if you had any available benchmark numbers comparing with libraqm/harfbuzz?

In my current due diligence, for libraqm I'm using this example (by modifying their unit test to do the same):

static void DoTest(const char* name, raqm_t* rq, FT_Face face, const uint32_t* text, uint32_t len)
{
    raqm_clear_contents(rq);

    uint64_t tstart = GetTime();

    assert (raqm_set_text(rq, text, len));
    assert (raqm_set_freetype_face (rq, face));

    if (len < 600)
      raqm_set_par_direction (rq, RAQM_DIRECTION_LTR);
    else
      raqm_set_par_direction (rq, RAQM_DIRECTION_RTL);

    assert (raqm_layout (rq));

    size_t count;
    raqm_glyph_t *glyphs = raqm_get_glyphs (rq, &count);
    uint64_t tend = GetTime();

    // printf("glyph count: %zu\n", count);
    // for (size_t i = 0; i < count; i++)
    // {
    //     printf ("gid#%d off: (%d, %d) adv: (%d, %d) idx: %d\n",
    //             glyphs[i].index,
    //             glyphs[i].x_offset,
    //             glyphs[i].y_offset,
    //             glyphs[i].x_advance,
    //             glyphs[i].y_advance,
    //             glyphs[i].cluster);
    // }

    float t = (tend - tstart)/1000.0f;
    printf("RAQM: %s: Segmented %u codepoints into %zu glyphs in %.3fms (%.4f ms / glyph)\n", name, len, count, (tend - tstart)/1000.0f, t/len);
}

And I'm using your suggested SegmentText from the Readme.md. (I've attached the test app in the zip)

The numbers I get are a fair bit slower than the raqm/harfbuzz test, so I'm curious if I'm perhaps missing something obvious in my setup? Are the numbers what you'd expect compared to your benchmarks?

Thanks for any info! /Mathias

• The test text is the first paragraph of Lorem Ipsum, 309 codepoints
• Results are milliseconds (and milliseconds/glyph)
• Font is NotoSansArabic-Regulat.ttf

EDIT: By using `NotoSans-Regulat.ttf` the timing becomes a lot faster:

Segmented 309 codepoints into 309 glyphs in 0.293ms

Still, about 2x the time from raqm/harfbuzz.

The KB test app (contains both fonts for easy testing)

bench-kb.zip

May I see the rest of the raqm numbers, for reference's sake?

[JCash]

You're right, sorry. I'm being scatterbrained right now.

Heh, no worries!

We weren't doing binary searches for glyph tables yet, which indeed becomes a problem with large fonts.
This is fixed in 5d397d8

Thanks!
This makes me happy, as that was the main concern left on that front :)

We are now ~4.6x faster on average..!

Nice!
Still being puzzled by my local benchmarks though, as one is very slightly faster, and the other is ~20% slower 🤔
But for my use case, I consider them equal enough for me to be happy. (i..e not having to consider harfbuzz et al)

Before (for completeness):

After (with your the fix):

In either case, I think the build/memory/runtime part of my due diligence favors kb_text_shape as the winner.
Now I also have to finish integrating it into our engine for a final look, to actually verify it.

[JimmyLefevre]

Thanks.

There's a large difference between my numbers and yours for Arabic, which probably comes down to me removing calls to malloc and realloc from your example code, if I had to guess?

[JCash]

In my last set of numbers, I've replaced it with a global buffer with room for 1000 glyphs (as I know I only need 884 of them), so it shouldn't be the allocation that does it. :/

Perhaps it's an arm vs amd/intel thing, or memory alignment etc... 🤔

[JimmyLefevre]

There are definitely gains to be had on our Unicode normalization, which would speed up everything, but should be particularly noticeable for the Latin case.

[behdad]

Hi,

HarfBuzz author here. I don't think byteswapping all the time in HB can show such a slowdown whatsoever.

I haven't check your test harness, but it might be the case that your code is pulling the entire font into L2 cache before starting shaping, whereas with HarfBuzz that is done on demand. I'll add some code to HB to warm up the font into memory before running tests and see.

And if anyone feels like writing a hb-kb shape backend to HB so we can run the test/perf suites against it, please do.

[JimmyLefevre]

That's a definite possibility. The allocation behavior probably plays a big part in these tiny tests, too.

[behdad]

I'm also curious to know how you do the endian-swapping in the GSUB/GPOS tables? Are you careful to NOT swap the same bytes more than once?

UPDATE: I see it does track duplicates for structs beginning at the same offset. That's good enough for existing compilers...