Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance issue when used from ImGui as a SVG font loader #150

Open
pthom opened this issue Jan 2, 2024 · 37 comments
Open

Performance issue when used from ImGui as a SVG font loader #150

pthom opened this issue Jan 2, 2024 · 37 comments
Labels
enhancement New feature or request

Comments

@pthom
Copy link

pthom commented Jan 2, 2024

Hello,

I saw some performance issues with ImGui when it uses LunaSvg in order to load some Svg fonts:

It can load some fonts with a very good performance. However, with some others, the performance drops and it takes about 2 seconds per glyph to load.

In order to facilitate the analysis I prepared a repro repository, here: https://github.com/pthom/lunasvg_perf_issue

    {
        // will load 1408 colored glyphs, with lunasvg. Fast!
        std::string fontFile = ThisDir() + "/fonts/noto-untouchedsvg.ttf";
        gEmojiFont  = ImGui::GetIO().Fonts->AddFontFromFileTTF(fontFile.c_str(), 30.0f, &cfg, ranges);
    }
    {
        // will load 1428 glyphs, with lunasvg. Slow. About 2 seconds per glyph
        std::string fontFile = ThisDir() + "/fonts/NotoColorEmoji-Regular.ttf";
        gEmojiFont  = ImGui::GetIO().Fonts->AddFontFromFileTTF(fontFile.c_str(), 30.0f, &cfg, ranges);
    }

As explained is the repro repository, a quick analysis with a profiler shows that time seems to be spent in std::map::find + std::vector::emplace_back.

PS: Happy new year! I'm sorry to bother you on Jan 2nd!

@sammycage
Copy link
Owner

sammycage commented Jan 2, 2024

@pthom After conducting a thorough investigation into the issue, I discovered that a large SVG file (approximately 12MB) from https://github.com/cppfw/svgren/blob/master/tests/unit/samples_data/back.svg takes about 2.5 seconds to load and render using LunaSvg alone. Interestingly, it loads faster than smaller files like glyphs in the repro profile analysis. This suggests that the problem might be associated with FreeType, ImGui, or other components in the reproduction process. Your ongoing collaboration is crucial for improving this library. Thank you for your support!

PS: Happy new year! I'm sorry to bother you on Jan 2nd!

Thanks for the New Year wishes! No bother at all, because clearly, dealing with GitHub issues is the best way to kick off the year. 😜

@pthom
Copy link
Author

pthom commented Jan 4, 2024

@sammycage :

You are probably right, there is something sniffy in the various components chain (ImGui => FreeType => LunaSvg).

Here is how I came to this conclusion:

I saw that for example, the glyph number 18 is extremely slow to load.

  1. I put a breakpoint inside ImFontAtlasBuildWithFreeTypeEx(ImGui), when glyph_i==18, just before it calls src_tmp.Font.LoadGlyph()
  2. This loop will call ImGuiLunasvgPortPresetSlot, which will call lunasvg::Document::loadFromData (a few lines into this function)
  3. Just before the call to lunasvg::Document::loadFromData, I wrote the SVG document with this code:
        // Write document->svg_document to a file for debugging
        std::ofstream file("svg_document.svg");
        file.write((const char*)document->svg_document, document->svg_document_length);

And then I looked at the saved svg document for the glyph number 18.
And then, surprise, surprise:

  • It is big: 14.5MB!!! (Even too big to be attached to this issue)
  • Even inkscape struggles with it
  • When opened inside inkscape, one sees that it contains a copy of all the glyphs (from glyph 4 to glyph 2808 in the screenshot below)
image

@pthom
Copy link
Author

pthom commented Jan 4, 2024

@ocornut: what do you think? Is is related to Freetype? Or maybe to ImGui?

@pthom
Copy link
Author

pthom commented Jan 4, 2024

I have a suspect inside Freetype, namely ttsvg.c.

More info:

If we look at the evolution of the svg document sizes per glyph, we have:

glyph_i: 0 / 1427
glyph_i: 1 / 1427 document->svg_document_length: 1650
glyph_i: 2 / 1427 document->svg_document_length: 1060
glyph_i: 3 / 1427 document->svg_document_length: 686
glyph_i: 4 / 1427 document->svg_document_length: 534
glyph_i: 5 / 1427 document->svg_document_length: 837
glyph_i: 6 / 1427 document->svg_document_length: 1087
glyph_i: 7 / 1427 document->svg_document_length: 917
glyph_i: 8 / 1427 document->svg_document_length: 922
glyph_i: 9 / 1427 document->svg_document_length: 908
glyph_i: 10 / 1427 document->svg_document_length: 612
glyph_i: 11 / 1427 document->svg_document_length: 941
glyph_i: 12 / 1427 document->svg_document_length: 884
glyph_i: 13 / 1427 document->svg_document_length: 7870
glyph_i: 14 / 1427 document->svg_document_length: 7870
glyph_i: 15 / 1427
glyph_i: 16 / 1427 document->svg_document_length: 14516311
glyph_i: 17 / 1427 document->svg_document_length: 14516311
glyph_i: 18 / 1427 document->svg_document_length: 14516311
glyph_i: 19 / 1427 document->svg_document_length: 4774
glyph_i: 20 / 1427 document->svg_document_length: 14516311
glyph_i: 21 / 1427 document->svg_document_length: 14516311
glyph_i: 22 / 1427 document->svg_document_length: 14516311
glyph_i: 23 / 1427 document->svg_document_length: 14516311
glyph_i: 24 / 1427 document->svg_document_length: 14516311
glyph_i: 25 / 1427 document->svg_document_length: 14516311
glyph_i: 26 / 1427 document->svg_document_length: 14516311
glyph_i: 27 / 1427 document->svg_document_length: 14516311
glyph_i: 28 / 1427 document->svg_document_length: 14516311
glyph_i: 29 / 1427 document->svg_document_length: 14516311
glyph_i: 30 / 1427 document->svg_document_length: 6554
glyph_i: 31 / 1427 document->svg_document_length: 8158
glyph_i: 32 / 1427 document->svg_document_length: 14516311
glyph_i: 33 / 1427 document->svg_document_length: 14516311
...

So, I investigated what happens when glyph_i=16, and somewhere inside the long callstack, we have:

external/freetype/src/sfnt/ttsvg.c, line 283:

  FT_LOCAL_DEF( FT_Error )
  tt_face_load_svg_doc( FT_GlyphSlot  glyph,
                        FT_UInt       glyph_index )
  {
    FT_Error   error  = FT_Err_Ok;
    TT_Face    face   = (TT_Face)glyph->face;
    FT_Memory  memory = face->root.memory;
    Svg*       svg    = (Svg*)face->svg;

    FT_Byte*  doc_list;
    FT_ULong  doc_limit;

    FT_Byte*   doc;
    FT_ULong   doc_offset;
    FT_ULong   doc_length;
    FT_UShort  doc_start_glyph_id;
    FT_UShort  doc_end_glyph_id;

    FT_SVG_Document  svg_document = (FT_SVG_Document)glyph->other;


    FT_ASSERT( !( svg == NULL ) );

    doc_list = svg->svg_doc_list;


    error = find_doc( doc_list + 2, svg->num_entries, glyph_index,
                                    &doc_offset, &doc_length,
                                    &doc_start_glyph_id, &doc_end_glyph_id );

And error = find_doc(...) did set these values:

svg = {Svg *} 0x6000004171c0
 version = {FT_UShort} 0
 num_entries = {FT_UShort} 674
glyph_index = {FT_UInt} 6
doc_offset = {FT_ULong} 9365
doc_length = {FT_ULong} 14516311           // ARGH.... 14.5MB
doc_start_glyph_id = {FT_UShort} 4.         // Glyph number 4...
doc_end_glyph_id = {FT_UShort} 2808.    // ... to 2808... Exactly what I saw inside Inkscape

I'm using Freetype VER-2-13-2 (i.e. the latest tag)

@pthom
Copy link
Author

pthom commented Jan 4, 2024

And finally, there is a probably related issue for freetype about the same font (NotoColorEmoji):

Freetype does not support 'COLR' v1 tables (whatever that means), but as it appears, it still tries to load fonts with such a format, instead of giving an error.

See https://gitlab.freedesktop.org/freetype/freetype-demos/-/issues/35

Werner Lemberg
@wl · 6 months ago
Owner

The new NotoColorEmoji.ttf font comes with a 'COLR' v1 table. While FreeType provides routines to read and parse fonts with 'COLR' v1 tables, it has no possibility to actually render them. It is rather simple to provide a simple default rendering for 'COLR' v0, so FreeType has it. However, the version 1 format is far too involved to be handled within FreeType – it is almost as complex as SVG.

If someone provides a library for rendering 'COLR' v1, FreeType can provide hooks in a similar way to SVG. Until then, it is unfortunately not possible to get something better.

@pthom
Copy link
Author

pthom commented Jan 4, 2024

I posted the issue to Freetype: https://gitlab.freedesktop.org/freetype/freetype/-/issues/1265

@sammycage
Copy link
Owner

@pthom Well done 👍

@pthom
Copy link
Author

pthom commented Jan 4, 2024

Summary of the current situation:

23M   NotoColorEmoji-Regular.ttf.      // probable issue with Freetype (colored, probably COLR v1)
10M   OpenMoji-color-colr0_svg.ttf     // probable issue with Freetype (colored, probably COLR v0)

858K  NotoEmoji-Regular.ttf            // works with Freetype (not colored)
14M   TwitterColorEmoji-SVGinOT.ttf    // works with Freetype (colored)
39M   noto-untouchedsvg.ttf            // works with Freetype (colored)
660K  seguiemj.ttf                     // works with Freetype

@pthom
Copy link
Author

pthom commented Jan 5, 2024

I can confirm the issue is in Freetype only, since I created a repro that depends only on it.
See https://gitlab.freedesktop.org/freetype/freetype/-/issues/1265#note_2226979

@pthom
Copy link
Author

pthom commented Jan 6, 2024

If a deep dive into FreeType internals interests you, you can look at https://gitlab.freedesktop.org/freetype/freetype/-/issues/1265#note_2228459

@sammycage
Copy link
Owner

@pthom Hi buddy,

Thanks for reaching out. After extensive research, I discovered that FreeType is working as expected. The performance issue seems to originate from the following functions in the ImGui FreeType implementation:

It appears that the glyph ID is omitted, which causes LunaSvg to render the entire document, leading to the performance drop you observed.

@ocornut
Copy link

ocornut commented Aug 4, 2024

Where would the glyph id be used/useful?
It’s not clear.

@sammycage
Copy link
Owner

@ocornut @pthom According to the FreeType rsvg demo (see source lines 326 to 336):

  • If the document contains only one glyph, start_glyph_id and end_glyph_id will have the same value.
  • If the document contains multiple glyphs, end_glyph_id will be larger than start_glyph_id.

In the code snippet:

if (start_glyph_id < end_glyph_id) {
    // Render only the element with its ID equal to `glyph<ID>`.
    sprintf(str, "#glyph%u", slot->glyph_index);
    id = str;
} else {
    // If NULL, render the whole document.
    id = NULL;
}
  • When start_glyph_id is less than end_glyph_id, the function renders only the element with the ID glyph<ID>.
  • When start_glyph_id equals end_glyph_id, id is set to NULL, indicating that the whole document should be rendered.

@ocornut
Copy link

ocornut commented Aug 4, 2024

Thank you for kindly investigating! Much appreciable.
I’ll let @pthom see if they want to further rework our LunaSVG code.

@pthom
Copy link
Author

pthom commented Aug 4, 2024

@omar, @sammycage
Thanks to both of you, and many thanks to you Sammy for pursuing this investigation!
I'm going to investigate this, using the repro repository. I will let you know what I find.

I will also need to re-investigate if the issue I found when using Freetype only (when not using neither ImGui, nor LunaSvg) still holds.

As a reminder, I had instrumented the Freetype code and found this:

codepoint: 57 glyph_index: 3190   - Hook_PresetSlot: svg_document_length: 884
codepoint: 169 glyph_index: 2811   - Hook_PresetSlot: svg_document_length: 7870
codepoint: 174 glyph_index: 2812   - Hook_PresetSlot: svg_document_length: 7870
codepoint: 8205 glyph_index: 3849  
codepoint: 8252 glyph_index: 6   - Hook_PresetSlot: svg_document_length: 14516311
codepoint: 8265 glyph_index: 2717   - Hook_PresetSlot: svg_document_length: 14516311
codepoint: 8419 glyph_index: 2718   - Hook_PresetSlot: svg_document_length: 14516311
codepoint: 8482 glyph_index: 3197   - Hook_PresetSlot: svg_document_length: 4774

i.e. when using Freetype in a standalone program with a SVG font (see code), some glyphs are using the full document size.

Anyway, I'm going to dive into it and let you know.

@sammycage
Copy link
Owner

@sammycage
Copy link
Owner

sammycage commented Aug 4, 2024

@pthom I have rendered glyph index 16 of the NotoColorEmoji-Regular.ttf font using plutosvg.

Rendering with start_glyph_id and end_glyph_id acknowledged:

Finished parsing: 14516311 bytes in 0.101836000 seconds
Finished bounding: 47 elements in 0.000261000 seconds
Finished rendering: 47 elements in 0.002626000 seconds

emoji-16

Rendering with start_glyph_id and end_glyph_id not acknowledged:

Finished parsing: 14516311 bytes in 0.111179000 seconds
Finished bounding: 151328 elements in 0.809536000 seconds
Finished rendering: 151328 elements in 16.835835000 seconds

emoji-16

Analysis

  1. Parsing Time: The parsing time remains relatively consistent, with a slight increase when start_glyph_id and end_glyph_id are not acknowledged.
  2. Bounding Time: Bounding time increases drastically when the glyph IDs are not acknowledged, indicating that unnecessary elements are being processed.
  3. Rendering Time: The rendering time without acknowledging the glyph IDs is substantially longer, showing the efficiency loss.
  4. Number of Elements: There is a significant difference in the number of elements processed, which directly impacts the rendering performance.

The significant difference in performance metrics and output demonstrates that not acknowledging start_glyph_id and end_glyph_id results in the renderer processing unnecessary elements.

Conclusion

Using start_glyph_id and end_glyph_id optimizes the rendering process by reducing the number of elements processed, leading to faster bounding and rendering times. This approach should be preferred for efficient rendering of SVG documents.

@pthom
Copy link
Author

pthom commented Aug 4, 2024

@sammycage, I might need a little more of your time.

I'm trying to find a strategy on how to apply your advices. I'm considering the two functions you mentioned "ImGuiLunasvgPortPresetSlot" and "ImGuiLunasvgPortRender".

They are both called by Freetype, in the order: ImGuiLunasvgPortPresetSlot, then ImGuiLunasvgPortRender

The solution might be separated in three steps:

  • Add a field char glyph_id[32] to LunasvgPortState (empty string if the whole document shall be rendered)
  • Fill it inside ImGuiLunasvgPortPresetSlot
  • Use it inside ImGuiLunasvgPortRender

I have several questions (I'm giving you below more details with my current status in the code):

  • Do you agree with this approach?
  • How could I render only one element with the desired ID inside ImGuiLunasvgPortRender, which calls state->svg->render
  • Do you agree with me that the parsing will slow us a bit but it's not a big issue?

LunasvgPortState

First I think we should add an information whether we should render the whole document or just a specific glyph.
and fill this info inside LunasvgPortState (by adding a glyph_id field for example).

        struct LunasvgPortState
        {
            FT_Error            err = FT_Err_Ok;
            lunasvg::Matrix     matrix;
            std::unique_ptr<lunasvg::Document> svg = nullptr;
            char glyph_id[32];                      // <====                 added this 
        };

ImGuiLunasvgPortPresetSlot

ImGuiLunasvgPortPresetSlot would be responsible for filling glyph_id. See the "WIP" section below.

static FT_Error ImGuiLunasvgPortPresetSlot(FT_GlyphSlot slot, FT_Bool cache, FT_Pointer* _state)
{
    FT_SVG_Document   document = (FT_SVG_Document)slot->other;

    LunasvgPortState* state = *(LunasvgPortState**)_state;
    FT_Size_Metrics&  metrics = document->metrics;

    // This function is called twice, once in the FT_Load_Glyph() and another right before ImGuiLunasvgPortRender().
    // If it's the latter, don't do anything because it's // already done in the former.
    if (cache)
        return state->err;

    // WIP: I have one question for you Samuel:
    // When reading your message I understand that parsing will be a bit slower, but it's not that big of a deal. Am I right?
    state->svg = lunasvg::Document::loadFromData((const char*)document->svg_document, document->svg_document_length);
    if (state->svg == nullptr)
    {
        state->err = FT_Err_Invalid_SVG_Document;
        return state->err;
    }

    // WIP
    {
        // If I instrument the code such as in rsvg_port.c (as you hinted)
        // I can see that we have a strong slowdown when start_glyph_id < end_glyph_id
        // which is a hint that you were right.
        FT_UShort  end_glyph_id   = document->end_glyph_id;
        FT_UShort  start_glyph_id = document->start_glyph_id;
        if ( start_glyph_id < end_glyph_id )
        {
            snprintf(state->glyph_id, sizeof(state->glyph_id), "glyph%u", slot->glyph_index);
            printf("ImGuiLunasvgPortPresetSlot glyph_index=%i / Should Render only the element with its ID equal to `glyph%i`\n", slot->glyph_index);
        }
        else
        {
            state->glyph_id[0] = '\0';
            printf("ImGuiLunasvgPortPresetSlot glyph_index=%i / Should Render the whole document\n", slot->glyph_index);
        }
    }

    lunasvg::Box box = state->svg->box();
    double scale = std::min(metrics.x_ppem / box.w, metrics.y_ppem / box.h);
    double xx = (double)document->transform.xx / (1 << 16);
    double xy = -(double)document->transform.xy / (1 << 16);
    double yx = -(double)document->transform.yx / (1 << 16);
    double yy = (double)document->transform.yy / (1 << 16);
    double x0 = (double)document->delta.x / 64 * box.w / metrics.x_ppem;
    double y0 = -(double)document->delta.y / 64 * box.h / metrics.y_ppem;

    // Scale and transform, we don't translate the svg yet
    state->matrix.identity();
    state->matrix.scale(scale, scale);
    state->matrix.transform(xx, xy, yx, yy, x0, y0);
    state->svg->setMatrix(state->matrix);

    // Pre-translate the matrix for the rendering step
    state->matrix.translate(-box.x, -box.y);

    // Get the box again after the transformation
    box = state->svg->box();

    // Calculate the bitmap size
    slot->bitmap_left = FT_Int(box.x);
    slot->bitmap_top = FT_Int(-box.y);
    slot->bitmap.rows = (unsigned int)(ImCeil((float)box.h));
    slot->bitmap.width = (unsigned int)(ImCeil((float)box.w));
    slot->bitmap.pitch = slot->bitmap.width * 4;
    slot->bitmap.pixel_mode = FT_PIXEL_MODE_BGRA;

    // Compute all the bearings and set them correctly. The outline is scaled already, we just need to use the bounding box.
    double metrics_width = box.w;
    double metrics_height = box.h;
    double horiBearingX = box.x;
    double horiBearingY = -box.y;
    double vertBearingX = slot->metrics.horiBearingX / 64.0 - slot->metrics.horiAdvance / 64.0 / 2.0;
    double vertBearingY = (slot->metrics.vertAdvance / 64.0 - slot->metrics.height / 64.0) / 2.0;
    slot->metrics.width = FT_Pos(IM_ROUND(metrics_width * 64.0));   // Using IM_ROUND() assume width and height are positive
    slot->metrics.height = FT_Pos(IM_ROUND(metrics_height * 64.0));
    slot->metrics.horiBearingX = FT_Pos(horiBearingX * 64);
    slot->metrics.horiBearingY = FT_Pos(horiBearingY * 64);
    slot->metrics.vertBearingX = FT_Pos(vertBearingX * 64);
    slot->metrics.vertBearingY = FT_Pos(vertBearingY * 64);

    if (slot->metrics.vertAdvance == 0)
        slot->metrics.vertAdvance = FT_Pos(metrics_height * 1.2 * 64.0);

    state->err = FT_Err_Ok;
    return state->err;
}

ImGuiLunasvgPortRender

This function should be able to render only the desired ID. How can this be achieved?

static FT_Error ImGuiLunasvgPortRender(FT_GlyphSlot slot, FT_Pointer* _state)
{
    printf("ImGuiLunasvgPortRender glyph_index=%i\n", slot->glyph_index);
    LunasvgPortState* state = *(LunasvgPortState**)_state;

    // If there was an error while loading the svg in ImGuiLunasvgPortPresetSlot(), the renderer hook still get called, so just returns the error.
    if (state->err != FT_Err_Ok)
        return state->err;

    // rows is height, pitch (or stride) equals to width * sizeof(int32)
    lunasvg::Bitmap bitmap((uint8_t*)slot->bitmap.buffer, slot->bitmap.width, slot->bitmap.rows, slot->bitmap.pitch);
    state->svg->setMatrix(state->svg->matrix().identity()); // Reset the svg matrix to the default value

    // WIP:
    // Another question for you, Samuel:
    // How could we limit the rendering to one specific ID (i.e. state->glyph_id) instead of the whole document?
    state->svg->render(bitmap, state->matrix);              // state->matrix is already scaled and translated

    state->err = FT_Err_Ok;
    return state->err;
}

Anyway, thanks again for your efforts. It is a big help!

@sammycage
Copy link
Owner

sammycage commented Aug 4, 2024

When reading your message I understand that parsing will be a bit slower, but it's not that big of a deal. Am I right?

Yes, parsing generally much faster than rendering, so the impact is minimal.

How could we limit the rendering to one specific ID

Limiting the rendering to one specific ID with a specific matrix isn't possible with the current version. However, I am currently working on this feature.

@sammycage
Copy link
Owner

@pthom In the meantime, I'd love to hear your thoughts on plutosvg. PlutoSVG is specifically designed for rendering SVG documents embedded in OpenType fonts. It uses block allocation to minimize memory fragmentation and is several times faster. Additionally, it supports Color Palette Table and image rendering. It implements all the elements in the specifications except for clipPath.

@pthom
Copy link
Author

pthom commented Aug 4, 2024

@sammycage
I have several questions.

  • about the issue I posted on the freestype repository: Do you agree that I should close it?

  • about plutosvg
    If I understand correctly, This library is very close to lunasvg, with 2 majors differences : 1/ it is written in C and 2/ integration with freetype hooks can be written with one single call. Am I correct ? Are there other reasons for it to be distinct from lunasvg ?

As far as using plutosvg inside Dear ImGui, It is not a decision I can make by myself. We will have to discuss this with Omar. There are potential consequences to consider in the decision , especially for Dear ImGui users who have already set up a build system using LunaSvg.

Thanks!

@sammycage
Copy link
Owner

about the issue I posted on the freestype repository: Do you agree that I should close it?

I believe it would be best to leave it open until the issue is completely resolved.

There are potential consequences to consider in the decision , especially for Dear ImGui users who have already set up a build system using LunaSvg.

I understand if this is the case.

Limiting the rendering to one specific ID with a specific matrix isn't possible with the current version. However, I am currently working on this feature.

I will update you when I am done. Thank you

@pthom
Copy link
Author

pthom commented Aug 5, 2024

Give me a few days, I'll give a try to plutosvg, so that Omar can have a look at it and see whether the switch might be interesting (However, I think that adding this to LunaSVG might be interesting as well)

@pthom
Copy link
Author

pthom commented Aug 5, 2024

I believe it would be best to leave it open until the issue is completely resolved.

I'll post an update anyhow, to inform them. It seems important, since Werner Lemberg seems to be looking for feedback about it.

image

Update: here is the message I posted:
https://gitlab.freedesktop.org/freetype/freetype/-/issues/1265#note_2513814

@pthom
Copy link
Author

pthom commented Aug 8, 2024

test plutosvg, current status: here are some first results after a first quick try Plutosvg: I could get it to compile and to replacer the Freetype hooks

However, the rendering is now faster, but still too slow (see below for profile analysis). It takes about 1 minute to load the font "NotoColorEmoji-Regular".

Compile and link with ImGui

This is slightly more complex that lunasvg, since it requires plutosvg + plutovg. There is no vcpkg support for the moment (as opposed to lunasvg).

Also, note that plutovg comes with its own version of stb_truetype.h, which is distinct from the one provided with ImGui (imstb_truetype.h). Both versions are almost similar (except for some warning correction on ImGui side)

When using CMake, the difference boils down to:

if (USE_PLUTO_INSTEAD_OF_LUNA_SVG)
    #
    # Add plutovg and plutosvg
    #
    # plutovg
    file(GLOB plutovg_sources external/plutovg/source/*.c)
    add_library(plutovg STATIC ${plutovg_sources})
    target_include_directories(plutovg PUBLIC external/plutovg/include)
    target_include_directories(plutovg PRIVATE external/plutovg/stb)  # Note: two versions of stb_truetype.h (one in plutovg, one in imgui)
    # plutosvg
    add_library(plutosvg STATIC external/plutosvg/source/plutosvg.c)
    target_include_directories(plutosvg PUBLIC external/plutosvg/source)
    target_compile_definitions(plutosvg PUBLIC PLUTOSVG_HAS_FREETYPE)
    target_link_libraries(plutosvg PUBLIC plutovg freetype)
else()
    # Add lunasvg
    add_subdirectory(external/lunasvg)
endif()

This difference in setup might interest @ocornut in deciding whether or not the switch is worth it.

Integration with ImGui

The integration is simple at the moment, see modifications inside ImGui: pthom/imgui@be7fbdf

Status

However this first commit is not enough, since the rendering is still slow

Repro code

See the branch plutosvg in the repro repository:
https://github.com/pthom/lunasvg_perf_issue/tree/plutosvg

Profiling result

It seems that we spend too much time inside plutosvg_load_document_from_data

image

I do not think there is much to do on the ImGui side to make it faster, since the hooks are now handled by plutosvg.

@sammycage
Copy link
Owner

sammycage commented Aug 8, 2024

However, the rendering is now faster, but still too slow (see below for profile analysis). It takes about 1 minute to load the font "NotoColorEmoji-Regular".

@pthom With the latest commit of plutosvg, it can now load and render all the glyphs in less than 4 seconds.

This is slightly more complex that lunasvg, since it requires plutosvg + plutovg.

In the next major release of lunasvg, lunasvg will be using plutovg as a dependency #110.

There is no vcpkg support for the moment (as opposed to lunasvg).

Once I am done with the next major release, I will talk to vcpkg to add plutovg and plutosvg.

@pthom
Copy link
Author

pthom commented Aug 8, 2024

I confirm that using the latest commit will reduce the font loading time down to 3 seconds (with FontHeight=60, 1428 glyphs); which is OK I think.
The time is consumed mainly in the parsing

image

I now have a nice running gui in the ImGui window:

image

Next steps

@sammycage :

  • would you consider adding a CMakeList besides the meson build file. I think more people know about it that CMake.
  • If not, could you explain the compilation flag "PLUTOSVG_HAS_FREETYPE" in the Readme. I almost gave up on the integration because my setup was failing, and I stumbled on this option by reverse engineering.
  • could you make the inclusion of stb_truetype.h customizable (via a preprocessor definition), to accomodate for the fact that Dear ImGui's true type file is name imstb_truetype.h? @ocornut : do you have any thoughts on this?

@ocornut: A decision will likely have to be made about the next steps. I'm open to work on it, but need more guidance.

Let's clarify what we know in order to be able to decide with accuracy:

  1. There is an actual performance issue + rendering issue in the current LunaSvg setup for Dear ImGui
  2. Correcting it will require either an upgrade to LunaSvg or to switch to PlutoSvg. It seems that the next version of LunaSvg will depend on PlutoSvg anyhow
  3. Integrating PlutoSvg into Dear ImGui is actually easy, and reduces the number of font rendering code inside ImGui (since PlutoSvg will handle the Freetye hooks). See pthom/imgui@be7fbdf

However, there are several points to consider:
1/ Should plutosvg come as an alternative to lunasvg, or as a replacement (both decisions are possible)
2/ Is it a concern that imstb_truetype is provided with potentially two version inside plutosvg and imgui
3/ There are two places inside ImGui code that make assumptions when using Freetype + Lunasvg: do they still hold when/if switching to PlutoSvg? They are summarized below:

#if !((FREETYPE_MAJOR >= 2) && (FREETYPE_MINOR >= 12))
#error IMGUI_ENABLE_FREETYPE_LUNASVG requires FreeType version >= 2.12
#endif
#ifdef IMGUI_ENABLE_FREETYPE_LUNASVG
        IM_ASSERT(slot->format == FT_GLYPH_FORMAT_OUTLINE || slot->format == FT_GLYPH_FORMAT_BITMAP || slot->format == FT_GLYPH_FORMAT_SVG);

@sammycage
Copy link
Owner

I confirm that using the latest commit will reduce the font loading time down to 3 seconds (with FontHeight=60, 1428 glyphs); which is OK I think.
The time is consumed mainly in the parsing.

3 seconds? Isn't that cool 😎. librsvg, the one used in the FreeType demo, takes like infinity!

would you consider adding a CMakeList besides the meson build file. I think more people know about it that CMake.

Thank you for the suggestion. At this time, we do not plan to add a CMakeLists file alongside the Meson build file.

If not, could you explain the compilation flag "PLUTOSVG_HAS_FREETYPE" in the Readme. I almost gave up on the integration because my setup was failing, and I stumbled on this option by reverse engineering.

Sorry, but the Meson options should handle that for you. I recommend checking the Meson build configuration for details on the available options.

@pthom
Copy link
Author

pthom commented Aug 8, 2024

Sorry, but the Meson options should handle that for you. I recommend checking the Meson build configuration for details on the available options.

Not everyone will use meson. I did not, and will not in this particular case (where another build system was in place) :-) Anyway, this is not that important.

3 seconds? Isn't that cool 😎. librsvg, the one used in the FreeType demo, takes like infinity!

1.5 seconds in release builds. Nice!

pthom added a commit to pthom/imgui that referenced this issue Aug 27, 2024
…native to lunasvg (fix ocornut#7187)

<SVG Fonts include a set of SVG documents. As per the [OpenType specification](https://learn.microsoft.com/en-us/typography/opentype/spec/svg#glyph-identifiers),
some SVG fonts (such as NotoColorEmoji) may group several glyphs in a common svg document (by selecting a subset of the elements in this document).

LunaSvg does support fonts where each glyph is associated to a distinct document. Unfortunately, it is not able to render
a subset of a svg document, and will likely not be able to do so in the future.

Its cousin project plutosvg (by the same author), is able to do it, and provides ready to use freetype hooks.

Example: sammycage/lunasvg#150 shows an example where a single svg document
included in the font may contains thousands of glyphs (each glyph is a subset of the svg document).

Pros and Cons
-------------
Since this commit adds some complexity in the code here is a study of the pros and cons:

Pros:
- plutosvg is faster than lunasvg
- freetype hooks are provided by plutosvg (no need to provide them in imgui_freetype.cpp)

Cons:
- the compilation setup for plutosvg is a bit more complex than for lunasvg (requires plutovg + plutosvg)
- no offical release is available yet for plutosvg. No vcpkg package available yet.
- having two competing compilation flags is not ideal (IMGUI_ENABLE_FREETYPE_LUNASVG vs IMGUI_ENABLE_FREETYPE_PLUTOSVG)
  (it may be possible to remove IMGUI_ENABLE_FREETYPE_LUNASVG in the future, at the cost of breaking some users build upon upgrade)

Compilation hints for plutovg/plutosvg
--------------------------------------
_Compilation hints for plutovg_
- Compile all source files in `plutovg/source/*.c`
- Add include directory: `plutovg/include` + `plutovg/stb`

_Compilation hints for plutosvg_
- Compile `plutosvg/source/plutosvg.c`
- Add include directory: `plutosvg/source`
- Add define: `PLUTOSVG_HAS_FREETYPE`
- Link with: plutovg, freetype

Demonstration repository
-------------------------
https://github.com/pthom/lunasvg_perf_issue
@pthom
Copy link
Author

pthom commented Aug 27, 2024

See ImGui new PR where I propose the integration of plutosvg.

@sammycage : It seems that you did some more performance improvement, since the application now starts in 0.45 seconds (this includes font loading + all application setup).

@sammycage
Copy link
Owner

Hey @pthom, how's it going?

I wanted to let you know that element subsetting is fully supported in version 3.0.0 of LunaSVG! Although PlutoSVG is several times faster than LunaSVG, as you mentioned, there are indeed potential consequences to consider, especially for Dear ImGui users who have already set up a build system using LunaSVG. If you decide to stick with LunaSVG, here’s some code that might help with the fix:

#include <cstdio>
#include <cmath>

#include <ft2build.h>
#include FT_FREETYPE_H
#include FT_MODULE_H
#include FT_OTSVG_H

#include <lunasvg.h>

struct ImGuiLunasvgPortEntry
{
    ImGuiLunasvgPortEntry(FT_UShort start_glyph_id, FT_UShort end_glyph_id, std::unique_ptr<lunasvg::Document> document);
    FT_UShort start_glyph_id;
    FT_UShort end_glyph_id;
    std::unique_ptr<lunasvg::Document> document;
};

inline ImGuiLunasvgPortEntry::ImGuiLunasvgPortEntry(FT_UShort start_glyph_id, FT_UShort end_glyph_id, std::unique_ptr<lunasvg::Document> document)
    : start_glyph_id(start_glyph_id), end_glyph_id(end_glyph_id), document(std::move(document))
{
}

using ImGuiLunasvgPortEntries = std::vector<ImGuiLunasvgPortEntry>;

struct LunasvgPortState
{
    lunasvg::Matrix matrix;
    lunasvg::Box extents;
    lunasvg::Element element;
    ImGuiLunasvgPortEntries entries;
};

static FT_Error ImGuiLunasvgPortInit(FT_Pointer* _state)
{
    *_state = new LunasvgPortState;
    return FT_Err_Ok;
}

static void ImGuiLunasvgPortFree(FT_Pointer* _state)
{
    delete (*(LunasvgPortState**)_state);
}

static FT_Error ImGuiLunasvgPortRender(FT_GlyphSlot slot, FT_Pointer* _state)
{
    LunasvgPortState* state = *(LunasvgPortState**)_state;
    if(state->element.isNull()) {
        return FT_Err_Invalid_SVG_Document;
    }

    lunasvg::Bitmap bitmap((uint8_t*)slot->bitmap.buffer, slot->bitmap.width, slot->bitmap.rows, slot->bitmap.pitch);
    state->element.render(bitmap, lunasvg::Matrix::translated(-state->extents.x, -state->extents.y) * state->matrix);

    slot->bitmap.pixel_mode = FT_PIXEL_MODE_BGRA;
    slot->bitmap.num_grays = 256;
    slot->format = FT_GLYPH_FORMAT_BITMAP;
    return FT_Err_Ok;
}

static lunasvg::Document* ImGuiLunasvgPortLoad(LunasvgPortState* state, FT_SVG_Document ft_document, FT_UInt index)
{
    if(ft_document->start_glyph_id < ft_document->end_glyph_id) {
        for(const auto& entry : state->entries) {
            if(index >= entry.start_glyph_id && index <= entry.end_glyph_id) {
                return entry.document.get();
            }
        }
    }

    auto document = lunasvg::Document::loadFromData(reinterpret_cast<const char*>(ft_document->svg_document), static_cast<size_t>(ft_document->svg_document_length));
    if(document == nullptr)
        return nullptr;
    state->entries.emplace_back(ft_document->start_glyph_id, ft_document->end_glyph_id, std::move(document));
    return state->entries.back().document.get();
}

static FT_Error ImGuiLunasvgPortPresetSlot(FT_GlyphSlot slot, FT_Bool cache, FT_Pointer* _state)
{
    FT_SVG_Document   ft_document = (FT_SVG_Document)slot->other;
    FT_Size_Metrics&  ft_metrics = ft_document->metrics;

    LunasvgPortState* state = *(LunasvgPortState**)_state;
    state->element = lunasvg::Element();
    state->matrix = lunasvg::Matrix();
    state->extents = lunasvg::Box();

    auto document = ImGuiLunasvgPortLoad(state, ft_document, slot->glyph_index);
    if(document == nullptr) {
        return FT_Err_Invalid_SVG_Document;
    }

    lunasvg::Element element;
    if(ft_document->start_glyph_id < ft_document->end_glyph_id) {
        char id[64];
        std::sprintf(id, "glyph%u", slot->glyph_index);
        element = document->getElementById(id);
    } else {
        element = document->documentElement();
    }

    if(element.isNull()) {
        return FT_Err_Invalid_SVG_Document;
    }

    auto extents = element.getLocalBoundingBox();
    auto scale = std::min(ft_metrics.x_ppem / extents.w, ft_metrics.y_ppem / extents.h);
    auto matrix = lunasvg::Matrix::scaled(scale, scale);
    matrix *= lunasvg::Matrix {
        (float)ft_document->transform.xx / (1 << 16),
        -(float)ft_document->transform.xy / (1 << 16),
        -(float)ft_document->transform.yx / (1 << 16),
        (float)ft_document->transform.yy / (1 << 16),
        (float)ft_document->delta.x / 64 * extents.w / ft_metrics.x_ppem,
        -(float)ft_document->delta.y / 64 * extents.h / ft_metrics.y_ppem
    };

    extents.transform(matrix);

    slot->bitmap_left = (FT_Int)extents.x;
    slot->bitmap_top = (FT_Int)-extents.y;

    slot->bitmap.rows = (unsigned int)ceilf(extents.h);
    slot->bitmap.width = (unsigned int)ceilf(extents.w);
    slot->bitmap.pitch = (int)slot->bitmap.width * 4;
    slot->bitmap.pixel_mode = FT_PIXEL_MODE_BGRA;

    float metrics_width = extents.w;
    float metrics_height = extents.h;

    float horiBearingX = extents.x;
    float horiBearingY = -extents.y;

    float vertBearingX = slot->metrics.horiBearingX / 64.f - slot->metrics.horiAdvance / 64.f / 2;
    float vertBearingY = (slot->metrics.vertAdvance / 64.f - slot->metrics.height / 64.f) / 2;

    slot->metrics.width = (FT_Pos)roundf(metrics_width * 64);
    slot->metrics.height = (FT_Pos)roundf(metrics_height * 64);

    slot->metrics.horiBearingX = (FT_Pos)(horiBearingX * 64);
    slot->metrics.horiBearingY = (FT_Pos)(horiBearingY * 64);
    slot->metrics.vertBearingX = (FT_Pos)(vertBearingX * 64);
    slot->metrics.vertBearingY = (FT_Pos)(vertBearingY * 64);
    if(slot->metrics.vertAdvance == 0)
        slot->metrics.vertAdvance = (FT_Pos)(metrics_height * 1.2f * 64);
    if(cache) {
        state->element = element;
        state->matrix = matrix;
        state->extents = extents;
    }

    return FT_Err_Ok;
}

SVG_RendererHooks hooks = { ImGuiLunasvgPortInit, ImGuiLunasvgPortFree, ImGuiLunasvgPortRender, ImGuiLunasvgPortPresetSlot };

Let me know if you have any other questions or need further assistance!

pthom added a commit to pthom/imgui that referenced this issue Oct 3, 2024
This is a follow up to these related issues:
- ocornut#7187 (which may need to be reopened, since there is a real rendering issue in ImGui)
- sammycage/lunasvg#150

----
SVG Fonts include a set of SVG documents. As per the [OpenType specification](https://learn.microsoft.com/en-us/typography/opentype/spec/svg#glyph-identifiers), some SVG fonts (such as NotoColorEmoji) may group several glyphs in a common svg document (by selecting a subset of the elements in this document).

LunaSvg did not originally support fonts where each glyph is associated to a distinct document. LunaSvg 3.0.0 now supports this feature

This PR thus adds support for LunaSvg (3.0.0), and its author (@sammycage) was kind enough to provide a patch for ImGui:
sammycage/lunasvg#150 (comment)

Notes
-----
- WARNING: the API of LunaSVG has changed inside the latest version (3.0.0): the current code inside imgui_freetype.cpp
  (without this PR) is not compatible with it!
  A test for this was added in imgui_freetype.cpp:
```cpp
#ifndef LUNASVG_VERSION_MAJOR
#error IMGUI_ENABLE_FREETYPE_LUNASVG requires LunaSvg version >= 3.0
#endif
#if !(LUNASVG_VERSION_MAJOR >= 3)
#error IMGUI_ENABLE_FREETYPE_LUNASVG requires LunaSvg version >= 3.0
#endif
```

- Performance: NotoColorEmoji-Regular is now correctly loaded in approx 1 second (versus > 1 hour with current ImGui)

- Alternative PR, using PlutoSvg instead of LunaSvg:
  There is an alternative PR that would replace LunaSvg by PlutoSvg (ocornut#7927).
  We will likely have to choose between those two.
  The present PR does not propose to replace LunaSvg by PlutoSvg, but to update LunaSvg to the latest version.

- Demonstration repository: https://github.com/pthom/lunasvg_perf_issue / branch "lunasvg_patch_oct24"
@pthom
Copy link
Author

pthom commented Oct 3, 2024

Hello Sammy,

I looked at your code which uses LunaSvg, and used it "as is" inside imgui_freetype.cpp.
It did work without any issue, and loaded the font and about 1 second (versus 0.5 with plutosvg). This is a very acceptable performance :-)

I think it is possible to open an alternative PR in the ImGui repo, this time using the updated LunaSvg.
Could you have a look at this possible PR (before we post it to ImGui):

https://github.com/ocornut/imgui/compare/master...pthom:imgui:update_lunasvg?expand=1

Notes:

  • I did use the code you wrote without any modification, and it worked. I mentioned your name in the commit message.
  • I did only one change, which is to check for LunaSvg's version (since the API changed, and is not backwards compatible)
  • When posting the PR, since you are the author of most modification, I will likely mention you as the main author and ask you to give more explanation to @ocornut if he has questions / comments.
  • If you want attribution for those modifications, feel free to do it: it is possible that you commits those modifications onto your own fork of ImGui, and later send the PR.

Please keep me posted. Cheers!


Below is the commit message I wrote:

Title: Fix OpenType SVG fonts rendering with LunaSVG 3.0 (by @sammycage)

This is a follow up to these related issues:


SVG Fonts include a set of SVG documents. As per the OpenType specification, some SVG fonts (such as NotoColorEmoji) may group several glyphs in a common svg document (by selecting a subset of the elements in this document).

LunaSvg did not originally support fonts where each glyph is associated to a distinct document. LunaSvg 3.0.0 now supports this feature

This PR thus adds support for LunaSvg (3.0.0), and its author (@sammycage) was kind enough to provide a patch for ImGui: #150 (comment)

Notes

  • WARNING: the API of LunaSVG has changed inside the latest version (3.0.0): the current code inside imgui_freetype.cpp (without this PR) is not compatible with it! A test for this was added in imgui_freetype.cpp:
#ifndef LUNASVG_VERSION_MAJOR
#error IMGUI_ENABLE_FREETYPE_LUNASVG requires LunaSvg version >= 3.0
#endif
#if !(LUNASVG_VERSION_MAJOR >= 3)
#error IMGUI_ENABLE_FREETYPE_LUNASVG requires LunaSvg version >= 3.0
#endif

@sammycage
Copy link
Owner

I appreciate your mention in the commit.

Regarding the code, I would like to clarify that I expect it not to be used as-is in the final PR. ImGui uses custom macros and functions, such as ImCeil and IM_NEW, which are not present in this code. Instead, I see it more as a test or demo to illustrate the potential benefits and to guide the integration.

I'm happy to help review and refine this further before it’s finalized.

@pthom
Copy link
Author

pthom commented Oct 24, 2024

Hello Sammy,

Omar did merge the PR that added support for plutosvg.

Posting a PR that adds support for the LunaSVG 3.0 is less urgent now that plutosvg support is included (and perhaps not needed after all).

However, the fact that the code currently in imgui_freetype.cpp is not compatible with LunaSVG 3.0 is a possible issue in the future, as it seems parts of the API has changed in V3.0.

Here is a list of the errors raised when compiling imgui_freeetype.cpp (as of now) with LunaSVG 3.0:

lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:865:17: error: no member named 'setMatrix' in 'lunasvg::Document'
  865 |     state->svg->setMatrix(state->svg->matrix().identity()); // Reset the svg matrix to the default value
      |     ~~~~~~~~~~~~^
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:865:39: error: no member named 'matrix' in 'lunasvg::Document'
  865 |     state->svg->setMatrix(state->svg->matrix().identity()); // Reset the svg matrix to the default value
      |                           ~~~~~~~~~~~~^
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:889:36: error: no member named 'box' in 'lunasvg::Document'
  889 |     lunasvg::Box box = state->svg->box();
      |                        ~~~~~~~~~~~~^
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:899:19: error: no member named 'identity' in 'lunasvg::Matrix'
  899 |     state->matrix.identity();
      |     ~~~~~~~~~~~~~ ^
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:901:19: error: no member named 'transform' in 'lunasvg::Matrix'
  901 |     state->matrix.transform(xx, xy, yx, yy, x0, y0);
      |     ~~~~~~~~~~~~~ ^
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:902:17: error: no member named 'setMatrix' in 'lunasvg::Document'
  902 |     state->svg->setMatrix(state->matrix);
      |     ~~~~~~~~~~~~^
lunasvg_perf_issue/external/imgui/misc/freetype/imgui_freetype.cpp:908:23: error: no member named 'box' in 'lunasvg::Document'
  908 |     box = state->svg->box();
      |           ~~~~~~~~~~~~^

Given this, several solutions can be explored:

  1. Do nothing (and may be advise users to switch to PlutoSVG + PlutoVG)
  2. Try to restore back compatibility with the old API in LunaSVG
  3. Add a check for LunaSVG < 3.0 in imgui_freetype.cpp (i.e #if (LUNASVG_VERSION_MAJOR >= 3) #error "imgui_freetype.cpp is compatible with LunaSVG < 3.0" #endif)
  4. Update imgui_freetype.cpp for LunaSVG3.0 (however, this would break compatibility for user who installed LunaSVG 1.0 via vcpkg)

At the moment, my bet would be on 2. or 3.

@sammycage: do you think option 2 would be feasible?

@ocornut: if you happen to read this, do you have an opinion?

Thanks

@sammycage
Copy link
Owner

@pthom After considering the options, I believe going with Option 1 would be the best approach for now, and I personally don't think Option 2 is feasible, at least for me. Trying to maintain backward compatibility with older APIs might introduce unnecessary complexity and maintenance overhead.

Additionally, for the sake of code quality and simplicity, maintaining two libraries that essentially do the same thing doesn't seem like the best idea. Streamlining by sticking with PlutoSVG, now that support has been merged, could avoid potential conflicts and redundancies in the future.

@hmaarrfk
Copy link

hmaarrfk commented Nov 30, 2024

@pthom does vcpkg support dependencies in the same way that conda(-forge) does?

edit: it seems reasonable to say "ok this new version of imgui now requires lunasvg > 3.0"

@pthom
Copy link
Author

pthom commented Nov 30, 2024

@hmaarrfk : I will answer in your PR at pthom/imgui_bundle#284

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants