Is it possible before PDF output to remove unused fonts that has been added as fallback?

[evilaliv3]

Hello!

At @globaleaks we are integrating fpdf2 to generate PDF reports. This library is so magic! thank you.

Since one of the goals of our software is to support high internationalization and localization we are using the Inter font family + adding the entire noto-sans font family.

This is done so that a message included inside a PDF in a specific language should be output correctly.
More specifically we are using the noto-sans family to ensure to use fonts studied for accessibility reasons and the result has an uniform quality and we are making using a for loop of FPDF adding .add_fonts() over ~90 required fonts

Then we set a default font using FPDF.set_font() and we add all the rest to fallback fonts using FPDF.set_fallback_fonts()

The question is: is there any possibility before outputing the PDF to strip out the fonts which contains only glyps that have not been used?

Thank you!

[andersonhc]

Hi @evilaliv3 ,

Supporting internationalization has been one of our recent focuses, and it's great to see it being useful :)

I have looked into this issue before, and the only barrier to adding only the used fonts in the final document is the page number substitution feature.

Currently, the code replaces the alias in the output function after the text has been rendered, which means we no longer have control over the font and glyphs. To ensure the glyphs exist after the alias replacement, fpdf2 adds numbers for every font included in add_fonts() to the final document.

I will start working on this feature soon to fix #1090 and implement #1138. One of my goals is to limit the final PDF to include only the used fonts/glyphs.

[evilaliv3]

Thank you! One idea could be to staring generating the pdf after checking every single character, determining ranges and then starting creating the PDF knowing the exact set of fonts that we will need. Do you have any recommendations that we could follow to try to make this? (We are using 2.7.6 since it is included in Debian) Do you think this is feasible?

[andersonhc]

You can use the code below - it shouldn't have any side effect except some very rare edge cases using page number alias.

Do this after generating the whole PDF and before calling output():

for font in list(pdf.fonts.keys()):
    if isinstance(pdf.fonts[font], fpdf.fonts.TTFFont) and len(pdf.fonts[font].subset) <= 17:
        pdf.fonts.pop(font)

[Lucas-C]

Nice, well done!

Maybe fpdf2 should always perform this optimization when .alias_nb_pages(None) is called (meaning .str_alias_nb_pages is None)?

What do you think @andersonhc?

Else, maybe we should document this recipe in the Fonts and Unicode section of our documentation?

[evilaliv3]

Thank you! You are amazing!

what does <= 17 represent? I see the removed fonts have precisely value 2 or exactly 17.

Of course i'm in the edge case since we are using the number alias to print the numbers in the footer so your code works and let us remove any unused fonts for us only if we disable the page numbering.

Do you have any suggestion to get around this last issue?

[andersonhc]

17 is the number of default characters fpdf2 adds for each font. To be safe you can read and store the subset length right after you do all the add_font() and then compare before output and remove the fonts if the subset size didn't change.

If you save somewhere what font is being used to write the page number alias and never remove that font, you should be OK on all cases.

[evilaliv3]

Thank you thank you thank you!

Will do this:

• override the __init__ to perform the initial count
• override the output to perform the comparison and remove unused fonts and never remove the default font that is the one that we use for the number as well.

Will come back and publish the code and a link to our commit so that other users could use this implementation.

[evilaliv3]

Thank you @andersonhc

I leave here how we managed to implement a solution for this discussion.
At the moment we are just trying to understand why the {nb} alias is not replaced (do you have any idea of the possible causes?
except from this with the the localizaion we should be set.

    class REPORTPDF(fpdf.FPDF):
        report_default_font = 'inter-latin-400-normal.ttf'
        report_direction = 'ltr'
        report_line_height = 3
        report_margin = 10
        fonts_characters = {}

        def __init__(self, *args, **kwargs):
            super(REPORTPDF, self).__init__(*args, **kwargs)

            fontspath = os.path.join(Settings.client_path, 'fonts')

            for filename in os.listdir(fontspath):
                if not re.match(r".*400.*.ttf", filename):
                    continue

                self.add_font(family=filename.upper(), style='', fname=os.path.join(fontspath, filename))

            for font in list(self.fonts.keys()):
                if font != self.report_default_font and isinstance(self.fonts[font], fpdf.fonts.TTFFont):
                    self.fonts_characters[font] = len(self.fonts[font].subset)

            self.set_font(self.report_default_font, "", 11)
            self.set_fallback_fonts([font for font in self.fonts.keys() if font != self.report_default_font])

            self.set_auto_page_break(auto=True, margin=15)

            self.set_author("GLOBALEAKS")
            self.set_creator("GLOBALEAKS")
            self.set_lang("EN")

            self.set_right_margin(self.report_margin)
            self.set_left_margin(self.report_margin)

            self.set_text_shaping(use_shaping_engine=True, direction="ltr")

        def header(self):
            self.set_font(self.report_default_font, "", 9)
            self.cell(80)
            self.set_text_shaping(use_shaping_engine=True, direction="ltr")
            self.cell(30, 10, self.title, align="C")
            self.set_text_shaping(use_shaping_engine=True, direction=self.report_direction)
            self.ln(20)


        def footer(self):
            self.set_y(-15)
            self.set_font(self.report_default_font, "", 9)
            self.set_text_shaping(use_shaping_engine=True, direction="ltr")
            self.cell(0, 10, f"{self.page_no()}/{{nb}}", align="C")
            self.set_text_shaping(use_shaping_engine=True, direction=self.report_direction)

        def output(self, *args, **kwargs):
            for font in list(self.fonts.keys()):
                if font != self.report_default_font and isinstance(self.fonts[font], fpdf.fonts.TTFFont) and \
                  self.fonts_characters[font] == len(self.fonts[font].subset):
                    self.fonts.pop(font)

            return super(REPORTPDF, self).output(*args, **kwargs)

[Lucas-C]

Thank you for sharing! 👍

[evilaliv3]

You are very welcome, happy to contribute.

[evilaliv3]

@Lucas-C @andersonhc : actually i'm noticing a strange behaviour that do not meet what you told me.

Right know you told that any font included is included as it is and that you are considering for the future instead to include only the used glyphs,

If i got it right now, if i add 1 fonts and i keep 1 fonts i would have expected to have a PDF file with a size bigger than that font.

I just tried to include GoNotoKurrent-Regular.ttf that is a special compressed font of 15MB that includes all the NotoSans fonts for International Languages (we are trying to do this because using add_font ~1000 times can give a to many files open errors), but the resulting PDF is of only 65kb; this is of course great, but how is this possible?

[evilaliv3]

Here the PDF of which i'm talking about
report.pdf

[Lucas-C]

This is because we extract a subset of the fonts used, and only insert in the final PDF file the required glyphs 🙂

The relevant code is there:
https://github.com/py-pdf/fpdf2/blob/2.7.9/fpdf/output.py#L559

[evilaliv3]

This is absolutely so good!

So the recipe for internationalization could be one of the following:

• document how to generate a combined file that includes multiple fonts
• offer a function that enable to achieve this goal

You may think: load the font that you need only, but what we are trying to manage is to print well a text of which we do not know the language.