improve the README?

[Pomax]

Given that http://docs.mathjax.org/en/latest/font-support.html directs users to this repository for when they need to have mathjax deal with more glyphs than the default fonts support (in my case, I need to create custom ja-JP and zh-CN versions that have support for basic Japanese and mainland Chinese writing) it would be super helpful if the README was improved to be more... README-like =)

• project name and description
• "prerequisites" section (assuming someone has a blank system)
• "how to install" section (with exact syntax)
• "how to use" section (with exact syntax and examples)
• where to get help (irc, other repos, website, forum, etc)
• credits and license

Right now it reads very much lilke no one is supposed to even know this repo exists, which is a little contradictory given that people are told to go here in the official MathJax docs =)

[pkra]

You're right that this repository is in a bad state. The reasons is simple: it saw extremely little activity throughout its lifetime. Essentially, when the original code as pushed back in the day and when the newer webfonts were added.

Adding support for a new fonts is difficult and requires very good understanding of MathJax itself. Adding a non-math font is even harder.

However, this will likely all become moot in the next couple of months -- v3 is taking shape and it looks the changes will affect the font handling. My best guess is: we'll create new repos for those tools that don't move into other repos and retire this one. All in all, I think it's unlikely we'll spend time fixing up this repo, I'm afraid.

[Pomax]

hm, but MathJax knows how to deal with "normal" OpenType fonts like STIX or Cambira just fine, so generating a metrics file for a new font that is also a standard OpenType font shouldn't really be more difficult than what is already done to ensure there are metrics for those fonts. How are the metrics for Cambria etc. generated right now? (I assume this is a task that runs whenever a new version of MathJax is generated, for instance).

I ask because I run https://pomax.github.io/bezierinfo which has a ton of LaTeX, and is currently seeing community-driven localisation efforts to Japanese and (mainland) Chinese. I use MathJax in an offline setting to convert all LaTeX blocks to SVG files (https://github.com/Pomax/BezierInfo-2/blob/master/tools/mathjax.js, which uses the mathjax-node library) so in order to let the community localize the LaTeX code as well, I need to point MathJax at locale-specific fonts (each build task is a separate thing so it thankfully doesn't need to load all the fonts at the same time. It'll just be loading a specific font for a single specific LaTeX block call).

This can't really wait months, I need to figure out a solution that works today even if a much nicer thing comes out when v3 is released at some point in the future, so if you know how I can take something like, say, Noto-Sans-CJK-JP.otf and generate the additional data that MathJax needs to do its job, I can simply roll with that until some future version of MathJax comes out with built-in support for "as long as it's an OpenType font, MathJax can read in the metrics because OpenType is a fixed spec" that's what lets opentype.js, fontkit.js, etc. etc. do their job, too)

[pkra]

How are the metrics [...] generated right now?

Using https://github.com/mathjax/MathJax-dev/tree/master/fonts/OpenTypeMath.

(I assume this is a task that runs whenever a new version of MathJax is generated, for instance).

It is actually run quite rarely (every other year according to git history).

in order to let the community localize the LaTeX code as well, I need to point MathJax at locale-specific fonts

Not necessarily. You can just use Unicode in the TeX directly and it will fallback to system fonts. With CJK the result on mathjax-node should be quite ok, especially, if the main task is to have it in \text{} but even beyond that.

[Pomax]

I tried some Chinese text outside of \tex{} but MathJax didn't seem to get that right, with quad fitting being done wrong (yielding overlapping glyphs rather than separated glyphs):

Original source

\[
\begin{align*}
  只不过是正弦函数 &= BLUE[a] + RED[b] \\
  只不过是正弦函数 &= BLUE[a] \cdot BLUE[a] + BLUE[a] \cdot RED[b] + RED[b] \cdot RED[b] \\
  只不过是正弦函数 &= BLUE[a] \cdot BLUE[a] \cdot BLUE[a] + BLUE[a] \cdot BLUE[a] \cdot RED[b] + BLUE[a] \cdot RED[b] \cdot RED[b] + RED[b] \cdot RED[b] \cdot RED[b]\\
\end{align*}
\]

(The RED[...] and BLUE[...] are preprocessor macros that turn into {\color{macro color}...})

Presumably this is because it doesn't put the "which font got used" information inside the SVG, which means if the browser picks a different font than what MathJax got, the metrics are guaranteed off. Or they're both using the same font but somehow these characters are getting metrics computed incorrectly.

I'll have a look at the python font build script. Adding a python dependency isn't really a problem if it lets my project build metrics for MathJax based on (subset) fonts for Japanese, Chinese, and hopefully a few more non-western languages.

[dpvc]

Presumably this is because it doesn't put the "which font got used" information inside the SVG, which means if the browser picks a different font than what MathJax got, the metrics are guaranteed off. Or they're both using the same font but somehow these characters are getting metrics computed incorrectly.

MathJax-node can't measure the width of elements, like MathJax can in the browser, because the DOM implementation used in MathJax-node doesn't support that (and no native javascript DOM replacement does), so MathJax-node can't tell how wide the characters are if they aren't in its own fonts (where it has the metric information directly). In MathJax in a browser, you can ask the browser to measure the width, but not in MathJax-node. So MathJax-node has to guess. It does this by using the "monospace" font, and making an assumption about the width of that font. For standard ASCII characters, this assumption is reasonable, but for non-western characters, this doesn't work so well. In your case, the glyphs are wider than the MathJax-node estimate, and there is no way for it to know that. So the output is crushed together, as you see.

You can adjust the width that MathJax-node uses by changing the 8.5 in line 318 of lib/mj-single.js to be something larger; a value of 13 worked for me. If you change the monospace to serif two lines later, then you might need to go to 15 or 16. Of course, this will be applied to all characters that aren't in the font you have selected, so it might be that other text is spaced too widely if you do this, but it might be the easiest fix for you.

MathJax knows how to deal with "normal" OpenType fonts like STIX or Cambira just fine

No, MathJax doesn't know about Cambria. It does have the data for STIX, but not Cambria, which is a proprietary font, and we would not be allowed to distribute a web-based version of it, so we can't include it in the MathJax font set. MathJax includes data for 7 open-source fonts.

Note that MathJax needs more than just the OpenType data. The MathJax font data files include hand-edited information as well, and the creation of the data files is not fully automated (for the fonts other than STIX and MathJax TeX, it is mostly automated, but the results still have problems and need some hand adjustment). Plus there is no current mechanism for mixing several fonts (e.g., for text and for math), so unless your Chinese font has all the needed math glyphs, you would still not be able to use them effectively, or would have to hand combine the data for several fonts. In any case, producing your own font data files is not a straight-forward task, and (as you point out), not well documented.

As Peter says, we do hope to improve this in v3, but the current situation in v2 is not great.

[Pomax]

I see, that's all good information. If I run the metrics system on a custom font to which I have use rights, would MathJax be able to generate the widths accurately? (with the same font served as webfont on the same family name)?

[dpvc]

The font data files include the metric information, so MathJax simply uses that (it doesn't generate the widths itself).

The SVG output doesn't actually use web fonts, but rather it uses paths generated from the font itself. (It only uses actual fonts for characters not in its own fonts, like the Chinese ones you are using, and then only system fonts, not web fonts, though you could load a web font yourself and request that as your default system font.)

Part of the font creation process that Peter points to involves doing that conversion to SVG paths. So if you did that, it would be possible to use the generated font data files within MathJax-node's SVG output. But I expect you would still need to combine that with an existing font data file, since I doubt your font will contain the require math glyphs. If you are only interested in the Chinese unicode blocks, then adding that into an existing font file would be easier than having to make a completely new font collection, but it will still take some hand work, and knowledge of the font data file format and font directory layout. But it could be done, yes.