Where should font-import data go?

[rmkaplan]

Font information that comes from external (like bdf, html) or legacy (strike, ac) is typically transformed in several stages before it can be made available for online use. The information in the original sources has to be converted into medleyfont format, the encoding scheme of the font has to be translated to mccs, and perhaps then various coercions have to be done to fill in character or charset information from other fonts.

The question is, where should the source and intermediate-stage files be stored? This information is not needed and should not be distributed as part of a release (and maybe it's needed only by someone who is working on fonts improvements).

One possibility is in a set of subdirectories in internal/fonts, for example internal/fonts/bdf/mccs/ for the mccs-translated bdf font files, and internal/fonts/bdf/completed/ for the final ready-to-distribute files. On this scheme, our current strike/ac display files would be moved to internal/fonts/legacydisplay/source/ and the transformation would be stored at .../legacydisplay/mccs/ and .../legacydisplay/completed. The completed fonts from different sources would eventually be copied/merged into the online fonts/displayfonts/.

Or, would it be better to somehow put the various font sources and intermediate translations in a submodule hanging off at some other level in the hierarchy?

[MattHeffron]

Copied from email that I sent to @rmkaplan. (I was composing it as he wrote this Issue):

I'm not sure committing these 3,250 files to the medley repo is a good idea. These files total over 900 MB, which is almost the same size as the whole medley repo (most of which is the .git folder). This would then be about doubled when added to the repo data in .git.

At this point in time, these files are data, so I think that they should go into a separate repo. It probably shouldn't be a submodule, because removing a submodule from a project reportedly is difficult. Once the fonts are completed, there should be fewer of them, as the files which contain the add-ons, e.g., Hebrew, SC, TC, and JP, have many Latin_-ish_ glyphs in common, so only the completed font files will be necessary to keep. (But we need to keep the CJK variants as separate completed fonts.) Also, we should consider the font point-sizes that we want to keep.

Even so, I wonder if, eventually, all of the ready-to-use fonts should be moved to a (different) repo, maybe submodule.

[rmkaplan]

Is it the bitmaps that take up all of that space? Would a better compression scheme make a difference?

[MattHeffron]

Would a better compression scheme make a difference?

It might, but that would be trading font file reading/decoding time against disk/repo space.
As I mentioned above, after font completion, many of these files will no longer be needed, but the files kept will be larger.
Also, do we need the 92 point size fonts? (total 185MB)

[rmkaplan]

Unless reading/decoding compressed bitmaps is an order of magnitude slower or more, I don't think the time would be something to worry about. It would only happen once per charset per font.

I personally don't have a use for size 92, but that's just me.