Compile to JSON

[mikaelkaron]

I've created a fairly simple RX pipeline to transform VFiles that looks like this:

import html from "rehype-stringify";
import extract from "remark-extract-frontmatter";
import frontmatter from "remark-frontmatter";
import markdown from "remark-parse";
import markdown2html from "remark-rehype";
import { mergeMap } from "rxjs/operators";
import { parse as toml } from "toml";
import unified from "unified";
import { VFile, VFileCompatible } from "vfile";
import { parse as yaml } from "yaml";

const transformer = unified()
  .use(frontmatter, ["yaml", "toml"])
  .use(extract, { yaml, toml, throws: true })
  .use(markdown)
  .use(markdown2html)
  .use(html);

export const transform = (processor = transformer) =>
  mergeMap<VFileCompatible, Promise<VFile>>(async (vfile) => {
    // process to VFile
    const processed = await processor.process(vfile);
    // destruct from processed
    const { data, contents: html } = processed;
    // set contents to stringified JSON
    processed.contents = JSON.stringify({ data, html });
    // return processed VFile
    return processed;
  });

Now this works fine, but I'd like to know if there's a way to move the JSON serialization into the default transformer rather than the mergeMap.

Ideally I'd probably want to replace html with a new json plugin (feels like both are redundant). I was fiddling a little bit with rehype-stringify but never really got it working as I wanted.

[wooorm]

html in transform is a string of HTML. Is that what you want in JSON?
Why do you want serialized JSON as output?

[mikaelkaron]

html in transform is a string of HTML. Is that what you want in JSON?

yes

Why do you want serialized JSON as output?

I'm fetching these JSON files from the browser where I'll be replacing innerHTML with the html prop and possibly <title> and/or other elements with values from the data prop.

technically I could even serialize the whole VFile as a json and read that, but it contains a bit more info than I need.

[wooorm]

hmm.

unified is for string -> syntax trees -> string, and it ends there. So I don’t think this should go in the unified pipeline.
Of course, vfile is for files, so what your doing outside of the pipeline is fine.
And there might sometime come an abstraction on top of unified and vfile where things like this are possible, but that doesn‘t exist yet.
I’d say your code is fine! 🤷‍♂️

[mikaelkaron]

one thing worth mentioning is that it's stringified JSON stored in content so it's still string -> tree -> string. But I can solve it outside the pipeline. Easiest is probably to have a separate RX operator that manipulates the VFile.