Ditching the .archive, .data, .draft, and .page File Extension(s)

[taufik-nurrohman]

I’m thinking about adding support for multiple page file extensions that are widely known as data store files, such as .ini, .json, .php, .xml, and .yaml. Those are commonly used to store key-value pair data. Some allow nesting, some don’t. But the most important thing is that some of them do have their own parser, which is available by default provided by PHP. This makes it possible to store a page file with the .json extension, for example, so that parsing would be as simple as:

$lot = json_decode(file_get_contents($folder . D . 'article.json'), true);

Pros

• A native syntax highlighter, auto-complete, automatic brace closer and brace pair detection will work right away with your preferred source code editor simply because it already knows the file type, so no special configuration needed.
• With support for popular file extensions such as .json and .yaml, you can basically just throw random JSON and YAML files into the .\lot\page folder, and then they will become valid page files.
• Native INI parsing with parse_ini_string() function.
• Native JSON parsing with json_decode() function.
• Native PHP parsing with include() or require() function.
• Native XML parsing with simplexml_load_string() function.
• Native YAML parsing with yaml_parse() function. ¹

Cons

• It’s difficult to optimize JSON and PHP parsing for a single data entry —or, at least, to verify the existence of a data key in the file— without parsing the whole file content. This is in contrast to YAML and INI files, which can be streamed line by line for loose detection before the actual parsing phase.
• It breaks the existing page files. You can’t simply change the file extension to .yaml and expect it to work.
• Using different file extensions for pages makes the CMS work harder. For example, listing page files requires more effort, and the file extension list may become difficult to maintain. New file types can be added or removed, but the list mostly needs to be hard-coded.
• It becomes more difficult to differentiate between archived, drafted, and published pages. Parsing the file content may be necessary just to skip the file path from the published page file list. Previously, it could be done very quickly because the page publication status was included in the file name extension.

Footnotes

1. This requires the PHP yaml extension, which is not installed by default, as far as I know. ↩

[taufik-nurrohman]

Typical page listing method:

// Native PHP
foreach (glob(LOT . D . 'page' . D . 'article' . D . '*.{ini,json,php,xml,yaml,yml}', GLOB_BRACE | GLOB_NOSORT) as $k => $v) { /* … */ }

// Feature in the CMS
foreach (g(LOT . D . 'page' . D . 'article', 'ini,json,php,xml,yaml,yml') as $k => $v) { /* … */ }

The problem arises:

• What if I want to remove .yml file extension support in the future?
• What if I want to treat files with extensions .conf, .desktop, .editorconfig, and .service the same way I treat files with the .ini extension?
• What if I want to support the .json5 file extension in the future because it somehow becomes a web standard?

My current solution is to use a mechanism similar to the one in my Image extension to create a list of functions named after the file extension it will support:

namespace {
    if (!\function_exists("\\yaml_parse")) {
        function yaml_parse(string $input, int $pos = 0, int &$ndocs = 0, ?array $callbacks = null) {
            // <https://github.com/mecha-cms/x.y-a-m-l/blob/v3.1.2/engine/plug/from.php#L7>
            return \From::YAML($input, true);
        }
    }
}

namespace x\page\from {
    function ini(string $path) {
        $r = \parse_ini_string(\file_get_contents($path), true, \INI_SCANNER_NORMAL);
        return \is_array($r) ? $r : [];
    }
    function json(string $path) {
        $r = \json_decode(\file_get_contents($path), true);
        return \is_array($r) ? $r : [];
    }
    function php(string $path) {
        $r = require $path;
        return \is_array($r) ? $r : [];
    }
    function xml(string $path) { /* … */ }
    function yaml(string $path) {
        $r = yaml_parse(\file_get_contents($path), 0);
        return \is_array($r) ? $r : [];
    }
    function yml(string $path) {
        return yaml($path);
    }
}

[taufik-nurrohman]

The following is an example of how the content of the page file might change if the .archive, .draft, and .page extensions were no longer supported (in which case, you would use the .yaml extension instead):

Before

---
title: Page Title
description: Page description.
type: Markdown
...

### Lorem Ipsum

Lorem ipsum **dolor** sit amet.

After

title: Page Title
description: Page description.
type: Markdown
content: |

  ### Lorem Ipsum

  Lorem ipsum **dolor** sit amet.

[taufik-nurrohman]

To generate list of supported file extensions:

function get_supported_page_extensions() {
    static $cache = null;
    if (null !== $cache) {
        return $cache;
    }
    $prefix = "x\\page\\from\\";    
    $r = [];
    foreach (get_defined_functions()['user'] as $v) {
        if (0 === strpos($v, $prefix)) {
            $r[] = substr($v, strlen($prefix));
        }
    }
    return ($cache = implode(',', $r));
}

[taufik-nurrohman]

INI

title = Page Title
description = Page description.
type = Markdown
content = "
### Lorem Ipsum

Lorem ipsum **dolor** sit amet.
"

JSON

{
  "title": "Page Title",
  "description": "Page description.",
  "type": "Markdown",
  "content": "### Lorem Ipsum\n\nLorem ipsum **dolor** sit amet."
}

PHP

<?php return [
    'title' => 'Page Title',
    'description' => 'Page description.',
    'type' => 'Markdown',
    'content' => <<<'S'
### Lorem Ipsum

Lorem ipsum **dolor** sit amet.
S
];

XML

<page>
  <title>Page Title</title>
  <description>Page description.</description>
  <type>Markdown</type>
  <content><![CDATA[
### Lorem Ipsum

Lorem ipsum **dolor** sit amet.
]]></content>
</page>

[taufik-nurrohman]

Cons

INI

• May display sensitive constants.
• It allows for a maximum of two levels of nesting by using the section syntax. Once you enter a section, you cannot return to the root; you can only move to a new section with a different name.

PHP

• It’s too robust; data should be constructed through safe form inputs. Only literal values can be written to the file.

XML

• The name of the root element is still open to discussion.

[taufik-nurrohman]

Hmmm…

In this situation, how to tell if time.yaml is a data or a page children of .\lot\page\lorem-ipsum.yaml file? 🤔

.\
└── lot\
    └── page\
        ├── lorem-ipsum\
        │   └── time.yaml ✔
        └── lorem-ipsum.yaml

[taufik-nurrohman]

Solution 1

.\
└── lot\
    └── page\
        ├── lorem-ipsum\
        │   ├── +time.yaml ✔
        │   └── time.yaml ✔
        └── lorem-ipsum.yaml

Now, time.yaml is a child of lorem-ipsum.yaml, +time.yaml is a data of lorem-ipsum.yaml.

Pros

• Even without documentation, it’s fairly obvious that files with a + prefix represent an addition to another file.
• People don’t intentionally design URLs that allow the + sign in their routes. While it’s technically possible to create a working URL with a + sign, it’s generally not popular.

Cons

• It takes more effort to list the page’s children:

  $pages = [];
  foreach (glob($folder . D . '*.yaml', GLOB_NOSORT) as $v) {
      if ('+' === (basename($v)[0] ?? 0)) {
          continue;
      }
      $pages[] = $v;
  }

  $pages = [];
  foreach (g($folder, 'yaml') as $k => $v) {
      if ('+' === (basename($k)[0] ?? 0)) {
          continue;
      }
      $pages[] = $k;
  }

  $pages = glob($folder . D . '[-.0-9_a-z]*.yaml', GLOB_NOSORT);

[taufik-nurrohman]

Solution 2

.\
└── lot\
    └── page\
        ├── lorem-ipsum\
        │   ├── +\
        │   │   └── time.yaml ✔
        │   └── time.yaml ✔
        └── lorem-ipsum.yaml

Use a folder named +\ to store all data.

Pros

• Listing the page’s children is the same as usual, no effort.

Cons

• Maybe it’s just a matter of time before I start messing around with characters like ~, !, @, $, and _ without caring about issues like operating system compatibility.

[taufik-nurrohman]

I have an idea that would keep the existing page files working by simply replacing the file extension with .yaml. That is to make the From::YAML() method treat YAML string containing --- prefix as a multiple document stream.

Input	Output
asdf: asdf	{ "asdf": "asdf" }
--- asdf: asdf	[ { "asdf": "asdf" } ]
asdf: asdf ---	[ { "asdf": "asdf" }, null ]
asdf: asdf --- asdf: asdf	[ { "asdf": "asdf" }, { "asdf": "asdf" } ]
	null
---	[ null ]
--- ---	[ null, null ]

With the new YAML parsing result specification, both formats in the example below would produce the same result. However, special handling is needed to parse the older format, which I want to keep since it is the most ✨ aesthetic ✨, in my opinion.

Style 1

This is a valid YAML document. No additional processing is necessary. Parsing it with From::YAML() method would produce the correct result immediately:

title: Page Title
description: Page description.
type: Markdown
content: |

  ### Lorem Ipsum

  Lorem ipsum **dolor** sit amet.

$lot = From::YAML($content, true);

Style 2

This is also a valid YAML document that will produce an array consisting of a single document. Any text after the ... will be left out, so the final result will only contain the description, title, and type data. To obtain the text after the end of the stream, a manual handling is needed:

---
title: Page Title
description: Page description.
type: Markdown
...

### Lorem Ipsum

Lorem ipsum **dolor** sit amet.

$lot = From::YAML($content, true);

if (is_array($lot = $lot[0] ?? [])) {
    if (!array_key_exists('content', $lot)) {
        $lot['content'] = trim(explode("\n...\n", $content . "\n", 2)[1] ?? "", "\n");
    }
} else {
    $lot = [];
}

[taufik-nurrohman]

I have been trying to convince ChatGPT to treat the ... marker as the “end of the stream” so that the parser can discard whatever comes after it. However, according to this example, the parser will resume parsing when the next --- marker is found in the rest of the stream.

It said that only certain things were allowed before reaching the next --- marker:

1. White-space(s)
2. Comment(s)
3. Another YAML document (started with ---)
4. End of the stream.

It said that giving a page file the extension .yaml is not compliant with the specification because the text after the ... marker may contain invalid YAML syntax due to my preferred page formatting style. A YAML file is supposed to contain only valid YAML syntax.

Consider the following example:

---
title: Page Title
description: Page description.
type: Markdown
...

> Lorem ipsum dolor sit amet.

From a hybrid point of view, it seems valid. It contains a mixed YAML syntax and a raw text after the ... marker that will be converted into HTML by the Markdown parser. From a YAML point of view, however, it is invalid because the text after the ... marker is not considered valid YAML syntax.

If you have a strict source code editor with YAMLLint installed, the junk text after the ... marker may prevent the editor from saving. It will throw an error stating that > asdf is not a valid YAML token. Removing the > prefix will fix the issue because it will then become a valid YAML string. However, that’s not the point. The point is that inconsistencies must be addressed to ensure the format survives future standards.

This is not a single YAML document followed by junk text after the ... marker:

---
title: Page Title
description: Page description.
type: Markdown
...

### Section 1

Content of section 1 goes here.

---

### Section 2

Content of section 2 goes here.

From a YAML point of view, it will be read as three documents, as follows:

[
  {
    "title": "Page Title",
    "description": "Page description.",
    "type": "Markdown"
  },
  "Content of section 1 goes here.",
  "Content of section 2 goes here."
]

There are only two safe solutions:

Solution 1

Give the page file a less common extension, such as .page, and allow the CMS to treat the first part as YAML and the rest as plain text, as it already does. The second solution is to switch to the most common plain text file extension, .txt. In either case, there is a trade-off: You won’t get proper YAML syntax highlighting, auto-closing, or auto-indenting features natively provided by your preferred source code editor.

Solution 2

Stick with the .yaml extension, but make sure that the second part is guaranteed to be returned as a string. A format that most closely resembles the existing ones and requires the least effort to adopt is as follows:

---
title: Page Title
description: Page description.
type: Markdown
--- |

 ### Section 1

 Content of section 1 goes here.

 ---

 ### Section 2

 Content of section 2 goes here.

The only trade-off with this new format is that you must add at least one space to every non-empty line in order to combine them into a single block of literal-style string.

[taufik-nurrohman]

The new specification for drafted files is to name them with a ~ prefix:

.\
└── lot\
    └── page\
        ├── lorem-ipsum\
        ├── ~test.yaml ✔
        └── lorem-ipsum.yaml

There is still no specific convention for archived files.

[taufik-nurrohman]

Or, like this, which gives us other potential features:

.\
└── lot\
    └── page\
        ├── .archive\
        │   └── test.yaml ✔
        ├── .draft\
        │   └── test.yaml ✔
        ├── .history\
        │   └── 2026\
        │       └── 01\
        │           ├── 19\
        │           │   └── test.yaml
        │           └── 20\
        │               └── test.yaml
        ├── .language\
        │   ├── en\
        │   ├── id\
        │   └── zh\
        ├── .sticky\
        │   └── test.yaml
        ├── .version\
        │   ├── 1.0\
        │   ├── 2.0\
        │   ├── 3.0\
        │   └── 3.1\
        └── lorem-ipsum.yaml

[taufik-nurrohman]

Hmm, the folder naming convention will break the child-parent relationship. Because files are no longer in the same tree level.

[taufik-nurrohman]

Final decision for the new naming convention:

• Archive page → #page-name.txt
• Draft page → ~page-name.txt
• Public page → page-name.txt