CLI tool to merge TOML files

[halloleo]

Often I have to merge two TOML files by table header. Here an example what I mean by this:

file1.toml:

[header_a]
key1 = "information"
key2 = "content"

[header_b]
key1 = "data"

file2.toml:

[header_a]
key3 = "details"

Merged fileresult.toml:

[header_a]
key1 = "information"
key2 = "content"
key3 = "details"

[header_b]
key1 = "data"

In order to not invent the wheel twice, does a readymade CLI tool for this exist?

[ChristianSi]

I'm not aware for an CLI tool – googling found some old stuff, but nothing that looked up-to-date. But it should be fairly easy to write, let's say for your example files in Python (untested code):

import tomllib
import tomli_w

with open("file1.toml", "rb") as fh:
    file1data = tomllib.load(fh)

with open("file2.toml", "rb") as fh:
    file2data = tomllib.load(fh)

fileresult_data = file1data | file2data

with open("fileresult_data.toml", "wb") as fh:
    tomli_w.dump(fileresult_data, fh)

Reading the file names from CLI arguments instead is left as exercise for the reader.

[halloleo]

Thanks @ChristianSi.

Makes total sense. I like the union operator on the dictionaries! :-)

[eksortso]

@ChristianSi Quick note... Should this be the line defining the result?

fileresult_data = file1data | file2data

[ChristianSi]

@eksortso Indeed, thanks for spotting that small typo (now corrected!) I guess that's what inevitably happens when you publish untested code...

[DivineDominion]

I tried this today, but it didn't produce the expected output:

cat > a.toml <<- EOF
[header_a]
key1 = "information"
key2 = "content"

[header_b]
key1 = "data"
EOF

cat > b.toml <<- EOF
[header_a]
key3 = "details"
EOF

import tomllib
import tomli_w

with open("a.toml", "rb") as fh:
    file1data = tomllib.load(fh)

with open("b.toml", "rb") as fh:
    file2data = tomllib.load(fh)

fileresult_data = file1data | file2data

print(tomli_w.dumps(fileresult_data))

Result:

[header_a]
key3 = "details"

[header_b]
key1 = "data"

Am I holding something wrong?

[ChristianSi]

@DivineDominion: Good catch, I had not considered that, while | merges dicts, it does so shallowly: if both dicts contain a key, the value from the second one is kept. In our case, however, we want to merge the values, provided they are both dicts (i.e., tables). This code should do the trick:

from typing import Any, Dict

def merge_dicts(a: Dict[Any, Any], b: Dict[Any, Any]) -> Dict[Any, Any]:
    """Merge two dictionaries recursively.

    If both have the same key and their values are dictionaries, merge them recursively.
    Otherwise, b overrides a.
    """
    result = a.copy()
    for key, b_val in b.items():
        a_val = result.get(key)
        if isinstance(a_val, dict) and isinstance(b_val, dict):
            result[key] = merge_dicts(a_val, b_val)
        else:
            result[key] = b_val
    return result

# Read the files first, as above
fileresult_data = merge_dicts(file1data, file2data)