feat: add TOML store by jcmfernandes · Pull Request #2031 · getsops/sops

jcmfernandes · 2026-01-02T11:10:43Z

Fixes #369. Supersedes #812 (that in turn superseded #533). Hopefully, the third time's the charm.

Uses github.com/pelletier/go-toml/v2. It discards comments.

Signed-off-by: João Fernandes <joao@bckground.com>

felixfontein

Thanks for picking this up! It would be great if this store could also handle comments, but it seems the library doesn't really handle them yet.

felixfontein · 2026-01-11T14:43:45Z

stores/toml/store.go

+
+	// Sort each group independently.
+	sortKeysNaturally(simpleKeys)
+	sortKeysNaturally(complexKeys)


I don't think it is a good idea to sort keys. I as a SOPS user expect a store to not modify the key order.

felixfontein · 2026-01-11T14:43:58Z

stores/toml/store.go

+// mapToTreeBranch converts a map[string]any to a sops.TreeBranch.
+func mapToTreeBranch(m map[string]any) (sops.TreeBranch, error) {
+	// Separate keys by type: simple values first, then complex types
+	// (tables/arrays).


What is the rationale for this?

felixfontein · 2026-01-11T14:52:13Z

stores/toml/store.go

+	}
+
+	// Replace single quotes with double quotes for string values.
+	result := bytes.ReplaceAll(buf.Bytes(), []byte("'"), []byte("\""))


felixfontein · 2026-01-11T14:56:30Z

stores/toml/store_test.go

+
+[2]
+  21 = [21.1, 21.2]
+  22 = 22


Is it possible to make the indentation configurable?

Unfortunately, no, when using the "normal" API. Please check #2031 (comment) - it might be possible if I go for a different approach.

jcmfernandes · 2026-01-11T21:26:43Z

Thanks for picking this up! It would be great if this store could also handle comments, but it seems the library doesn't really handle them yet.

Happy to contribute!

I have good and bad news: the good news is that, to some extent, it's possible to handle comments. Being specific, it's possible to handle start-of-the-line comments, but not end-of-the-line comments, i.e.,

# We can handle these
xpto = 42 # But we can't handle these

The bad news is that this requires using the unstable API and results in significantly more complex code. Should I give it a try?

felixfontein · 2026-01-13T21:46:06Z

Hmm, good question... Do you know whether the marshaller supports comments (at least start-of-line ones) as well? If it does not, then I guess it doesn't make sense - yet - to support them.

jcmfernandes · 2026-01-13T23:51:14Z

Hmm, good question... Do you know whether the marshaller supports comments (at least start-of-line ones) as well? If it does not, then I guess it doesn't make sense - yet - to support them.

The marshaler, i.e., the "normal" github.com/pelletier/go-toml/v2 package does not support marshaling comments. The unstable API does support comments, but only start-of-the-line comments. I can give it a try! It will be more complex, and the unstable API is, well, unstable, making updates possibly more complicated.

felixfontein · 2026-01-14T06:13:26Z

I can only see parsing code in the unstable API. Can you point me to the unstable API that allows creating TOML files with start-of-the-line comments?

germandiagogomez · 2026-02-05T15:37:11Z

Is this patch going to be merged? I need this badly to onboard sops :)

Otherwise I will have to find alternatives for my toml files.

felixfontein · 2026-02-05T19:59:26Z

That depends. Right now the PR is not mergable because not all commits are signed off. I'm also curious on the answer to my question, I was not able to find code in the library that allows to create TOML files with comments (but then, I didn't search very intensely).

felixfontein · 2026-02-07T13:48:10Z

@jcmfernandes @bckground was deleting the branch (which seems to have been a consequence of deleting the repository) intentionally?

jcmfernandes · 2026-02-13T11:03:47Z

@jcmfernandes @bckground was deleting the branch (which seems to have been a consequence of deleting the repository) intentionally?

I'm deeply sorry, @felixfontein. My associate deleted the repo from our org as he didn't know I was working on this. He owes us all a drink!

I have good and bad news.

The good news is that I was able to rewrite the TOML store and have it keep structure (order) and comments. Before I open a PR, please check https://github.com/bckground/sops/commits/toml-alternative and let me know what you think (disclaimer: AI-assisted but not AI-written).

The bad news is that I had to modify https://github.com/pelletier/go-toml and my changes aren't yet upstream — you can find them at https://github.com/bckground/go-toml/commits/marshal-ast — and I had to use the unstable package.

I'm hopeful my changes will be accepted. I'll then attempt to get this TOML store merged.

felixfontein · 2026-02-15T12:49:24Z

@jcmfernandes thanks for the update! I glanced over the code changes (both for SOPS and go-toml), and I think it would be great to have them. (I haven't looked in detail, and obviously I can't speak for go-toml whether they want them.)

The bad news is that I had to modify https://github.com/pelletier/go-toml and my changes aren't yet upstream — you can find them at https://github.com/bckground/go-toml/commits/marshal-ast — and I had to use the unstable package.

I think it would benefit go-toml a lot to have a more general formatter API, even if it's "just" in unstable. Maybe this would also bring "unstable" one more step in the direction of eventually becoming stable?

I saw you've started with a first PR: pelletier/go-toml#1031 🎉

A general note: I'd probably also add some "round-trip" tests for parser + formatter, where you have a set of TOML documents that are first parsed to an AST, and then the AST is formatted, and the result is compared to the original input. This obviously won't work for all TOML files, but it can help a lot during development to see what a change is causing (resp. that a change is not breaking anything on a higher level).

jcmfernandes added 5 commits January 2, 2026 20:11

chore: add go-toml library to the module

240015e

Signed-off-by: João Fernandes <joao@bckground.com>

feat: add TOML store

46ed1b4

Signed-off-by: João Fernandes <joao@bckground.com>

chore: wire things up

76038c7

Signed-off-by: João Fernandes <joao@bckground.com>

refactor: use go-toml/v2 instead

16f62ab

Signed-off-by: João Fernandes <joao@bckground.com>

test: add more tests by taking inspiration from the YAML suite

d7cbc32

Signed-off-by: João Fernandes <joao@bckground.com>

jcmfernandes force-pushed the toml-store branch 2 times, most recently from e3b5072 to d0bf7b4 Compare January 3, 2026 18:00

chore: keep alphabetical order

39cd216

jcmfernandes force-pushed the toml-store branch from d0bf7b4 to 39cd216 Compare January 3, 2026 18:01

test: types

8da3551

felixfontein reviewed Jan 11, 2026

View reviewed changes

felixfontein added enhancement area/stores labels Jan 11, 2026

bckground closed this by deleting the head repository Feb 7, 2026

felixfontein mentioned this pull request Feb 16, 2026

add toml store #812

Closed

Conversation

jcmfernandes commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixfontein left a comment

Choose a reason for hiding this comment

Uh oh!

felixfontein Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

felixfontein Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

felixfontein Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

felixfontein Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

jcmfernandes Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

jcmfernandes commented Jan 11, 2026

Uh oh!

felixfontein commented Jan 13, 2026

Uh oh!

jcmfernandes commented Jan 13, 2026

Uh oh!

felixfontein commented Jan 14, 2026

Uh oh!

germandiagogomez commented Feb 5, 2026

Uh oh!

felixfontein commented Feb 5, 2026

Uh oh!

felixfontein commented Feb 7, 2026

Uh oh!

jcmfernandes commented Feb 13, 2026

Uh oh!

felixfontein commented Feb 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

jcmfernandes commented Jan 2, 2026 •

edited

Loading