Fix #881 (#464): Added JSON encoder to disable escaping html symbols

[Moskovych]

Fixes #881
Fixes #464

[Moskovych]

@felixfontein , @autrilla , could you please review PR and approve builds?
Any additional changes are required? More tests?
Functionality is not changed - as you said, output file is semantically the same.

[autrilla]

LGTM. Because this is technically a breaking change, I'd like @ajvb's LGTM too. Like I've mentioned before, I think this kind of breaking change is not really breaking, since we make no guarantees about the syntax of the output, and only about the semantics.

[autrilla]

A test for this would be nice to have

[Moskovych]

@autrilla A test for this would be nice to have

Ok, will work on it.

[autrilla]

Hmmm, looks like the tests are broken

[Moskovych]

@autrilla , and looks like it's pgp one, which I didn't changed.

[Moskovych]

I can't proceed with adding new tests as existing are not working properly (see #882)

[tanandy]

hi guys we have the same issue, any updates here ?

[Moskovych]

@tanandy , sorry for delaying, but this PR still require unit test to be covered. I'll try to add them asap.

[akshaypatidar1999]

@ajvb @autrilla Can you please release this? We are also facing similar issue

[Moskovych]

Still blocked by #977.

[hiddeco]

Have you tried one of the keys from this history tree? https://github.com/getsops/sops/commits/main/pgp/sops_functional_tests_key.asc

[Moskovych]

@hiddeco , thanks for pointing me out, but correct me if I'm wrong:
to fix the tests: stores/json/store_test.go
I need to decrypt the file: stores/json/test_resources/example.json
where it contains aws kms key (for which I don't have access) or gpg E5297818703249D0C60E19E6824612478D1A4CCD

From what I see, there are no such key (the last one, gpg), even in history of repository.

Did I missed something?

[felixfontein]

stores/json/test_resources/example.json isn't used in the tests.

[Moskovych]

@felixfontein , github workflows are not running without any approvals. Can you help with it?

[felixfontein]

Encode appends a newline (https://pkg.go.dev/encoding/json#Encoder.Encode), which json.Marshal apparently does not do. (This makes the unit tests fail.) This is apparently because Encoder is for JSON streams, while Marshal produces a single JSON file.

[felixfontein]

Simply adding code which removes a trailing newline (if exists) probably suffices to fix this. I don't see another way to do this with Go's API.

[Moskovych]

Shouldn't we stick with newline in new files instead?

[felixfontein]

This function is called from encodeValue and used during serialization of any JSON value, so if you use this to create JSON of a map, you get someting like

{
  "foo"
: "bar"
,
  "baz"
: 42
,
  "bam"
: null
}

[bscaleb]

Any updates here?