Some questions about migrating from v5 to v6

[marians]

I tried migrating a project of ours from v5 to v6 and can't figure out some of the API changes. Is there a migration guide somewhere? If yes, it would be great to have it linked in the v6.0.0 release notes.

• With httploader no longer there, is there anything to do to resolve URLs from $ref?
• What is the new equivalent of Compiler.ExtractAnnotations?
• What is the new equivalent of Schema.Deprecated?
• What is the new equivalent of Schema.RecursiveRef?
• What is the new equivalent of Schema.Constant?

Once I figured these out, I guess I'll run into more questions.

Thanks for your help!

[santhosh-tekuri]

With httploader no longer there, is there anything to do to resolve URLs from $ref?

you have to implement UrlLoader as shown in https://github.com/santhosh-tekuri/jsonschema/blob/boon/example_http_test.go

[santhosh-tekuri]

What is the new equivalent of Compiler.ExtractAnnotations

annotations are always extracted. so you no longer need Compiler.ExtractAnnotations

[santhosh-tekuri]

What is the new equivalent of Schema.Deprecated?

I missed that field. added in v6.0.1

[santhosh-tekuri]

What is the new equivalent of Schema.RecursiveRef?

I misspelled the field name as RecusiveRef. corrected in v6.0.1

[santhosh-tekuri]

What is the new equivalent of Schema.Constant?

it is now Schema.Const

[santhosh-tekuri]

Once I figured these out, I guess I'll run into more questions.

sure. it will be great help for me

[marians]

Thanks for the response so far, and for the quick mitigation! This helped a great deal.

I also ran into Schema.Format always being nil, and found out that I had to call Compiler.AssertFormat after compiler creation to enable parsing that.

[santhosh-tekuri]

I also ran into Schema.Format always being nil

this is same behaviour as in v5.0.0. for draft >7, Compiler.AssertFormat must be enabled

[santhosh-tekuri]

NOTE: second argument to compiler.AddResource in v5 is io.Reader where as in v6 is json object. so beware of what you are passing to second argument, because it compiles but fails at runtime.

[MattBrittan]

Further to the above comment "second argument to compiler.AddResource in v5 is io.Reader where as in v6 is json object" . The docs say:

The argument doc must be valid json value.

However I don't think this is all that clear, what is a "valid json value"? This is particularly important because the use of any means that code written for v5 compiles without error when using v6, but then may fail at runtime.

This caused an unexpected runtime issue for me (caught by tests) when upgrading from V5 (not sure if this is intentional or a bug). For example the following works in V5 (playground:

c := jsonschema.NewCompiler()
if err := c.AddResource(url, strings.NewReader(`{
	"properties": {
		"uniqueKeys": { "type": "string" }
	}
}`)); err != nil {
	log.Fatal(err)
}
_, err := c.Compile(url)

With v6 it compiles OK, but then panics (playground) when run.

To fix this it appears you need to use jsonschema.UnmarshalJSON to parse the schema before passing it to AddResource (with that change things appear to work (playground). Could I suggest that the docs be updated to reflect this, and possibly a check be made in AddResource so it fails fast (ideally returning an error instead of the panic)?

Thanks very much for all of your work on this!

[jachym-tousek-keboola]

I also have some questions about the migration. I found several things our old code was using that seem to have been removed or changed significantly:

1. We were using RegisterExtension which was removed. What should I use instead?
2. Types has been changed to integer with most of the API not exported, I don't see a way to replace this line?
3. ValidationError previously had AbsoluteKeywordLocation which is removed. What should I do with this code then?
4. InstanceLocation on ValidationError was changed to array. We were using the string for sorting.
5. Message on ValidationError was removed entirely? We were using that quite a bit too.

[santhosh-tekuri]

Types has been changed to integer with most of the API not exported, I don't see a way to replace this line?

you can convert Types to []string as follows: schema.Types.ToString(). but this allows to only read the types.
in your code you are updating Types field. currently it is private. I did not expect someone might be editing complied schema.
it is actually bit flag. you can manually do it by referring to the source code. I can add public API to modify it if you want.

[santhosh-tekuri]

ValidationError previously had AbsoluteKeywordLocation which is removed. What should I do with this code then?

it is changed to method. use verr.absoluteKeywordLocation() but it is not exported. I have to export it.

[santhosh-tekuri]

It seems you are using ValidationError api much. I suggest to use ValidationError.DetailedOutput(). it returns OutputUnit which is more friendlier to use and follows json specification.

ValidationError API is designed not to generate much garbage and to produce different styles of output. so it is not that friendlier api for consumers.

you can see the same error output in cli if you pass --output detailed. it prints in json format

[jachym-tousek-keboola]

you can convert Types to []string as follows: schema.Types.ToString(). but this allows to only read the types.
in your code you are updating Types field. currently it is private. I did not expect someone might be editing complied schema.
it is actually bit flag. you can manually do it by referring to the source code. I can add public API to modify it if you want.

No need. I managed to adjust the code to no longer need that (keboola/keboola-as-code@e457957).

[jachym-tousek-keboola]

it is changed to method. use verr.absoluteKeywordLocation() but it is not exported. I have to export it.

Maybe I can use .DetailedOutput().AbsoluteKeywordLocation instead?

[santhosh-tekuri]

InstanceLocation on ValidationError was changed to array. We were using the string for sorting.

yes. it is now array which actually contains the instaceLocation splitter by /. I will add InstanceLocationString() method which you can use

[jachym-tousek-keboola]

Maybe I can use DetailedOutput().InstanceLocation instead? 🤔

[santhosh-tekuri]

Message on ValidationError was removed entirely? We were using that quite a bit too.

use the following:

import	"golang.org/x/text/message"

var defaultPrinter = message.NewPrinter(language.English)
message := verr.ErrorKind.LocalizedString(defaultPrinter)

[jachym-tousek-keboola]

Thank you!

[santhosh-tekuri]

I suggest to use ValidationError.DetailedOutput() method. this has stable api as per specification.

[jachym-tousek-keboola]

So in this case... DetailedOutput().Error.Kind.LocalizedString(defaultPrinter)?

[santhosh-tekuri]

I mean not just of error message. it has all fields you want: keywordLocation, absoluteKeywordLocation, InstanceLocation as strings
currently it also does not has nice way to get error message. you have to use OutputUnit.Error.MarshalJSON(). but I just added OutputUnit.Error.String() in 82f3994

[jachym-tousek-keboola]

Okay I updated everything using your advice: https://github.com/keboola/keboola-as-code/pull/2369/files

But it seems there is still something wrong. jsonschema v6 now panics in our tests. Even if I misconfigured it somehow - which is entirely possible - it should return an error, rather than panic, shouldn't it?

Panic details

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc11afc]

goroutine 7 [running]:
testing.tRunner.func1.2({0x3221d00, 0x54220c0})
	/opt/hostedtoolcache/go/1.24.2/x64/src/testing/testing.go:1734 +0x3eb
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.24.2/x64/src/testing/testing.go:1737 +0x696
panic({0x3221d00?, 0x54220c0?})
	/opt/hostedtoolcache/go/1.24.2/x64/src/runtime/panic.go:792 +0x132
github.com/santhosh-tekuri/jsonschema/v6.(*roots).validate(0xc000b1f340, 0xc0008141e0, {0x356cfc0, 0xc0008141b0}, {0x0, 0x0})
	/home/runner/.cache/go/modcache/github.com/santhosh-tekuri/jsonschema/[email protected]/roots.go:241 +0x7c
github.com/santhosh-tekuri/jsonschema/v6.(*roots).addRoot(0xc000b1f340, {0x36e4a80, 0x13}, {0x356cfc0, 0xc0008141b0})
	/home/runner/.cache/go/modcache/github.com/santhosh-tekuri/jsonschema/[email protected]/roots.go:53 +0x277
github.com/santhosh-tekuri/jsonschema/v6.(*roots).orLoad(0xc000b1f340, {0x36e4a80, 0x13})
	/home/runner/.cache/go/modcache/github.com/santhosh-tekuri/jsonschema/[email protected]/roots.go:38 +0xec
github.com/santhosh-tekuri/jsonschema/v6.(*roots).resolveFragment(0xc000b1f340, {{0x36e4a80?, 0xc0008141b0?}, {0x0?, 0x36e4a80?}})
	/home/runner/.cache/go/modcache/github.com/santhosh-tekuri/jsonschema/[email protected]/roots.go:63 +0x5f
github.com/santhosh-tekuri/jsonschema/v6.(*Compiler).Compile(0xc0007f5da0, {0x36e4a80, 0x13})
	/home/runner/.cache/go/modcache/github.com/santhosh-tekuri/jsonschema/[email protected]/compiler.go:183 +0xa6
github.com/keboola/keboola-as-code/internal/pkg/encoding/json/schema.compileSchema({0xc000b181a0, 0x190, 0x190}, 0x1)
	/home/runner/work/keboola-as-code/keboola-as-code/internal/pkg/encoding/json/schema/schema.go:231 +0x856
github.com/keboola/keboola-as-code/internal/pkg/encoding/json/schema.GenerateDocument({0xc000b181a0, 0x190, 0x190})
	/home/runner/work/keboola-as-code/keboola-as-code/internal/pkg/encoding/json/schema/generate.go:18 +0x12f
github.com/keboola/keboola-as-code/internal/pkg/state/local.generateContent({0xc000b181a0, 0x190, 0x190}, 0xc000044560)
	/home/runner/work/keboola-as-code/keboola-as-code/internal/pkg/state/local/create.go:67 +0x1a5
github.com/keboola/keboola-as-code/internal/pkg/state/local.(*Manager).createObject(0xc000b36690, {0x3da4170, 0xc000b1f300}, {0x36cb826, 0x7})
	/home/runner/work/keboola-as-code/keboola-as-code/internal/pkg/state/local/create.go:41 +0x245
github.com/keboola/keboola-as-code/internal/pkg/state/local.TestLocalCreateConfigRowContentFromSchema(0xc00056d340)
	/home/runner/work/keboola-as-code/keboola-as-code/internal/pkg/state/local/create_test.go:132 +0x525
testing.tRunner(0xc00056d340, 0x379f7c0)
	/opt/hostedtoolcache/go/1.24.2/x64/src/testing/testing.go:1792 +0x226
created by testing.(*T).Run in goroutine 1
	/opt/hostedtoolcache/go/1.24.2/x64/src/testing/testing.go:1851 +0x8f3

[santhosh-tekuri]

I just published v6.0.2
this release has OutputUnit.Error.String() API

[jachym-tousek-keboola]

Thank you! I'll try it today and let you know. :)

[jachym-tousek-keboola]

Okay with 6.0.2 it no longer panics, it just returned an errror:

        	Error Trace:	/tmp/9b74cb56-b3c0-4cad-91a0-e19bca158c04/internal/pkg/encoding/json/schema/generate_test.go:17
        	Error:      	Received unexpected error:
        	            	"file:///schema.json#" is not valid against metaschema: jsonschema validation failed with 'https://json-schema.org/draft/2020-12/schema#'
        	            	- at '': invalid jsonType *bytes.Reader

So I adjusted the AddResource call according to your test from 86cca28 but it only changed the error a little bit:

        	Error Trace:	/tmp/78921945-7176-43d5-a9e6-f48a527c4f25/internal/pkg/encoding/json/schema/generate_test.go:17
        	Error:      	Received unexpected error:
        	            	"file:///schema.json#" is not valid against metaschema: jsonschema validation failed with 'https://json-schema.org/draft/2020-12/schema#'
        	            	- at '': invalid jsonType *strings.Reader

I must be missing something but the error doesn't quite clarify what type I'm supposed to pass to AddResource and the argument type is just any which doesn't help either. 🤔

[jachym-tousek-keboola]

Ah, okay... I added jsonschema.UnmarshalJSON call (inspiration from AddResource call in the vocabulary example). That looks like it might be the right way. It triggered some new errors but seem to be from our code.

[santhosh-tekuri]

Sometimes users of this library may want to use third party library to parse json for performance. Or may already have parsed json. So to accomodate those usecases, AddResource api is changed. Similarly Validate also takes parsed json

[jantari]

What is the v6 equivalent of jsonschema.CompileString?

I want to embed my JSON schema using:

//go:embed settingsFileSchema.json
var settingsFileSchema string

[jantari]

I attempted to copy-paste the last v5 implementation: https://github.com/santhosh-tekuri/jsonschema/blob/v5.3.1/compiler.go#L67

//go:embed settingsFileSchema.json
var settingsFileSchema string

compiler := jsonschema.NewCompiler()
if compiler.AddResource("settingsFileSchema.json", strings.NewReader(settingsFileSchema)); err != nil {
    log.Fatal(err)
}
schema, err := compiler.Compile("settingsFileSchema.json")
if err != nil {
    log.Fatal(err)
}

but I am getting the error:

"file:///home/jantari/repos/rewinged/settingsFileSchema#\" is not valid against metaschema: jsonschema validation failed with 'https://json-schema.org/draft/2020-12/schema#'\n- at '': invalid jsonType *strings.Reader"

[santhosh-tekuri]

the second argument to compiler.AddResource is parsed json

see example https://github.com/santhosh-tekuri/jsonschema/blob/boon/example_test.go#L39