Skip to content

Commit 0c37a62

Browse files
Ericson2314Ericson2314
committed
Change JSON derivation format in two ways
- Use canonical content address JSON format for floating content addressed derivation outputs This keeps it more consistent. - Reorganize inputs into nested structure (`inputs.srcs` and `inputs.drvs`) This will allow for an easier to use, but less compact, alternative where `srcs` is just a list of derived paths. It also allows for other experiments for derivations with a different input structure, as I suspect will be needed for secure build traces.
1 parent 147e183 commit 0c37a62

21 files changed

+298
-251
lines changed

‎doc/manual/source/protocols/json/derivation.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
{{#include derivation-v3-fixed.md}}
1+
{{#include derivation-v4-fixed.md}}
22

33
<!-- need to convert YAML to JSON first
44
## Raw Schema
55
6-
[JSON Schema for Derivation v3](schema/derivation-v3.json)
6+
[JSON Schema for Derivation v3](schema/derivation-v4.json)
77
-->

‎doc/manual/source/protocols/json/meson.build‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ schemas = [
1313
'content-address-v1',
1414
'store-path-v1',
1515
'store-object-info-v1',
16-
'derivation-v3',
16+
'derivation-v4',
1717
'deriving-path-v1',
1818
'build-trace-entry-v1',
1919
'build-result-v1',

‎doc/manual/source/protocols/json/schema/derivation-v3.yaml‎ renamed to ‎doc/manual/source/protocols/json/schema/derivation-v4.yaml‎

Lines changed: 61 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
"$schema": "http://json-schema.org/draft-04/schema"
2-
"$id": "https://nix.dev/manual/nix/latest/protocols/json/schema/derivation-v3.json"
2+
"$id": "https://nix.dev/manual/nix/latest/protocols/json/schema/derivation-v4.json"
33
title: Derivation
44
description: |
5-
Experimental JSON representation of a Nix derivation (version 3).
5+
Experimental JSON representation of a Nix derivation (version 4).
66
77
This schema describes the JSON representation of Nix's `Derivation` type.
88
@@ -17,8 +17,7 @@ required:
1717
- name
1818
- version
1919
- outputs
20-
- inputSrcs
21-
- inputDrvs
20+
- inputs
2221
- system
2322
- builder
2423
- args
@@ -32,10 +31,10 @@ properties:
3231
Used when calculating store paths for the derivation’s outputs.
3332
3433
version:
35-
const: 3
36-
title: Format version (must be 3)
34+
const: 4
35+
title: Format version (must be 4)
3736
description: |
38-
Must be `3`.
37+
Must be `4`.
3938
This is a guard that allows us to continue evolving this format.
4039
The choice of `3` is fairly arbitrary, but corresponds to this informal version:
4140
@@ -47,6 +46,12 @@ properties:
4746
4847
- Version 3: Drop store dir from store paths, just include base name.
4948
49+
- Version 4: Two cleanups, batched together to lesson churn:
50+
51+
- Reorganize inputs into nested structure (`inputs.srcs` and `inputs.drvs`)
52+
53+
- Use canonical content address JSON format for floating content addressed derivation outputs.
54+
5055
Note that while this format is experimental, the maintenance of versions is best-effort, and not promised to identify every change.
5156
5257
outputs:
@@ -70,47 +75,56 @@ properties:
7075
additionalProperties:
7176
"$ref": "#/$defs/output/overall"
7277

73-
inputSrcs:
74-
type: array
75-
title: Input source paths
76-
description: |
77-
List of store paths on which this derivation depends.
78-
79-
> **Example**
80-
>
81-
> ```json
82-
> "inputSrcs": [
83-
> "47y241wqdhac3jm5l7nv0x4975mb1975-separate-debug-info.sh",
84-
> "56d0w71pjj9bdr363ym3wj1zkwyqq97j-fix-pop-var-context-error.patch"
85-
> ]
86-
> ```
87-
items:
88-
$ref: "store-path-v1.yaml"
89-
90-
inputDrvs:
78+
inputs:
9179
type: object
92-
title: Input derivations
80+
title: Derivation inputs
9381
description: |
94-
Mapping of derivation paths to lists of output names they provide.
95-
96-
> **Example**
97-
>
98-
> ```json
99-
> "inputDrvs": {
100-
> "6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
101-
> "fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
102-
> }
103-
> ```
104-
>
105-
> specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`.
106-
patternProperties:
107-
"^[0123456789abcdfghijklmnpqrsvwxyz]{32}-.+\\.drv$":
108-
title: Store Path
82+
Input dependencies for the derivation, organized into source paths and derivation dependencies.
83+
required:
84+
- srcs
85+
- drvs
86+
properties:
87+
srcs:
88+
type: array
89+
title: Input source paths
90+
description: |
91+
List of store paths on which this derivation depends.
92+
93+
> **Example**
94+
>
95+
> ```json
96+
> "srcs": [
97+
> "47y241wqdhac3jm5l7nv0x4975mb1975-separate-debug-info.sh",
98+
> "56d0w71pjj9bdr363ym3wj1zkwyqq97j-fix-pop-var-context-error.patch"
99+
> ]
100+
> ```
101+
items:
102+
$ref: "store-path-v1.yaml"
103+
drvs:
104+
type: object
105+
title: Input derivations
109106
description: |
110-
A store path to a derivation, mapped to the outputs of that derivation.
111-
oneOf:
112-
- "$ref": "#/$defs/outputNames"
113-
- "$ref": "#/$defs/dynamicOutputs"
107+
Mapping of derivation paths to lists of output names they provide.
108+
109+
> **Example**
110+
>
111+
> ```json
112+
> "drvs": {
113+
> "6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
114+
> "fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
115+
> }
116+
> ```
117+
>
118+
> specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`.
119+
patternProperties:
120+
"^[0123456789abcdfghijklmnpqrsvwxyz]{32}-.+\\.drv$":
121+
title: Store Path
122+
description: |
123+
A store path to a derivation, mapped to the outputs of that derivation.
124+
oneOf:
125+
- "$ref": "#/$defs/outputNames"
126+
- "$ref": "#/$defs/dynamicOutputs"
127+
additionalProperties: false
114128
additionalProperties: false
115129

116130
system:
@@ -189,24 +203,18 @@ properties:
189203
The output is content-addressed, and the content-address is fixed in advance.
190204
191205
See [Fixed-output content-addressing](@docroot@/store/derivation/outputs/content-address.md#fixed) for more details.
192-
type: object
206+
"$ref": "./content-address-v1.yaml"
193207
required:
194208
- method
195-
- hashAlgo
196209
- hash
197210
properties:
198211
method:
199-
"$ref": "./content-address-v1.yaml#/$defs/method"
200212
description: |
201213
Method of content addressing used for this output.
202-
hashAlgo:
203-
title: Hash algorithm
204-
"$ref": "./hash-v1.yaml#/$defs/algorithm"
205214
hash:
206-
type: string
207215
title: Expected hash value
208216
description: |
209-
The expected content hash in base-16.
217+
The expected content hash.
210218
additionalProperties: false
211219

212220
caFloating:

‎src/json-schema-checks/meson.build‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -70,15 +70,15 @@ schemas += [
7070
# Match overall
7171
{
7272
'stem' : 'derivation',
73-
'schema' : schema_dir / 'derivation-v3.yaml',
73+
'schema' : schema_dir / 'derivation-v4.yaml',
7474
'files' : [
7575
'dyn-dep-derivation.json',
7676
'simple-derivation.json',
7777
],
7878
},
7979
{
8080
'stem' : 'derivation',
81-
'schema' : schema_dir / 'derivation-v3.yaml#/$defs/output/overall',
81+
'schema' : schema_dir / 'derivation-v4.yaml#/$defs/output/overall',
8282
'files' : [
8383
'output-caFixedFlat.json',
8484
'output-caFixedNAR.json',
@@ -92,14 +92,14 @@ schemas += [
9292
# Match exact variant
9393
{
9494
'stem' : 'derivation',
95-
'schema' : schema_dir / 'derivation-v3.yaml#/$defs/output/inputAddressed',
95+
'schema' : schema_dir / 'derivation-v4.yaml#/$defs/output/inputAddressed',
9696
'files' : [
9797
'output-inputAddressed.json',
9898
],
9999
},
100100
{
101101
'stem' : 'derivation',
102-
'schema' : schema_dir / 'derivation-v3.yaml#/$defs/output/caFixed',
102+
'schema' : schema_dir / 'derivation-v4.yaml#/$defs/output/caFixed',
103103
'files' : [
104104
'output-caFixedFlat.json',
105105
'output-caFixedNAR.json',
@@ -108,21 +108,21 @@ schemas += [
108108
},
109109
{
110110
'stem' : 'derivation',
111-
'schema' : schema_dir / 'derivation-v3.yaml#/$defs/output/caFloating',
111+
'schema' : schema_dir / 'derivation-v4.yaml#/$defs/output/caFloating',
112112
'files' : [
113113
'output-caFloating.json',
114114
],
115115
},
116116
{
117117
'stem' : 'derivation',
118-
'schema' : schema_dir / 'derivation-v3.yaml#/$defs/output/deferred',
118+
'schema' : schema_dir / 'derivation-v4.yaml#/$defs/output/deferred',
119119
'files' : [
120120
'output-deferred.json',
121121
],
122122
},
123123
{
124124
'stem' : 'derivation',
125-
'schema' : schema_dir / 'derivation-v3.yaml#/$defs/output/impure',
125+
'schema' : schema_dir / 'derivation-v4.yaml#/$defs/output/impure',
126126
'files' : [
127127
'output-impure.json',
128128
],

‎src/libstore-tests/data/derivation/ca/advanced-attributes-defaults.json‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,10 @@
1212
"outputHashMode": "recursive",
1313
"system": "my-system"
1414
},
15-
"inputDrvs": {},
16-
"inputSrcs": [],
15+
"inputs": {
16+
"drvs": {},
17+
"srcs": []
18+
},
1719
"name": "advanced-attributes-defaults",
1820
"outputs": {
1921
"out": {
@@ -22,5 +24,5 @@
2224
}
2325
},
2426
"system": "my-system",
25-
"version": 3
27+
"version": 4
2628
}

‎src/libstore-tests/data/derivation/ca/advanced-attributes-structured-attrs-defaults.json‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,10 @@
88
"dev": "/02qcpld1y6xhs5gz9bchpxaw0xdhmsp5dv88lh25r2ss44kh8dxz",
99
"out": "/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9"
1010
},
11-
"inputDrvs": {},
12-
"inputSrcs": [],
11+
"inputs": {
12+
"drvs": {},
13+
"srcs": []
14+
},
1315
"name": "advanced-attributes-structured-attrs-defaults",
1416
"outputs": {
1517
"dev": {
@@ -33,5 +35,5 @@
3335
"system": "my-system"
3436
},
3537
"system": "my-system",
36-
"version": 3
38+
"version": 4
3739
}

‎src/libstore-tests/data/derivation/ca/advanced-attributes-structured-attrs.json‎

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -9,25 +9,27 @@
99
"dev": "/02qcpld1y6xhs5gz9bchpxaw0xdhmsp5dv88lh25r2ss44kh8dxz",
1010
"out": "/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9"
1111
},
12-
"inputDrvs": {
13-
"j56sf12rxpcv5swr14vsjn5cwm6bj03h-foo.drv": {
14-
"dynamicOutputs": {},
15-
"outputs": [
16-
"dev",
17-
"out"
18-
]
12+
"inputs": {
13+
"drvs": {
14+
"j56sf12rxpcv5swr14vsjn5cwm6bj03h-foo.drv": {
15+
"dynamicOutputs": {},
16+
"outputs": [
17+
"dev",
18+
"out"
19+
]
20+
},
21+
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv": {
22+
"dynamicOutputs": {},
23+
"outputs": [
24+
"dev",
25+
"out"
26+
]
27+
}
1928
},
20-
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv": {
21-
"dynamicOutputs": {},
22-
"outputs": [
23-
"dev",
24-
"out"
25-
]
26-
}
29+
"srcs": [
30+
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv"
31+
]
2732
},
28-
"inputSrcs": [
29-
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv"
30-
],
3133
"name": "advanced-attributes-structured-attrs",
3234
"outputs": {
3335
"bin": {
@@ -101,5 +103,5 @@
101103
"system": "my-system"
102104
},
103105
"system": "my-system",
104-
"version": 3
106+
"version": 4
105107
}

‎src/libstore-tests/data/derivation/ca/advanced-attributes.json‎

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -25,25 +25,27 @@
2525
"requiredSystemFeatures": "rainbow uid-range",
2626
"system": "my-system"
2727
},
28-
"inputDrvs": {
29-
"j56sf12rxpcv5swr14vsjn5cwm6bj03h-foo.drv": {
30-
"dynamicOutputs": {},
31-
"outputs": [
32-
"dev",
33-
"out"
34-
]
28+
"inputs": {
29+
"drvs": {
30+
"j56sf12rxpcv5swr14vsjn5cwm6bj03h-foo.drv": {
31+
"dynamicOutputs": {},
32+
"outputs": [
33+
"dev",
34+
"out"
35+
]
36+
},
37+
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv": {
38+
"dynamicOutputs": {},
39+
"outputs": [
40+
"dev",
41+
"out"
42+
]
43+
}
3544
},
36-
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv": {
37-
"dynamicOutputs": {},
38-
"outputs": [
39-
"dev",
40-
"out"
41-
]
42-
}
45+
"srcs": [
46+
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv"
47+
]
4348
},
44-
"inputSrcs": [
45-
"qnml92yh97a6fbrs2m5qg5cqlc8vni58-bar.drv"
46-
],
4749
"name": "advanced-attributes",
4850
"outputs": {
4951
"out": {
@@ -52,5 +54,5 @@
5254
}
5355
},
5456
"system": "my-system",
55-
"version": 3
57+
"version": 4
5658
}

0 commit comments

Comments
 (0)