Skip to content

Commit fc1f428

Browse files
authored
Tool to fetch API specs. Document API support (#61)
Recent k8s servers publish OpenAPI v3 specs via a set of API discovery URLs, instead of the earlier consolidated specification. Adding a tool that can be used to prepare a consolidated OpenAPI v3 specification from multiple discovered API endpoints. The API version support is via a module and is pluggable. The included API module supports a default set of APIs and versions. Also documenting the included API versions support in this PR.
1 parent d25bda1 commit fc1f428

File tree

6 files changed

+326
-3
lines changed

6 files changed

+326
-3
lines changed

Project.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ authors = ["JuliaHub Inc."]
44
keywords = ["kubernetes", "client"]
55
license = "MIT"
66
desc = "Julia Kubernetes Client"
7-
version = "0.7.4"
7+
version = "0.7.5"
88

99
[deps]
1010
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ A Julia Kubernetes Client.
66

77
An easy to use API to access Kubernetes clusters from Julia. The `Kuber.ApiImpl.Kubernetes` submodule has the complete set of low level APIs and entities.
88

9+
[Supported API Versions](SupportedAPIVersions.md)
10+
911
Most of the low level APIs fit into a common usage pattern. Kuber.jl makes it possible to use all of them with only a few intuitive verb based APIs. Verbs act on entities. Entities can be identified by names or selector patterns, or otherwise can apply to all entities of that class. Verbs can take additional parameters, e.g. when creating or updating entities.
1012

1113
API and Entity naming convention follows the standard Kubernetes API and Model naming conventions.

SupportedAPIVersions.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
## Supported API Versions
2+
3+
The default API client included in this version of Kuber.jl supports the following API versions:
4+
5+
- `admissionregistration`
6+
- `admissionregistration_v1`
7+
- `admissionregistration_v1beta1`
8+
- `apiextensions`
9+
- `apiextensions_v1`
10+
- `apiextensions_v1beta1`
11+
- `apiregistration`
12+
- `apiregistration_v1`
13+
- `apiregistration_v1beta1`
14+
- `apis`
15+
- `apps`
16+
- `apps_v1`
17+
- `apps_v1beta1`
18+
- `apps_v1beta2`
19+
- `auditregistration`
20+
- `auditregistration_v1alpha1`
21+
- `authentication`
22+
- `authentication_v1`
23+
- `authentication_v1beta1`
24+
- `authorization`
25+
- `authorization_v1`
26+
- `authorization_v1beta1`
27+
- `autoscaling`
28+
- `autoscaling_v1`
29+
- `autoscaling_v2beta1`
30+
- `autoscaling_v2beta2`
31+
- `batch`
32+
- `batch_v1`
33+
- `batch_v1beta1`
34+
- `batch_v2alpha1`
35+
- `certificates`
36+
- `certificates_v1beta1`
37+
- `coordination`
38+
- `coordination_v1`
39+
- `coordination_v1beta1`
40+
- `core`
41+
- `core_v1`
42+
- `custom_metrics_v1beta1`
43+
- `discovery`
44+
- `discovery_v1beta1`
45+
- `events`
46+
- `events_v1beta1`
47+
- `extensions`
48+
- `extensions_v1beta1`
49+
- `flowcontrolApiserver`
50+
- `flowcontrolApiserver_v1alpha1`
51+
- `karpenterSh_v1alpha5`
52+
- `logs`
53+
- `metrics_v1beta1`
54+
- `networking`
55+
- `networking_v1`
56+
- `networking_v1beta1`
57+
- `node`
58+
- `node_v1alpha1`
59+
- `node_v1beta1`
60+
- `policy`
61+
- `policy_v1beta1`
62+
- `rbacAuthorization`
63+
- `rbacAuthorization_v1`
64+
- `rbacAuthorization_v1alpha1`
65+
- `rbacAuthorization_v1beta1`
66+
- `scheduling`
67+
- `scheduling_v1`
68+
- `scheduling_v1alpha1`
69+
- `scheduling_v1beta1`
70+
- `settings`
71+
- `settings_v1alpha1`
72+
- `storage`
73+
- `storage_v1`
74+
- `storage_v1alpha1`
75+
- `storage_v1beta1`
76+
- `version`
77+

gen/K8sOpenAPISpec.jl

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
"""
2+
To consolidate all discovered OpenAPI endpoint specifications:
3+
- Ensure all required CRDs are applied on the k8s server
4+
- Preferably use `kubectl proxy` temporarily to avoid having to install certificates
5+
- Run `julia -e 'include("K8sOpenAPISpec.jl"); K8sOpenAPISpec.get_specification()'`. This should create a file named `k8s.json` with all discovered OpenAPI specifications included.
6+
7+
To display all discovered API versions in a generated `k8s.json` file:
8+
- Run `julia -e 'include("K8sOpenAPISpec.jl"); K8sOpenAPISpec.show_api_versions()'`
9+
"""
10+
module K8sOpenAPISpec
11+
12+
using Downloads
13+
using JSON
14+
using YAML
15+
16+
const DEFAULT_API_SERVER = "http://localhost:8001"
17+
const API_DISCOVERY = "/openapi/v3"
18+
19+
function fetch_json(url::String)
20+
iob = IOBuffer()
21+
Downloads.download(url, iob)
22+
return JSON.parse(String(take!(iob)))
23+
end
24+
25+
function discover_apis(api_server::String=DEFAULT_API_SERVER)
26+
api_discovery_url = string(api_server, API_DISCOVERY)
27+
api_discovery = fetch_json(api_discovery_url)
28+
api_discovery["paths"]
29+
end
30+
31+
function merge_l1!(merged_spec::Dict{String,Any}, spec::Dict{String,Any}, key::String)
32+
haskey(spec, key) || return
33+
if haskey(merged_spec, key)
34+
merge!(merged_spec[key], spec[key])
35+
else
36+
merged_spec[key] = spec[key]
37+
end
38+
return merged_spec
39+
end
40+
41+
function merge_l2(merged_spec::Dict{String,Any}, spec::Dict{String,Any}, key1::String)
42+
haskey(spec, key1) || return
43+
44+
if !haskey(merged_spec, key1)
45+
merged_spec[key1] = Dict{String,Any}()
46+
end
47+
48+
for key2 in keys(spec[key1])
49+
merge_l1!(merged_spec[key1], spec[key1], key2)
50+
end
51+
return merged_spec
52+
end
53+
54+
function merge_array!(merged_spec::Dict{String,Any}, spec::Dict{String,Any}, key::String)
55+
if haskey(spec, key)
56+
if haskey(merged_spec, key)
57+
merged_spec[key] = vcat(merged_spec[key], spec[key])
58+
else
59+
merged_spec[key] = spec[key]
60+
end
61+
end
62+
return merged_spec
63+
end
64+
65+
function merge_spec!(merged_spec::Dict{String,Any}, spec::Dict{String,Any})
66+
# /openapi
67+
haskey(spec, "openapi") && (merged_spec["openapi"] = spec["openapi"])
68+
69+
# /info
70+
merge_l1!(merged_spec, spec, "info")
71+
# /externalDocs
72+
merge_l1!(merged_spec, spec, "externalDocs")
73+
74+
# /servers
75+
merge_array!(merged_spec, spec, "servers")
76+
# /tags
77+
merge_array!(merged_spec, spec, "tags")
78+
79+
# /paths/{path}/{method}
80+
merge_l2(merged_spec, spec, "paths")
81+
82+
# /components/schemas/{schema}
83+
# /components/responses/{response}
84+
# /components/parameters/{parameter}
85+
# /components/securitySchemes/{securityScheme}
86+
merge_l2(merged_spec, spec, "components")
87+
88+
# /definitions/{definition}
89+
merge_l1!(merged_spec, spec, "definitions")
90+
# /securityDefinitions/{securityDefinition}
91+
merge_l1!(merged_spec, spec, "securityDefinitions")
92+
# /security (merge arrays)
93+
merge_array!(merged_spec, spec, "security")
94+
95+
return merged_spec
96+
end
97+
98+
function create_merged_spec(api_server::String=DEFAULT_API_SERVER)
99+
paths = discover_apis(api_server)
100+
merged_spec = Dict{String,Any}()
101+
for name in keys(paths)
102+
@info("merging $name")
103+
url = string(api_server, paths[name]["serverRelativeURL"])
104+
api_spec = fetch_json(url)
105+
merge_spec!(merged_spec, api_spec)
106+
end
107+
if !haskey(merged_spec, "openapi")
108+
merged_spec["openapi"] = "3.0.0"
109+
end
110+
return merged_spec
111+
end
112+
113+
"""
114+
get_specification(; outfile::String="k8s.json", api_server::String=DEFAULT_API_SERVER)
115+
116+
Consolidate all discovered OpenAPI endpoint specifications and write them to a file.
117+
118+
Keyword arguments:
119+
- `outfile::String`: The file to write the OpenAPI specification to. Defaults to `k8s.json`.
120+
- `api_server::String`: The URL of the k8s API server. Defaults to `http://localhost:8001`.
121+
"""
122+
function get_specification(; outfile::String="k8s.json", api_server::String=DEFAULT_API_SERVER)
123+
merged_spec = create_merged_spec(api_server)
124+
open(outfile, "w") do f
125+
JSON.print(f, merged_spec, 2)
126+
end
127+
end
128+
129+
"""
130+
merge_specifications(folder::String; outfile::String="k8s.json")
131+
132+
Merge all JSON files in a given folder into a single OpenAPI specification.
133+
134+
Arguments:
135+
- `folder::String`: The folder to read the JSON files from.
136+
137+
Keyword arguments:
138+
- `outfile::String`: The file to write the OpenAPI specification to. Defaults to `k8s.json`.
139+
"""
140+
function merge_specifications(folder::String; outfile::String="k8s.json")
141+
merged_spec = Dict{String,Any}()
142+
for file in readdir(folder)
143+
if endswith(file, ".json")
144+
spec = JSON.parsefile(joinpath(folder, file))
145+
elseif endswith(file, ".yaml")
146+
spec = YAML.load_file(joinpath(folder, file))
147+
else
148+
continue
149+
end
150+
@info("merging $file")
151+
merge_spec!(merged_spec, convert(Dict{String,Any}, spec))
152+
end
153+
if !haskey(merged_spec, "openapi")
154+
merged_spec["openapi"] = "3.0.0"
155+
end
156+
open(outfile, "w") do f
157+
JSON.print(f, merged_spec, 2)
158+
end
159+
end
160+
161+
"""
162+
show_api_versions(specification::Dict{String,Any})
163+
164+
Display all discovered API versions in a given OpenAPI specification.
165+
166+
Arguments:
167+
- `specification::Dict{String,Any}`: The OpenAPI specification to read the API versions from.
168+
"""
169+
function show_api_versions(specification::Dict{String,Any})
170+
tags = Set{String}()
171+
for path in keys(specification["paths"])
172+
for op in keys(specification["paths"][path])
173+
if !isa(specification["paths"][path][op], Dict)
174+
# @warn("possible invalid specification: /paths/$path/$op")
175+
continue
176+
end
177+
if haskey(specification["paths"][path][op], "tags")
178+
for tag in specification["paths"][path][op]["tags"]
179+
push!(tags, tag)
180+
end
181+
end
182+
end
183+
end
184+
sorted_tags = sort(collect(tags))
185+
for tag in sorted_tags
186+
println(tag)
187+
end
188+
end
189+
190+
"""
191+
show_api_versions(; specification_file::String="k8s.json")
192+
193+
Display all discovered API versions in a generated `k8s.json` file.
194+
195+
Keyword arguments:
196+
- `specification_file::String`: The JSON file to read the OpenAPI specification from. Defaults to `k8s.json`.
197+
"""
198+
function show_api_versions(; specification_file::String="k8s.json")
199+
specification = JSON.parsefile(specification_file)
200+
show_api_versions(specification)
201+
end
202+
203+
end # module K8sOpenAPISpec

gen/README.md

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Use the bundled `generate.sh` script to generate OpenAPI client implementations.
44

5-
- Fetch the OpenAPI v2 (Swagger) specification from the `/openapi/v2` endpoint of a running k8s api server
5+
- Fetch the OpenAPI v2 (Swagger) specification from the `/openapi/v2` endpoint of a running k8s api server. For K8s servers that serve only the OpenAPI v3 specifications, use the included [K8sOpenAPISpec.jl](K8sOpenAPISpec.jl) script to consolidate specs from the API discovery endpoints into a single specfification file.
66
- Ensure the k8s server has all the required CRDs installed
77
- The specification file must be named `swagger.json`. It can be stored in any location, but store it in the `spec` folder if you wish to update the Kuber.jl package itself
88
- The k8s OpenAPI spec uses a custom `int-or-string` format, that needs to be tweaked in the specification to be able to generate it correctly (see: https://github.com/kubernetes/kube-openapi/issues/52)
@@ -17,3 +17,43 @@ Use the bundled `generate.sh` script to generate OpenAPI client implementations.
1717
- Note:
1818
- the `api` folder in the output path will be renamed to `api_bak`
1919
- existing `api_bak` folder if any in output folder will be deleted
20+
21+
## List Supported API Versions
22+
23+
The included [K8sOpenAPISpec.jl](K8sOpenAPISpec.jl) script can be used to list the supported APIs and their versions from a given K8s OpenAPI specification.
24+
25+
## K8sOpenAPISpec
26+
27+
A standalone tool included to help in generating the API OpenAPI client from Kubernetes OpenAPI specifications.
28+
29+
### `get_specification(; outfile::String="k8s.json", api_server::String=DEFAULT_API_SERVER)`
30+
31+
Consolidate all discovered OpenAPI endpoint specifications and write them to a file.
32+
33+
Keyword arguments:
34+
- `outfile::String`: The file to write the OpenAPI specification to. Defaults to `k8s.json`.
35+
- `api_server::String`: The URL of the k8s API server. Defaults to `http://localhost:8001`.
36+
37+
### `merge_specifications(folder::String; outfile::String="k8s.json")`
38+
39+
Merge all JSON files in a given folder into a single OpenAPI specification.
40+
41+
Arguments:
42+
- `folder::String`: The folder to read the JSON files from.
43+
44+
Keyword arguments:
45+
- `outfile::String`: The file to write the OpenAPI specification to. Defaults to `k8s.json`.
46+
47+
### `show_api_versions(; specification_file::String="k8s.json")`
48+
49+
Display all discovered API versions in a generated `k8s.json` file.
50+
51+
Keyword arguments:
52+
- `specification_file::String`: The JSON file to read the OpenAPI specification from. Defaults to `k8s.json`.
53+
54+
### show_api_versions(specification::Dict{String,Any})
55+
56+
Display all discovered API versions in a given OpenAPI specification.
57+
58+
Arguments:
59+
- `specification::Dict{String,Any}`: The OpenAPI specification to read the API versions from.

src/helpers.jl

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -431,7 +431,8 @@ function build_model_api_map(ctx::KuberContext)
431431
for name in names(types; all=true)
432432
(name in [:eval, Symbol("#eval"), :include, Symbol("#include"), Symbol(split(string(types), '.')[end])]) && continue
433433
# de-prioritize extensions for the default simpleapi mapping (so if a model already has a dedicated api version, do not use extensions)
434-
haskey(modelapi, name) && (types === apimodule(ctx).Typedefs.ExtensionsV1beta1) && continue
434+
# extensions are deprecated and not supported in k8s versions after v1.16
435+
# haskey(modelapi, name) && (types === apimodule(ctx).Typedefs.ExtensionsV1beta1) && continue
435436
modelapi[name] = apiver
436437
end
437438
end

0 commit comments

Comments
 (0)