Skip to content

Commit f7968d9

Browse files
ablaomablaom
committed
add update and other management tools. Some testing still to go.
1 parent e6dd9eb commit f7968d9

File tree

12 files changed

+5230
-51
lines changed

12 files changed

+5230
-51
lines changed

‎Manifest.toml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
julia_version = "1.11.3"
44
manifest_format = "2.0"
5-
project_hash = "c9ed719b81be6cd3c78325086f6126c1ee10b86e"
5+
project_hash = "7757131567033d8a17d491e4b5b4803e1018bf86"
66

77
[[deps.ArgTools]]
88
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
@@ -153,6 +153,12 @@ version = "3.4.0"
153153
uuid = "f489334b-da3d-4c2e-b8f0-e476e12c162b"
154154
version = "1.11.0"
155155

156+
[[deps.Suppressor]]
157+
deps = ["Logging"]
158+
git-tree-sha1 = "6dbb5b635c5437c68c28c2ac9e39b87138f37c0a"
159+
uuid = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
160+
version = "0.2.8"
161+
156162
[[deps.TOML]]
157163
deps = ["Dates"]
158164
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

‎Project.toml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
99
MLJModelInterface = "e80e1ace-859a-464e-9ed9-23947d8ae3ea"
1010
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
1111
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
12+
Suppressor = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
1213

1314
[compat]
1415
Distributed = "1.11.0"
@@ -20,8 +21,7 @@ Suppressor = "0.2"
2021
julia = "1.10"
2122

2223
[extras]
23-
Suppressor = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
2424
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
2525

2626
[targets]
27-
test = ["Test", "Suppressor"]
27+
test = ["Test",]

‎docs/make.jl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ makedocs(
1313
),
1414
pages=[
1515
"Home" => "index.md",
16+
"Registry management tools" => "registry_management_tools.md",
1617
"Internals" => "internals.md",
1718
],
1819
sitename="MLJModelRegistry.jl",

‎docs/src/index.md‎

Lines changed: 2 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -27,17 +27,6 @@ accessed in one of two ways:
2727
- By developers wishing to update the registry, but using software tools provided by the
2828
MLJModelRegistry.jl package, as described in this documentation.
2929

30-
As it is for MLJ developers, none of the code provided in this repository is part of the
31-
standard MLJ distribution.
32-
33-
## Contents
34-
35-
- [Registering a new model](@ref)
36-
37-
- [Updating metadata for existing models](@ref)
38-
39-
- [Complete list of registry maintenance tools](@ref)
40-
41-
- [Internals](@ref)
42-
30+
As it is intended for MLJ developers only, none of the code provided in this repository is
31+
part of the standard MLJ distribution.
4332

‎docs/src/internals.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,15 @@
11
# Internals
22

3-
This page contains privates method documentation for maintainers of the
4-
MLJModelRegistry.jl software tools.
3+
This page contains documentation for non-public API, for maintainers of
4+
MLJModelRegistry.jl.
55

66
```@autodocs
7-
Modules = [MLJModelRegistry, GenericRegistry]
7+
Modules = [GenericRegistry]
8+
```
9+
```@docs
10+
MLJModelRegistry.metadata
11+
MLJModelRegistry.encode_dic
12+
MLJModelRegistry.modeltype_given_constructor(modeltypes)
13+
MLJModelRegistry.traits_given_constructor_name
814
```
915

‎docs/src/registry_management_tools.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Registry management tools
2+
3+
## Overview
4+
5+
```@docs
6+
MLJModelRegistry
7+
```
8+
9+
## Methods
10+
11+
- [`update`](@ref)
12+
- [`ModelRegistry.gc`](@ref)
13+
- [`ModelRegistry.get`](@ref)
14+
15+
```@docs
16+
update
17+
MLJModelRegistry.gc
18+
MLJModelRegistry.get
19+
```

‎registry/Metadata.toml‎

Lines changed: 4896 additions & 0 deletions
Large diffs are not rendered by default.

‎registry/Project.toml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,39 @@
11
[deps]
2+
BetaML = "024491cd-cc6b-443e-8034-08ea7eb7db2b"
3+
CatBoost = "e2e10f9a-a85d-4fa9-b6b2-639a32100a12"
4+
EvoLinear = "ab853011-1780-437f-b4b5-5de6f4777246"
5+
EvoTrees = "f6006082-12f8-11e9-0c9c-0d5d367ab1e5"
6+
FeatureSelection = "33837fe5-dbff-4c9e-8c2f-c5612fe2b8b6"
7+
Imbalance = "c709b415-507b-45b7-9a3d-1767c89fde68"
8+
InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
9+
LaplaceRedux = "c52c1a26-f7c5-402b-80be-ba1e638ad478"
10+
LightGBM = "7acf609c-83a4-11e9-1ffb-b912bcd3b04a"
11+
MLJBalancing = "45f359ea-796d-4f51-95a5-deb1a414c586"
12+
MLJBase = "a7f614a8-145f-11e9-1d2a-a57a1082229d"
13+
MLJClusteringInterface = "d354fa79-ed1c-40d4-88ef-b8c7bd1568af"
214
MLJDecisionTreeInterface = "c6f25543-311c-4c74-83dc-3ea6d1015661"
15+
MLJEnsembles = "50ed68f4-41fd-4504-931a-ed422449fee0"
16+
MLJFlux = "094fc8d1-fd35-5302-93ea-dabda2abf845"
17+
MLJGLMInterface = "caf8df21-4939-456d-ac9c-5fefbfb04c0c"
18+
MLJIteration = "614be32b-d00c-4edb-bd02-1eb411ab5e55"
19+
MLJLIBSVMInterface = "61c7150f-6c77-4bb1-949c-13197eac2a52"
20+
MLJLinearModels = "6ee0df7b-362f-4a72-a706-9e79364fb692"
21+
MLJModels = "d491faf4-2d78-11e9-2867-c94bc002c0b7"
22+
MLJMultivariateStatsInterface = "1b6a4a23-ba22-4f51-9698-8599985d3728"
23+
MLJNaiveBayesInterface = "33e4bacb-b9e2-458e-9a13-5d9a90b235fa"
24+
MLJScikitLearnInterface = "5ae90465-5518-4432-b9d2-8a1def2f0cab"
25+
MLJTSVDInterface = "7fa162e1-0e29-41ca-a6fa-c000ca4e7e7e"
26+
MLJTestInterface = "72560011-54dd-4dc2-94f3-c5de45b75ecd"
27+
MLJText = "5e27fcf9-6bac-46ba-8580-b5712f3d6387"
28+
MLJTuning = "03970b2e-30c4-11ea-3135-d1576263f10f"
29+
MLJXGBoostInterface = "54119dfa-1dab-4055-a167-80440f4f7a91"
30+
Maxnet = "81f79f80-22f2-4e41-ab86-00c11cf0f26f"
331
NearestNeighborModels = "636a865e-7cf4-491e-846c-de09b730eb36"
32+
OneRule = "90484964-6d6a-4979-af09-8657dbed84ff"
33+
OutlierDetectionNeighbors = "51249a0a-cb36-4849-8e04-30c7f8d311bb"
34+
OutlierDetectionPython = "2449c660-d36c-460e-a68b-92ab3c865b3e"
35+
ParallelKMeans = "42b8e9d4-006b-409a-8472-7f34b3fb58af"
36+
PartitionedLS = "19f41c5e-8610-11e9-2f2a-0d67e7c5027f"
37+
SIRUS = "cdeec39e-fb35-4959-aadb-a1dd5dede958"
38+
SelfOrganizingMaps = "ba4b7379-301a-4be0-bee6-171e4e152787"
39+
SymbolicRegression = "8254be44-1295-4e6a-a16d-46603ac705cb"

‎src/MLJModelRegistry.jl‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,77 @@
1+
"""
2+
MLJModelRegistry
3+
4+
Module providing methods for managing the MLJ Model Registry. To modify the registry:
5+
6+
- Create a local clone of the [MLJModelRegistry.jl
7+
repository](https://github.com/JuliaAI/MLJModelRegistry.jl)
8+
9+
- Use Julia's package manager to add or remove items from the list of registered packages
10+
in the environment "/registry/". If adding a new item, see the protocol below.
11+
12+
- In a fresh temporary environment, run `Pkg.develop("path_to_clone")` and `using
13+
MLJModelRegistry`.
14+
15+
- To add or update the metadata associated with a package (compulsory for new packages)
16+
run [`update(pkg)`](@ref)
17+
18+
- To update the metadata for *all* packages in the registry, run [`update()`](@ref).
19+
20+
- When satisfied, commit your changes to the clone and make a pull request to the master
21+
MLJModelRegistry.jl repository.
22+
23+
!!! important
24+
25+
Removing a package from the "/registry/" enviroment does not remove its metadata from
26+
the Model Registry (from "/registry/Metatdata.toml"). Unless you later call `update()`
27+
to update all package metadata (slow), you must call [`MLJModelRegistry.gc()`](@ref) to
28+
specifically remove metadata for all orphaned packages (fast).
29+
30+
# Protocol for adding new packages to the registry environment
31+
32+
!!! important
33+
34+
In any pull request to update the Model Registry you should note the final
35+
output of `Pkg.status(outdated=true)`.
36+
37+
1. In your local clone of MLJModelRegistry.jl, `activate` the environment at "/registry/".
38+
39+
2. `update` the environment
40+
41+
3. Note the output of `Pkg.status(outdated=true)`
42+
43+
3. `add` the new package
44+
45+
4. Repeat steps 2 and 3 above, and investigate any dependency downgrades for which your addition may be the cause.
46+
47+
If adding the new package results in downgrades to existing dependencies because your
48+
package is not up to date with it's compatibility bounds, then your pull request to
49+
register the new models may be rejected.
50+
51+
"""
152
module MLJModelRegistry
253

354
import MLJModelInterface
455
using OrderedCollections
556
using InteractiveUtils
57+
using Suppressor
58+
using Distributed
659

760
# Location of the MLJ model registry (a Julia pkg environment + metadata):
861
const ROOT = joinpath(@__DIR__, "..")
962
const REGISTRY = joinpath(ROOT, "registry")
1063

64+
# for controlling logging:
65+
struct Loud end
66+
struct Quiet end
67+
1168
# The MLJ Model Registry is a special case of a "generic model registry", as described in
1269
# this file, defining the `GenericRegistry` module (which has methods, no types):
1370
include("GenericRegistry.jl")
1471
include("check_traits.jl")
1572
include("remote_methods.jl")
1673
include("methods.jl")
1774

75+
export update
76+
1877
end # module

0 commit comments

Comments
 (0)