Skip to content

Commit 6a7e1ac

Browse files
authored
update docs (#7)
1 parent 58bc198 commit 6a7e1ac

File tree

2 files changed

+151
-136
lines changed

2 files changed

+151
-136
lines changed

README.md

Lines changed: 73 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
**H**_yper_**P**_arameter_
1+
Hyperparameter
22
===========================
33

44
<h3 align="center">
@@ -7,88 +7,99 @@
77
</p>
88
</h3>
99

10-
HyperParameter is a pythonic configuration framework designed to simplify the massive configuration in complex applications. The key feature is a dynamic hierarchical parameter tree composited with scopes. HyperParameter is particularly well suited for machine learning experiments and related systems, which have many parameters and nested codes.
10+
<p align="center">
1111

12-
Key Conceptions
13-
---------------
14-
15-
1. `parameter tree`, a nested python dict with support for default values and object-style API;
16-
1. `param_scope`, a context manager for compositing the ` parameter tree` with nested `param_scope`;
17-
2. `auto_param`, a decorator allowing default parameters of a function or class to be supplied from `param_scope`;
12+
**Hyperparameter, Make configurable AI applications.Build for Python hackers.**
1813

14+
</p>
1915

2016
Quick Start
2117
-----------
2218

23-
A quick example for defining a model with HyperParameter:
19+
`Hyperparameter` uses `auto _ param` decorator to convert keywords arguments into configurable parameters:
2420

2521
```python
26-
@auto_param
27-
def dnn(input, layers=3, activation="relu"):
28-
"""
29-
build a DNN model with the following configurations:
30-
- dnn.layers(default: 3)
31-
- dnn.activation(default: "relu")
32-
"""
33-
for i in range(layers):
34-
input = Linear(input)
35-
input = activation_fn(
36-
activation,
37-
input
38-
)
39-
return input
40-
41-
# call dnn with default configuration
42-
# and create a 3 layer dnn with relu activation
43-
dnn(x)
44-
45-
# passing parameter using param_scope
46-
with param_scope(**{
47-
"dnn.layers": 4,
48-
"dnn.activation": "sigmoid"}):
49-
# create a 4 layer dnn with sigmoid activation
50-
dnn()
22+
from hyperparameter import auto_param
23+
24+
@auto_param("foo")
25+
def foo(x, y=1, z="a"):
26+
return f"x={x}, y={y}, z={z}"
5127
```
5228

53-
Another example for building ML system:
29+
The parameters can be controlled with `param_scope`
5430

5531
```python
56-
@auto_param
57-
def inference(x, backend="tvm"):
58-
...
32+
from hyperparameter import param_scope
5933

60-
with param_scope(backend="onnx"):
61-
inference(x)
34+
foo(1) # x=1, y=1, z='a'
35+
with param_scope(**{"foo.y":2}):
36+
foo(1) # x=1, y=2, z='a'
6237
```
6338

6439
Advanced Usage
6540
--------------
66-
### Nested Scope and Configuration Composition
6741

68-
HyperParameter uses nested `param_scope` for configuration composition :
42+
### Read/Write Parameters
6943

70-
``` python
44+
```python
7145
from hyperparameter import param_scope
72-
# on initialization, the parameter tree is empty: {}
73-
with param_scope(a=1) as ps:
74-
# in the with context, the composited parameter tree is {"a": 1}
75-
ps == {"a": 1}
76-
with param_scope(a=2, b=3) as ps2:
77-
# in the nested scope, the composited parameter tree is {"a": 2, "b": 3}
78-
# param `b` is a new, and param `a` is overwrite by new value
79-
ps2 == {"a": 2, "b": 3}
80-
# when exit the inner scope, the modification of inner scope is cleaned up
81-
# the composited parameter tree is {"a": 1}
82-
ps == {"a": 1}
46+
47+
# create param_scope
48+
with param_scope():
49+
pass
50+
51+
with param_scope("foo.y=1", "foo.z=b"):
52+
pass
53+
54+
with param_scope(**{"foo.y":1, "foo.z":2}):
55+
pass
56+
57+
# read param with default value
58+
with param_scope(**{"foo.y":2}) as ps:
59+
y = ps.foo.y(1)
60+
y = ps.foo.y | 1
61+
y = param_scope.foo.y(1)
62+
y = param_scope.foo.y | 1
63+
foo(1) # x=1, y=2, z='a'
64+
65+
# wite values to param_scope
66+
with param_scope(**{"foo.y":2}) as ps:
67+
ps.foo.y = 2
68+
param_scope.foo.y = 2
8369
```
8470

85-
### Manage Parameters from CMD Line
71+
### Nested Scope
8672

87-
It is recommended to use three-layer configuration for complex programmes:
73+
`Hyperparameter` support nested `param_scope`:
8874

89-
1. `inline default values`;
90-
2. `config file`, which will override `inline default values`;
91-
3. `cmdline arguments` that override both `config file` and `inline default values`;
75+
``` python
76+
from hyperparameter import param_scope
77+
78+
# no param_scope, use the default value defined in foo
79+
foo(1) # x=1, y=1, z='a'
80+
81+
# start a new param_scope
82+
# and set the default value of `foo.y` to `2`
83+
with param_scope(**{"foo.y":2}) as ps:
84+
# found one param_scope `ps`,
85+
# and receive default value of `foo.y` from `ps`
86+
foo(1) # x=1, y=2, z='a'
87+
88+
# start another param_scope
89+
# and set the default value of `foo.y` to `3`
90+
with param_scope(**{"foo.z": "b"}) as ps2:
91+
# found nested param_scope `ps2`,
92+
# and receive default values of `foo.z` from `ps2`
93+
foo(1) # x=1, y=2, z='b'
94+
# `ps2` ends here, and `foo.y` is restored to `2`
95+
foo(1) # x=1, y=2, z='a'
96+
# `ps` ends here, and `foo.y` is restored to `1`
97+
foo(1) # x=1, y=1, z='a'
98+
```
99+
100+
### CMD Line Arguments
101+
102+
An example CLI app:
92103

93104
```python
94105
from hyperparameter import param_scope, auto_param
@@ -99,17 +110,13 @@ def main(a=0, b=1): # `inline default values`
99110

100111
if __name__ == "__main__":
101112
import argparse
102-
import json
103113
parser = argparse.ArgumentParser()
104-
parser.add_argument("--config", type=str, default=None)
114+
105115
parser.add_argument("-D", "--define", nargs="*", default=[], action="extend")
106116
args = parser.parse_args()
107117

108-
with open(args.config) as f:
109-
cfg = json.load(f) # read config file
110-
with param_scope(**cfg): # scope for `config file`
111-
with param_scope(*args.define): # scope for `cmdline args`
112-
main()
118+
with param_scope(*args.define):
119+
main()
113120
```
114121

115122
Examples

README.zh.md

Lines changed: 78 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -1,122 +1,130 @@
1-
**H**_yper_**P**_arameter_
2-
===========================
1+
Hyperparameter
2+
===============
33

44
<h3 align="center">
55
<p style="text-align: center;">
66
<a href="README.md" target="_blank">ENGLISH</a> | <a href="README.zh.md">中文文档</a>
77
</p>
88
</h3>
99

10-
HyperParameter 是轻量级python代码配置框架,用户仅需对代码添加标记即可实现配置化。特别适用于机器学习模型的参数管理,帮助开发者对接MLOps工具;也适用于大型Python应用的配置管理。=====
10+
<p align="center">
1111

12-
### 一个示例
12+
**Hyperparameter, Make configurable AI applications.Build for Python hackers.**
1313

14-
假设我们开发了一个MLP结构,并使用在某个模型中:
14+
</p>
1515

16-
```python
17-
class MLP(nn.Module):
18-
def __init__(self, units=[64, 32, 16], activation="ReLU", use_bn=False):
19-
...
20-
21-
class MyAwesomeModel(nn.Module):
22-
def __init__(self):
23-
self.mlp = MLP()
24-
...
25-
26-
model = MyAwesomeModel()
27-
```
28-
29-
如果我们想尝试修改MLP结构的参数来提升模型效果,通常要修改其代码,或者在MyAwesomeModel中添加额外参数。前者代码难以维护,而后者需要大量冗余代码。
16+
快速开始
17+
-------
3018

31-
HyperParameter 通过 `auto_param`装饰器自动抽取配置参数
19+
`Hyperparameter` 使用装饰器 `auto _ param` 自动将keyword参数转化为超参
3220

3321
```python
3422
from hyperparameter import auto_param
3523

36-
@auto_param
37-
class MLP(nn.Module):
38-
def __init__(self, units=[64, 32, 16], activation="ReLU", use_bn=False):
39-
...
24+
@auto_param("foo")
25+
def foo(x, y=1, z="a"):
26+
return f"x={x}, y={y}, z={z}"
4027
```
4128

42-
上述代码将自动生成三个参数,分别对应`MLP.__init__`方法的keyword参数:
43-
44-
1. `MLP.units`,默认值`[64, 32, 16]`;
45-
2. `MLP.activation`,默认值`ReLU`;
46-
3. `MLP.use_bn`,默认值`False`;
47-
48-
三个参数与keyword参数同名,并被放在`MLP`这个namespace下。参数值通过`param_scope`来控制:
29+
超参数可以通过 `param_scope` 控制:
4930

5031
```python
5132
from hyperparameter import param_scope
5233

53-
with param_scope(**{"MLP.activation": "sigmoid"}):
54-
model = MyAwesomeModel()
34+
foo(1) # x=1, y=1, z='a'
35+
with param_scope(**{"foo.y":2}):
36+
foo(1) # x=1, y=2, z='a'
5537
```
5638

5739
高级用法
5840
-------
59-
### 嵌套scope
6041

61-
我们可以通过嵌套`param_scope`来精细控制参数,当退出一个`param_scope`,该scope对参数的修改自动失效:
42+
### 读写超参
6243

63-
``` python
64-
from hyperparameter import auto_param, param_scope
44+
```python
45+
from hyperparameter import param_scope
6546

66-
@auto_param
67-
def foo(a=1, b=2):
68-
print(f"a={a}, b={b}")
69-
70-
with param_scope(a=3):
71-
foo() # a=3, b=2!
72-
with param_scope(a=2, b=3):
73-
foo() # a=2, b=3!
74-
foo() # a=3, b=2!
47+
# 创建param_scope
48+
with param_scope():
49+
pass
50+
51+
with param_scope("foo.y=1", "foo.z=b"):
52+
pass
53+
54+
with param_scope(**{"foo.y":1, "foo.z":2}):
55+
pass
56+
57+
# 读取超参(待默认值)
58+
with param_scope(**{"foo.y":2}) as ps:
59+
y = ps.foo.y(1)
60+
y = ps.foo.y | 1
61+
y = param_scope.foo.y(1)
62+
y = param_scope.foo.y | 1
63+
foo(1) # x=1, y=2, z='a'
64+
65+
# 写入超参
66+
with param_scope(**{"foo.y":2}) as ps:
67+
ps.foo.y = 2
68+
param_scope.foo.y = 2
7569
```
7670

77-
### 从命令行管理参数
71+
### 嵌套Scope
7872

79-
我们通常推荐使用如下三层结构管理参数:
73+
`Hyperparameter` support nested `param_scope`:
8074

81-
1. 代码的inline默认值,即写在函数或者类定义中的默认值;
82-
2. 配置文件,会覆盖inline默认值;
83-
3. 命令行参数,会覆盖配置文件和inline默认值;
75+
``` python
76+
from hyperparameter import param_scope
8477

85-
我们推荐的命令脚本示例如下:
78+
# 当前没有param_scope,使用函数定义的默认值
79+
foo(1) # x=1, y=1, z='a'
80+
81+
# 开启新的param_scope
82+
# 并将`foo.y`默认值设为`2`
83+
with param_scope(**{"foo.y":2}) as ps:
84+
# 发现一个param_scope `ps`,
85+
# 并从`ps`中获取`foo.y`的默认值
86+
foo(1) # x=1, y=2, z='a'
87+
88+
# 开启另一个新的param_scope
89+
# 并将`foo.y`默认值设为`3`
90+
with param_scope(**{"foo.z": "b"}) as ps2:
91+
# 发现一个嵌套param_scope `ps2`,
92+
# 并从`ps2`中获取`foo.y`的默认值
93+
foo(1) # x=1, y=2, z='b'
94+
# `ps2` 结束,参数清理,`foo.y` 恢复为`2`
95+
foo(1) # x=1, y=2, z='a'
96+
# `ps` 结束,参数清理,`foo.y` 恢复为`1`
97+
foo(1) # x=1, y=1, z='a'
98+
```
99+
100+
### CMD Line Arguments
101+
102+
CLI应用示例代码:
86103

87104
```python
88105
from hyperparameter import param_scope, auto_param
89106

90107
@auto_param
91-
def main(a=0, b=1): # inline默认值
108+
def main(a=0, b=1): # `inline default values`
92109
print(a, b)
93110

94111
if __name__ == "__main__":
95112
import argparse
96-
import json
97113
parser = argparse.ArgumentParser()
98-
parser.add_argument("--config", type=str, default=None)
114+
99115
parser.add_argument("-D", "--define", nargs="*", default=[], action="extend")
100116
args = parser.parse_args()
101117

102-
if args.config is not None: # 加载配置文件
103-
with open(args.config) as f:
104-
cfg = json.load(f)
105-
else:
106-
cfg = {}
107-
108-
with param_scope(**cfg): # 配置文件的scope
109-
with param_scope(*args.define): # 命令行参数的scope
110-
main()
118+
with param_scope(*args.define):
119+
main()
111120
```
112121

113122
更多示例
114-
-------
115-
116-
### [超参调节](examples/sparse_lr/README.md)
123+
--------
117124

118-
该示例展示了如何使用`hyperparameter`来搭建机器学习项目,并保证复现性。
125+
### [超参搜索](examples/sparse_lr/README.md)
119126

120-
### [试验跟踪](examples/mnist/README.md)
127+
该示例展示如何在研究项目中使用hyperparameter,并让模型实验可以复现。
128+
### [实验管理](examples/mnist/README.md)
121129

122-
该例子说明了如何通过`hyperparameter`进行试验管理,并跟踪试验结果
130+
该示例演示如何使用hyperparameter进行实验管理,并对接mlflow的tracing系统

0 commit comments

Comments
 (0)