Skip to content

Commit 61f69c1

Browse files
authored
Minor tweaks
1 parent e23256b commit 61f69c1

File tree

2 files changed

+17
-6
lines changed

2 files changed

+17
-6
lines changed

docs/getting-started/advanced-usage.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ parallel_backend = init_parallel_backend(backend_name="joblib")
5757
of pyDVL. They are used internally as part of the computations of the
5858
different methods.
5959

60-
!!! info
60+
!!! danger "Deprecation notice"
6161

6262
We are currently planning to deprecate
6363
[MapReduceJob][pydvl.parallel.map_reduce.MapReduceJob] in favour of the
@@ -88,7 +88,7 @@ with joblib.parallel_config(backend="loky", verbose=100):
8888

8989
#### Ray
9090

91-
!!! info
91+
!!! warning "Additional dependencies"
9292

9393
The Ray parallel backend requires optional dependencies.
9494
See [Extras][installation-extras] for more information.
@@ -155,13 +155,13 @@ classes.
155155
[1, 2, 3]
156156
```
157157

158-
#### Map reduce
158+
#### Map-reduce
159159

160-
The map reduce interface is older and more limited in the patterns
160+
The map-reduce interface is older and more limited in the patterns
161161
it allows us to use.
162162

163163
To reproduce the previous example using
164-
[MapReduceJob][pydvl.parallel.map_reduce.MapReduceJob]we would use:
164+
[MapReduceJob][pydvl.parallel.map_reduce.MapReduceJob], we would use:
165165

166166
```pycon
167167
>>> from pydvl.parallel import JoblibParallelBackend, MapReduceJob

src/pydvl/parallel/__init__.py

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,18 @@
2727
results = list(executor.map(lambda x: x + 1, range(5)))
2828
assert results == [1, 2, 3, 4, 5]
2929
```
30-
30+
!!! tip "Passsing large objects"
31+
When running tasks which accept heavy inputs, it is important
32+
to first use `put()` on the object and use the returned reference
33+
as argument to the callable within `submit()`. For example:
34+
```python
35+
u_ref = parallel_backend.put(u)
36+
...
37+
executor.submit(task, utility=u)
38+
```
39+
Note that `task()` does not need to be changed in any way:
40+
the backend will `get()` the object and pass it to the function
41+
upon invocation.
3142
There is an alternative map-reduce implementation
3243
[MapReduceJob][pydvl.parallel.map_reduce.MapReduceJob] which internally
3344
uses joblib's higher level API with `Parallel()` which then indirectly also

0 commit comments

Comments
 (0)