Skip to content
This repository was archived by the owner on Jul 10, 2025. It is now read-only.

Commit 4569980

Browse files
committed
Various updates
1 parent c22e495 commit 4569980

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

rfcs/20201121-keras-model-fit-ps.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ To take advantage of TF2 support of parameter server training, a `ClusterCoordin
206206

207207
##### Advanced use case: `ClusterCoordinator` as a singleton
208208

209-
Since `ClusterCoordinator` instance spins off worker and failure handling threads, there should only be one `ClusterCoordinator` at any given time, and making it a singleton ensures that those threads are only created once:
209+
Now, let's consider a more advanced use case where the `ClusterCoordinator` instance is needed by users. Since `ClusterCoordinator` instance spins off worker and failure handling threads, there should only be one `ClusterCoordinator` at any given time, and making it a singleton ensures that those threads are only created once:
210210

211211
```
212212
class ClusterCoordinator(object):
@@ -216,11 +216,11 @@ class ClusterCoordinator(object):
216216
return strategy.cluster_coordinator
217217
```
218218

219-
Being a singleton is important considering there are power users who would like to `schedule` functions themselves in addition to `model.fit` usage. That is, they can instantiate one before `model.fit` does, or use one after `model.fit` has instantiated one. In either case, they should access the same `ClusterCoordinator` instance.
219+
Being a singleton is important considering there are power users who would like to `schedule` functions themselves in addition to `model.fit` usage. That is, they can instantiate one before `model.fit` does, or use one after `model.fit` has instantiated one. In either case, they should access the same `ClusterCoordinator` instance, as the one `model.fit` uses.
220220

221221
##### Have an attribute in `ParameterServerStrategy` that holds the `ClusterCoordinator`
222222

223-
We propose that an attribute is added to `ParameterServerStrategy` to keep track of the `ClusterCoordinator`. When a `ClusterCooridinator` is instantiated, such attribute will be set. Here, we assume that the distribution `Strategy` object can determine whether or not it is supposed to be used with a `ClusterCoordinator`. See below “Changes in tf.distribute” section for more information.
223+
We propose that an attribute is added to `ParameterServerStrategy` to keep track of the `ClusterCoordinator`. When a `ClusterCoordinator` is instantiated, such attribute will be set. Here, we assume that the distribution `Strategy` object can determine whether or not it is supposed to be used with a `ClusterCoordinator`. See below “Changes in tf.distribute” section for more information.
224224

225225
```
226226
class ClusterCoordinator(...):
@@ -229,7 +229,7 @@ class ClusterCoordinator(...):
229229
strategy.cluster_coordinator = self
230230
```
231231

232-
And, we instantiate the `ClusterCoordinator` as soon as `model.fit` is called for the first time. It will then be reused for the next `fit`, or on a different model.
232+
And, we instantiate the `ClusterCoordinator` as soon as `model.fit` is called for the first time. Note that if users have instantiated it prior to `model.fit` calls, the same instance is returned from the `ClusterCoordinator` constructor. It will then be reused for the next `fit`, or on a different model.
233233

234234
```
235235
class Model(...):

0 commit comments

Comments
 (0)