-With above, users' model code doesn't need to change w.r.t whether the variable is sharded or not. By conforming to the variable interface, we could also easily swap the implementation to other infrastructure that supports more general variable partitioning and model parallelism . Having that in mind, to avoid API churn, we'd like to refrain from exposing `ShardedVariable` as a public symbol. This, however, makes some advanced cases harder, e.g, creating a custom Keras layer that behaves differently for sharded and non-sharded variables. `ShardedVariable` symbol will be visible to Keras, since Keras libraries inevitably need to do instance checking of `ShardedVariable`.
0 commit comments