Skip to content

Commit b5fbfb5

Browse files
image center
1 parent 362d113 commit b5fbfb5

File tree

1 file changed

+3
-1
lines changed
  • cloud-infrastructure/ai-infra-gpu/ai-infrastructure/dstack

1 file changed

+3
-1
lines changed

cloud-infrastructure/ai-infra-gpu/ai-infrastructure/dstack/README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,6 @@ fsdp_config:
122122
* `fsdp_config`: These set up how FSDP operates, such as how the model is sharded (`fsdp_sharding_strategy`) and whether parameters are offloaded to CPU (`fsdp_offload_params`).
123123

124124
![Hybrid shards](assets/images/image2.png "Hybrid shards")
125-
<img src="https://github.com/oracle-devrel/technology-engineering/blob/dstack-tutorial/cloud-infrastructure/ai-infra-gpu/ai-infrastructure/dstack/assets/images/image1.png" width="400">
126125

127126
With the `FSDP` of `distributed_type` and `FULL_SHARD` of `fsdp_config`’s `fsdp_sharding_strategy`, a model will be sharded across multiple GPUs in a single machine. When dealing with multiple compute nodes, each node will host an identical copy of the model, which is itself split across multiple GPUs within that node. This means each partitioned model instance on each node processes different sections or batches of your dataset. To distribute a single model across multiple GPUs spanning across multiple nodes, configure the parameter `fsdp_sharding_strategy` as `HYBRID_SHARD`.
128127

@@ -210,6 +209,9 @@ curl -X POST https://black-octopus-1.mycustomdomain.com/generate \
210209
Additionally, for a deployed model, dstack automatically provides a user interface to directly interact with the model:
211210

212211
![User interface](assets/images/image1.png "User interface")
212+
<p align="center">
213+
<img src="https://github.com/oracle-devrel/technology-engineering/blob/dstack-tutorial/cloud-infrastructure/ai-infra-gpu/ai-infrastructure/dstack/assets/images/image1.png" width="600">
214+
</p>
213215

214216
## Conclusion
215217

0 commit comments

Comments
 (0)