Deploying to gh-pages from @ dstackai/dstack@f148a3e 🚀

Author: jvstme

assets/images/social/blog/category/community.png

@@ -926,7 +926,7 @@
       <ul class="md-nav__list">
 
           <li class="md-nav__item">
-  <a href="#placement" class="md-nav__link">
+  <a href="#cloud-placement" class="md-nav__link">
     <span class="md-ellipsis">
 
         <span class="md-typeset">
@@ -1042,7 +1042,7 @@
       <ul class="md-nav__list">
 
           <li class="md-nav__item">
-  <a href="#placement_1" class="md-nav__link">
+  <a href="#ssh-placement" class="md-nav__link">
     <span class="md-ellipsis">
 
         <span class="md-typeset">
@@ -3563,7 +3563,7 @@
       <ul class="md-nav__list">
 
           <li class="md-nav__item">
-  <a href="#placement" class="md-nav__link">
+  <a href="#cloud-placement" class="md-nav__link">
     <span class="md-ellipsis">
 
         <span class="md-typeset">
@@ -3679,7 +3679,7 @@
       <ul class="md-nav__list">
 
           <li class="md-nav__item">
-  <a href="#placement_1" class="md-nav__link">
+  <a href="#ssh-placement" class="md-nav__link">
     <span class="md-ellipsis">
 
         <span class="md-typeset">
@@ -3867,22 +3867,19 @@ <h3 id="define-a-configuration">Define a configuration<a class="headerlink" href
 
 </div>
 
-<h4 id="placement">Placement<a class="headerlink" href="#placement" title="Permanent link">&para;</a></h4>
+<h4 id="cloud-placement">Placement<a class="headerlink" href="#cloud-placement" title="Permanent link">&para;</a></h4>
 <p>To ensure instances are interconnected (e.g., for
 <a href="../tasks/#distributed-tasks">distributed tasks</a>), set <code>placement</code> to <code>cluster</code>. 
 This ensures all instances are provisioned in the same backend and region with optimal inter-node connectivity</p>
 <details class="info">
 <summary>AWS</summary>
-<p><code>dstack</code> automatically enables <a href="https://aws.amazon.com/hpc/efa/" target="_blank">Elastic Fabric Adapter <span class="twemoji external"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z"/></svg></span></a>
-for the instance types that support it:
-<code>p5.48xlarge</code>, <code>p4d.24xlarge</code>, <code>g4dn.12xlarge</code>, <code>g4dn.16xlarge</code>, <code>g4dn.8xlarge</code>, <code>g4dn.metal</code>,
-<code>g5.12xlarge</code>, <code>g5.16xlarge</code>, <code>g5.24xlarge</code>, <code>g5.48xlarge</code>, <code>g5.8xlarge</code>, <code>g6.12xlarge</code>,
-<code>g6.16xlarge</code>, <code>g6.24xlarge</code>, <code>g6.48xlarge</code>, <code>g6.8xlarge</code>, and <code>gr6.8xlarge</code>.</p>
-<p>Currently, only one EFA interface is enabled per instance, regardless of its maximum capacity.
+<p><code>dstack</code> automatically enables the Elastic Fabric Adapter for all
+<a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html#efa-instance-types" target="_blank">EFA-capable instance types <span class="twemoji external"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z"/></svg></span></a>.
+Currently, only one EFA interface is enabled per instance, regardless of its maximum capacity.
 This will change once <a href="https://github.com/dstackai/dstack/issues/1804" target="_blank">this issue <span class="twemoji external"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z"/></svg></span></a> is resolved.</p>
 </details>
 <blockquote>
-<p>The <code>cluster</code> placement is supported only for <code>aws</code>, <code>azure</code>, <code>gcp</code>, and <code>oci</code>
+<p>The <code>cluster</code> placement is supported only for <code>aws</code>, <code>azure</code>, <code>gcp</code>, <code>oci</code>, and <code>vultr</code>
 backends.</p>
 </blockquote>
 <h4 id="resources">Resources<a class="headerlink" href="#resources" title="Permanent link">&para;</a></h4>
@@ -4052,7 +4049,7 @@ <h3 id="define-a-configuration_1">Define a configuration<a class="headerlink" hr
 </div>
 <p>3.&nbsp;The user specified should have passwordless <code>sudo</code> access.</p>
 </details>
-<h4 id="placement_1">Placement<a class="headerlink" href="#placement_1" title="Permanent link">&para;</a></h4>
+<h4 id="ssh-placement">Placement<a class="headerlink" href="#ssh-placement" title="Permanent link">&para;</a></h4>
 <p>If the hosts are interconnected (i.e. share the same network), set <code>placement</code> to <code>cluster</code>. 
 This is required if you'd like to use the fleet for <a href="../tasks/#distributed-tasks">distributed tasks</a>.</p>
 <h5 id="network">Network<a class="headerlink" href="#network" title="Permanent link">&para;</a></h5>

assets/images/social/community.png

@@ -3821,7 +3821,7 @@ <h3 id="ports">Ports<a class="headerlink" href="#ports" title="Permanent link">&
 <h3 id="distributed-tasks">Distributed tasks<a class="headerlink" href="#distributed-tasks" title="Permanent link">&para;</a></h3>
 <p>By default, a task runs on a single node.
 However, you can run it on a cluster of nodes by specifying <code>nodes</code>.</p>
-<div editor-title="examples/fine-tuning/train.dstack.yml">
+<div editor-title="train.dstack.yml">
 
 <div class="highlight"><pre><span></span><code><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">task</span>
 <span class="c1"># The name is optional, if not specified, generated randomly</span>
@@ -3830,33 +3830,57 @@ <h3 id="distributed-tasks">Distributed tasks<a class="headerlink" href="#distrib
 <span class="c1"># The size of the cluster</span>
 <span class="nt">nodes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
 
-<span class="nt">python</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;3.10&quot;</span>
+<span class="nt">python</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;3.12&quot;</span>
 
-<span class="c1"># Commands of the task</span>
+<span class="c1"># Commands to run on each node</span>
 <span class="nt">commands</span><span class="p">:</span>
+<span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git clone https://github.com/pytorch/examples.git</span>
+<span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">cd examples/distributed/ddp-tutorial-series</span>
 <span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pip install -r requirements.txt</span>
 <span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">torchrun</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--nproc_per_node=$DSTACK_GPUS_PER_NODE</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--node_rank=$DSTACK_NODE_RANK</span>
+<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--nproc-per-node=$DSTACK_GPUS_PER_NODE</span>
+<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--node-rank=$DSTACK_NODE_RANK</span>
 <span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--nnodes=$DSTACK_NODES_NUM</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--master_addr=$DSTACK_MASTER_NODE_IP</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--master_port=8008 resnet_ddp.py</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--num_epochs 20</span>
+<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--master-addr=$DSTACK_MASTER_NODE_IP</span>
+<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--master-port=12345</span>
+<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">multinode.py 50 10</span>
 
 <span class="nt">resources</span><span class="p">:</span>
 <span class="w">  </span><span class="nt">gpu</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">24GB</span>
+<span class="w">  </span><span class="c1"># Uncomment if using multiple GPUs</span>
+<span class="w">  </span><span class="c1">#shm_size: 24GB</span>
 </code></pre></div>
 
 </div>
 
-<p>All you need to do is pass the corresponding environment variables such as 
-<code>DSTACK_GPUS_PER_NODE</code>, <code>DSTACK_NODE_RANK</code>, <code>DSTACK_NODES_NUM</code>,
-<code>DSTACK_MASTER_NODE_IP</code>, and <code>DSTACK_GPUS_NUM</code> (see <a href="#system-environment-variables">System environment variables</a>).</p>
+<p>Nodes can communicate using their private IP addresses.
+Use <code>DSTACK_MASTER_NODE_IP</code>, <code>$DSTACK_NODE_RANK</code>, and other
+<a href="#system-environment-variables">System environment variables</a>
+to discover IP addresses and other details.</p>
+<details class="info">
+<summary>Network interface</summary>
+<p>Distributed frameworks usually detect the correct network interface automatically,
+but sometimes you need to specify it explicitly.</p>
+<p>For example, with PyTorch and the NCCL backend, you may need
+to add these commands to tell NCCL to use the private interface:</p>
+<div class="highlight"><pre><span></span><code><span class="nt">commands</span><span class="p">:</span>
+<span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">apt-get install -y iproute2</span>
+<span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="p p-Indicator">&gt;</span>
+<span class="w">    </span><span class="no">if [[ $DSTACK_NODE_RANK == 0 ]]; then</span>
+<span class="w">      </span><span class="no">export NCCL_SOCKET_IFNAME=$(ip -4 -o addr show | fgrep $DSTACK_MASTER_NODE_IP | awk &#39;{print $2}&#39;)</span>
+<span class="w">    </span><span class="no">else</span>
+<span class="w">      </span><span class="no">export NCCL_SOCKET_IFNAME=$(ip route get $DSTACK_MASTER_NODE_IP | sed -E &#39;s/.*?dev (\S+) .*/\1/;t;d&#39;)</span>
+<span class="w">    </span><span class="no">fi</span>
+<span class="w">  </span><span class="c1"># ... The rest of the commands</span>
+</code></pre></div>
+</details>
 <div class="admonition info">
 <p class="admonition-title">Fleets</p>
-<p>To ensure all nodes are provisioned into a cluster placement group and to enable the highest level of inter-node 
-connectivity (incl. support for <a href="https://aws.amazon.com/hpc/efa/" target="_blank">EFA <span class="twemoji external"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z"/></svg></span></a>),
-create a <a href="../fleets/">fleet</a> via a configuration before running a disstributed task.</p>
+<p>Distributed tasks can only run on fleets with
+<a href="../fleets/#cloud-placement">cluster placement</a>.
+While <code>dstack</code> can provision such fleets automatically, it is
+recommended to create them via a fleet configuration
+to ensure the highest level of inter-node connectivity.</p>
 </div>
 <p><code>dstack</code> is easy to use with <code>accelerate</code>, <code>torchrun</code>, Ray, Spark, and any other distributed frameworks.</p>
 <h3 id="resources">Resources<a class="headerlink" href="#resources" title="Permanent link">&para;</a></h3>
@@ -4061,7 +4085,7 @@ <h3 id="environment-variables">Environment variables<a class="headerlink" href="
 </tr>
 <tr>
 <td><code>DSTACK_MASTER_NODE_IP</code></td>
-<td>The internal IP address the master node</td>
+<td>The internal IP address of the master node</td>
 </tr>
 <tr>
 <td><code>DSTACK_NODES_IPS</code></td>

assets/images/social/examples.png

@@ -3398,28 +3398,30 @@ <h2 id="dstackyml">.dstack.yml<a class="headerlink" href="#dstackyml" title="Per
 <li><code id="DSTACK_GPUS_PER_NODE">DSTACK_GPUS_PER_NODE</code> – The number of GPUs per node</li>
 <li><code id="DSTACK_NODE_RANK">DSTACK_NODE_RANK</code> – The rank of the node</li>
 <li>
-<p><code id="DSTACK_NODE_RANK">DSTACK_NODE_RANK</code> – The internal IP address the master node.</p>
-<p>Below is an example of using <code>DSTACK_NODES_NUM</code>, <code>DSTACK_GPUS_PER_NODE</code>, <code>DSTACK_NODE_RANK</code>, and <code>DSTACK_NODE_RANK</code>
+<p><code id="DSTACK_NODE_RANK">DSTACK_MASTER_NODE_IP</code> – The internal IP address of the master node.</p>
+<p>Below is an example of using <code>DSTACK_NODES_NUM</code>, <code>DSTACK_GPUS_PER_NODE</code>, <code>DSTACK_NODE_RANK</code>, and <code>DSTACK_MASTER_NODE_IP</code>
  for distributed training:</p>
-<div class="highlight"><pre><span></span><code><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">task</span>
-<span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">train-distrib</span>
-
-<span class="c1"># The number of instances in the cluster</span>
-<span class="nt">nodes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
-
-<span class="nt">python</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;3.10&quot;</span>
-<span class="nt">commands</span><span class="p">:</span>
-<span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pip install -r requirements.txt</span>
-<span class="w">  </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">torchrun</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--nproc_per_node=$DSTACK_GPUS_PER_NODE</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--node_rank=$DSTACK_NODE_RANK</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--nnodes=$DSTACK_NODES_NUM</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--master_addr=$DSTACK_MASTER_NODE_IP</span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">--master_port=8008</span><span class="w"> </span>
-<span class="w">    </span><span class="l l-Scalar l-Scalar-Plain">resnet_ddp.py --num_epochs 20</span>
-
-<span class="nt">resources</span><span class="p">:</span>
-<span class="w">  </span><span class="nt">gpu</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">24GB</span>
+<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">task</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">train-distrib</span>
+
+<span class="w"> </span><span class="nt">nodes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
+<span class="w"> </span><span class="nt">python</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;3.12&quot;</span>
+
+<span class="w"> </span><span class="nt">commands</span><span class="p">:</span>
+<span class="w">   </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git clone https://github.com/pytorch/examples.git</span>
+<span class="w">   </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">cd examples/distributed/ddp-tutorial-series</span>
+<span class="w">   </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pip install -r requirements.txt</span>
+<span class="w">   </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">torchrun</span>
+<span class="w">     </span><span class="l l-Scalar l-Scalar-Plain">--nproc-per-node=$DSTACK_GPUS_PER_NODE</span>
+<span class="w">     </span><span class="l l-Scalar l-Scalar-Plain">--node-rank=$DSTACK_NODE_RANK</span>
+<span class="w">     </span><span class="l l-Scalar l-Scalar-Plain">--nnodes=$DSTACK_NODES_NUM</span>
+<span class="w">     </span><span class="l l-Scalar l-Scalar-Plain">--master-addr=$DSTACK_MASTER_NODE_IP</span>
+<span class="w">     </span><span class="l l-Scalar l-Scalar-Plain">--master-port=12345</span>
+<span class="w">     </span><span class="l l-Scalar l-Scalar-Plain">multinode.py 50 10</span>
+
+<span class="w"> </span><span class="nt">resources</span><span class="p">:</span>
+<span class="w">   </span><span class="nt">gpu</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">24GB</span>
+<span class="w">   </span><span class="nt">shm_size</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">24GB</span>
 </code></pre></div>
 </li>
 <li>