doc

Author: yueyinqiu

RELEASENOTES.md

@@ -16,6 +16,8 @@ __API Changes__:
 - #1291 `Tensor.grad()` and `Tensor.set_grad()` have been replaced by a new property `Tensor.grad`.
     - A potential memory leak caused by `set_grad` has been resolved.
 - `Include` method of dispose scopes has been removed. Use `Attach` instead.
+- Two more `Attach` methods that accepts `IEnumerable<IDisposable>`s and arrays as the parameter have been added into dispose scopes.
+- A new property `torch.CurrentDisposeScope` has been added to provide the ability to get the current dispose scope.
 
 __Bug Fixes__:
 
@@ -27,7 +29,7 @@ __Bug Fixes__:
 __Bug Fixes__:
 
 - `TensorDataset` will now keep the aliases detached from dispose scopes, to avoid the unexpected disposal.
-- `DataLoaderEnumerator` has been completely rewritten to resolve the unexpected shuffler disposal, the ignorance of drop last and the incorrect count of worker.
+- `DataLoaderEnumerator` has been completely rewritten to resolve the unexpected shuffler disposal, the ignorance of `drop_last`, the incorrect count of worker, and the potential leak cause by multithreading.
 - #1303 Allow dispose scopes to be disposed out of LIFO order.
 
 # NuGet Version 0.102.4

docfx/articles/memory.md

@@ -295,6 +295,107 @@ public override Tensor entropy() => torch.WrappedTensorDisposeScope(() => ((scal
 
 ```
 
+## Data loaders and datasets
+
+Sometimes we'd like to train our model in the following pattern:
+
+```csharp
+using var dataLoader = torch.utils.data.DataLoader(...);
+for (int epoch = 0; epoch < 100; epoch++)
+{
+    foreach (var batch in dataLoader)
+    {
+        using (torch.NewDisposeScope())
+        {
+            ...
+        }
+    }
+}
+```
+
+In this case, you may notice that `batch` (at least the first batch) is created outside the dispose scope, which would cause a potential memory leak.
+
+Of course we could manually dispose them. But actually we don't have to care about that, because the data loader will automatically dispose it before the next iteration.
+
+However this might cause another problem. For example, we will get disposed tensors when using Linq. The behavior could be modified by setting `disposeBatch` to `false`:
+
+```csharp
+using TorchSharp;
+
+using var dataset = torch.utils.data.TensorDataset(torch.zeros([3]));
+
+using var dataLoader1 = torch.utils.data.DataLoader(dataset, batchSize: 1);
+using var dataLoader2 = torch.utils.data.DataLoader(dataset, batchSize: 1, disposeBatch: false);
+
+Console.WriteLine(dataLoader1.First()[0].IsInvalid); // True
+Console.WriteLine(dataLoader2.First()[0].IsInvalid); // False
+```
+
+But those tensors would be detached from all the dispose scopes, even if the whole process is wrapped by a scope. (Otherwise it may lead to confusion since the iterations may not happen in the same dispose scope.) So don't forget to dispose them later or manually attach them to a scope. Also, be aware that enumerating the same `IEnumerable` twice could produce different instances:
+
+```csharp
+// DON'T DO THIS:
+using TorchSharp;
+
+using var dataset = torch.utils.data.TensorDataset(torch.zeros([3]));
+using var dataLoader = torch.utils.data.DataLoader(dataset, batchSize: 1, disposeBatch: false);
+
+var tensors = dataLoader.Select(x => x[0]);
+DoSomeThing(tensors.ToArray());
+
+foreach (var tensor in tensors)
+{
+    tensor.Dispose();
+    // DON'T DO THIS.
+    // The tensor is not the one you have passed into `DoSomeThing`.
+}
+```
+
+Meanwhile, if you want do write a dataset on your own, you shall notice that data loaders will dispose the tensors got from `GetTensor` after collation. So a dataset like this will not work because the saved tensor is disposed:
+
+```csharp
+using TorchSharp;
+
+using var dataLoader = torch.utils.data.DataLoader(new MyDataset(), batchSize: 1);
+foreach (var _ in dataLoader) ;
+// System.InvalidOperationException:
+// Tensor invalid -- empty handle.
+
+class MyDataset : torch.utils.data.Dataset
+{
+    private torch.Tensor tensor = torch.zeros([]);
+    public override Dictionary<string, torch.Tensor> GetTensor(long index)
+    {
+        tensor = tensor + 1;
+        // The new tensor is attached to the dispose scope in the data loader,
+        // and it will be disposed after collation,
+        // so in the next iteration it becomes invalid.
+        return new() { ["tensor"] = tensor };
+    }
+
+    public override long Count => 3;
+}
+```
+
+Since the actual technique to "catch" the tensors is just a simple dispose scope. So we can write the class like this to avoid the disposal:
+
+```csharp
+class MyDataset : torch.utils.data.Dataset
+{
+    private torch.Tensor tensor = torch.zeros([]);
+    public override Dictionary<string, torch.Tensor> GetTensor(long index)
+    {
+        var previous = tensor;
+        tensor = (previous + 1).DetachFromDisposeScope();
+        previous.Dispose(); // Don't forget to dispose the previous one.
+        return new() { ["tensor"] = tensor };
+    }
+
+    public override long Count => 3;
+}
+```
+
+```
 
 ## Links and resources
 

src/TorchSharp/DisposeScope.cs

@@ -218,15 +218,31 @@ public void Detach(IEnumerable<IDisposable> disposables)
         }
 
         public void Attach(IDisposable disposable)
+        {
+            _ = Attach((IEnumerable<IDisposable>)new[] { disposable });
+        }
+
+        public void Attach(params IDisposable[] disposables)
+        {
+            _ = Attach((IEnumerable<IDisposable>)disposables);
+        }
+
+        public IReadOnlyList<IDisposable> Attach(IEnumerable<IDisposable> disposables)
         {
             if (this._disposeScopeManager is null)
                 throw new ObjectDisposedException(this.GetType().FullName);
-            if (disposable is torch.Tensor tensor) {
-                if (tensor.OwningDisposeScope == null && !tensor.IsInvalid) {
-                    _disposeScopeManager.StatisticsInstance.DetachedFromScopeCount--;
+
+            var result = new List<IDisposable>();
+            foreach (var disposable in disposables) {
+                if (disposable is torch.Tensor tensor) {
+                    if (tensor.OwningDisposeScope == null && !tensor.IsInvalid) {
+                        _disposeScopeManager.StatisticsInstance.DetachedFromScopeCount--;
+                    }
                 }
+                AddToOther(this, disposable);
+                result.Add(disposable);
             }
-            AddToOther(this, disposable);
+            return result;
         }
 
         /// <summary>

src/TorchSharp/Tensor/Tensor.cs

@@ -7377,6 +7377,12 @@ public static long max_int_value(ScalarType type)
         public static DisposeScope NewDisposeScope() =>
             DisposeScopeManager.ThreadSingleton.NewDisposeScope();
 
+        /// <summary>
+        /// Get the current dispose scope for the current thread.
+        /// </summary>
+        public static DisposeScope? CurrentDisposeScope =>
+            DisposeScopeManager.ThreadSingleton.CurrentDisposeScope;
+
         /// <summary>
         /// Creates a new dispose scope for the current thread, wrapping an expression.
         /// </summary>