Deprecate last_dim_is_batch (bump PyTorch version to >= 2.0) (#2549)

* Update PyTorch requirement to >=2.0

* Add sum_interaction_terms utility to replace functionality of
NewtonGirardAdditiveKernel

This kernel will be deprecated and removed as part of GPyTorch 2.0.

* Add tutorial on additive and product kernels.

Include instructions for alternative implementation for the following deprecated kernels:
- AdditiveStructureKernel
- ProductStructureKernel
- NewtonGirardAdditiveKernel

* Add deprecation warnings for removal of last_dim_is_batch:

- Kernels called with last_dim_is_batch raise a DeprecationWarning
- AdditiveStructureKernel, ProductStructureKernel, and
NewtonGirardAdditiveKernel raise DeprecationWarnings when instantiated.
- Update example notebooks to remove usage of these deprecated kernels.

* Apply suggestions from code review

Co-authored-by: Jonathan Wenger <[email protected]>

* Fix linting error

---------

Co-authored-by: Jonathan Wenger <[email protected]>

Author: gpleiss

.conda/meta.yaml

@@ -17,7 +17,7 @@ requirements:
 
   run:
     - python>=3.8
-    - pytorch>=1.11
+    - pytorch>=2.0
     - scikit-learn
     - jaxtyping>=0.2.9
     - linear_operator>=0.5.2

.github/workflows/run_test_suite.yml

@@ -50,7 +50,7 @@ jobs:
         if [[ ${{ matrix.pytorch-version }} = "master" ]]; then
           pip install --pre torch -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html;
         else
-          pip install torch==1.11+cpu -f https://download.pytorch.org/whl/torch_stable.html;
+          pip install torch==2.0.1 --index-url https://download.pytorch.org/whl/cpu
         fi
         pip install -e .
         if [[ ${{ matrix.extras }} == "with-extras" ]]; then

README.md

@@ -30,7 +30,7 @@ See our [**documentation, examples, tutorials**](https://gpytorch.readthedocs.io
 
 **Requirements**:
 - Python >= 3.8
-- PyTorch >= 1.11
+- PyTorch >= 2.0
 
 Install GPyTorch using pip or conda:
 

docs/source/utils.rst

@@ -20,6 +20,12 @@ Interpolation Utilities
 .. automodule:: gpytorch.utils.interpolation
    :members:
 
+Nearest Neighbors Utilities
+---------------------------------
+
+.. automodule:: gpytorch.utils.nearest_neighbors
+   :members:
+
 Quadrature Utilities
 ----------------------------
 
@@ -31,9 +37,3 @@ Transform Utilities
 
 .. automodule:: gpytorch.utils.transforms
    :members:
-
-Nearest Neighbors Utilities
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. automodule:: gpytorch.utils.nearest_neighbors
-   :members:

examples/00_Basic_Usage/index.rst

@@ -6,17 +6,22 @@ parameter constraints and priors, and saving and loading models.
 
 Before checking these out, you may want to check out our `simple GP regression tutorial`_ that details the anatomy of a GPyTorch model.
 
-- Check out our `Tutorial on Hyperparameters`_ for information on things like raw versus actual
+* Check out our `Tutorial on Hyperparameters`_ for information on things like raw versus actual
   parameters, constraints, priors and more.
-- The `Saving and Loading Models`_ notebook details how to save and load GPyTorch models
+* The `Saving and Loading Models`_ notebook details how to save and load GPyTorch models
   on disk.
+* The `Kernels with Additive or Product Structure`_ notebook describes how to compose kernels additively or multiplicatively,
+  whether for expressivity, sample efficiency, or scalability.
+* The `Implementing a Custom Kernel`_ notebook details how to write your own custom kernel in GPyTorch.
+* The `Tutorial on Metrics`_ describes various metrics provided by GPyTorch for assessing the generalization of GP models.
 
 .. toctree::
    :maxdepth: 1
    :hidden:
 
    Hyperparameters.ipynb
    Saving_and_Loading_Models.ipynb
+   kernels_with_additive_or_product_structure.ipynb
    Implementing_a_custom_Kernel.ipynb
    Metrics.ipynb
 
@@ -29,6 +34,9 @@ Before checking these out, you may want to check out our `simple GP regression t
 .. _Saving and Loading Models:
   Saving_and_Loading_Models.ipynb
 
+.. _Kernels with Additive or Product Structure:
+  kernels_with_additive_or_product_structure.ipynb
+
 .. _Implementing a custom Kernel:
   Implementing_a_custom_Kernel.ipynb