From 42ccf700e3cb0481b46a7b79cb6d26a920cafe83 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20Galy-Fajou?= Date: Thu, 31 Dec 2020 13:30:26 +0100 Subject: [PATCH 1/5] Update userguide.md --- docs/src/userguide.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/src/userguide.md b/docs/src/userguide.md index 59bdda9ec..e2d259d4b 100644 --- a/docs/src/userguide.md +++ b/docs/src/userguide.md @@ -7,14 +7,15 @@ For example to create a square exponential kernel ```julia k = SqExponentialKernel() ``` -Instead of having lengthscale(s) for each kernel we use `Transform` objects (see [Transform](@ref)) which are directly going to act on the inputs before passing them to the kernel. -For example to premultiply the input by 2.0 we create the kernel the following options are possible +!!! tip "How do I set the lengthscale?" Instead of having lengthscale(s) for each kernel we use `Transform` objects (see [Transform](@ref)) which are directly going to act on the inputs before passing them to the kernel. +For example, if you want to to premultiply the input by 2.0, you can create your kernel with the following options: ```julia - k = transform(SqExponentialKernel(),ScaleTransform(2.0)) # returns a TransformedKernel - k = @kernel SqExponentialKernel() l=2.0 # Will be available soon - k = TransformedKernel(SqExponentialKernel(),ScaleTransform(2.0)) + k = transform(SqExponentialKernel(), 2.0)) # returns a TransformedKernel + k = TransformedKernel(SqExponentialKernel(), ScaleTransform(2.0)) ``` +In the example of the [SqExponentialKernel](@ref), you can reproduce the usual definition, $$\exp\left(-\frac{\|x-x'\|^2}{\rho^2}$$, by using the following `Transform` : `transform(SqExponentialKernel(), 1 / ρ`. Check the [`Transform`](@ref) page to see the other options. + To premultiply the kernel by a variance, you can use `*` or create a `ScaledKernel` ```julia k = 3.0*SqExponentialKernel() @@ -79,9 +80,12 @@ For example : What if you want to differentiate through the kernel parameters? Even in a highly nested structure such as : ```julia - k = transform(0.5*SqExponentialKernel()*MaternKernel()+0.2*(transform(LinearKernel(),2.0)+PolynomialKernel()),[0.1,0.5]) + k = transform( + 0.5 * SqExponentialKernel() * MaternKernel() + + 0.2 * (transform(LinearKernel(), 2.0) + PolynomialKernel()), + [0.1, 0.5]) ``` -One can get the array of parameters to optimize via `params` from `Flux.jl` +One can access the array of trainable parameters via `params` from `Flux.jl` ```julia using Flux From 6c89b8d8a7b5d8ce8398135a32089bd20d22aa53 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20Galy-Fajou?= Date: Thu, 31 Dec 2020 13:39:34 +0100 Subject: [PATCH 2/5] Apply suggestions from code review Co-authored-by: David Widmann --- docs/src/userguide.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/src/userguide.md b/docs/src/userguide.md index e2d259d4b..93f88e18e 100644 --- a/docs/src/userguide.md +++ b/docs/src/userguide.md @@ -8,12 +8,12 @@ For example to create a square exponential kernel k = SqExponentialKernel() ``` !!! tip "How do I set the lengthscale?" Instead of having lengthscale(s) for each kernel we use `Transform` objects (see [Transform](@ref)) which are directly going to act on the inputs before passing them to the kernel. -For example, if you want to to premultiply the input by 2.0, you can create your kernel with the following options: +For example, if you want to premultiply the input by 2.0, you can create your kernel with the following options: ```julia k = transform(SqExponentialKernel(), 2.0)) # returns a TransformedKernel k = TransformedKernel(SqExponentialKernel(), ScaleTransform(2.0)) ``` -In the example of the [SqExponentialKernel](@ref), you can reproduce the usual definition, $$\exp\left(-\frac{\|x-x'\|^2}{\rho^2}$$, by using the following `Transform` : `transform(SqExponentialKernel(), 1 / ρ`. +In the example of the [SqExponentialKernel](@ref), you can reproduce the usual definition, $$\exp\left(-\frac{\|x-x'\|^2}{\rho^2}\right)$$, by using `transform(SqExponentialKernel(), 1 / ρ)`. Check the [`Transform`](@ref) page to see the other options. To premultiply the kernel by a variance, you can use `*` or create a `ScaledKernel` From efa4d40e33e81834f1e37191b8f3b45f3341c1cc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20Galy-Fajou?= Date: Thu, 31 Dec 2020 18:44:31 +0100 Subject: [PATCH 3/5] Update docs/src/userguide.md --- docs/src/userguide.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/src/userguide.md b/docs/src/userguide.md index 93f88e18e..3316f5343 100644 --- a/docs/src/userguide.md +++ b/docs/src/userguide.md @@ -85,7 +85,14 @@ What if you want to differentiate through the kernel parameters? Even in a highl + 0.2 * (transform(LinearKernel(), 2.0) + PolynomialKernel()), [0.1, 0.5]) ``` -One can access the array of trainable parameters via `params` from `Flux.jl` +One can access the named tuple of trainable parameters via `Functors.functor` from `Functors.jl`. +This means that in practice you can implicitly optimize the kernel parameters by calling: +```julia +using Flux +hp = params(k) +gradient(hp) do + ... some loss function on the kernel .... +end ```julia using Flux From 132fda0b74c1586dc83965b0b0fb8698cd03e6c7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20Galy-Fajou?= Date: Thu, 31 Dec 2020 18:44:37 +0100 Subject: [PATCH 4/5] Update docs/src/userguide.md --- docs/src/userguide.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/src/userguide.md b/docs/src/userguide.md index 3316f5343..e0b27033c 100644 --- a/docs/src/userguide.md +++ b/docs/src/userguide.md @@ -11,7 +11,6 @@ For example to create a square exponential kernel For example, if you want to premultiply the input by 2.0, you can create your kernel with the following options: ```julia k = transform(SqExponentialKernel(), 2.0)) # returns a TransformedKernel - k = TransformedKernel(SqExponentialKernel(), ScaleTransform(2.0)) ``` In the example of the [SqExponentialKernel](@ref), you can reproduce the usual definition, $$\exp\left(-\frac{\|x-x'\|^2}{\rho^2}\right)$$, by using `transform(SqExponentialKernel(), 1 / ρ)`. Check the [`Transform`](@ref) page to see the other options. From fa256f026c6537909671e367099964e04b0730ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20Galy-Fajou?= Date: Fri, 1 Jan 2021 14:41:32 +0100 Subject: [PATCH 5/5] Update userguide.md --- docs/src/userguide.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/docs/src/userguide.md b/docs/src/userguide.md index e0b27033c..5073fd936 100644 --- a/docs/src/userguide.md +++ b/docs/src/userguide.md @@ -88,12 +88,8 @@ One can access the named tuple of trainable parameters via `Functors.functor` fr This means that in practice you can implicitly optimize the kernel parameters by calling: ```julia using Flux -hp = params(k) -gradient(hp) do - ... some loss function on the kernel .... +kernelparams = Flux.params(k) +Flux.gradient(kernelparams) do + ... some loss function on the kernel .... end - -```julia - using Flux - params(k) ```