@@ -44,17 +44,34 @@ mat1k = rand(3,1000);
44
44
@btime Zygote. gradient (m -> sum (MapCols {3} (fun, m)), $ mat1k); # 28.229 μs, 164.63 KiB
45
45
```
46
46
47
- For such a simple function, timing ` sum(sin, MapCols{3}(fun, m)) ` takes 3 to 10 times longer!
47
+ On recent versions of Julia, ` mapcols ` has become much faster, 5- 10 times.
48
48
49
49
## Other packages
50
50
51
- This package also provides Zygote gradients for the slice/glue functions in
51
+ This package also provides Zygote gradients for the Slice/Align functions in
52
+ [ JuliennedArrays] ( https://github.com/bramtayl/JuliennedArrays.jl ) ,
53
+ which can be used to write many mapslices-like operations:
54
+
55
+ ``` julia
56
+ using JuliennedArrays
57
+ jumap (f,m) = Align (map (f, Slices (m, True (), False ())), True (), False ())
58
+ jumap1 (f,m) = Align (map (f, Slices (m, 1 )), 1 )
59
+ jumap (fun, mat) # same as mapcols
60
+ jumap1 (fun, mat)
61
+ Zygote. gradient (m -> sum (sin, jumap (fun, m)), mat)[1 ]
62
+
63
+ @btime jumap (fun, $ mat1k); # 44.823 μs
64
+ @btime jumap1 (fun, $ mat1k); # 11.805 μs, really?
65
+ @btime Zygote. gradient (m -> sum (jumap (fun, m)), $ mat1k); # 26.110 ms
66
+ @btime Zygote. gradient (m -> sum (jumap1 (fun, m)), $ mat1k) # 412.904 μs, really?
67
+ ```
68
+
69
+ It used to do the same thing for the slice/glue functions in
52
70
[ TensorCast] ( https://github.com/mcabbott/TensorCast.jl ) ,
53
- which can be used to write many mapslices-like operations.
54
- (The function ` slicemap(f, A, dims) ` uses these functions, without having to write index notation.)
71
+ but but that should soon be part of that package:
55
72
56
73
``` julia
57
- using TensorCast
74
+ using TensorCast# two
58
75
@cast [i,j] := fun (mat[:,j])[i] # same as mapcols
59
76
60
77
tcm (mat) = @cast out[i,j] := fun (mat[:,j])[i]
@@ -64,22 +81,6 @@ Zygote.gradient(m -> sum(sin, tcm(m)), mat)[1]
64
81
@btime Zygote. gradient (m -> sum (tcm (m)), $ mat1k); # 18.358 ms
65
82
```
66
83
67
- Similar gradients work for the Slice/Align functions in
68
- [ JuliennedArrays] ( https://github.com/bramtayl/JuliennedArrays.jl ) ,
69
- so it defines these too:
70
-
71
- ``` julia
72
- using JuliennedArrays
73
- jumap (f,m) = Align (map (f, Slices (m, True (), False ())), True (), False ())
74
- jumap (fun, mat) # same as mapcols
75
- Zygote. gradient (m -> sum (sin, jumap (fun, m)), mat)[1 ]
76
-
77
- @btime jumap (fun, $ mat1k); # 421.061 μs
78
- @btime Zygote. gradient (m -> sum (jumap (fun, m)), $ mat1k); # 18.383 ms
79
- ```
80
-
81
- That's a 2-line gradient definition, so borrowing it may be easier than depending on this package.
82
-
83
84
The original purpose of ` MapCols ` , with ForwardDiff on slices, was that this works well when
84
85
the function being mapped integrates some differential equation.
85
86
@@ -128,9 +129,3 @@ Other packages which define gradients of possible interest:
128
129
* https://github.com/GiggleLiu/LinalgBackwards.jl
129
130
* https://github.com/mcabbott/ArrayAllez.jl
130
131
131
- Differentiation packages this could perhaps support, quite the zoo:
132
- * https://github.com/dfdx/Yota.jl
133
- * https://github.com/invenia/Nabla.jl
134
- * https://github.com/denizyuret/AutoGrad.jl
135
- * https://github.com/Roger-luo/YAAD.jl
136
- * And perhaps one day, just https://github.com/JuliaDiff/ChainRules.jl
0 commit comments