Skip to content

Start testing reverse mode #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Feb 8, 2024
Merged

Start testing reverse mode #254

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
eb76681
add FiniteDifferences.jl
nmheim Jan 25, 2024
6bff01d
mark as broken
nmheim Jan 25, 2024
fc6b7fc
start adding gradcheck tests from zygote
nmheim Jan 25, 2024
11ca009
more tests
nmheim Jan 26, 2024
6b4579b
pmap tests
nmheim Jan 26, 2024
a514f7d
add some more testsets
nmheim Jan 26, 2024
4344987
remove complex/misc tests
nmheim Jan 29, 2024
be1df82
wrap in module; import prime
nmheim Jan 29, 2024
9a764c9
adjoint fix
nmheim Jan 29, 2024
29a856b
rename to value_and_pullback
nmheim Jan 31, 2024
9334ddf
remove comment
nmheim Jan 31, 2024
de07109
fix incorrect getindex gradient
nmheim Feb 6, 2024
6c9048c
include comment to explain view
nmheim Feb 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,13 @@ StructArrays = "0.6"
julia = "1.10"

[extras]
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
FiniteDifferences = "26cc04aa-876d-5657-8c51-4c34ba976000"
ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Symbolics = "0c5d862f-8b57-4792-8d23-62f2024744c7"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["ForwardDiff", "LinearAlgebra", "Random", "Symbolics", "Test"]
test = ["Distributed", "FiniteDifferences", "ForwardDiff", "LinearAlgebra", "Random", "Symbolics", "Test"]
2 changes: 1 addition & 1 deletion src/stage1/generated.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ function (::∂⃖{N})(::typeof(Base.getindex), a::Array{<:Number}, inds...) whe
(@Base.constprop :aggressive Δ->begin
Δ isa AbstractZero && return (NoTangent(), Δ, map(Returns(Δ), inds)...)
BB = zero(a)
BB[inds...] = unthunk(Δ)
view(BB, inds...) .+= unthunk(Δ)
Copy link
Member

@oxinabox oxinabox Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this different?
0 .+ Δ = Δ at least assuming they are the same size

Copy link
Contributor Author

@nmheim nmheim Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, I should have added a comment here. this is for cases like

julia> gradient(sum ∘ x -> x[:,[1,1,2]], rand(3,4))[1]
3×4 Matrix{Float64}:
 2.0  1.0  0.0  0.0
 2.0  1.0  0.0  0.0
 2.0  1.0  0.0  0.0

The view lets one add duplicate indices

julia> x = zeros(3,4)
3×4 Matrix{Float64}:
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0

julia> view(x, :, [1,1,2]) .+= ones(3,3)
3×3 view(::Matrix{Float64}, :, [1, 1, 2]) with eltype Float64:
 2.0  2.0  1.0
 2.0  2.0  1.0
 2.0  2.0  1.0

julia> x
3×4 Matrix{Float64}:
 2.0  1.0  0.0  0.0
 2.0  1.0  0.0  0.0
 2.0  1.0  0.0  0.0

Copy link
Member

@oxinabox oxinabox Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh cool trick. I think last time i needed to handle that my solution was way more complicated.

(NoTangent(), BB, map(x->NoTangent(), inds)...)
end),
(@Base.constprop :aggressive (_, Δ, _)->begin
Expand Down
Loading