Skip to content

make InterceptTerms generate Bools to avoid unnecessary promotion of other columns #294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

make InterceptTerms generate Bools to avoid unnecessary promotion of other columns #294

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
05516d9
test
kleinschmidt May 26, 2023
995146e
with no intercept
kleinschmidt May 26, 2023
a90e512
bool
kleinschmidt May 26, 2023
00915d7
test this behavior too
kleinschmidt May 26, 2023
9e1fae6
unused
kleinschmidt May 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions src/terms.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -547,8 +547,10 @@ function modelcols(t::InteractionTerm, d::ColumnTable)
row_kron_insideout(*, (modelcols(term, d) for term in t.terms)...)
end

modelcols(t::InterceptTerm{true}, d::NamedTuple) = ones(size(first(d)))
modelcols(t::InterceptTerm{false}, d) = Matrix{Float64}(undef, size(first(d),1), 0)
# use Bool as the eltype here to avoid promoting e.g. Float32s in other columns
# to Float64 unless it's really necessary
modelcols(t::InterceptTerm{true}, d::NamedTuple) = ones(Bool, size(first(d)))
modelcols(t::InterceptTerm{false}, d) = Matrix{Bool}(undef, size(first(d),1), 0)
Comment on lines +552 to +553
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to generate BitArrays here? I guess it doesn't matter much, but it would save some memory.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd kinda hoped to be able to just use a scalar but hcat doesn't work with that


modelcols(t::FormulaTerm, d::NamedTuple) = (modelcols(t.lhs,d), modelcols(t.rhs, d))

Expand Down
30 changes: 30 additions & 0 deletions test/modelmatrix.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,5 +402,35 @@
f = apply_schema(@formula(0 ~ a&b&c), schema(t))
@test vec(modelcols(f.rhs, t)) == modelcols.(Ref(f.rhs), Tables.rowtable(t))
end

@testset "#293 conversion of Float32s" begin
d = (; y=rand(Float32, 4), x=rand(Float32, 4), z=[1:4; ])
f = @formula(y ~ 1 + x + log(x) * z)
y, x = modelcols(apply_schema(f, schema(d)), d)
@test eltype(y) == eltype(x) == Float32

f0 = @formula(y ~ 0 + x + log(x) * z)
y, x = modelcols(apply_schema(f0, schema(d)), d)
@test eltype(y) == eltype(x) == Float32

fint = @formula(y ~ 1 + z)
y, x = modelcols(apply_schema(fint, schema(d)), d)
@test eltype(x) == Int

contr = DummyCoding(; base=4, levels=1:4)
d = (; y=rand(Float32, 4), x=rand(Float32, 4), z=[1:4; ])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't yet used below, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is used but it's redundant with the definition above...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait the contr part? yeah that's right.


# currently this is the best way to construct contrasts with Float32s...
dummy_cmat = Float32[1 0 0
0 1 0
0 0 1
0 0 0]
contr = StatsModels.ContrastsCoding(dummy_cmat, [1:4;])

sch = schema(f, d, Dict(:z => contr))
y, x = modelcols(apply_schema(f, sch), d)
@test size(x) == (4, 1 + 1 + 1 + 3 + 3)
@test eltype(x) == eltype(y) == Float32
end

end