Skip to content

rowwise(): Detect and reject non-atomic, non-list column values with clear error message #7250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

rowwise(): Detect and reject non-atomic, non-list column values with clear error message #7250

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
aaa09aa
added check
venom1204 Aug 12, 2025
2110d98
..
venom1204 Aug 12, 2025
6b271bb
Merge branch 'master' into issue_7219
venom1204 Aug 13, 2025
2f777d5
removed empty lines
venom1204 Aug 15, 2025
d19bc2a
Merge branch 'master' into issue_7219
venom1204 Aug 15, 2025
dd36cf7
Merge branch 'issue_7219' of https://github.com/Rdatatable/data.table…
venom1204 Aug 15, 2025
08fe92e
Merge branch 'master' into issue_7219
venom1204 Aug 24, 2025
669f6ad
vapply_1b
venom1204 Aug 25, 2025
3ece463
Merge branch 'issue_7219' of https://github.com/Rdatatable/data.table…
venom1204 Aug 25, 2025
e2de9c0
type-> class
venom1204 Aug 30, 2025
dddcedb
Merge branch 'master' of https://github.com/Rdatatable/data.table int…
venom1204 Aug 30, 2025
dafc34f
lintr + error
venom1204 Aug 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions R/rowwiseDT.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,22 @@ rowwiseDT = function(...) {
nrows = length(body) %/% ncols
if (length(body) != nrows * ncols)
stopf("There are %d columns but the number of cells is %d, which is not an integer multiple of the columns", ncols, length(body))
is_problematic = vapply(
body,
function(v) !is.atomic(v) && !is.null(v) && typeof(v) != "list",
Copy link
Member

@ben-schwen ben-schwen Aug 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this check really what @aitap wrote about catering for that is.atomic(NULL)=TRUE on older R versions?

it should also be faster to check !(is.atomic(v) || is.null(v) || typeof(v) == "list")

Note that we have our internal versions of vapply e.g. vapply_1b

FUN.VALUE = logical(1L)
)
if (any(is_problematic)) {
first_problem_idx = which(is_problematic)[1L]
col_idx = (first_problem_idx - 1L) %% ncols + 1L
col_name = header[col_idx]
obj_type = typeof(body[[first_problem_idx]])
Copy link
Member

@ben-schwen ben-schwen Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if typeof is the right choice here. For the problem raised in #7219 class would be a better choice since class would be "function" but typeof would be still "closure".

stopf(
"In column '%s', received an object of type '%s'.\nComplex objects (like functions, models, etc.) must be wrapped in list() to be stored in a data.table column.\nPlease use `list(...)` for this value.",
Copy link
Member

@ben-schwen ben-schwen Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really true? Many models are indeed lists, e.g. lm(mpg~., data=mtcars).

Other thing on top of my head would be expressions which we do not fully support. Storing environments into a data.table seems uncommon to me.

col_name,
obj_type
)
}
# make all the non-scalar elements to a list
needs_list = lengths(body) != 1L
body[needs_list] = lapply(body[needs_list], list)
Expand Down
12 changes: 12 additions & 0 deletions inst/tests/tests.Rraw
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21620,3 +21620,15 @@ local({
test(2338.9, {fwrite(dd, f, forceDecimal=FALSE); fread(f)}, di)
})

# rowwiseDT() valid and invalid handling of complex objects #7219
test(2339.1, rowwiseDT(x =, y =, 1, 2, 3, 4), data.table(x = c(1, 3), y = c(2, 4)))
test(2339.2, rowwiseDT(x =, func =,
1, list(\(x) x + 1),
2, list(function(z) z * 2)),
data.table(x = c(1, 2), func = list(\(x) x + 1, function(z) z * 2)))
test(2339.3, rowwiseDT(x =, func =, 1, \(x) x + 1),
error = "In column 'func', received an object of type 'closure'.*wrap.*list")
test(2339.4, rowwiseDT(x =, expr =, 1, quote(a + b)),
error = "In column 'expr', received an object of type 'language'.*wrap.*list")
test(2339.5, rowwiseDT(x =, plist =, 1, as.pairlist(list(123))),
error = "In column 'plist', received an object of type 'pairlist'.*wrap.*list")
Loading