[cbindlist/mergelist] Implement cbindlist (#6435)

MichaelChirico · ben-schwen · jangorecki · web-flow · commit ae2b8155d1fc · 2025-07-01T10:53:43.000-07:00
* cbindlist add cbind by reference, timing R prototype of mergelist wording use lower overhead funs stick to int32 for now, correct R_alloc bmerge C refactor for codecov and one loop for speed address revealed codecov gaps refactor vecseq for codecov seqexp helper, some alloccol export on C bmerge codecov, types handled in R bmerge already better comment seqexp bmerge mult=error #655 multiple new C utils swap if branches explain new C utils comments mostly reduce conflicts to PR #4386 comment C code address multiple matches during update-on-join #3747 Revert "address multiple matches during update-on-join #3747" This reverts commit b64c0c3. merge.dt has temporarily mult arg, for testing minor changes to cbindlist c dev mergelist, for single pair now add quiet option to cc() mergelist tests add check for names to perhaps.dt rm mult from merge.dt method rework, clean, polish multer, fix righ and full joins make full join symmetric mergepair inner function to loop on extra check for symmetric mergelist manual ensure no df-dt passed where list expected comments and manual handle 0 cols tables more tests more tests and debugging move more logic closer to bmerge, simplify mergepair more tests revert not used changes reduce not needed checks, cleanup copy arg behavior, manual, no tests yet cbindlist manual, export both cleanup processing bmerge to dtmatch test function match order for easier preview vecseq gets short-circuit batch test allow browser big cleanup remmove unneeded stuff, reduce diff more cleanup, minor manual fixes add proper test scripts Merge branch 'master' into cbind-merge-list comment out not used code for coverage more tests, some nocopy opts rename sql test script, should fix codecov simplify dtmatch inner branch more precise copy, now copy only T or F unused arg not yet in api, wording comments and refer issues codecov hasindex coverage codecov gap tests for join using key, cols argument fix missing import forderv more tests, improve missing on handling more tests for order of inner and full join for long keys new allow.cartesian option, #4383, #914 reduce diff, improve codecov reduce diff, comments need more DT, not lists, mergelist 3+ tbls proper escape heavy check unit tests more tests, address overalloc failure mergelist and cbindlist retain index manual, examples fix manual minor clarify in manual retain keys, right outer join for snowflake schema joins duplicates in cbindlist recycling in cbindlist escape 0 input in copyCols empty input handling closing cbindlist vectorized _on_ and _join.many_ arg rename dtmatch to dtmerge vectorized args: how, mult push down input validation add support for cross join, semi join, anti join full join, reduce overhead for mult=error mult default value dynamic fix manual add "see details" to Rd mention shared on in arg description amend feedback from Michael semi and anti joins will not reorder x columns Merge branch 'master' into cbind-merge-list spelling, thx to @jan-glx check all new funs used and add comments bugfix, sort=T needed for now Merge branch 'master' into cbind-merge-list Update NEWS.md Merge branch 'master' into cbind-merge-list Merge branch 'master' into cbind-merge-list NEWS placement numbering ascArg->order Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list attempt to restore from master Update to stopf() error style Need isFrame for now More quality checks: any(!x)->!all(x); use vapply_1{b,c,i} really restore from master try to PROTECT() before duplicate() update error message in test appease the rchk gods extraneous space missing ';' use catf simplify perhapsDataTableR move sqlite.Rraw.manual into other.Rraw simplify for loop Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list * cbindlist add cbind by reference, timing R prototype of mergelist wording use lower overhead funs stick to int32 for now, correct R_alloc bmerge C refactor for codecov and one loop for speed address revealed codecov gaps refactor vecseq for codecov seqexp helper, some alloccol export on C bmerge codecov, types handled in R bmerge already better comment seqexp bmerge mult=error #655 multiple new C utils swap if branches explain new C utils comments mostly reduce conflicts to PR #4386 comment C code address multiple matches during update-on-join #3747 Revert "address multiple matches during update-on-join #3747" This reverts commit b64c0c3. merge.dt has temporarily mult arg, for testing minor changes to cbindlist c dev mergelist, for single pair now add quiet option to cc() mergelist tests add check for names to perhaps.dt rm mult from merge.dt method rework, clean, polish multer, fix righ and full joins make full join symmetric mergepair inner function to loop on extra check for symmetric mergelist manual ensure no df-dt passed where list expected comments and manual handle 0 cols tables more tests more tests and debugging move more logic closer to bmerge, simplify mergepair more tests revert not used changes reduce not needed checks, cleanup copy arg behavior, manual, no tests yet cbindlist manual, export both cleanup processing bmerge to dtmatch test function match order for easier preview vecseq gets short-circuit batch test allow browser big cleanup remmove unneeded stuff, reduce diff more cleanup, minor manual fixes add proper test scripts Merge branch 'master' into cbind-merge-list comment out not used code for coverage more tests, some nocopy opts rename sql test script, should fix codecov simplify dtmatch inner branch more precise copy, now copy only T or F unused arg not yet in api, wording comments and refer issues codecov hasindex coverage codecov gap tests for join using key, cols argument fix missing import forderv more tests, improve missing on handling more tests for order of inner and full join for long keys new allow.cartesian option, #4383, #914 reduce diff, improve codecov reduce diff, comments need more DT, not lists, mergelist 3+ tbls proper escape heavy check unit tests more tests, address overalloc failure mergelist and cbindlist retain index manual, examples fix manual minor clarify in manual retain keys, right outer join for snowflake schema joins duplicates in cbindlist recycling in cbindlist escape 0 input in copyCols empty input handling closing cbindlist vectorized _on_ and _join.many_ arg rename dtmatch to dtmerge vectorized args: how, mult push down input validation add support for cross join, semi join, anti join full join, reduce overhead for mult=error mult default value dynamic fix manual add "see details" to Rd mention shared on in arg description amend feedback from Michael semi and anti joins will not reorder x columns Merge branch 'master' into cbind-merge-list spelling, thx to @jan-glx check all new funs used and add comments bugfix, sort=T needed for now Merge branch 'master' into cbind-merge-list Update NEWS.md Merge branch 'master' into cbind-merge-list Merge branch 'master' into cbind-merge-list NEWS placement numbering ascArg->order Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list attempt to restore from master Update to stopf() error style Need isFrame for now More quality checks: any(!x)->!all(x); use vapply_1{b,c,i} really restore from master try to PROTECT() before duplicate() update error message in test appease the rchk gods extraneous space missing ';' use catf simplify perhapsDataTableR move sqlite.Rraw.manual into other.Rraw simplify for loop Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list * restore ws change * cbindlist add cbind by reference, timing R prototype of mergelist wording use lower overhead funs stick to int32 for now, correct R_alloc bmerge C refactor for codecov and one loop for speed address revealed codecov gaps refactor vecseq for codecov seqexp helper, some alloccol export on C bmerge codecov, types handled in R bmerge already better comment seqexp bmerge mult=error #655 multiple new C utils swap if branches explain new C utils comments mostly reduce conflicts to PR #4386 comment C code address multiple matches during update-on-join #3747 Revert "address multiple matches during update-on-join #3747" This reverts commit b64c0c3. merge.dt has temporarily mult arg, for testing minor changes to cbindlist c dev mergelist, for single pair now add quiet option to cc() mergelist tests add check for names to perhaps.dt rm mult from merge.dt method rework, clean, polish multer, fix righ and full joins make full join symmetric mergepair inner function to loop on extra check for symmetric mergelist manual ensure no df-dt passed where list expected comments and manual handle 0 cols tables more tests more tests and debugging move more logic closer to bmerge, simplify mergepair more tests revert not used changes reduce not needed checks, cleanup copy arg behavior, manual, no tests yet cbindlist manual, export both cleanup processing bmerge to dtmatch test function match order for easier preview vecseq gets short-circuit batch test allow browser big cleanup remmove unneeded stuff, reduce diff more cleanup, minor manual fixes add proper test scripts Merge branch 'master' into cbind-merge-list comment out not used code for coverage more tests, some nocopy opts rename sql test script, should fix codecov simplify dtmatch inner branch more precise copy, now copy only T or F unused arg not yet in api, wording comments and refer issues codecov hasindex coverage codecov gap tests for join using key, cols argument fix missing import forderv more tests, improve missing on handling more tests for order of inner and full join for long keys new allow.cartesian option, #4383, #914 reduce diff, improve codecov reduce diff, comments need more DT, not lists, mergelist 3+ tbls proper escape heavy check unit tests more tests, address overalloc failure mergelist and cbindlist retain index manual, examples fix manual minor clarify in manual retain keys, right outer join for snowflake schema joins duplicates in cbindlist recycling in cbindlist escape 0 input in copyCols empty input handling closing cbindlist vectorized _on_ and _join.many_ arg rename dtmatch to dtmerge vectorized args: how, mult push down input validation add support for cross join, semi join, anti join full join, reduce overhead for mult=error mult default value dynamic fix manual add "see details" to Rd mention shared on in arg description amend feedback from Michael semi and anti joins will not reorder x columns Merge branch 'master' into cbind-merge-list spelling, thx to @jan-glx check all new funs used and add comments bugfix, sort=T needed for now Merge branch 'master' into cbind-merge-list Update NEWS.md Merge branch 'master' into cbind-merge-list Merge branch 'master' into cbind-merge-list NEWS placement numbering ascArg->order Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list attempt to restore from master Update to stopf() error style Need isFrame for now More quality checks: any(!x)->!all(x); use vapply_1{b,c,i} really restore from master try to PROTECT() before duplicate() update error message in test appease the rchk gods extraneous space missing ';' use catf simplify perhapsDataTableR move sqlite.Rraw.manual into other.Rraw simplify for loop Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list * cbindlist add cbind by reference, timing R prototype of mergelist wording use lower overhead funs stick to int32 for now, correct R_alloc bmerge C refactor for codecov and one loop for speed address revealed codecov gaps refactor vecseq for codecov seqexp helper, some alloccol export on C bmerge codecov, types handled in R bmerge already better comment seqexp bmerge mult=error #655 multiple new C utils swap if branches explain new C utils comments mostly reduce conflicts to PR #4386 comment C code address multiple matches during update-on-join #3747 Revert "address multiple matches during update-on-join #3747" This reverts commit b64c0c3. merge.dt has temporarily mult arg, for testing minor changes to cbindlist c dev mergelist, for single pair now add quiet option to cc() mergelist tests add check for names to perhaps.dt rm mult from merge.dt method rework, clean, polish multer, fix righ and full joins make full join symmetric mergepair inner function to loop on extra check for symmetric mergelist manual ensure no df-dt passed where list expected comments and manual handle 0 cols tables more tests more tests and debugging move more logic closer to bmerge, simplify mergepair more tests revert not used changes reduce not needed checks, cleanup copy arg behavior, manual, no tests yet cbindlist manual, export both cleanup processing bmerge to dtmatch test function match order for easier preview vecseq gets short-circuit batch test allow browser big cleanup remmove unneeded stuff, reduce diff more cleanup, minor manual fixes add proper test scripts Merge branch 'master' into cbind-merge-list comment out not used code for coverage more tests, some nocopy opts rename sql test script, should fix codecov simplify dtmatch inner branch more precise copy, now copy only T or F unused arg not yet in api, wording comments and refer issues codecov hasindex coverage codecov gap tests for join using key, cols argument fix missing import forderv more tests, improve missing on handling more tests for order of inner and full join for long keys new allow.cartesian option, #4383, #914 reduce diff, improve codecov reduce diff, comments need more DT, not lists, mergelist 3+ tbls proper escape heavy check unit tests more tests, address overalloc failure mergelist and cbindlist retain index manual, examples fix manual minor clarify in manual retain keys, right outer join for snowflake schema joins duplicates in cbindlist recycling in cbindlist escape 0 input in copyCols empty input handling closing cbindlist vectorized _on_ and _join.many_ arg rename dtmatch to dtmerge vectorized args: how, mult push down input validation add support for cross join, semi join, anti join full join, reduce overhead for mult=error mult default value dynamic fix manual add "see details" to Rd mention shared on in arg description amend feedback from Michael semi and anti joins will not reorder x columns Merge branch 'master' into cbind-merge-list spelling, thx to @jan-glx check all new funs used and add comments bugfix, sort=T needed for now Merge branch 'master' into cbind-merge-list Update NEWS.md Merge branch 'master' into cbind-merge-list Merge branch 'master' into cbind-merge-list NEWS placement numbering ascArg->order Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list attempt to restore from master Update to stopf() error style Need isFrame for now More quality checks: any(!x)->!all(x); use vapply_1{b,c,i} really restore from master try to PROTECT() before duplicate() update error message in test appease the rchk gods extraneous space missing ';' use catf simplify perhapsDataTableR move sqlite.Rraw.manual into other.Rraw simplify for loop Merge remote-tracking branch 'origin/cbind-merge-list' into cbind-merge-list * Apply Ben's suggested changes Co-authored-by: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com> * Test address overlap with original 'l' directly * Use local() blocks to "seal off" local variables from other current+future tests * whitespace style * test that key is wiped from duplicates * revert: next test already tests wiped keys; add comment * rm redundant test * no local() block needed anymore * retain multiple keys * refine description of copy= for now * missing ')' Co-authored-by: Jan Gorecki <J.Gorecki@wit.edu.pl> * r->R * fixed approach for reading addresses * mis * more updated error expectations * disable new test for now * move setDT behavior test into tests.Rraw * split out setcbindlist * missed setcbindlist in Rd * Add as TODO * attempt to fix tests * fix setDT test * one last ironing out * ugh * drop unreachable code * Also unreachable by known code paths * Annotate UNPROTECT --------- Co-authored-by: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com> Co-authored-by: Jan Gorecki <J.Gorecki@wit.edu.pl>
diff --git a/NAMESPACE b/NAMESPACE
@@ -59,6 +59,7 @@ export(nafill)
 export(setnafill)
 export(.Last.updated)
 export(fcoalesce)
+export(cbindlist, setcbindlist)
 export(substitute2)
 #export(DT)  # mtcars |> DT(i,j,by)  #4872 #5472
 export(fctr)
diff --git a/R/mergelist.R b/R/mergelist.R
@@ -0,0 +1,12 @@
+cbindlist_impl_ = function(l, copy) {
+  ans = .Call(Ccbindlist, l, copy)
+  if (anyDuplicated(names(ans))) { ## invalidate key and index
+    setattr(ans, "sorted", NULL)
+    setattr(ans, "index", NULL)
+  }
+  setDT(ans)
+  ans
+}
+
+cbindlist = function(l) cbindlist_impl_(l, copy=TRUE)
+setcbindlist = function(l) cbindlist_impl_(l, copy=FALSE)
diff --git a/inst/tests/mergelist.Rraw b/inst/tests/mergelist.Rraw
@@ -0,0 +1,71 @@
+require(methods)
+
+if (exists("test.data.table", .GlobalEnv, inherits=FALSE)) {
+  if ((tt<-compiler::enableJIT(-1))>0)
+    cat("This is dev mode and JIT is enabled (level ", tt, ") so there will be a brief pause around the first test.\n", sep="")
+} else {
+  require(data.table)
+  test = data.table:::test
+}
+
+# cbindlist, setcbindlist
+
+local({
+  l = list(
+    d1 = data.table(x=1:3, v1=1L),
+    d2 = data.table(y=3:1, v2=2L),
+    d3 = data.table(z=2:4, v3=3L)
+  )
+  ans = cbindlist(l)
+  expected = data.table(l$d1, l$d2, l$d3)
+  test(11.01, ans, expected)
+  test(11.02, intersect(vapply(ans, address, ""), unlist(lapply(l, vapply, address, ""))), character())
+  ans = setcbindlist(l)
+  expected = setDT(c(l$d1, l$d2, l$d3))
+  test(11.03, ans, expected)
+  test(11.04, length(intersect(vapply(ans, address, ""), unlist(lapply(l, vapply, address, "")))), ncol(expected))
+})
+
+test(11.05, cbindlist(list(data.table(a=1L), data.table(), data.table(d=2L), data.table(f=3L))), data.table(a=1L, d=2L, f=3L))
+## codecov
+test(12.01, cbindlist(data.frame(a=1L)), error="must be a list")
+test(12.02, cbindlist(TRUE), error="must be a list")
+test(12.03, cbindlist(list(data.table(a=1L), 1L)), error="is not a data.table")
+test(12.04, options = c(datatable.verbose=TRUE), cbindlist(list(data.table(a=1:2), data.table(b=1:2))), data.table(a=1:2, b=1:2), output="cbindlist.*took")
+test(12.05, cbindlist(list(data.table(), data.table(a=1:2), data.table(b=1:2))), data.table(a=1:2, b=1:2))
+test(12.06, cbindlist(list(data.table(), data.table(a=1:2), list(b=1:2))), data.table(a=1:2, b=1:2))
+test(12.07, cbindlist(list(data.table(a=integer()), list(b=integer()))), data.table(a=integer(), b=integer()))
+## duplicated names
+test(12.08, cbindlist(list(data.table(a=1L, b=2L), data.table(b=3L, d=4L))), data.table(a=1L, b=2L, b=3L, d=4L))
+local({
+  # also test that keys, indices are wiped
+  ans = cbindlist(list(setindexv(data.table(a=2:1, b=1:2), "a"), data.table(a=1:2, b=2:1, key="a"), data.table(a=2:1, b=1:2)))
+  test(12.09, ans, data.table(a=2:1, b=1:2, a=1:2, b=2:1, a=2:1, b=1:2))
+  test(12.10, indices(ans), NULL)
+})
+## recycling, first ensure cbind recycling that we want to match to
+test(12.11, cbind(data.table(x=integer()), data.table(a=1:2)), data.table(x=c(NA_integer_, NA), a=1:2))
+test(12.12, cbind(data.table(x=1L), data.table(a=1:2)), data.table(x=c(1L, 1L), a=1:2))
+test(12.13, cbindlist(list(data.table(a=integer()), data.table(b=1:2))), error="Recycling.*not yet implemented")
+test(12.14, cbindlist(list(data.table(a=1L), data.table(b=1:2))), error="Recycling.*not yet implemented")
+test(12.15, setcbindlist(list(data.table(a=integer()), data.table(b=1:2))), error="have to have the same number of rows")
+test(12.16, setcbindlist(list(data.table(a=1L), data.table(b=1:2))), error="have to have the same number of rows")
+
+## retain indices
+local({
+  l = list(
+    data.table(id1=1:5, id2=5:1, id3=1:5, v1=1:5),
+    data.table(id4=5:1, id5=1:5, v2=1:5),
+    data.table(id6=5:1, id7=1:5, v3=1:5),
+    data.table(id8=5:1, id9=5:1, v4=1:5)
+  )
+  setkeyv(l[[1L]], "id1"); setindexv(l[[1L]], list("id1", "id2", "id3", c("id1","id2","id3"))); setindexv(l[[3L]], list("id6", "id7")); setindexv(l[[4L]], "id9")
+  ii = lapply(l, indices)
+  ans = cbindlist(l)
+  test(13.1, key(ans), "id1")
+  test(13.2, indices(ans), c("id1", "id2", "id3", "id1__id2__id3", "id6", "id7", "id9"))
+  test(13.3, ii, lapply(l, indices)) ## this tests that original indices have not been touched, shallow_duplicate in mergeIndexAttrib
+})
+test(13.4, cbindlist(list(data.table(a=1:2), data.table(b=3:4, key="b"))), data.table(a=1:2, b=3:4, key="b"))
+# TODO(#7116): this could be supported
+# test(13.5, cbindlist(list(data.table(a=1:2, key="a"), data.table(b=3:4, key="b"))), data.table(a=1:2, b=3:4, key=c("a", "b")))
diff --git a/inst/tests/tests.Rraw b/inst/tests/tests.Rraw
@@ -21291,3 +21291,15 @@ unlink(f)
 test(2325.2,
      fread('"foo","bar","baz"\n"a","b","c"', na.strings=c('"foo"', '"bar"', '"baz"'), header=FALSE),
      data.table(V1=c(NA, "a"), V2=c(NA, "b"), V3=c(NA, "c")))
+
+## ensure setDT will retain key and indices when it is called on the list (cbindlist assumes this)
+local({
+  d = data.table(x=1:2, y=2:1, z=2:1, v1=1:2)
+  setkeyv(d, "x"); setindexv(d, list("y", "z"))
+  a = attributes(d)
+  attributes(d) = a[!names(a) %in% c("class", ".internal.selfref", "row.names")]
+  test(2326.1, class(d), "list")
+  setDT(d)
+  test(2326.2, key(d), "x")
+  test(2326.3, indices(d), c("y", "z"))
+})
diff --git a/man/cbindlist.Rd b/man/cbindlist.Rd
@@ -0,0 +1,41 @@
+\name{cbindlist}
+\alias{cbindlist}
+\alias{setcbindlist}
+\alias{cbind}
+\alias{cbind.data.table}
+\title{Column bind multiple data.tables}
+\description{
+  Column bind multiple \code{data.table}s.
+}
+\usage{
+  cbindlist(l)
+  setcbindlist(l)
+}
+\arguments{
+  \item{l}{ \code{list} of \code{data.table}s to merge. }
+}
+\details{
+  Column bind only stacks input elements. Works like \code{\link{data.table}}, but takes \code{list} type on input. Zero-column tables in \code{l} are omitted. Tables in \code{l} should have matching row count; recycling of length-1 rows is not yet implemented. Indices of the input tables are transferred to the resulting table, as well as the \emph{key} of the first keyed table.
+}
+\value{
+  A new \code{data.table} based on the stacked objects.
+  
+  For \code{setcbindlist}, columns in the output will be shared with the input, i.e., \emph{no copy is made}.
+}
+\note{
+  No attempt is made to deduplicate resulting names. If the result has any duplicate names, keys and indices are removed.
+}
+\seealso{
+  \code{\link{data.table}}, \code{\link{rbindlist}}, \code{\link{setDT}}
+}
+\examples{
+d1 = data.table(x=1:3, v1=1L, key="x")
+d2 = data.table(y=3:1, v2=2L, key="y")
+d3 = data.table(z=2:4, v3=3L)
+cbindlist(list(d1, d2, d3))
+cbindlist(list(d1, d1))
+d4 = setcbindlist(list(d1))
+d4[, v1:=2L]
+identical(d4, d1)
+}
+\keyword{ data }
diff --git a/src/data.table.h b/src/data.table.h
@@ -298,6 +298,9 @@ SEXP substitute_call_arg_namesR(SEXP expr, SEXP env);
 //negate.c
 SEXP notchin(SEXP x, SEXP table);
 
+// mergelist.c
+SEXP cbindlist(SEXP x, SEXP copyArg);
+
 // functions called from R level .Call/.External and registered in init.c
 // these now live here to pass -Wstrict-prototypes, #5477
 // all arguments must be SEXP since they are called from R level
diff --git a/src/init.c b/src/init.c
@@ -149,6 +149,7 @@ R_CallMethodDef callMethods[] = {
 {"CstartsWithAny", (DL_FUNC)&startsWithAny, -1},
 {"CconvertDate", (DL_FUNC)&convertDate, -1},
 {"Cnotchin", (DL_FUNC)&notchin, -1},
+{"Ccbindlist", (DL_FUNC) &cbindlist, -1},
 {"Cwarn_matrix_column_r", (DL_FUNC)&warn_matrix_column_r, -1},
 {NULL, NULL, 0}
 };
diff --git a/src/mergelist.c b/src/mergelist.c
@@ -0,0 +1,78 @@
+#include "data.table.h"
+
+void mergeIndexAttrib(SEXP to, SEXP from) {
+  if (!isInteger(to) || LENGTH(to)!=0)
+    internal_error(__func__, "'to' must be integer() already"); // # nocov
+  if (isNull(from))
+    return;
+  SEXP t = ATTRIB(to), f = ATTRIB(from);
+  if (isNull(t)) // target has no attributes -> overwrite
+    SET_ATTRIB(to, shallow_duplicate(f));
+  else {
+    for (t = ATTRIB(to); CDR(t) != R_NilValue; t = CDR(t)); // traverse to end of attributes list of to
+    SETCDR(t, shallow_duplicate(f));
+  }
+}
+
+SEXP cbindlist(SEXP x, SEXP copyArg) {
+  if (!isNewList(x) || isFrame(x))
+    error(_("'%s' must be a list"), "x");
+  bool copy = (bool)LOGICAL(copyArg)[0];
+  const bool verbose = GetVerbose();
+  double tic = 0;
+  if (verbose)
+    tic = omp_get_wtime();
+  int nx = length(x), nans = 0, nr = -1, *nnx = (int*)R_alloc(nx, sizeof(int));
+  bool recycle = false;
+  for (int i=0; i<nx; ++i) {
+    SEXP thisx = VECTOR_ELT(x, i);
+    if (!perhapsDataTable(thisx))
+      error(_("Element %d of 'l' list is not a data.table."), i+1);
+    nnx[i] = n_columns(thisx);
+    if (!nnx[i])
+      continue;
+    int thisnr = n_rows(thisx);
+    if (nr < 0) // first (non-zero length table) iteration
+      nr = thisnr;
+    else if (nr != thisnr) {
+      if (!copy)
+        error(_("For copy=FALSE all non-empty tables in 'l' have to have the same number of rows, but l[[%d]] has %d rows which differs from the previous non-zero number of rows (%d)."), i+1, thisnr, nr);
+      recycle = true;
+    }
+    nans += nnx[i];
+  }
+  if (recycle)
+    error(_("Recycling rows is not yet implemented.")); // dont we have a routines for that already somewhere?
+  SEXP ans = PROTECT(allocVector(VECSXP, nans));
+  SEXP index = PROTECT(allocVector(INTSXP, 0));
+  SEXP key = R_NilValue;
+  setAttrib(ans, sym_index, index);
+  SEXP names = PROTECT(allocVector(STRSXP, nans));
+  for (int i=0, ians=0; i<nx; ++i) {
+    int protecti =0;
+    SEXP thisx = VECTOR_ELT(x, i);
+    SEXP thisnames = PROTECT(getAttrib(thisx, R_NamesSymbol)); protecti++;
+    for (int j=0; j<nnx[i]; ++j, ++ians) {
+      SEXP thisxcol;
+      if (copy) {
+        thisxcol = PROTECT(duplicate(VECTOR_ELT(thisx, j))); protecti++;
+      } else {
+        thisxcol = VECTOR_ELT(thisx, j);
+      }
+      SET_VECTOR_ELT(ans, ians, thisxcol);
+      SET_STRING_ELT(names, ians, STRING_ELT(thisnames, j));
+    }
+    mergeIndexAttrib(index, getAttrib(thisx, sym_index));
+    if (isNull(key)) // first key is retained
+      key = getAttrib(thisx, sym_sorted);
+    UNPROTECT(protecti); // thisnames, thisxcol
+  }
+  if (isNull(ATTRIB(index)))
+    setAttrib(ans, sym_index, R_NilValue);
+  setAttrib(ans, R_NamesSymbol, names);
+  setAttrib(ans, sym_sorted, key);
+  if (verbose)
+    Rprintf(_("cbindlist: took %.3fs\n"), omp_get_wtime()-tic);
+  UNPROTECT(3); // ans, index, names
+  return ans;
+}
diff --git a/tests/mergelist.R b/tests/mergelist.R
@@ -0,0 +1,2 @@
+require(data.table)
+test.data.table(script="mergelist.Rraw")

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+require(data.table)`
	`2`	`+test.data.table(script="mergelist.Rraw")`