Skip to content

Fix dense [<- writes and support [i, j, k] and selected_ranges() - #896

Merged
mojaveazure merged 2 commits into
mainfrom
fix/dense-assign-nd
Oct 26, 2026
Merged

mojaveazure merged 2 commits into
mainfrom
fix/dense-assign-nd

Conversation

@mojaveazure

Copy link
Copy Markdown
Member

Fixes the dense [<- write bugs reported in #877, and makes dense writes work for 3D+ arrays.

Bugs fixed

ncol Before, [i, j] Before, selected_ranges() Now
1 silent no-op silent no-op written
2 written error: buffer sizes (writes full domain) written
3 error: "does not contain all required… columns" same error written
4 written written (only because it fills the domain) written
  • ncol = 1: the single-attribute flatten step required ncol > 1, but its outer if still matched, so the list case was skipped as well. The value kept the name V1, the name check failed, and the if had no else, so the method returned without writing. A name mismatch is now an error.
  • ncol = 3: the data.frame-of-dimensions-and-attributes check matched on column count alone (2 dims + 1 attr = 3). For dense arrays it now also requires a matching column name.
  • Subarray: was as.integer(c(range(i), range(j))), set only when both i and j were given and there was a single attribute. It was 2D and INT32 only, ignored selected_ranges(), and widened non-contiguous indices. Dense writes with multiple attributes never set a subarray.

New behavior

  • arr[i, j, k] <- value on dense arrays, capturing k the same way [ does (nd_index_from_syscall()). A 4th index is an error, not a message as on the read side, because ignoring it would write to the wrong cells. Sparse writes still use i and j only.
  • Dense writes honor selected_ranges(), named or unnamed, for any number of dimensions. Dimensions with no selection span their full domain.
  • The subarray is typed per dimension (libtiledb_subarray_add_range_with_type()), so INT64 and datetime dimensions work.
  • nd_index_from_syscall() no longer evaluates value a second time.

Behavior changes (in NEWS and ?"[<-,tiledb_array-method")

These are now errors:

  • dense indices that aren't contiguous and increasing (they used to be widened to their range);
  • an index for a dimension the array lacks, e.g. a third index on a 2D array, which used to be silently dropped;
  • more than one selected range for a dimension;
  • selected_points() on a write;
  • a value that doesn't match the array (used to be a silent no-op).

Also: selected_ranges() left on an array object from an earlier read now apply to writes through it.

Known limitation

A dimension with no index or selected range spans its full domain, not its current domain. This only matters for dense arrays that have a current domain set.

Testing

  • New inst/tinytest/test_densewrite.R (32 tests):

    • 2D with 1 to 4 columns, for both [i, j] and selected_ranges();
    • an offset block;
    • 3D [i, j, k] and [i, , k], and an array value;
    • named, out-of-order selected_ranges() on a 4D array;
    • 1D, INT64 dimensions, and multiple attributes;
    • every new error.

    It's a separate file because test_tiledbarray.R calls exit_file() partway through on macOS and r-universe.

  • Full suite: 1790 tests, 1 failure (test_config_profile.R, rest.token). The same failure occurs on main (1758 tests, 1 failure). This was run against the R changes loaded into an installed 0.34.0 build, not a fresh compile; CI will cover that.

Closes #877

🤖 Generated with Claude Code

Dense `[<-` silently wrote nothing for one-column right-hand sides,
rejected three-column ones as dimension/attribute columns, ignored
`selected_ranges()`, and built an INT32-only 2-d subarray from `range(i)`
and `range(j)`. Writes now build a typed subarray for any number of
dimensions from `[i, j, k]` or `selected_ranges()`, matching `[`.

Closes #877

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mojaveazure
mojaveazure requested a review from jp-dark October 6, 2026 19:52
@jp-dark
jp-dark requested review from jdblischak and removed request for jp-dark October 7, 2026 13:48

@jdblischak jdblischak left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mojaveazure!

I recommend bumping the version in DESCRIPTION to 0.34.0.1

I confirmed that this PR fixes the errors from the example provided in #877 (comment)

git log -n 1 --oneline
## 43cabd22 (HEAD -> fix/dense-assign-nd) Fix dense `[<-` writes and support `[i, j, k]` and `selected_ranges()`
docker run --rm -it -v $(pwd):/TileDB-R rocker/r2u:latest bash
cd TileDB-R/
apt update -qq && apt install --yes --no-install-recommends cmake curl git
Rscript -e 'remotes::install_deps(".", dependencies=TRUE)'
R CMD build --no-build-vignettes --no-manual .
R CMD INSTALL --use-C17 tiledb_0.34.0.tar.gz
library("tiledb")
## TileDB R 0.34.0 with TileDB Embedded 2.30.0 on Ubuntu 24.04.4 LTS.
## See https://tiledb.com for more information about TileDB.

tmp <- tempfile()
dom <- tiledb_domain(dims = list(
    tiledb_dim("d1", c(1L, 4L), 4L, type = "INT32"),
    tiledb_dim("d2", c(1L, 3L), 3L, type = "INT32")
))
schema <- tiledb_array_schema(
    dom,
    attrs = list(tiledb_attr("x", type = "FLOAT64")),
    sparse = FALSE
)
tiledb_array_create(tmp, schema)

# ncol=1: no more silent failure
arr <- tiledb_array(tmp, query_type = "WRITE")
selected_ranges(arr) <- list(cbind(1, 4), cbind(1, 1))
arr[] <- matrix(runif(4), 4, 1)
tiledb_array_close(arr)
arr <- tiledb_array(tmp, query_type = "READ")
arr[1:4, 1:3]$x
## [1] 0.9549284 0.1356648 0.5171040 0.3093992       NaN       NaN       NaN
## [8]       NaN       NaN       NaN       NaN       NaN
tiledb_array_close(arr)

# ncol=3: no more error
arr2 <- tiledb_array(tmp, query_type = "WRITE")
arr2[1:4, 1:3] <- matrix(runif(12), 4, 3)
arr2[1:4, 1:3]$x
## [1] 0.35634239 0.13263939 0.45245257 0.03139209 0.38876229 0.27229030
## [7] 0.39942395 0.25701818 0.37835856 0.88043968 0.93553177 0.50401161
tiledb_array_close(arr2)

sessionInfo()
## R version 4.6.1 (2026-06-24)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.4 LTS
##
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0  LAPACK version 3.12.0
##
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## time zone: UTC
## tzcode source: system (glibc)
##
## attached base packages:
## [1] stats     graphics  grDevices datasets  utils     methods   base
##
## other attached packages:
## [1] RcppSpdlog_0.0.29 tiledb_0.34.0
##
## loaded via a namespace (and not attached):
##  [1] zoo_1.9-1           bit_4.6.0           compiler_4.6.1
##  [4] tools_4.6.1         RcppCCTZ_0.2.14     spdl_0.0.5
##  [7] Rcpp_1.1.2          bspm_0.5.8          nanoarrow_0.9.0
## [10] bit64_4.8.6         nanotime_0.3.15     grid_4.6.1
## [13] data.table_1.18.6.1 lattice_0.23-1

@cgiachalis

cgiachalis commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Perhaps, the documentation for [<- method should state that dense writes no need dim indexes to be supplied on the right hand side object; also the error message could inform user to drop the dims - for comparison the sparse array informs with error message: "...column index has to be supplied".

Now that we got this fixed, the helper fromDataFrame for dense arrays and ingest mode can be fixed too; maybe in a follow up PR?

library(tiledb)
uri <- tempfile()
df1 <- data.frame(index = as.Date(0:1),
                  a = c(100, 120),
                  b = c(80, 120))

tiledb::fromDataFrame(df1,
                      uri,
                      col_index = 1,
                      tile_domain = c(0, 10),
                      sparse = FALSE,
                      mode = "schema_only")

arr <- tiledb::tiledb_array(uri, return_as = "data.frame")

# nope because we supplied dims
arr[0:1, ] <- df1
#> Error:
#> ! [TileDB::Query] Error: Dense write queries cannot set dimension buffers

# drop dim: index
arr[0:1, ] <- df1[, -1] # NOTE before this PR -> Error: WriterBase: Buffer sizes check failed; Invalid number of cells given for attribute 'b' (2 != 11)


arr[]
#>        index   a   b
#> 1 1970-01-01 100  80
#> 2 1970-01-02 120 120


# Using tiledb::fromDataFrame "ingest"  - fails
uri <- tempfile()
tiledb::fromDataFrame(df1,
                      uri,
                      col_index = 1,
                      tile_domain = c(0, 10),
                      sparse = FALSE,
                      mode = "ingest")
#> Error:
#> ! [TileDB::Query] Error: Unordered writes are only possible for sparse arrays

@mojaveazure
mojaveazure merged commit 9812fc4 into main Oct 26, 2026
1 check passed
@mojaveazure
mojaveazure deleted the fix/dense-assign-nd branch October 26, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dense [<- write fails for 3D arrays ("Subarray should have num_dims * 2 values")

4 participants