Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# fireSense_dataPrepPredict (development version)

* Inside one year (the reps of a hindcast, `fireTimeStep` < 1) the ignition and spread covariates are built once, not at every step. They are rebuilt when the year, the climate, `cohortData`, `pixelGroupMap`, `flammableRTM`, `landcoverDT`, `rstLCC_RTM`, the non-forest time since disturbance, `lightningMaps` (ignition) or the fitted rows (`studyAreaWithSpreadParams`: ELF, polygons, fitted parameter names) changed; inputs are compared by rlang::hash() of their values (`covariatesBuilt()`, `inputDigest()`). Same covariates; their `year` label is the first step's time. About 12 s per rep saved on a hindcast year.
* The land cover of the per-ELF fuel sets (`mod$ELFrstLCC`, `mod$ELFlandcoverDTs`) was made once per run and never refreshed, so a change of `rstLCC_RTM` or `flammableRTM` inside a run was not seen; it is now made again when either changed. `rlang` is in `reqdPkgs`.

* `dataYear` defaults to 2020 (was 2011), the default of Biomass_core, Biomass_borealDataPrep, Biomass_speciesData and burnSummaries, and may be up to 2025. `.inputObjects` stops when another module has a different `dataYear` (`SpaDES.core::paramCheckOtherMods()`). It is read only when `fireSense_dataPrepFit` does not supply the land cover, stand age and non-forest time since disturbance (a run that only predicts), so runs with `fireSense_dataPrepFit` predict as before.

* A run without `fireSense_dataPrepFit` (one that only predicts) now works: `.inputObjects` reads the stored fits of the ELFs the study area touches from the SpreadFit ledger (`predictInputsFromLedger()`, with `fireSenseUtils::readSpreadFitLedger()`), sets the per-ELF `sppEquivs`, `nonForestedLCCGroupsList` and `missingLCCgroupList`, asks canClimateData for the fits' climate variables (`climateVariables`, `climateVariablesForFire`; the 'MDC' default made the ignition covariates fail with "[subset] invalid name(s)"), and makes `lightningMaps` (`fireSenseUtils::prepare_LightningData()`). `prepIgAndEscPredictData` makes `ignitionFitRTM`, the coarse grid of the ignition covariates, unless another module supplies it (`fireSense_ignitionPredict` stopped on a NULL one). New parameters `fitFileSuffix`, `spreadFitFilename`, `spreadFitGoogleDriveFolder` (as in `fireSense_dataPrepFit`). A `studyAreaWithSpreadParams` supplied as `NULL` predicts without stored fits, as before.
Expand Down
88 changes: 79 additions & 9 deletions fireSense_dataPrepPredict.R
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ defineModule(sim, list(
"PredictiveEcology/LandR@development",
"PredictiveEcology/reproducible@development",
"PredictiveEcology/SpaDES.core@development (>= 3.0.4)",
"rlang",
"sf",
"terra"
),
Expand Down Expand Up @@ -325,13 +326,14 @@ doEvent.fireSense_dataPrepPredict <- function(sim, eventTime, eventType) {
)
},
prepIgAndEscPredictData = {
sim <- prepare_IgnitionAndEscapePredict(sim)
if (!covariatesBuilt(sim, "ignition", list(sim$lightningMaps[["lightningDays"]])))
sim <- prepare_IgnitionAndEscapePredict(sim)
sim <- scheduleEvent(sim, time(sim) + P(sim)$fireTimeStep,
"fireSense_dataPrepPredict", "prepIgAndEscPredictData"
)
},
prepSpreadPredictData = {
sim <- prepare_SpreadPredict(sim)
if (!covariatesBuilt(sim, "spread")) sim <- prepare_SpreadPredict(sim)
sim <- scheduleEvent(sim, time(sim) + P(sim)$fireTimeStep,
"fireSense_dataPrepPredict", "prepSpreadPredictData"
)
Expand Down Expand Up @@ -734,6 +736,73 @@ prepare_SpreadPredict <- function(sim) {
}


#' A digest of an input, to tell whether it changed
#'
#' `rlang::hash()` (xxhash128) of the values (a `SpatRaster`: its names and values; a table: its columns), so a change that keeps
#' the sum (two pixels swapped) is seen too.
#'
#' @param x a `SpatRaster`, table, list or `NULL`.
#' @return a character.
inputDigest <- function(x) {
if (inherits(x, "SpatRaster")) x <- if (terra::hasValues(x)) list(names(x), terra::values(x, mat = FALSE))
if (is.data.frame(x)) x <- as.list(x)
rlang::hash(x)
}

#' A stamp of the vegetation the fuel classes are made from
#'
#' Digests of `cohortData`, `pixelGroupMap` and `flammableRTM`: equal stamps mean the same inputs.
#'
#' @param sim A `simList`.
#' @return A list of digests.
vegetationStamp <- function(sim) {
lapply(list(sim$cohortData, sim$pixelGroupMap, sim$flammableRTM), inputDigest)
}

#' Are this year's covariates already built from these inputs?
#'
#' The ignition and spread covariates depend only on the year, the climate, the vegetation, the land cover,
#' the non-forest time since disturbance and the fitted rows (which ELF, which polygons, which fitted
#' parameter names, from which the fuel-class roles are read), so the steps inside one year (the reps of a
#' hindcast, `fireTimeStep` < 1) would rebuild the same tables. The stamp is of the actual inputs, not of
#' time alone: land cover and vegetation can change within a run. It is kept in `mod` per `what`; it is
#' replaced when it differs, and `FALSE` is returned so the caller builds. The rows are reduced to the own
#' ELF's first (`useOwnELFRows()`, as the builders do), so the stamp sees what the builders see.
#'
#' @param sim A `simList`.
#' @param what `"ignition"` or `"spread"`: which covariates, and so which climate variables.
#' @param extra a list of further inputs of those covariates.
#'
#' @return `TRUE` if the covariates in `sim` were built from these inputs, else `FALSE`.
covariatesBuilt <- function(sim, what, extra = list()) {
sim <- useOwnELFRows(sim)
sa <- sim$studyAreaWithSpreadParams
rows <- list(as.character(sim$.ELFind), as.character(sa[[fireSenseUtils::polygonIDTxt]]),
if (!is.null(sa$params)) lapply(sa$params, colnames))
climate <- sim$currentClimateRasters[[sim$climateVariablesForFire[[what]]]]
stamp <- list(floor(as.numeric(time(sim))), sharedInputStamp(sim), lapply(c(list(climate), extra), inputDigest), rows)
built <- sim[[c(ignition = "fireSense_igAndEscapePred_Covariates", spread = "fireSense_SpreadCovariates")[[what]]]]
if (!is.null(built) && identical(mod$covariateStamps[[what]], stamp)) return(TRUE)
mod$covariateStamps[[what]] <- stamp
FALSE
}

#' The stamp of the inputs both covariate events read, made once per step
#'
#' The ignition and the spread event of one step read the same vegetation, time since disturbance, `landcoverDT`
#' and `rstLCC_RTM`; hashing them is as costly as a rebuild on a large landscape, so the second event of a step
#' takes the first's stamp (kept in `mod`, keyed on `time(sim)`).
#'
#' @param sim A `simList`.
#' @return A list of digests.
sharedInputStamp <- function(sim) {
now <- as.numeric(time(sim))
if (!identical(mod$sharedInputStamp$time, now))
mod$sharedInputStamp <- list(time = now, stamp = list(
vegetationStamp(sim), lapply(list(sim$nonForest_timeSinceDisturbance, sim$landcoverDT, sim$rstLCC_RTM), inputDigest)))
mod$sharedInputStamp$stamp
}

#' The fuel-class tables for this year, one per fuel set, shared by the ignition and spread covariates
#'
#' `fireSenseUtils::fireSenseCovariatesCreate()` is called for ignition (through
Expand All @@ -752,13 +821,7 @@ prepare_SpreadPredict <- function(sim) {
#' @return list of `data.table`s, one per element of `fuelSets`.
fuelClassTablesThisYear <- function(sim, fuelSets) {
year <- time(sim)
stamp <- function() {
cd <- sim$cohortData
list(NROW(cd), sum(as.numeric(cd$B), na.rm = TRUE), sum(as.numeric(cd$age), na.rm = TRUE),
sum(as.numeric(cd$pixelGroup), na.rm = TRUE),
sum(terra::values(sim$pixelGroupMap, mat = FALSE), na.rm = TRUE),
sum(terra::values(sim$flammableRTM, mat = FALSE), na.rm = TRUE))
}
stamp <- function() vegetationStamp(sim)
make <- function() lapply(fuelSets, function(fs) fireSenseUtils::cohortsToFuelClasses(
cohortData = sim$cohortData,
pixelGroupMap = sim$pixelGroupMap,
Expand Down Expand Up @@ -952,6 +1015,13 @@ useOwnELFRows <- function(sim) {

ELFfuelSets <- function(sim) {
fcc <- P(sim)$fuelClassCol
## the land cover the fuel sets are made from is kept, and made again when `rstLCC_RTM` or `flammableRTM` changed
lcStamp <- lapply(list(sim$rstLCC_RTM, sim$flammableRTM), inputDigest)
if (!identical(mod$ELFlcStamp, lcStamp)) {
mod$ELFrstLCC <- NULL
mod$ELFlandcoverDTs <- NULL
mod$ELFlcStamp <- lcStamp
}
if (length(sim$sppEquivs) > 1L || length(sim$nonForestedLCCGroupsList)) {
n <- length(sim$sppEquivs)
if (length(sim$nonForestedLCCGroupsList) != n || length(sim$missingLCCgroupList) != n)
Expand Down
103 changes: 103 additions & 0 deletions tests/testthat/test-covariatesOncePerYear.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
## Inside one year (the reps of a hindcast, `fireTimeStep` < 1) the covariates do not change, so they are
## built once. A change to an input they are made from (land cover, vegetation, climate) rebuilds them.

countFuelClassCalls <- function(code) {
n <- 0L
orig <- fireSenseUtils::cohortsToFuelClasses
testthat::local_mocked_bindings(
cohortsToFuelClasses = function(...) { n <<- n + 1L; orig(...) },
.package = "fireSenseUtils")
force(code)
n
}

repTimes <- list(start = 2001, end = 2001.75)
repParams <- function(what) list(fireTimeStep = 0.25, whichModulesToPrepare = what)

for (what in c("fireSense_ignitionPredict", "fireSense_spreadPredict")) {
covName <- if (what == "fireSense_ignitionPredict") "fireSense_igAndEscapePred_Covariates" else "fireSense_SpreadCovariates"

test_that(paste("four same-year steps build the covariates once, as one step does:", what), {
n <- countFuelClassCalls(reps <- toyPrepRun(toyObjects(), times = repTimes, params = repParams(what)))
once <- toyPrepRun(toyObjects(), times = list(start = 2001, end = 2001),
params = list(whichModulesToPrepare = what))
## the tables are the first step's (their `year` label is 2001), not rebuilt at 2001.75
expect_identical(n, 1L) # was 4
expect_identical(covDF(reps[[covName]]), covDF(once[[covName]]))
})

test_that(paste("a vegetation change between steps of one year rebuilds the covariates:", what), {
sim <- SpaDES.core::spades(toyPrepSim(toyObjects(), times = list(start = 2001, end = 2001),
params = repParams(what)), debug = FALSE)
first <- data.table::copy(sim[[covName]])
sim$cohortData$B <- sim$cohortData$B * 2L
## the three more steps of the year: the first rebuilds, the others reuse it
SpaDES.core::end(sim) <- 2001.75
n <- countFuelClassCalls(sim <- SpaDES.core::spades(sim, debug = FALSE))
expect_identical(n, 1L)
expect_false(isTRUE(all.equal(covDF(first), covDF(sim[[covName]]))))
})
}

test_that("a new year rebuilds the covariates", {
n <- countFuelClassCalls(toyPrepRun(toyObjects(), times = list(start = 2001, end = 2003),
params = list(whichModulesToPrepare = "fireSense_spreadPredict")))
expect_identical(n, 3L)
})

## the first step of the year, then `change(sim)`, then the other three: the number of fuel-class builds
## they make, and the covariates before and after
stepsAfterChange <- function(objs, change, what, params = NULL) {
p <- c(list(fireTimeStep = 0.25), if (!is.null(what)) list(whichModulesToPrepare = what), params)
sim <- SpaDES.core::spades(toyPrepSim(objs, times = list(start = 2001, end = 2001), params = p), debug = FALSE)
first <- list(ig = data.table::copy(sim$fireSense_igAndEscapePred_Covariates),
sp = data.table::copy(sim$fireSense_SpreadCovariates))
sim <- change(sim)
SpaDES.core::end(sim) <- 2001.75
n <- countFuelClassCalls(sim <- SpaDES.core::spades(sim, debug = FALSE))
list(n = n, first = first, after = list(ig = sim$fireSense_igAndEscapePred_Covariates,
sp = sim$fireSense_SpreadCovariates))
}

test_that("ignition and spread prepared together build once per year, as in a hindcast", {
n <- countFuelClassCalls(reps <- toyPrepRun(toyObjects(), times = repTimes,
params = list(fireTimeStep = 0.25)))
once <- toyPrepRun(toyObjects(), times = list(start = 2001, end = 2001))
expect_identical(n, 1L) # one year, both events, was 8 events' worth of builds (2 per step, shared)
expect_identical(covDF(reps$fireSense_SpreadCovariates), covDF(once$fireSense_SpreadCovariates))
expect_identical(covDF(reps$fireSense_igAndEscapePred_Covariates), covDF(once$fireSense_igAndEscapePred_Covariates))
})

test_that("a change that keeps the sum (two pixels swapped) rebuilds", {
r <- stepsAfterChange(toyObjects(), function(sim) {
v <- terra::values(sim$pixelGroupMap, mat = FALSE)
v[c(1, 3)] <- v[c(3, 1)]
sim$pixelGroupMap <- terra::setValues(sim$pixelGroupMap, v)
sim
}, "fireSense_spreadPredict")
expect_identical(r$n, 1L)
expect_false(isTRUE(all.equal(covDF(r$first$sp), covDF(r$after$sp))))
})

test_that("a land-cover change between steps of one year rebuilds, with several ELFs too", {
o <- toyObjects()
o$landcoverDT <- NULL
o$sppEquivs <- list(
data.table::data.table(LandR = c("Pice_mar", "Pinu_ban", "Popu_tre"), FuelClass = c("class1", "class1", "class2")),
data.table::data.table(LandR = c("Pice_mar", "Pinu_ban", "Popu_tre"), FuelClass = c("Pice_mar", "Pinu_ban", "decid")))
o$nonForestedLCCGroupsList <- list(list(nfLCC_wetland = 19L, nfLCC_grass = 16L), list(nfLCC_wetgrs = c(16L, 19L)))
o$missingLCCgroupList <- list("nfLCC_grass", "nfLCC_wetgrs")
r <- stepsAfterChange(o, function(sim) { # a forest pixel and a wetland pixel swap land cover
v <- terra::values(sim$rstLCC_RTM, mat = FALSE)
v[c(1, 9)] <- v[c(9, 1)]
sim$rstLCC_RTM <- terra::setValues(sim$rstLCC_RTM, v)
sim
}, "fireSense_spreadPredict")
expect_identical(r$n, 2L) # the rebuild, one per ELF
expect_false(isTRUE(all.equal(covDF(r$first$sp), covDF(r$after$sp))))
})

test_that("a change of the simulated ELF rebuilds", {
r <- stepsAfterChange(toyObjects(), function(sim) { sim$.ELFind <- "7"; sim }, "fireSense_spreadPredict")
expect_identical(r$n, 1L)
})
Loading