-
Notifications
You must be signed in to change notification settings - Fork 1
Strategic decoupling
Strategic decoupling is the practice of explicitly writing stable intermediate results of a model to disk, and letting subsequent runs read those results instead of recalculating them. It is the current answer to a need that the former CalcCache (an automatic disk cache, retired since the GeoDMS 8 series) used to address, but with a deliberate design difference: you choose what persists, and that choice is part of the configuration.
- Iteration speed: while developing or calibrating one part of a model, the parts you are not touching are read from disk instead of recalculated. Iterating on a national model then takes minutes, not hours.
- Reuse: decoupled results are written once and reused by every subsequent run, every scenario variant that shares them, and every colleague working with the same data.
- Distribution: decoupled files are ordinary, open storage formats. They can be shipped to users who want to run only the downstream part of a model, or who may not access the (private) source data behind them.
- Transparency: which items are decoupled, to which files and in which format is declared in the model text itself. Unlike ad-hoc save-calls scattered through scripts, the decoupling is visible, reviewable and versioned along with the rest of the model.
Decoupling uses the regular storage facilities of the GeoDMS (see Data Source and StorageManager):
- Declare a write-container: a container with items that reference the results to be decoupled, each configured with a StorageName pointing to the target file. Updating this container writes the files.
- Declare the read-side: items that read those same files as a data source, from which the downstream part of the model continues to calculate.
Typical formats are the native .fss format for fast reads, GeoTIFF for grid data and parquet or csv for tabular exchange with other tools.
Production models organise this with dedicated write-containers per data layer. The RSopen land-use model, for instance, decouples its base data (identical across all scenario variants) and its variant data separately, so each is built once and reused in every run thereafter; see its documentation.
Decoupling is deliberate, and so is refreshing it. There is no automatic invalidation: when source data or expressions upstream of a decoupled result change, re-running the write-container is a conscious step in the workflow. Two habits keep this manageable:
- Decouple only stable intermediates: data that changes rarely (source data integration, base layers), not the parts under active development.
- Make the refresh step explicit in the project workflow, for instance one write-container per data layer, updated whenever that layer's inputs change.
Within a session no bookkeeping is needed at all: the engine's calculation management tracks dependencies, calculates lazily and never computes the same result twice.
GeoDMS ©Object Vision BV. Source code distributed under GNU GPL-3. Documentation distributed under CC BY-SA 4.0.