The algorithm

The general weight

The framework computes a weighting point WP for each input variable as the product of its relevance factor (RF, an integer 1–5 published in the paper’s Tables 1 and 2) and a function of how the variable was actually measured: number of observations, layers, replications, depth, distance from the meteorological station, and elevation difference.

For the weather block specifically, two penalties apply when the nearest station is far from the field. A distance penalty reduces the weight proportionally past an optimal cut-off (1 km for precipitation, 5 km for temperature and humidity, 10 km for wind, 20 km for radiation). An altitude penalty kicks in once the station and the field differ by more than 30 m:

Per-item formulas

Most blocks just sum observations × weight: management, phenology, previous crop, site, and the “observation” state variables (lodging, pests, damages, weeds) work this way.

Three blocks have richer formulas. Initial values rewards both depth and observation count:

Soil items split into two methods by depth scope. Full-profile measurements (texture, bulk density, water-retention curve, hydraulic-conductivity curve, field capacity, wilting point) use Method 1 with depth normalised by 1.5 m:

Shallow-soil measurements (organic carbon, organic nitrogen, pH) use Method 2 with depth normalised by 0.3 m:

State variables have one bespoke formula per crop / soil / flux item — twenty-three in total. They share a family resemblance — clipped-min terms for layers, observations, and replicates, multiplied by the weight — but each one’s denominator is chosen for the typical measurement frequency of that variable. See backend/engine/scoring.py for the full set.

Tier thresholds

Block totals are cut into tiers at fixed values. The thresholds below are what the shipped 2014 binary uses and what the web port reproduces — see the discrepancy note below the table.

The seasons multiplier

Treatments, sites, and seasons aren’t a tier-classified block of their own — they instead produce a single sum that multiplies the dataset’s raw block total to give an adjusted total. More seasons / treatments increases the multiplier (up to a cap of 1.3); a sparser experiment decreases it. Paper Table 3:

Discrepancies between paper and shipped binary

The published paper and the shipped 2014 Windows binary disagree in nine places. The web port is faithful to the shipped binary by default (version key 2015-kersebaum in backend/engine/weights.yaml). Listed here so anyone reading both knows which is which:

• Overall tier ladder — paper: Platinum ≥ 173 / Gold ≥ 135 / Silver ≥ 90 / Copper ≥ 60; binary: Platinum ≥ 180 / Gold ≥ 145 / same Silver / same Copper.
• Multiplier clamp — paper caps the seasons sum at 28 (so the multiplier maxes at 1.3); the binary has no clamp.
• Slope / exposition — paper Table 1 includes it (RF 1); recovered from the binary’s Münchenberg example, which scores it as obs × weight.
• Leaf wetness & soil temperature — in the binary’s weather block but not in the paper.
• Deep percolation flux & CO₂ flux weights — paper says RF 3; binary defaults to 2.
• Weather optimal-distance defaults — the paper specifies per-variable values (1, 5, 10, 20 km); the binary expects the user to enter them.
• Altitude-penalty factor — paper Eq. 2 uses 1.0; the binary’s default is 0.5.

A paper-faithful weights version is now shipped as 2015-kersebaum-paper. The Müncheberg example shifts from Gold (binary defaults) to Platinum under it — ≈174.4 adjusted total, still comfortably above the paper’s 173 threshold even after dropping soilTemp (−2 raw). To use the paper variant, pick it from the /score page’s weights-version dropdown, or pass weightsVersion: "2015-kersebaum-paper" in the POST /api/score body.

Importing AgMIP / ICASA ACE files

DataRanking accepts uploads in the AgMIP ACE crop-experiment format directly — both raw JSON (.json) and gzipped (.aceb). On the /score page the Upload ACE button does the conversion server-side via POST /api/score/ace and shows you the converted DataRanking dataset alongside the score — you can edit and re-score without re-uploading.

Multi-experiment. ACE files routinely carry multiple experiments sharing the same weather and soil records. DataRanking scores each experiment separately and returns a list of results. The web UI currently shows only the first; a multi-experiment switcher is on the roadmap.

Best-effort mapping. ACE fields map onto DataRanking sub-items where the alignment is obvious; absent data is silently skipped (block-tier-average classification is the right mode for partial inputs — an absent previousCrop doesn’t score zero, it’s just not counted toward the tier average). High-level mapping:

• site ← fl_lat, fl_long, flele, flsl.
• weather ← presence of dailyWeather[].rain / tmax / tmin / rhumd / wind / srad in the linked weather record. Distance is the haversine between the field and the station, in km.
• soil ← per-layer presence of slbdm (bulk density), sloc (organic carbon, 0–30 cm), slphw (pH), sldul / slll (water-retention LL/DUL), slcly (texture), etc. in the linked soil record. cOrg, nOrg, and ph use the 0–30 cm soil method; the rest use the full-profile method.
• initialValues ← initial_conditions.soilLayer[].ich2o, icno3, icnh4.
• previousCrop ← initial_conditions.icpcr, icdat, icrag/icrn (residue management).
• management ← management.events[] filtered by event ∈ planting / harvest / fertilizer / irrigation / tillage. Variety and seed-density come from the planting event’s cul_name and plpop.
• phenology ← observed.pldae (emergence), adat (anthesis/flowering), mdat (maturity).
• stateVariables ← observed.hwah (yield), cwam + timeSeries[].cwad (above-ground biomass), laix + timeSeries[].laid (LAI), and the soil time series (swld, nosld, nhsld).
• seasons ← inferred from how many experiments in the same file share the same field and planting crop. Standard ACE doesn’t carry the paper’s explicit treatments-and-seasons block; users should override if their experiment has a richer design.

The data/examples/ace/ directory in the repo ships three BSD-3 demonstration files from the AgMIP json-translation-samples repository — maize Machakos, hard-spring-cereal wheat, rice paddy — that you can upload to see the importer in action.

Two classification methods

Paper p. 404 documents two ways to derive an overall tier from the eight per-block tiers. DataRanking computes both on every score and the /score page displays them side by side — when they disagree, that’s itself informative about how balanced the dataset is.

1. Total sum (primary — shown as the big tier card on the result page). Block points add up, the seasons multiplier is applied, and the result is bucketed via the overall ladder shown above. This is what the shipped 2014 binary uses.
2. Block-tier average (the “second-opinion” callout). Per-block tiers are mapped to integers (NA = 0, Copper = 1, Silver = 2, Gold = 3, Platinum = 4), averaged across all eight blocks, then rounded to a final tier via round half up: ≥ 3.5 → Platinum, ≥ 2.5 → Gold, ≥ 1.5 → Silver, ≥ 0.5 → Copper, otherwise NA. The seasons multiplier does not apply here — it’s an average over per-block classifications, not over raw points.

Worked example — Müncheberg under the binary defaults scores Gold by both methods: total sum 176.58, average (4+3+0+3+4+4+3+3)/8 = 3.0 → Gold. Under the paper version it scores Platinum (174.41) by total sum but Gold (3.0) by average — the per-block tiers are unchanged but the overall ladder differs. The disagreement honestly reflects that Müncheberg has three Platinum blocks pulling the total score up while the previousCrop NA pulls the average down.

2026-modern — the v2 extension experimental

2026-modern extends the 2015 framework with three new content blocks, a cross-cutting axis, a calibration-level mapping, and a tier-aware uncertainty band. Critically, it preserves the 2015 verdict: the original 8 blocks are scored byte-identically to 2015-kersebaum, so a dataset's tier_2015 under 2026-modern matches its overallTier under the canonical version. The v2 work adds a parallel tier_2026_composite reported alongside.

New blocks

Remote sensing

Six criteria covering spatial resolution, revisit cadence, cloud-mask quality, ground-truth pairing, variables retrieved (LAI / biomass / canopy), and retrieval-algorithm transparency. Anchored to Levitan, Char & Tucker (2019) and Boote et al. (2016).

In-situ sensor / IoT

Six criteria covering temporal resolution, sensor calibration, gap-filling method, variables captured (soil moisture / soil temperature / ET), spatial sampling, and data accessibility. Anchored to Wallach et al. (2021) and Janssen et al. (2017).

Genetic / cultivar

Four criteria covering cultivar identity (pedigree / SNP accession / name only), canonical model coefficients, genetic data depth, and phenotypic-trial linkage. Anchored to Hoogenboom et al. (2019) and Boote et al. (2016); two sub-criteria still need final citation anchors.

FAIR cross-cutting axis

Every block (original 8 + new 3) carries a FAIR sub-axis scored across Findability, Accessibility, Interoperability, and Reusability (Wilkinson et al. 2016). The block's FAIR tier produces a multiplier (Platinum=1.00, Gold=0.85, Silver=0.70, Copper=0.50, NA=0.50) applied to its raw contribution before the composite total is summed.

Wallach 2024 supportable-groups mapping

Each score also reports which Wallach 2024 calibration groups the dataset can support. Wallach et al. (2024) — the AgMIP Phase IV protocol — defines an 8-step procedure that calibrates variables in groups (phenology, biomass, final values, soil-water, soil-N) rather than a single “level” taxonomy. The dataset-side question the protocol poses is therefore which groups the dataset supports, not which numeric level it has reached.

For each group, the engine evaluates the relevant per-block tiers and checks whether the underlying sub-criteria are populated. The result is a row of supports / does-not-support flags surfaced alongside the tier verdict on /score.

Uncertainty band

Every v2 score includes a tier-aware uncertainty band: 10,000-sample Monte-Carlo resampling of per-block tier verdicts under a Dirichlet prior (concentration 0.85). The result is a p05 / p50 / p95 band around both tier_2015 and tier_2026_composite, so users see where the verdict is robust and where it sits on a tier boundary.

Snapshot URLs and citability

Every v2 score is anchored to an immutable URL of the form /score/{id}/v/{hash} where hash is the SHA256 over (payload ‖ weights_version). Identical inputs always produce the same URL; any revision yields a new one. Major 2026-modern releases are also deposited to Zenodo for a stable DOI.

Full per-criterion citation anchors live in the repo at docs/v2/lit-review/; per-criterion weights and tier-cut thresholds live in backend/engine/weights.yaml under the "2026-modern" key.