Skip to content

CMORise CCI Soil Moisture Gapfilled product #4281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
morobking wants to merge 4 commits into
base: main
Choose a base branch
from
+42 −7
Open

CMORise CCI Soil Moisture Gapfilled product #4281

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b10415e
coordinate issue fixed
Nov 25, 2025
35e9c88
Working version for gapfilled SM. need to tidy ready for PR
Dec 9, 2025
1fba34c
update after style check
Dec 10, 2025
93ff8c3
pre commit run
Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 22 additions & 3 deletions esmvaltool/cmorizers/data/cmor_config/ESACCI-SOILMOISTURE.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,35 @@ attributes:
modeling_realm: sat
institution: 'TU Wien (AUT); VanderSat B.V. (NL); Planet Labs (NL); CESBIO (FR), EODC Gmbh (AUT)'
reference: 'esacci-soilmoisture'
comment: ''

# COMBINED Attributes
source: 'ftp://anon-ftp.ceda.ac.uk/neodc/esacci/soil_moisture/data/'
title: 'ESA CCI Soil Moisture'
version: 'L3S-SSMV-COMBINED-v08.1'
comment: ''

# GAPFILLED Attributes
version 'L3S-SSMV-COMBINED_GAPFILLED-fv09.1'
Copy link
Contributor

@axel-lauer axel-lauer Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
version 'L3S-SSMV-COMBINED_GAPFILLED-fv09.1'
version: 'L3S-SSMV-COMBINED_GAPFILLED-fv09.1'

source: 'https://researchdata.tuwien.ac.at/records/s5j4q-rpd32'
title: 'ESA CCI Soil Moisture'

variables:
# sm:
# mip: Eday
# raw: sm
# filename: ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED-{year}????000000-fv08.1.nc
# smStderr:
# mip: Eday
# raw: sm_uncertainty
# filename: ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED-{year}????000000-fv08.1.nc

sm:
mip: Eday
raw: sm
filename: ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED-{year}????000000-fv08.1.nc
gapfilled: True
filename: ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED_GAPFILLED-{year}????000000-fv09.1.nc
smStderr:
mip: Eday
raw: sm_uncertainty
filename: ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED-{year}????000000-fv08.1.nc
gapfilled: True
filename: ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED_GAPFILLED-{year}????000000-fv09.1.nc
24 changes: 20 additions & 4 deletions esmvaltool/cmorizers/data/formatters/datasets/esacci_soilmoisture.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ def fix_coords(cube):
cube: iris.cube.Cube
data cube with fixed coordinates.
"""
# This makes fix_dim_coordnames work with GAPFILLED
try:
cube.coord("lat").standard_name = "latitude"
except iris.exceptions.CoordinateNotFoundError:
pass
try:
cube.coord("lon").standard_name = "longitude"
except iris.exceptions.CoordinateNotFoundError:
pass

# First fix any completely missing coord var names
fix_dim_coordnames(cube)

Expand Down Expand Up @@ -86,14 +96,18 @@ def fix_coords(cube):
return cube


def extract_variable(raw_info):
"""Extract variables."""
def extract_variable(raw_info, is_gapfilled):
"""Extract variables.
is_gapfilled is True or False to determine if the StdErr is going to
load as an ancillary variable or a seperate cube.
"""
rawvar = raw_info["name"]
constraint = iris.Constraint(name=rawvar)
if rawvar == "sm_uncertainty":
if rawvar == "sm_uncertainty" and not is_gapfilled:
sm_cube = iris.load_cube(
raw_info["file"], iris.NameConstraint(var_name="sm")
)

ancillary_var = sm_cube.ancillary_variable(
"Volumetric Soil Moisture Uncertainty"
)
Expand All @@ -102,6 +116,7 @@ def extract_variable(raw_info):
cube = iris.load_cube(raw_info["file"], constraint)

# Remove dysfunctional ancillary data without standard names

for ancillary_variable in cube.ancillary_variables():
cube.remove_ancillary_variable(ancillary_variable)

Expand All @@ -126,6 +141,7 @@ def cmorization(in_dir, out_dir, cfg, cfg_user, start_date, end_date):
var_info = cfg["cmor_table"].get_variable(vals["mip"], var_name)
glob_attrs["mip"] = vals["mip"]
raw_info = {"name": vals["raw"]}
is_gapfilled = vals["gapfilled"]
inpfile_pattern = os.path.join(in_dir, vals["filename"])
logger.info(
"CMORizing var %s from file type %s", var_name, inpfile_pattern
Expand All @@ -136,7 +152,7 @@ def cmorization(in_dir, out_dir, cfg, cfg_user, start_date, end_date):
inpfiles = sorted(glob.glob(year_inpfile_pattern))
for inpfile in inpfiles:
raw_info["file"] = inpfile
cube = extract_variable(raw_info)
cube = extract_variable(raw_info, is_gapfilled)
all_data_cubes.append(cube)
final_cube = concatenate(all_data_cubes)
fix_var_metadata(final_cube, var_info)
Expand Down