Skip to content

Commit 96b9185

Browse files
OriginalCopy2Ischa
OriginalCopy2
and
Ischa
authored
Bep ischa (#7)
* Test commit * Notebooks for Historical analysis - Historical drought analyser - HBV calibrator - HBV output generator - Extra data files for historical data * Commit for calibration changes Also new files are added: - Drought_analyser.ipynb - Combine_ERA5_data.ipynb - Qcrit_recalculator.ipynb * Periodic push * FInal commit to branch The notebooks are refactored, and descriptions are added --------- Co-authored-by: Ischa <[email protected]>
1 parent 07e9ba8 commit 96b9185

14 files changed

+49008
-0
lines changed

‎book/model_loire/6123500_Q_Month.txt‎

Lines changed: 793 additions & 0 deletions
Large diffs are not rendered by default.

‎book/model_loire/CMIP_Future_prediction.ipynb‎

Lines changed: 689 additions & 0 deletions
Large diffs are not rendered by default.

‎book/model_loire/Combine_ERA5_data.ipynb‎

Lines changed: 290 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,290 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "a8f886dd-2d0d-410f-bdba-8b29ffe957e0",
6+
"metadata": {},
7+
"source": [
8+
"# Combine ERA5 dataset\n",
9+
"\n",
10+
"This notebook shows how different periods of ERA5 data are merged as one forcing file."
11+
]
12+
},
13+
{
14+
"cell_type": "markdown",
15+
"id": "0cf57816-b1e3-4c54-9e47-f35194840e62",
16+
"metadata": {},
17+
"source": [
18+
"### 1. Imports"
19+
]
20+
},
21+
{
22+
"cell_type": "code",
23+
"execution_count": 1,
24+
"id": "54a100d8-1423-4afb-99ad-0c1a02a9c857",
25+
"metadata": {},
26+
"outputs": [],
27+
"source": [
28+
"# general python\n",
29+
"import warnings\n",
30+
"warnings.filterwarnings(\"ignore\", category=UserWarning)\n",
31+
"\n",
32+
"import numpy as np\n",
33+
"from pathlib import Path\n",
34+
"import pandas as pd\n",
35+
"import matplotlib.pyplot as plt\n",
36+
"import xarray as xr\n",
37+
"import geopandas as gpd\n",
38+
"import pandas as pd\n",
39+
"\n",
40+
"#niceties\n",
41+
"from rich import print\n",
42+
"\n",
43+
"# Needed\n",
44+
"from ipywidgets import IntProgress\n",
45+
"from IPython.display import display\n",
46+
"from scipy.stats import qmc\n",
47+
"import yaml"
48+
]
49+
},
50+
{
51+
"cell_type": "code",
52+
"execution_count": 2,
53+
"id": "88fd5378-4b7b-4a54-a9cc-d91e818a8b52",
54+
"metadata": {},
55+
"outputs": [],
56+
"source": [
57+
"# general eWaterCycle\n",
58+
"import ewatercycle\n",
59+
"import ewatercycle.models\n",
60+
"import ewatercycle.forcing"
61+
]
62+
},
63+
{
64+
"cell_type": "markdown",
65+
"id": "a133328d-e894-4b65-8a6c-7fb28b05e2a7",
66+
"metadata": {},
67+
"source": [
68+
"### 2. Load the datasets\n",
69+
"\n",
70+
"The ERA5 is already generated using Historical_data_analysis.ipynb and loaded into this notebook."
71+
]
72+
},
73+
{
74+
"cell_type": "code",
75+
"execution_count": 3,
76+
"id": "8b967ab2-3cfe-4ed2-81f8-c04fd420c683",
77+
"metadata": {},
78+
"outputs": [],
79+
"source": [
80+
"# Load 1990 - 1994\n",
81+
"forcing_path_ERA5_90_94 = Path.home() / \"forcing\" / \"loire_river\" / \"ERA5-90-94\" / \"work\" / \"diagnostic\" / \"script\"\n",
82+
"ERA5_forcing_9094 = ewatercycle.forcing.sources[\"LumpedMakkinkForcing\"].load(directory=forcing_path_ERA5_90_94)\n",
83+
"\n",
84+
"# Load 1995 - 2004\n",
85+
"forcing_path_ERA5_95_04 = Path.home() / \"forcing\" / \"loire_river\" / \"ERA5-95-04\" / \"work\" / \"diagnostic\" / \"script\" \n",
86+
"ERA5_forcing_9504 = ewatercycle.forcing.sources[\"LumpedMakkinkForcing\"].load(directory=forcing_path_ERA5_95_04)\n",
87+
"\n",
88+
"# Load 2005 - 2012\n",
89+
"forcing_path_ERA5_05_12 = Path.home() / \"forcing\" / \"loire_river\" / \"ERA5-5-12\" / \"work\" / \"diagnostic\" / \"script\"\n",
90+
"ERA5_forcing_0512 = ewatercycle.forcing.sources[\"LumpedMakkinkForcing\"].load(directory=forcing_path_ERA5_05_12)\n",
91+
"\n",
92+
"#Load 2013 - 2019\n",
93+
"forcing_path_ERA5_13_19 = Path.home() / \"forcing\" / \"loire_river\" / \"ERA5-13-19\" / \"work\" / \"diagnostic\" / \"script\"\n",
94+
"ERA5_forcing_1319 = ewatercycle.forcing.sources[\"LumpedMakkinkForcing\"].load(directory=forcing_path_ERA5_13_19)"
95+
]
96+
},
97+
{
98+
"cell_type": "markdown",
99+
"id": "3c1bdae9-71cd-4848-a34f-201ae8f22d38",
100+
"metadata": {},
101+
"source": [
102+
"### 3. Merge the data\n",
103+
"\n",
104+
"Each forcing (pr, tas, rsds and evspsblpot) is loaded in the combine_variable function and combined using the xarray method 'concat'."
105+
]
106+
},
107+
{
108+
"cell_type": "code",
109+
"execution_count": 4,
110+
"id": "06808055-81a9-42e0-8386-88e589a318a8",
111+
"metadata": {},
112+
"outputs": [],
113+
"source": [
114+
"# List of periods and corresponding filenames\n",
115+
"folders = [\n",
116+
" \"ERA5-90-94\",\n",
117+
" \"ERA5-95-04\",\n",
118+
" \"ERA5-5-12\",\n",
119+
" \"ERA5-13-19\"\n",
120+
"]\n",
121+
"\n",
122+
"periods = [\n",
123+
" \"1990-1994\",\n",
124+
" \"1995-2004\",\n",
125+
" \"2005-2012\",\n",
126+
" \"2013-2019\"\n",
127+
"]\n",
128+
"\n",
129+
"# Combine function\n",
130+
"def combine_variable(var_name, file):\n",
131+
" datasets = []\n",
132+
" for i, period in enumerate(periods):\n",
133+
" if var_name == 'evspsblpot':\n",
134+
" file_path = Path.home() / \"forcing\" / \"loire_river\" / folders[i] / \"work\" / \"diagnostic\" / \"script\" / file\n",
135+
" else: \n",
136+
" file_name = 'OBS6_ERA5_reanaly_1_day_'+var_name+'_'+period+\".nc\"\n",
137+
" file_path = Path.home() / \"forcing\" / \"loire_river\" / folders[i] / \"work\" / \"diagnostic\" / \"script\" / file_name\n",
138+
" datasets.append(xr.open_dataset(file_path))\n",
139+
" \n",
140+
" combined = xr.concat(datasets, dim='time')\n",
141+
" return combined"
142+
]
143+
},
144+
{
145+
"cell_type": "markdown",
146+
"id": "0c64d331-f6be-4578-9e59-6829a2684c85",
147+
"metadata": {},
148+
"source": [
149+
"### 4. Save the data\n",
150+
"\n",
151+
"The combine_variable function from Chapter 3 is used and stored in a new file directory. The combined forcings are again stored as netcdf files."
152+
]
153+
},
154+
{
155+
"cell_type": "code",
156+
"execution_count": 5,
157+
"id": "cb14862e-5a11-4703-8465-82015a9f9114",
158+
"metadata": {},
159+
"outputs": [],
160+
"source": [
161+
"# Combine each variable\n",
162+
"combined_pr = combine_variable('pr', '_')\n",
163+
"combined_tas = combine_variable('tas', '_')\n",
164+
"combined_rsds = combine_variable('rsds', '_')\n",
165+
"combined_evspsblpot = combine_variable('evspsblpot', 'Derived_Makkink_evspsblpot.nc')\n",
166+
"\n",
167+
"# Save combined datasets\n",
168+
"output_path = Path.home() / \"forcing\" / \"loire_river\" / \"ERA5-90-19\"\n",
169+
"output_path.mkdir(parents=True, exist_ok=True)\n",
170+
"\n",
171+
"combined_pr.to_netcdf(output_path / \"combined_ERA5_1990_2019_pr.nc\")\n",
172+
"combined_tas.to_netcdf(output_path / \"combined_ERA5_1990_2019_tas.nc\")\n",
173+
"combined_rsds.to_netcdf(output_path / \"combined_ERA5_1990_2019_rsds.nc\")\n",
174+
"combined_evspsblpot.to_netcdf(output_path / \"combined_ERA5_1990_2019_evspsblpot.nc\")"
175+
]
176+
},
177+
{
178+
"cell_type": "markdown",
179+
"id": "82c97506-25b3-426b-9e2c-45884f1d9111",
180+
"metadata": {},
181+
"source": [
182+
"### 5. Create new yaml file \n",
183+
"\n",
184+
"A new yaml file needs to be added to the new directory in order make it readable by the eWaterCycle methods."
185+
]
186+
},
187+
{
188+
"cell_type": "code",
189+
"execution_count": 6,
190+
"id": "149f8c1b-9981-4a68-85f7-72e603274b84",
191+
"metadata": {},
192+
"outputs": [],
193+
"source": [
194+
"# Create yaml \n",
195+
"forcing_yaml = {\n",
196+
" 'start_time': '1990-01-01T00:00:00Z',\n",
197+
" 'end_time': '2019-12-31T23:59:59Z',\n",
198+
" 'shape': 'estreams_cb_FR003882.shp',\n",
199+
" 'filenames': {\n",
200+
" 'pr': 'combined_ERA5_1990_2019_pr.nc',\n",
201+
" 'tas': 'combined_ERA5_1990_2019_tas.nc',\n",
202+
" 'rsds': 'combined_ERA5_1990_2019_rsds.nc',\n",
203+
" 'evspsblpot': 'combined_ERA5_1990_2019_evspsblpot.nc'\n",
204+
" }\n",
205+
"}\n",
206+
"\n",
207+
"# Save the YAML file\n",
208+
"yaml_file_path = output_path / \"ewatercycle_forcing.yaml\"\n",
209+
"with open(yaml_file_path, 'w') as yaml_file:\n",
210+
" yaml.dump(forcing_yaml, yaml_file, default_flow_style=False)"
211+
]
212+
},
213+
{
214+
"cell_type": "markdown",
215+
"id": "a0aa73af-539e-457c-bdef-948a63338c48",
216+
"metadata": {},
217+
"source": [
218+
"### 6. Check if it works"
219+
]
220+
},
221+
{
222+
"cell_type": "code",
223+
"execution_count": 7,
224+
"id": "36cb51d8-30db-4fd9-81c8-0cb3f9443a4e",
225+
"metadata": {},
226+
"outputs": [
227+
{
228+
"data": {
229+
"text/html": [
230+
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">LumpedMakkinkForcing</span><span style=\"font-weight: bold\">(</span>\n",
231+
" <span style=\"color: #808000; text-decoration-color: #808000\">start_time</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'1990-01-01T00:00:00Z'</span>,\n",
232+
" <span style=\"color: #808000; text-decoration-color: #808000\">end_time</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'2019-12-31T23:59:59Z'</span>,\n",
233+
" <span style=\"color: #808000; text-decoration-color: #808000\">directory</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">PosixPath</span><span style=\"font-weight: bold\">(</span><span style=\"color: #008000; text-decoration-color: #008000\">'/home/ischa/forcing/loire_river/ERA5-90-19'</span><span style=\"font-weight: bold\">)</span>,\n",
234+
" <span style=\"color: #808000; text-decoration-color: #808000\">shape</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">PosixPath</span><span style=\"font-weight: bold\">(</span><span style=\"color: #008000; text-decoration-color: #008000\">'/home/ischa/forcing/loire_river/ERA5-90-19/estreams_cb_FR003882.shp'</span><span style=\"font-weight: bold\">)</span>,\n",
235+
" <span style=\"color: #808000; text-decoration-color: #808000\">filenames</span>=<span style=\"font-weight: bold\">{</span>\n",
236+
" <span style=\"color: #008000; text-decoration-color: #008000\">'evspsblpot'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'combined_ERA5_1990_2019_evspsblpot.nc'</span>,\n",
237+
" <span style=\"color: #008000; text-decoration-color: #008000\">'pr'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'combined_ERA5_1990_2019_pr.nc'</span>,\n",
238+
" <span style=\"color: #008000; text-decoration-color: #008000\">'rsds'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'combined_ERA5_1990_2019_rsds.nc'</span>,\n",
239+
" <span style=\"color: #008000; text-decoration-color: #008000\">'tas'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'combined_ERA5_1990_2019_tas.nc'</span>\n",
240+
" <span style=\"font-weight: bold\">}</span>\n",
241+
"<span style=\"font-weight: bold\">)</span>\n",
242+
"</pre>\n"
243+
],
244+
"text/plain": [
245+
"\u001b[1;35mLumpedMakkinkForcing\u001b[0m\u001b[1m(\u001b[0m\n",
246+
" \u001b[33mstart_time\u001b[0m=\u001b[32m'1990-01-01T00:00:00Z'\u001b[0m,\n",
247+
" \u001b[33mend_time\u001b[0m=\u001b[32m'2019-12-31T23:59:59Z'\u001b[0m,\n",
248+
" \u001b[33mdirectory\u001b[0m=\u001b[1;35mPosixPath\u001b[0m\u001b[1m(\u001b[0m\u001b[32m'/home/ischa/forcing/loire_river/ERA5-90-19'\u001b[0m\u001b[1m)\u001b[0m,\n",
249+
" \u001b[33mshape\u001b[0m=\u001b[1;35mPosixPath\u001b[0m\u001b[1m(\u001b[0m\u001b[32m'/home/ischa/forcing/loire_river/ERA5-90-19/estreams_cb_FR003882.shp'\u001b[0m\u001b[1m)\u001b[0m,\n",
250+
" \u001b[33mfilenames\u001b[0m=\u001b[1m{\u001b[0m\n",
251+
" \u001b[32m'evspsblpot'\u001b[0m: \u001b[32m'combined_ERA5_1990_2019_evspsblpot.nc'\u001b[0m,\n",
252+
" \u001b[32m'pr'\u001b[0m: \u001b[32m'combined_ERA5_1990_2019_pr.nc'\u001b[0m,\n",
253+
" \u001b[32m'rsds'\u001b[0m: \u001b[32m'combined_ERA5_1990_2019_rsds.nc'\u001b[0m,\n",
254+
" \u001b[32m'tas'\u001b[0m: \u001b[32m'combined_ERA5_1990_2019_tas.nc'\u001b[0m\n",
255+
" \u001b[1m}\u001b[0m\n",
256+
"\u001b[1m)\u001b[0m\n"
257+
]
258+
},
259+
"metadata": {},
260+
"output_type": "display_data"
261+
}
262+
],
263+
"source": [
264+
"forcing_combined = ewatercycle.forcing.sources[\"LumpedMakkinkForcing\"].load(directory=output_path)\n",
265+
"print(forcing_combined)"
266+
]
267+
}
268+
],
269+
"metadata": {
270+
"kernelspec": {
271+
"display_name": "Python 3 (ipykernel)",
272+
"language": "python",
273+
"name": "python3"
274+
},
275+
"language_info": {
276+
"codemirror_mode": {
277+
"name": "ipython",
278+
"version": 3
279+
},
280+
"file_extension": ".py",
281+
"mimetype": "text/x-python",
282+
"name": "python",
283+
"nbconvert_exporter": "python",
284+
"pygments_lexer": "ipython3",
285+
"version": "3.12.7"
286+
}
287+
},
288+
"nbformat": 4,
289+
"nbformat_minor": 5
290+
}

0 commit comments

Comments
 (0)