PWA-hydro-conditioning-main

Author: Idil Yaktubay ([email protected]), IISD-ELA

This repository provides a workflow to hydro-condition Prairie watersheds using the hydro_condition.py script. It is designed for use with the custom PWA-hydro-conditioning-tools Python package, and requires certain datasets as input.

Repository Structure

PWA-hydro-conditioning-main/                  
├── hydro_condition_py.py       # Main script to run the hydro-conditioning pipeline
├── README.md                   # This documentation
├── hydrocon_env.yml            # Environment file for the hydro-conditioning pipeline
└── .gitignore                  # File that tells Git to ignore the "Data" folder created by the user

Prerequisites

To be able to run this pipeline, the user must do the following:

1. Install Anaconda: You can install the latest version of Anaconda here. While any recent version of Conda should work, this documentation was prepared using Conda version 24.9.2. If you encounter issues that may be related to your Conda version, please reach out to us. We recommend installing the full Anaconda distribution (rather than Miniconda), as the steps in this documentation assume a full Conda installation.

2. Configure Git:

   2(a) If you don't already have one, create a GitHub account.

   2(b) Download and install Git to your desktop from the official site.

   2(c) Connect your local machine to GitHub using SSH. This allows you to securely clone and push repositories without typing your password everytime. For step-by-step instructions, see GitHub's official documentation.

   2(d) Install and enable the Python extension in VS Code from the Extensions panel in VS Code:

   

3. Install GDAL: GDAL is a geospatial data processing software that our hydro-conditioning product utilizes. You can install GDAL by running the conda install -c conda-forge gdalcommand on Anaconda PowerShell Prompt. You can verify the install with gdalinfo --version.

Setup Instructions

1. Clone this repository and the pwa-tools repository

1.1 Open the Visual Studio Code app on your desktop. 1.2 Open a new Powershell terminal by using the toolbar on the top left corner.

Your terminal should look something like this:

(base) PS C:\Users\iyaktubay>

1.3 Clone this repository to your workspace by running the following command:

(base) PS C:\Users\iyaktubay> git clone https://github.com/IISD-ELA/PWA-hydro-conditioning-main.git

1.4 In the same workspace, clone the pwa-tools repository by running the following command:

(base) PS C:\Users\iyaktubay> git clone https://github.com/IISD-ELA/PWA-hydro-conditioning-tools.git

1.5 Close the Visual Studio Code app.

2. Create your environment

2.1 Navigate to the Anaconda PowerShell Prompt App:

2.2 In the command line, change your working directory to the cloned PWA-hydro-conditioning-tools folder. You can do this with the cd command, followed by a space and the path to the folder (relative to your current location). For example, if the cloned folder is located C:\Users\iyaktubay\PWA-hydro-conditioning-main and your current location is C:\Users\iyaktubay, then the appropriate command would be:

(base) PS C:\Users\iyaktubay> cd PWA-hydro-conditioning-main

And, after running the command, your terminal would look like this:

(base) PS C:\Users\iyaktubay\PWA-hydro-conditioning-main>

2.3 Now that your working directory is the PWA-hydro-conditioning-tools folder, you can create your environment by running the following command:

(base) PS C:\Users\iyaktubay\PWA-hydro-conditioning-main> conda env create -f hydrocon_env.yml

2.4 To confirm that your environment has been successfully created, you can run the following command:

(base) PS C:\Users\iyaktubay\PWA-hydro-conditioning-main> conda env list

This should return something like this:

# conda environments:
#
base                  *  C:\Users\iyaktubay\AppData\Local\anaconda3
geotest                  C:\Users\iyaktubay\AppData\Local\anaconda3\envs\geotest
hydrocon_env             C:\Users\iyaktubay\AppData\Local\anaconda3\envs\hydrocon_env
pwa_dev                  C:\Users\iyaktubay\AppData\Local\anaconda3\envs\pwa_dev
test_env                 C:\Users\iyaktubay\AppData\Local\anaconda3\envs\test_env
(base) PS C:\Users\iyaktubay\PWA-hydro-conditioning-main>

3. Install the pwa-tools package

3.1 Reopen the Visual Studio Code app and open a new PowerShell terminal just as you did in step 1.2. Your terminal should look like:

(base) PS C:\Users\iyaktubay>

3.2 Activate the hydrocon_env environment you have created in step 2 by running the following command:

(base) PS C:\Users\iyaktubay> conda activate hydrocon_env

Your terminal should now look something like this:

(hydrocon_dev) PS C:\Users\iyaktubay\PWA-hydro-conditioning-main>

3.3 Change your working directory to the cloned PWA-hydro-conditioning-tools folder by running the following command (change the path to match your relative path):

(hydrocon_dev) PS C:\Users\iyaktubay> cd PWA-hydro-conditioning-tools

Your terminal should now look like this:

(hydrocon_dev) PS C:\Users\iyaktubay\PWA-hydro-conditioning-tools>

3.4 Install the custom pwa-tools package in editable mode by running the following command. You must install it in editable mode for the pipeline to work correctly.

(hydrocon_dev) PS C:\Users\iyaktubay\PWA-hydro-conditioning-tools> pip install -e .

3.5 After installation, if the pwa-tools package was updated in the remote repository, you can locally update the package by uninstalling the package and re-installing it in editing mode:

(hydrocon_env) PS C:\Users\iyaktubay\PWA-hydro-conditioning-tools> pip uninstall pwa-tools

(hydrocon_env) PS C:\Users\iyaktubay\PWA-hydro-conditioning-tools> pip install -e .

You can check whether the package was updated in the remote repository by running the following command:

(hydrocon_env) PS C:\Users\iyaktubay\PWA-hydro-conditioning-tools> git status

If the package is up to date, you should see something like this:

(hydrocon_env) PS C:\Users\iyaktubay\PWA-hydro-conditioning-tools> git status
On branch main
Your branch is up to date with 'origin/main'.

4. Prepare the input data

Create a Data/ folder inside the PWA-hydro-conditioning-main folder and download and extract the following zip files into it. Do not create any subfolders in the Data/ folder as the hydro_condition.py script expects all data files to not be in subfolders. The script will automatically organize input and output files into subfolders itself.

• Watershed of interest based on outlet point from the CLRH Hydrofabrics website (e.g., ID: 05OE006 for Manning Canal)
• Streams dataset of interest from NHN streams website.
  1. In the directory, open the folder named with the first two digits of your CLRH station ID (e.g., open the 05/ folder if your CLRH station ID is 05OE006).
  2. Download the .zip file whose name contains the third and fourth digits of your CLRH station ID (e.g., download nhn_rhn_05oe000_shp... if your CLRH station ID is 05OE006).
  3. If multiple .zip files match, download both and check which shapefile covers the geographic extent of your watershed. You can do this by using a GIS tool such as ArcGIS.
• Raster DEM(s) of interest from LiDAR DEMs (e.g., Seine & Rat 2016 for Manning Canal)

Your local workspace should now have the following structure:

your-working-directory/
├── PWA-hydro-conditioning-main/
    ├── hydro_condition.py
    ├── README.md
    ├── .gitignore
    └── Data
        └──  ...data files you downloaded...
└── PWA-hydro-conditioning-tools/
    └──  ...repository contents...

5. Run the hydroconditioning script

5.1 On Visual Studio Code's welcome page, open the PWA-hydro-conditioning-main folder by clicking "Open folder...":

5.2 Open up terminal on Visual Studio Code once again if it's not already open and change your working directory to the PWA-hydro-conditioning-main folder if it's not already there. 5.3 Run the following command to execute the hydro conditioning script. The script will ask you to input your watershed name as well as some file names.

python hydro_condition.py

5.4 Once the script has fully run, you will see the output files under the Data\<watershed name you entered when prompted>\HydroConditioning\Processed folder:

You can also see any intermediate files in the ...\Interim folder:

The output files include a depression depths raster .tif, depression depths shapefile, and a zonal statistics file for your watershed by default, as well as a wetland polygons shapefile with statistics (area, total storage, and median depth) if the user chooses to generate it.

6. Contact and support

For support or any other inquiries, please contact us at [email protected].