Difference between revisions of "WRF Hindcast"

From Agrineer.org Wiki
Jump to: navigation, search
(Platforms)
 
(20 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  
Agrineer's WRF Hindcast Project generates the data used by the [[Grow Degree Calculator]] and the [[Soil Moisture Estimator]] applications. WRF stands for Weather, Research, and Forecasting and is a program made available by [http://www.wrf-model.org/index.php UCAR/NCAR] and other research participants.  
+
Agrineer's WRF Hindcast Project generates the input data used by the [[Grow Degree Calculator]] (GDC) and the [[Soil Moisture Estimator]] (SME) applications. WRF stands for [http://www2.mmm.ucar.edu/wrf/users  Weather, Research, and Forecasting] and is a program made available by [https://www.mmm.ucar.edu/wrf-model-general UCAR/NCAR] and other research participants.  
  
 
WRF is normally used to forecast weather, but for our purposes we use it to simulate weather in the past.
 
WRF is normally used to forecast weather, but for our purposes we use it to simulate weather in the past.
Line 7: Line 7:
  
 
[[File:WRF_Sectors.png|300px|thumb|right|WCONUS sector grid, current operational sectors are shown in green.]]
 
[[File:WRF_Sectors.png|300px|thumb|right|WCONUS sector grid, current operational sectors are shown in green.]]
The area modeled by the WRF program is called a "domain", but since this implementation uses three domains (at 30km, 10km, and 3.3km resolution), the area of interest, the third domain, is called a "sector". Each sector is made up of 171x171 "pixels", 3.3km x 3.3km in size. Client programs can access the pixel's data by indicating latitude and longitude coordinates.
+
The area modeled by the WRF program is called a "domain", but since this implementation uses three nested domains (at 30km, 10km, and 3.3km resolution), the area of interest, the third domain, is called a "sector". Each sector is currently made up of 171x171 "pixels", at 3.3km x 3.3km in size. Client applications can access the pixel's simulated weather data by indicating latitude and longitude coordinates.
 +
<!-- TODO: insert image example of nested domains here -->
  
The image to the right depicts the sectors defined, and those currently active (operational) are shown in green. This set of sectors are designated WCONUS (Western Continental United States) with an appellation of grid location. Sector rows are given letters and columns are given numbers, with the origin on the bottom right. For example, the bottom right sector is WCONUS_A0 and the top left sector is WCONUS_E4. Grid coordinates are based on data delivery needs and computation capacity.
+
The image to the right depicts the sectors defined, and currently active (operational) sectors are shown in green. The defined set of sectors are designated WCONUS (Western CONtinental United States) with an appellation of grid location. Sector rows are given letters and columns are given numbers, with the origin on the bottom right. For example, the bottom right sector is WCONUS_A0 and the top left sector is WCONUS_E4.
The most eastern sector users will want data earlier than the western ones, by about two hours, and so computation starts with "0" column. Likewise, the most southern sectors will have an earlier planting season than northern ones, and so the most southern row is "A".
+
  
== Procedure ==
+
== Implementation ==
The WRF climate simulation program requires a sophisticated input stream and is usually executed on a High Performance Computer (HPC), that is, a parallel computing platform. See [[#Platforms|Platforms]] below. A description of input files, directories structures and output is given below.
+
The WRF climate simulation program requires a sophisticated input stream, consisting of daily data and configuration files, and is usually executed on a parallel computing platform. See [[#Platforms|Platforms]] below for Agrineer's implementation. Each sector's output data is calculated off-line and delivered to the web server on a daily basis. Users can expect a maximum of one day delay, with eastern sectors getting results in the morning and western sectors getting results later in the day. "Namelist" input files to WPS and WRF programs can be found in the sector data download [https://www.agrineer.org/downloads/sectors.php page].
  
=== Input Files ===
+
== Output ==
 +
Typical WRF output include hundreds of variable values amounting to about 4.5 GB (1.5 GB per domain) per sector daily. A portion of these are used to calculate the standard reference evaporation (ETo) values in the [[Soil Moisture Estimator|SME]] application. Since the daily [[ETo Calculation|ETo]] values are calculated off-line using WRF output, we could deliver to the server just the ETo and precipitation values for accumulation purposes. However, the [[Grow Degree Calculator|GDC]] application requires daily maximum and minimun temperatures and so we deliver these as well, in netCDF format. All three domains for each sector are available on a daily basis from [https://www.agrineer.org/downloads/sectors.php here]. Use the netCDF programs "ncdump" and "ncview" to inspect the headers and contents.
  
==== GFS ====
+
== Global Forecast System ==
Agrineer's implementation of the WRF program uses the [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs | Global Forecast System] (GFS) forcing files as input. These are files generated by NOAA (NCEP) which are used to force data interpolations to defined values at certain times, projected into the future at six hour intervals. Once those projected times are reached, the forcing file is regenerated using real data instead of projected data and a new set of projected files are then generated, etc. This project uses the re-analyzed files at six-hour intervals for historical evaluation.
+
Agrineer's implementation of the WRF program uses the [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs Global Forecast System] (GFS) forcing files as input. These are files generated by [https://www.ncdc.noaa.gov NOAA's National Centers for Environmental Information] (NCEI),which are used to force data interpolations to defined values at certain times, projected into the future at six hour intervals. Once a projected time is reached, the forcing file is regenerated using real data instead of projected data and a new set of projected files are generated, and the process repeats itself. This project uses the re-analyzed files at six-hour intervals for historical evaluation. The resolution of the data is 2.5 x 2.5 degrees, and the gridded data is interpolated down to a 0.25 X 0.25 degree resolution to provide finer detail. The data is retrieved from [ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/ here] where the folders are date and interval dependent. To retrieve re-analyzed data for 2017/03/03 interval 00, for example, go to gfs.2017030300, then find gfs.t00z.pgrb2.0p25.f000.
  
== WRF Output ==
+
== Platform ==
 +
Agrineer's WRF platform is Linux Mint as the operating system on an 8-core 64-bit AMD based computer, compiled under GNU gfortran with MPICH parallel implementation. The Python language is used to integrate all of the working parts, from retrieving input files to uploading results to the server. The dynamic solver model used in WRF is the ARW (Advanced Research WRF) core.
  
Domain sizes 171x171
+
The computer platform described can calculate 10 sectors per day, with rest periods for maintenance. For redundancy, Agrineer operates two WRF platforms in separate long distance locations. 
variables
+
  
 +
== WRF Docker Container ==
 +
A Docker container package is provided which generates WRF data used in the GDC and SME tools. Third party data contribution for specific sectors can be done with this package and is suitable for compute servers and multi-core desktops alike. The container package can be downloaded [https://gitlab.com/agrineer/WRF_container here]. Registration with Agrineer.org is required for data contributions used in the GDC and SME tools.
 +
 +
<!--
 
== FAQ ==
 
== FAQ ==
  
Line 30: Line 35:
 
=== Download  and Install WRF ===
 
=== Download  and Install WRF ===
 
=== Create Sector directories ===
 
=== Create Sector directories ===
== Platforms ==
 
Agrineer's WRF platform is Linux Mint as the operating system on 8-core AMD computers, compiled under GNU gfortran with MPICH parallel implementation. The Python language is used to integrate all of the working parts. The dynamic solver model used in WRF is ARW (Advanced Research WRF) core.
 
  
=== Scripts ===
+
== Input Files ==
 +
=== GFS ===
 +
=== WPS and WRF namelist ===
 +
== Scripts ==
  
 
== User Support ==
 
== User Support ==
 +
-->

Latest revision as of 20:28, 18 January 2019

Agrineer's WRF Hindcast Project generates the input data used by the Grow Degree Calculator (GDC) and the Soil Moisture Estimator (SME) applications. WRF stands for Weather, Research, and Forecasting and is a program made available by UCAR/NCAR and other research participants.

WRF is normally used to forecast weather, but for our purposes we use it to simulate weather in the past.

Sectors

WCONUS sector grid, current operational sectors are shown in green.

The area modeled by the WRF program is called a "domain", but since this implementation uses three nested domains (at 30km, 10km, and 3.3km resolution), the area of interest, the third domain, is called a "sector". Each sector is currently made up of 171x171 "pixels", at 3.3km x 3.3km in size. Client applications can access the pixel's simulated weather data by indicating latitude and longitude coordinates.

The image to the right depicts the sectors defined, and currently active (operational) sectors are shown in green. The defined set of sectors are designated WCONUS (Western CONtinental United States) with an appellation of grid location. Sector rows are given letters and columns are given numbers, with the origin on the bottom right. For example, the bottom right sector is WCONUS_A0 and the top left sector is WCONUS_E4.

Implementation

The WRF climate simulation program requires a sophisticated input stream, consisting of daily data and configuration files, and is usually executed on a parallel computing platform. See Platforms below for Agrineer's implementation. Each sector's output data is calculated off-line and delivered to the web server on a daily basis. Users can expect a maximum of one day delay, with eastern sectors getting results in the morning and western sectors getting results later in the day. "Namelist" input files to WPS and WRF programs can be found in the sector data download page.

Output

Typical WRF output include hundreds of variable values amounting to about 4.5 GB (1.5 GB per domain) per sector daily. A portion of these are used to calculate the standard reference evaporation (ETo) values in the SME application. Since the daily ETo values are calculated off-line using WRF output, we could deliver to the server just the ETo and precipitation values for accumulation purposes. However, the GDC application requires daily maximum and minimun temperatures and so we deliver these as well, in netCDF format. All three domains for each sector are available on a daily basis from here. Use the netCDF programs "ncdump" and "ncview" to inspect the headers and contents.

Global Forecast System

Agrineer's implementation of the WRF program uses the Global Forecast System (GFS) forcing files as input. These are files generated by NOAA's National Centers for Environmental Information (NCEI),which are used to force data interpolations to defined values at certain times, projected into the future at six hour intervals. Once a projected time is reached, the forcing file is regenerated using real data instead of projected data and a new set of projected files are generated, and the process repeats itself. This project uses the re-analyzed files at six-hour intervals for historical evaluation. The resolution of the data is 2.5 x 2.5 degrees, and the gridded data is interpolated down to a 0.25 X 0.25 degree resolution to provide finer detail. The data is retrieved from here where the folders are date and interval dependent. To retrieve re-analyzed data for 2017/03/03 interval 00, for example, go to gfs.2017030300, then find gfs.t00z.pgrb2.0p25.f000.

Platform

Agrineer's WRF platform is Linux Mint as the operating system on an 8-core 64-bit AMD based computer, compiled under GNU gfortran with MPICH parallel implementation. The Python language is used to integrate all of the working parts, from retrieving input files to uploading results to the server. The dynamic solver model used in WRF is the ARW (Advanced Research WRF) core.

The computer platform described can calculate 10 sectors per day, with rest periods for maintenance. For redundancy, Agrineer operates two WRF platforms in separate long distance locations.

WRF Docker Container

A Docker container package is provided which generates WRF data used in the GDC and SME tools. Third party data contribution for specific sectors can be done with this package and is suitable for compute servers and multi-core desktops alike. The container package can be downloaded here. Registration with Agrineer.org is required for data contributions used in the GDC and SME tools.