Skip to content

Commit c77d7b4

Browse files
author
Kristian Förster
committed
Initial commit.
1 parent d968804 commit c77d7b4

25 files changed

+4482
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
*.pyc
2+
docs/build

‎Licence_GPLv3.txt‎

Lines changed: 674 additions & 0 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Readme.txt for MELODIST
2+
3+
*Welcome to MELODIST - MEteoroLOgical observation time series DISaggregation Tool*
4+
5+
MELODIST is an open-source toolbox written in Python for disaggregating daily meteorological time series to hourly time steps. The software framework consists of disaggregation functions for each variable including temperature, humidity, precipitation, shortwave radiation, and wind speed. These functions can simply be called from a station object, which includes all relevant information about site characteristics. The data management of time series is handled using data frame objects as defined in the pandas package. In this way, input and output data can be easily prepared and processed. For instance, the pandas package is data i/o capable and includes functions to plot time series using the matplotlib library.
6+
7+
An example file (example.py) is provided along the package itself. This example demonstrates the usage of MELODIST for all variables. First, a station object is created providing some basic details on the site’s characteristics (e.g., latitude and longitude are relevant for radiation disaggregation). Once the station object is defined and filled with data, each disaggregation step is done through calling the designated function specified for each variable. Each of these functions requires a `method` argument and if needed additional parameters to work properly. Some of these methods (see below) require additional statistical evaluations of hourly time series prior to the disaggregation procedure. This information is stored in a station statistics object that is associated to the station object (see example file for further details).
8+
9+
## Station object:
10+
In the framework of MELODIST a station object includes all relevant information including metadata and time series. A station is generated using the constructor method:
11+
12+
s = melodist.Station(lon=longitude, lat=latitude, timezone=timezone)
13+
14+
Data is simply added by assignment (e.g., the data frame `data_obs_daily`):
15+
16+
s.data_daily = data_obs_daily
17+
18+
A station statistics object can be generated in a similar manner. As station statistics are derived through analysing hourly observations for calibration, a reference to the data frame including hourly observations is given:
19+
20+
s.statistics = melodist.StationStatistics(data_obs_hourly)
21+
22+
Statistics can be derived for each variable by calling the respective function of the statistics object `s.statistics`: `calc_wind_stats()`, `calc_humidity_stats()`, `calc_temperature_stats()`, `calc_precipitation_stats()`.
23+
24+
## Disaggregation methods:
25+
26+
The `Station` class provides functions to perform the disaggregation procedures for each variable: `disaggregate_temperature()`, `disaggregate_humidity()`, `disaggregate_wind()`, `disaggregate_radiation()`, and `disaggregate_precipitation()`. Moreover, an interpolation approach is also available using the `interpolate()` function.
27+
28+
Hint: It is worth noting that each of the implemented disaggregation methods is directly accessible, e.g., `melodist.precipitation.disagg_prec()`. In this case all relevant parameters (e.g., those derived through calibrations) need to provided in the function call. This method specific call of functions is not necessary if a station and the corresponding station statistic object is defined. Thus, it is recommended to define objects and to perform the disaggregation procedures using the object’s methods. Also, the names and signatures of these functions are likely subject to change in future versions of MELODIST.
29+
30+
At least the `method` argument is required for each of the disaggregation functions in a `Station`object. Please find below a list of available disaggregation methods for each variable:
31+
32+
### Temperature (based on minimum and maximum temperature recordings):
33+
* `method=='sine'` (T1) standard sine redistribution, requires 1 additional parameter `min_max_time=='fix'` (T1a): The diurnal course of temperature is fixed without any seasonal variations.
34+
* `min_max_time=='sun_loc'` (T1b): The diurnal course of temperature is modelled based on sunrise, noon and sunset calculations.
35+
* `min_max_time=='sun_loc_shift'` (T1c): This option activates empirical corrections of the ideal course modelled by `sun_loc` (requires calling `calc_temperature_stats()` prior to the disaggregation).
36+
* An optional parameter `mod_nighttime` (T1d, bool, default: `False`) allows one to apply a linear interpolation of night time values, which proves preferable during polar nights.
37+
38+
### Humidity:
39+
* `method=='minimal'` (H1): The dew point temperature is set to the minimum temperature on that day.
40+
* `method=='dewpoint_regression'` (H2): Based on hourly observations, a regression approach is applied to calculate daily dew point temperature. Regression parameters must be specified (which is automatically done if `calc_humidity_stats()` is called prior to disaggregation).
41+
* `method=='linear_dewpoint_variation'` (H3): This method extends H2 through linearly varying dew point temperature between consecutive days. The parameter `kr` needs to be specified (kr=6 if monthly radiation exceeds 100 W/m^2 else kr=12).
42+
* `method=='min_max'` (H4): this method requires minimum and maximum relative humidity for each day.
43+
44+
### Wind speed:
45+
* `method == 'equal'` (W1): If this method is chosen, the daily average wind speed is assumed to be valid for each hour on that day.
46+
* `method == 'cosine'` (W2): The cosine function option simulates a diurnal course of wind speed and requires calibration (`calc_wind_stats()`).
47+
* `method == 'random'` (W3): This option is a stochastic method that draws random numbers to disaggregate wind speed taking into account the daily average (no parameter estimation required).
48+
49+
### Radiation:
50+
* `method=='pot_rad'` (R1): This method allows one to disaggregate daily averages of shortwave radiation using hourly values of potential (clear-sky) radiation calculated for the location of the station.
51+
* `method=='pot_rad_via_ssd'` (R2): If daily sunshine recordings are available, the Angstrom model is applied to transform sunshine duration to shortwave radiation. An automatic calibration procedure is not yet implemented.
52+
* `method=='pot_rad_via_bc'` (R3): In this case, the Bristow-Campbell model is applied which relates minimum and maximum temperature to shortwave radiation. An automatic calibration procedure is not yet implemented.
53+
54+
### Precipitation
55+
* `method== 'equal'` (P1): In order to derive hourly from daily values, the daily total is simply divided by 24 resulting in an equal distribution.
56+
* `method=='cascade'` (P2): The cascade model is more complex and requires a parameter estimation method (`calc_precipitation_stats()`). Statistics can be calculated using different options (parameters). Using the keyword `months`, the seasons for which the statistics will be calculated independently can be specified (see example file). The keyword `percentile` allows one to adjust the threshold to separate precipitation intensities into two classes (low and high) for building the parameters. The default value is 50% (median). An additional optional argument `avg_stats` is used to decide whether statistics of all cascade levels will be averaged (default is `True`). All options previously listed are optional and can be changed to tune the disaggregation results.
57+
* `method=='masterstation'` (P3). If hourly values are available for another site in the vicinity of the station considered, the cumulative sub-daily mass curve can be transferred from the station that provides hourly values to the station of interest.
58+
59+
## Utilities:
60+
Among other features the `melodist.util` module includes some functions that might be useful for data analyses:
61+
62+
* `detect_gaps(dataframe, timestep, print_all=False, print_max=5, verbose=True)` can be used to find gaps in the data frame. A gap will be detected if any increment of time is not equal to the specified time step (in seconds).
63+
64+
* Some methods require time series with full days (24 h) only. `drop_incomplete_days(dataframe)` drops heading and trailing days if they are not complete (0-23h).
65+
66+
* For testing purposes an aggregation function is provided which aggregates hourly time series (data frames) to daily time series taking into account the characteristics of each meteorological variable (e.g., mean value for precipitation, daily total for precipitation, ...): `daily_from_hourly()`
67+
68+
## Data input/output:
69+
70+
MELODIST includes a feature to read and save parameter settings for all disaggregation methods in order to transfer settings or to continue work at a later time. This feature is based on the JSON format which provides a flexible and easily readable ASCII file format for different applications.
71+
72+
To save MELODIST parameters included in station statistics object you can simply call the `to_json(filename)` method of this object. At any time it is possible to recall this settings by creating a new station statistics object based on the settings stored in that file:
73+
74+
new_stationstatistics_object = melodist.StationStatistics.from_json(filename)
75+
76+
Since MELODIST is based on pandas, numerous ways to import and export pandas data frames are possible. The `to_csv()` function of pandas is ideal to save and load time series without any restriction with respect to MELODIST applications.
77+
78+
79+

‎docs/Makefile‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
# Makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line.
5+
SPHINXOPTS =
6+
SPHINXBUILD = sphinx-build
7+
PAPER =
8+
BUILDDIR = build
9+
10+
# User-friendly check for sphinx-build
11+
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
12+
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
13+
endif
14+
15+
# Internal variables.
16+
PAPEROPT_a4 = -D latex_paper_size=a4
17+
PAPEROPT_letter = -D latex_paper_size=letter
18+
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
19+
# the i18n builder cannot share the environment and doctrees with the others
20+
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
21+
22+
.PHONY: help
23+
help:
24+
@echo "Please use \`make <target>' where <target> is one of"
25+
@echo " html to make standalone HTML files"
26+
@echo " dirhtml to make HTML files named index.html in directories"
27+
@echo " singlehtml to make a single large HTML file"
28+
@echo " pickle to make pickle files"
29+
@echo " json to make JSON files"
30+
@echo " htmlhelp to make HTML files and a HTML help project"
31+
@echo " qthelp to make HTML files and a qthelp project"
32+
@echo " applehelp to make an Apple Help Book"
33+
@echo " devhelp to make HTML files and a Devhelp project"
34+
@echo " epub to make an epub"
35+
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
36+
@echo " latexpdf to make LaTeX files and run them through pdflatex"
37+
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
38+
@echo " text to make text files"
39+
@echo " man to make manual pages"
40+
@echo " texinfo to make Texinfo files"
41+
@echo " info to make Texinfo files and run them through makeinfo"
42+
@echo " gettext to make PO message catalogs"
43+
@echo " changes to make an overview of all changed/added/deprecated items"
44+
@echo " xml to make Docutils-native XML files"
45+
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
46+
@echo " linkcheck to check all external links for integrity"
47+
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
48+
@echo " coverage to run coverage check of the documentation (if enabled)"
49+
50+
.PHONY: clean
51+
clean:
52+
rm -rf $(BUILDDIR)/*
53+
54+
.PHONY: html
55+
html:
56+
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
57+
@echo
58+
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
59+
60+
.PHONY: dirhtml
61+
dirhtml:
62+
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
63+
@echo
64+
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
65+
66+
.PHONY: singlehtml
67+
singlehtml:
68+
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
69+
@echo
70+
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
71+
72+
.PHONY: pickle
73+
pickle:
74+
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
75+
@echo
76+
@echo "Build finished; now you can process the pickle files."
77+
78+
.PHONY: json
79+
json:
80+
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
81+
@echo
82+
@echo "Build finished; now you can process the JSON files."
83+
84+
.PHONY: htmlhelp
85+
htmlhelp:
86+
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
87+
@echo
88+
@echo "Build finished; now you can run HTML Help Workshop with the" \
89+
".hhp project file in $(BUILDDIR)/htmlhelp."
90+
91+
.PHONY: qthelp
92+
qthelp:
93+
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
94+
@echo
95+
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
96+
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
97+
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/melodist.qhcp"
98+
@echo "To view the help file:"
99+
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/melodist.qhc"
100+
101+
.PHONY: applehelp
102+
applehelp:
103+
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
104+
@echo
105+
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
106+
@echo "N.B. You won't be able to view it unless you put it in" \
107+
"~/Library/Documentation/Help or install it in your application" \
108+
"bundle."
109+
110+
.PHONY: devhelp
111+
devhelp:
112+
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
113+
@echo
114+
@echo "Build finished."
115+
@echo "To view the help file:"
116+
@echo "# mkdir -p $$HOME/.local/share/devhelp/melodist"
117+
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/melodist"
118+
@echo "# devhelp"
119+
120+
.PHONY: epub
121+
epub:
122+
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
123+
@echo
124+
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
125+
126+
.PHONY: latex
127+
latex:
128+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
129+
@echo
130+
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
131+
@echo "Run \`make' in that directory to run these through (pdf)latex" \
132+
"(use \`make latexpdf' here to do that automatically)."
133+
134+
.PHONY: latexpdf
135+
latexpdf:
136+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
137+
@echo "Running LaTeX files through pdflatex..."
138+
$(MAKE) -C $(BUILDDIR)/latex all-pdf
139+
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
140+
141+
.PHONY: latexpdfja
142+
latexpdfja:
143+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
144+
@echo "Running LaTeX files through platex and dvipdfmx..."
145+
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
146+
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
147+
148+
.PHONY: text
149+
text:
150+
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
151+
@echo
152+
@echo "Build finished. The text files are in $(BUILDDIR)/text."
153+
154+
.PHONY: man
155+
man:
156+
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
157+
@echo
158+
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
159+
160+
.PHONY: texinfo
161+
texinfo:
162+
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
163+
@echo
164+
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
165+
@echo "Run \`make' in that directory to run these through makeinfo" \
166+
"(use \`make info' here to do that automatically)."
167+
168+
.PHONY: info
169+
info:
170+
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
171+
@echo "Running Texinfo files through makeinfo..."
172+
make -C $(BUILDDIR)/texinfo info
173+
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
174+
175+
.PHONY: gettext
176+
gettext:
177+
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
178+
@echo
179+
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
180+
181+
.PHONY: changes
182+
changes:
183+
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
184+
@echo
185+
@echo "The overview file is in $(BUILDDIR)/changes."
186+
187+
.PHONY: linkcheck
188+
linkcheck:
189+
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
190+
@echo
191+
@echo "Link check complete; look for any errors in the above output " \
192+
"or in $(BUILDDIR)/linkcheck/output.txt."
193+
194+
.PHONY: doctest
195+
doctest:
196+
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
197+
@echo "Testing of doctests in the sources finished, look at the " \
198+
"results in $(BUILDDIR)/doctest/output.txt."
199+
200+
.PHONY: coverage
201+
coverage:
202+
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
203+
@echo "Testing of coverage in the sources finished, look at the " \
204+
"results in $(BUILDDIR)/coverage/python.txt."
205+
206+
.PHONY: xml
207+
xml:
208+
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
209+
@echo
210+
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
211+
212+
.PHONY: pseudoxml
213+
pseudoxml:
214+
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
215+
@echo
216+
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

0 commit comments

Comments
 (0)