Fix codacy coverage and report combined unit/acceptance coverage#1033
Merged
LaurenceBeard merged 2 commits intometoppv:masterfrom Dec 13, 2019
Merged
Fix codacy coverage and report combined unit/acceptance coverage#1033LaurenceBeard merged 2 commits intometoppv:masterfrom
LaurenceBeard merged 2 commits intometoppv:masterfrom
Conversation
bin/improver uses lib as the working directory, so the previous coverage path was wrong.
tjtg
changed the title
Contributor
Author
|
This change hasn't produced coverage information on codacy as expected, so will need further investigation. |
tjtg
force-pushed
the
codacy-coverage
branch
from
f353272 to
389273b
Compare
Contributor
Author
|
Coverage file is now being produced in the branch, and I've added a text report which will be visible in the Travis CI log. The Codacy API token is only available on branches within metoppv/improver, so coverage information of branches in personal forks won't appear on Codacy. Codacy should pick up the coverage once merged to master. Please go ahead and review this PR now. |
tjtg
changed the title
LaurenceBeard
approved these changes
Dec 12, 2019
Contributor
LaurenceBeard
left a comment
LaurenceBeard
left a comment
There was a problem hiding this comment.
This looks good.
But with a view to moving away from travis and improver-tests, do you think it would be worth doing new things in GitHub actions (as standalone scripts, rather than expanding on improver_tests)?
Contributor
Author
Yes, I agree with the intention to move from Travis to Github actions. |
gavinevans
added a commit
to gavinevans/improver
that referenced
this pull request
Dec 20, 2019
…_emos_documentation * upstream/master: (28 commits) IMPRO-1519: extrapolate clize (metoppv#1035) save atomically by writing to a temporary file and then renaming (metoppv#1065) Minor code style fixes (metoppv#1079) IMPRO-1534: convert extend radar mask to use clize (metoppv#1056) Pylint with Github Actions (metoppv#1066) Tests with Github Actions (metoppv#1030) Improve forecast times (metoppv#1027) Fix duplicate clize output and errors (metoppv#1041) Remove BATS tests and associated code (metoppv#1045) Fix codacy coverage and report combined unit/acceptance coverage (metoppv#1033) Added a cli to get a remainder so that we can calculate sleet in the … (metoppv#1020) IMPRO-1424: Making snow-falling-level more general (metoppv#1024) IMPRO-1476: Accumulate CLI (metoppv#1025) Pytest for 1195 (metoppv#1029) Conversion of acceptance tests to pytest (metoppv#1026) Mandatory attributes (metoppv#1019) IMPRO-1195: create convert_to_realizations CLI (metoppv#959) IMPRO-1490: Wind vector to UV (metoppv#1012) Enable Travis build config validation (metoppv#1004) Fixed bug caused by unnecessary processing of percentile values. (metoppv#1023) ...
cgsandford
added a commit
that referenced
this pull request
Jan 6, 2020
* Minor edits to reduce Read the Docs memory consumption (#1022) * Minor edits to try to reduce Read the Docs memory consumption. * Move environment.yml to cause less confusion, as this file is specific for Read the Docs. * Fixed bug caused by unnecessary processing of percentile values. (#1023) * Enable Travis build config validation (#1004) * Update .travis.yml * Update .travis.yml * IMPRO-1490: Wind vector to UV (#1012) * Added cli/vector_to_uv Added a cli to convert speed and direction to u and v. * Updated wind-vetor-to-uv to move imports locally * Update lib/improver/cli/wind_vector_to_uv.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Updated to remove wind from cli __init__.py * Update wind_vector_to_uv.py * Added tests for create constrainted inputcube converter * Updated to remove unused import * Update lib/improver/cli/__init__.py * Removed constraints loading. * Remved unused import * Removed old changes. * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * IMPRO-1195: create convert_to_realizations CLI (#959) * Added cli/to_realizations. * Update -to-realizations CLI to use clize and rename * Add BATS tests using probabilities-to-realizations and percentiles-to-realizations tests * Update lib/improver/cli/convert_to_realizations.py * Apply suggestions from code review * Mandatory attributes (#1019) * Added a function to generate default values for mandatory attributes. * Set up default title attribute and removal of grid attributes in spot extraction. * Handled and checked for mandatory attributes in cube creation utility. * Moved mandatory attribute definitions to constants module. * Added unit tests and tidied up mandatory attribute generation function. * Updated blending code to add mandatory attributes. * Used constant time types and units in forecast_times.py. * Added new title argument to StandardiseGridAndMetadata plugin, to be applied on regridding. * Added "regridded_title" argument to standardise CLI and updated tests. * Used global constant MOSG_GRID_ATTRIBUTES in standardisation and tidied up a few code style issues. * Fixed codacy issue. * Refactored forecast_period_coord to be acceptable to Codacy. * Fixed style errors. * Removed unnecessary type-cast. * Responded to review comments. * Responded to review comments. * Conversion of acceptance tests to pytest (#1026) * Squash pytest work * Update improver-tests CLI to use pytest * Specify pycodestyle error code * Fix pylint marker, mark more tests as slow * Bring back dechunking via temporary files * Remove coverage from acceptance tests * Backup skip in kgo_root function * Set time lagged ensemble basic acceptance test to xfail * Increase nccmp timeout to 5 minutes Some systems are really slow to run nccmp, as by default it reads small pieces from files and the -B option to buffer whole variables in memory is not available on older nccmp versions. * Change order of named args in cli/__init__.py * Convert acceptance tests to use clize * Fix standardise pytest after #1019 * Add blank line to quieten pylint * Pytest for 1195 (#1029) * Added pytests for convert to realizations * Deleted bats tests * Updated for clize * Update test_convert_to_realizations.py * Update test_convert_to_realizations.py * IMPRO-1476: Accumulate CLI (#1025) * Added an accumlate CLI. * Updated nowcast accumuate * Update lib/improver/cli/nowcast_accumulate.py * Update lib/improver/cli/nowcast_accumulate.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Apply suggestions from code review Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/cli/test_init.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/nowcast_accumulate.py * Updated nowcast accumuate * Apply suggestions from code review Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Updated for second review * Update lib/improver/cli/nowcast_accumulate.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Added pytest for accumulate * Added new pytest for accumulate * Updated pytests for accumulate * Removed bats test * Updated for codacy * IMPRO-1424: Making snow-falling-level more general (#1024) * Almost working apart from a mismatch for some points in the KGO. * Remove almost all mentions of snow from the psychrometric calculations. FallingSnowLevel plugin renamed to PhaseChangeLevel. * Update CLI to use PhaseChangeLevel plugin. * Most unit tests fixed. CLIs updated, CLI tests still needed. Committing prior to investigating PhaseChangeLevel process unit test failures. * Attribute tweak. * Additional check added to ensure wet bulb integral data is in descending height order within the input cube. This is no longer guaranteed as the wet bulb integral cube could notionally be created by some mechanism other than the IMPROVER plugin/CLI. * CLI tests updated. * Style fixes and CLI test changes. * New tests. * Remove last mentions of snow where inappropriate. * Pylint improvements. * Corrected a typo and expanded unit test doc-strings to be more descriptive. Also added an additional unit test for the sleet-rain phase change. * Review typo changes. * Merged in pytest changes and deleted redundant snow_falling_level test. * Replicate BATS test changes in pytest * Added a cli to get a remainder so that we can calculate sleet in the … (#1020) * Added a cli to get a remainder so that we can calculate sleet in the future. * Deleted trailing white space * Deleted pylint message * Putting back a line I accidentally deleted. * I added a plugin and rewrote the cli and tests to be able to filter sleet from snow and rain * Fixed a codacy error. * Added warning and test for negative cube values. * Deleting an else that wasn't needed. * Changes to the unit tests to use set_up_probability_cube * Deleting whitespaces. * Fixing indentation errors. * Changes to the docstrings. * Adding comments and sorting out variable names. * Modifying docstring in calculate_sleet_prob.py * Fixing codacy problem with test_calculate_sleet_probability.py * Modifying calculate_sleet_prob.py to run with MANDATORY ATTRIBUTES * Changes needed for pytest. * Fix codacy coverage and report combined unit/acceptance coverage (#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Remove BATS tests and associated code (#1045) * Fix duplicate clize output and errors (#1041) * Removes section that printed the error message to stdout just before it gets printed to stderr. * Adds filter for "Raises" sections in doc-strings which trigger unexplained error messages. * Improve forecast times (#1027) * Changed forecast_period_coord utility to create coordinate with expected datatype and units, which are not configurable - since we have decided to fail on save if these metadata do not conform. Updated time-lagging unit tests to set up appropriate cubes. * Rearranged unit tests to cover coordinate creation separately from top level manipulation. * Fixed trailing whitespace. * Tests with Github Actions (#1030) * Create tests.yml * Create environment.yml * Delete .travis.yml because GitHub actions > * Include style/doc build tests * Update environment.yml * Keep travis for the time being * conflict * Pylint with Github Actions (#1066) * Add pylint to actions * Update environment.yml * pin astroid fix pylint output * IMPRO-1534: convert extend radar mask to use clize (#1056) * Updated radar to use clize * Updated to remove float checking * Minor code style fixes (#1079) * Removed no-else-raise issues and reworked torturous if / else tangle in wind_gust_diagnostic.py. * Removed no-else-returns. * Removed unnecessary "pass" and "try-except-raise" statements. * Reworked loop in more pythonic way. * save atomically by writing to a temporary file and then renaming (#1065) * save atomically by writing to a temporary file and then renaming * stringify filename * IMPRO-1519: extrapolate clize (#1035) * Updated to remove speed and direction * Updated for clize * Updated nowcast acc tests. * Updated to move error handling into plugin * Updated to move error into plugin * IMPRO-1517: Update Orographic enhancement to use clize. (#1032) * Updated to use clize. * Updated type * Updated for errors * IMPRO-1523: Convert optical flow to clize (#1053) * Updated to clize and pytests Co-authored-by: @MoseleyS * Fixed extrapolate CLI tests. * Removed TODO. Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: Ben Fitzpatrick <ben.fitzpatrick@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: LaurenceBeard <laurence.beard@metoffice.gov.uk> Co-authored-by: Tomek Trzeciak <TomekTrzeciak@users.noreply.github.com>
arh89
added a commit
that referenced
this pull request
Jan 8, 2020
* Fix codacy coverage and report combined unit/acceptance coverage (#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Changes to recursive filter to allow it to work with clize. * Deleting whitespaces and an extra line. * Fixing indentation issues. * Simplify recursive-filter CLI Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk>
arh89
added a commit
to arh89/improver
that referenced
this pull request
Jan 8, 2020
* Fix codacy coverage and report combined unit/acceptance coverage (metoppv#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Changes to recursive filter to allow it to work with clize. * Deleting whitespaces and an extra line. * Fixing indentation issues. * Simplify recursive-filter CLI Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk>
arh89
added a commit
that referenced
this pull request
Jan 8, 2020
* Fix codacy coverage and report combined unit/acceptance coverage (#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Changes to recursive filter to allow it to work with clize. * Deleting whitespaces and an extra line. * Fixing indentation issues. * Simplify recursive-filter CLI Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk>
arh89
added a commit
that referenced
this pull request
Jan 13, 2020
* Clize: Combine CLI (#1034) * Removes argparser section from combine CLI and updates pytests with clize argument names. * Replaces = with blank in keyword args. * Removes "if __main__" and makes acceptance tests pass without the equals symbols. * Changes to convert feels like temperature to clize. (#1043) * Changes to convert feels like temperature to clize. * Deleting trailing whitespace. * Modified a line of code that was too long. * Put flag on new line as per Neil's suggestion. * Clizify weighted blending CLI (#1037) * First attempt at refactor. Pytests fail for the moment. * Fixed CLI and pytests. * Removed accidental whitespace. * Fixed pycodestyle issue. * Improved docstring. * Removed reference to "main". * Removed unnecessary numpy dependency. * Clize applied to apply-lapse-rate CLI (#1036) * Clize applied to apply-lapse-rate CLI * Replaces = with blank in keyword args. * Removes __main__ section and = separators in pytest argument setup. * Removes unused imports. * Small improvement following review. * Clizes create-grid-with-halo (#1046) * Clizes create-grid-with-halo * Removes default statement which is repeated in the help-string anyway. * Convert create-grid-with-halo CLI to clize (#1040) * Convert create-grid-with-halo CLI to clize * Update lib/improver/cli/create_grid_with_halo.py * Clize time lagging (2) (#1047) * Clizefied time lagging CLI. * Tidied up. * Updated several docstrings with correct types. * Removed unused "cycletime" argument and unnecessary __init__ and __repr__ methods. * Changed variable name and corrected plugin docstring. * Simplified function signature and call. * Convert wind-gust-diagnostic CLI to use Clize (#1054) * Convert wind-gust-diagnostic CLI to use Clize. * Minor edit to default arguments. * Removing stray trailing "o". * Conversion of extract to use clize. (#1052) * Conversion of extract to use clize. Required changes to user inputs due to treatment of comma separation by clize. * Pep fixes. * Remove __main__ calling. * change improver-extract CLI multiple constraint handling * Slightly expanded help to describe input requirements. * Modified to use clize. (#1055) * clize changes for uv-index (#1059) * Clize phase-change-level (#1058) * Modify phase-change-level to use clize. * Update lib/improver/cli/phase_change_level.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/acceptance/test_phase_change_level.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/acceptance/test_phase_change_level.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Clize wet-bulb-temperature-integral (#1060) * Modify wet-bulb-temperature-integral to use clize. * Update lib/improver/psychrometric_calculations/psychrometric_calculations.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Removed extraneous main. * Remove pylint warning about touching data having no effect. * Clize wind direction (#1062) * Modify wind-direction CLI to use clize. * Remove long comment. Rely upon plugin doc-string to retain information. * Convert apply-emos-coefficients to clize (#1038) * Convert apply-emos-coefficients to clize * apply-emos-coefficients pytest style changes * Convert blend-adjacent-points CLI to use clize (#1039) * Convert blend-adjacent-points CLI to use clize * Update mandatory kwargs * blend-adjacent-points pytest style update * Remove newline in blend-adjacent-points CLI tests * Clize threshold (#1064) * Removed unnecessary error handling. * Tidied up some names and errors. * Clizefied CLI and fixed tests. * Tidied up threshold CLI logic and docstrings. * Fixes pycodestyle errors (#1068) * Migrates spot-extract to use Clize. (#1048) * Migrates spot-extract to use Clize. * Clize standardise (#1049) * Updated CLI not to reuse names. * Removed unneeded test and moved error handling into "process" function. * Got everything working except list of coordinates to remove. * Fixed last test. * Removed "default is" from docstrings and tidied up some descriptions. * Made target grid and source landmask positional arguments, and removed unnecessary error. * Added type to landmask vicinity argument. * Clize topographic band CLIs (#1063) * Modify topographic band related CLIs to use clize. * Review changes. * Describes behaviour if optional output file is not supplied. (#1067) * between-thresholds CLI to clize (#1074) * Use clize for generate-landmask-ancillary CLI (#1071) * Impro 1497 clize percentiles to probabilities (#1069) * Changes to clize percentiles_to_probabilities, including updating docstrings. * Update lib/improver/cli/percentiles_to_probabilities.py Added additional * argument to separate out cube arguments and non-cube arguments Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/acceptance/test_percentiles_to_probabilities.py Adding --output-diagnostic-name before the value in test call. Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * neighbour-finding CLI to clize (#1073) * neighbour-finding CLI to clize * Update lib/improver/tests/acceptance/test_neighbour_finding.py Co-Authored-By: bayliffe <benjamin.ayliffe@metoffice.gov.uk> * wxcode CLI to clize (#1076) * wxcode CLI to clize * Adds back extended help as an option to the CLI. Code to generate help moved to wxcode utilities. This approach avoids the need to import the module unless requested and keeps the CLI tidy. * Refactor wxcode CLI help * make cli.wxcode.extend_help private * Added missing blank line that was making tests fail. * wind-downscaling CLI to clize (#1077) * wind-downscaling CLI to clize * Remove unused CLI option in wind downscaling * update-grid-metadata CLI to clize (#1075) * update-grid-metadata CLI to clize * update-grid-metadata changes following review * temporal-interpolate CLI to clize (#1078) * temporal-interpolate CLI to clize * Update lib/improver/cli/temporal_interpolate.py Co-Authored-By: fionaRust <fiona.rust@metoffice.gov.uk> * merge cubes at end of temporal-interpolate CLI Co-authored-by: fionaRust <fiona.rust@metoffice.gov.uk> * temp-lapse-rate CLI to clize (#1081) * Clizefied temperature lapse rate CLI. * Clarified why the temperature cube is a required argument. * Removed trailing whitespace. * temp-lapse-rate CLI, change landmask and orography to optional positional arguments Co-authored-by: cgsandford <35029690+cgsandford@users.noreply.github.com> * IMPRO-1516: clize nbhood land sea (#1042) * Updated to use clize * Updated pytests * Updated nbhood land and sea changed how it deals with radius/radii and deals with error handling. * Updated cli/nbhood names. * Updated radius doctrings. * Updated variable name * Updated to add new radii test * Updated kgo and changed tests input order * Create CLI for generating recursive filters alphas (#1083) * Update temporal_interpolate.py Removing a space that was causing an error * Updated to remove none-assignment and assert (#1094) * IMPRO-1539: clizify nbhood (#1092) * Clizify nbhood. This change includes the removal of the recursive filter options from the nbhood CLI and all associated tests. * Review changes. * Clize nbhood iterate with mask (#1096) * Clizify nbhood-iterate-with-mask. Remove a common utility that is no longer required. * IMPRO-1539: clizify nbhood (#1092) * Clizify nbhood. This change includes the removal of the recursive filter options from the nbhood CLI and all associated tests. * Review changes. * Review changes. * update generate-orographic-alphas test to reflect most common usage (#1098) * nbhood clizification tidy-up (#1100) * Changes following an inadvertently skipped review. * Removed redundant raises doc-string entries. * Convert recursive-filter CLI to use clize (#1099) * Fix codacy coverage and report combined unit/acceptance coverage (#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Changes to recursive filter to allow it to work with clize. * Deleting whitespaces and an extra line. * Fixing indentation issues. * Simplify recursive-filter CLI Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> * estimate-emos-coefficients CLI to clize (#1095) * Update estimate-emos-coefficients * fix tests * remove duplicated line in emos plugin * review feedback * set default tolerance to its value from argparser CLI Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> * Fix style in recursive filter CLI (#1102) * Silences "interpreted-text" warning when parsing apply-emos-coefficients CLI. (#1103) * Update lib/improver/cli/threshold.py * Clize porting review (#1104) * remove explicit defaults from docstrings * remove ' or None' from docstrings * orog -> orography * temp_lapse_rate renames, improver.constants without iris * warnings_on -> check_metadata * ecc_bounds_warning -> ignore_ecc_bounds, random_ordering -> randomise, reordering -> reorder, rebadging -> rebadge, num_realizations -> realizations_count, no_of_realizations -> realizations_count, no_of_percentiles -> percentiles_count * thresholds_dict -> bands_config * sum_or_fraction -> area_sum, re_mask -> remask * wts_calc_method -> weighting_method, wts_dict -> weighting_config * attributes_dict -> new_metadata * consistent naming of land-sea mask * cubelist -> cubes * minimum_dz -> similar_altitude * site list as positional argument * wind gust renames * extend_radar_mask renames * wind_downscaling renames * various input cube renames * combine: new_metadata -> bounds_config and drop one level of dict nesting * new_metadata -> attributes_config * rebadge or reorder based on presence of the raw ensemble argument * pep8 * remove tests subcommand, use improver-tests directly instead * one line command description * remove argparser * remove legacy support in clizefy decorator * document orography argument Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: cgsandford <35029690+cgsandford@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: fionaRust <fiona.rust@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net>
MoseleyS
added a commit
that referenced
this pull request
Jan 16, 2020
…ng_feature_branch_merge * commit '5f7b1d5b786fa98ed5de4c54ebaac971b67f0456': (23 commits) IMPRO-1523: Convert optical flow to clize (#1053) IMPRO-1517: Update Orographic enhancement to use clize. (#1032) IMPRO-1519: extrapolate clize (#1035) save atomically by writing to a temporary file and then renaming (#1065) Minor code style fixes (#1079) IMPRO-1534: convert extend radar mask to use clize (#1056) Pylint with Github Actions (#1066) Tests with Github Actions (#1030) Improve forecast times (#1027) Fix duplicate clize output and errors (#1041) Remove BATS tests and associated code (#1045) Fix codacy coverage and report combined unit/acceptance coverage (#1033) Added a cli to get a remainder so that we can calculate sleet in the … (#1020) IMPRO-1424: Making snow-falling-level more general (#1024) IMPRO-1476: Accumulate CLI (#1025) Pytest for 1195 (#1029) Conversion of acceptance tests to pytest (#1026) Mandatory attributes (#1019) IMPRO-1195: create convert_to_realizations CLI (#959) IMPRO-1490: Wind vector to UV (#1012) ...
bayliffe
added a commit
that referenced
this pull request
Sep 29, 2020
* Implement pysteps semi-Lagrangian advection scheme (#958) * Added wrapper for pysteps semi-Lagrangian advection method. * Added pysteps to conda path in travis, and removed hard-coded pysteps path from advection module. * IMPRO-1354 use PySteps extrapolate method in CLI (#1018) * Switches extrapolation technique to PySteps. * Fixes syntax error. * Adds option to PySteps for Orographic Enhancement data to be None if data are not precipitation. * Adds use of attributes_dict to PySteps advection class. * Suppresses stdout message when importing PySteps. * Moves new CLI KGO to unique locations relating to the feature branch so that we don't break master. * Moves argument "attributes_dict" from init to process method in PySteps Extrapolate class. * Deals with Codacy issues and comments. * Removes options for handling non-precipitation fields. * Uses contextmanager instead of creating a class to redirect stdout. * Small improvement to contextmanager * Removes unused import and adds doc-string. * Merge branch (#1085) * Minor edits to reduce Read the Docs memory consumption (#1022) * Minor edits to try to reduce Read the Docs memory consumption. * Move environment.yml to cause less confusion, as this file is specific for Read the Docs. * Fixed bug caused by unnecessary processing of percentile values. (#1023) * Enable Travis build config validation (#1004) * Update .travis.yml * Update .travis.yml * IMPRO-1490: Wind vector to UV (#1012) * Added cli/vector_to_uv Added a cli to convert speed and direction to u and v. * Updated wind-vetor-to-uv to move imports locally * Update lib/improver/cli/wind_vector_to_uv.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Updated to remove wind from cli __init__.py * Update wind_vector_to_uv.py * Added tests for create constrainted inputcube converter * Updated to remove unused import * Update lib/improver/cli/__init__.py * Removed constraints loading. * Remved unused import * Removed old changes. * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * IMPRO-1195: create convert_to_realizations CLI (#959) * Added cli/to_realizations. * Update -to-realizations CLI to use clize and rename * Add BATS tests using probabilities-to-realizations and percentiles-to-realizations tests * Update lib/improver/cli/convert_to_realizations.py * Apply suggestions from code review * Mandatory attributes (#1019) * Added a function to generate default values for mandatory attributes. * Set up default title attribute and removal of grid attributes in spot extraction. * Handled and checked for mandatory attributes in cube creation utility. * Moved mandatory attribute definitions to constants module. * Added unit tests and tidied up mandatory attribute generation function. * Updated blending code to add mandatory attributes. * Used constant time types and units in forecast_times.py. * Added new title argument to StandardiseGridAndMetadata plugin, to be applied on regridding. * Added "regridded_title" argument to standardise CLI and updated tests. * Used global constant MOSG_GRID_ATTRIBUTES in standardisation and tidied up a few code style issues. * Fixed codacy issue. * Refactored forecast_period_coord to be acceptable to Codacy. * Fixed style errors. * Removed unnecessary type-cast. * Responded to review comments. * Responded to review comments. * Conversion of acceptance tests to pytest (#1026) * Squash pytest work * Update improver-tests CLI to use pytest * Specify pycodestyle error code * Fix pylint marker, mark more tests as slow * Bring back dechunking via temporary files * Remove coverage from acceptance tests * Backup skip in kgo_root function * Set time lagged ensemble basic acceptance test to xfail * Increase nccmp timeout to 5 minutes Some systems are really slow to run nccmp, as by default it reads small pieces from files and the -B option to buffer whole variables in memory is not available on older nccmp versions. * Change order of named args in cli/__init__.py * Convert acceptance tests to use clize * Fix standardise pytest after #1019 * Add blank line to quieten pylint * Pytest for 1195 (#1029) * Added pytests for convert to realizations * Deleted bats tests * Updated for clize * Update test_convert_to_realizations.py * Update test_convert_to_realizations.py * IMPRO-1476: Accumulate CLI (#1025) * Added an accumlate CLI. * Updated nowcast accumuate * Update lib/improver/cli/nowcast_accumulate.py * Update lib/improver/cli/nowcast_accumulate.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Apply suggestions from code review Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/cli/test_init.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/nowcast_accumulate.py * Updated nowcast accumuate * Apply suggestions from code review Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Updated for second review * Update lib/improver/cli/nowcast_accumulate.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Added pytest for accumulate * Added new pytest for accumulate * Updated pytests for accumulate * Removed bats test * Updated for codacy * IMPRO-1424: Making snow-falling-level more general (#1024) * Almost working apart from a mismatch for some points in the KGO. * Remove almost all mentions of snow from the psychrometric calculations. FallingSnowLevel plugin renamed to PhaseChangeLevel. * Update CLI to use PhaseChangeLevel plugin. * Most unit tests fixed. CLIs updated, CLI tests still needed. Committing prior to investigating PhaseChangeLevel process unit test failures. * Attribute tweak. * Additional check added to ensure wet bulb integral data is in descending height order within the input cube. This is no longer guaranteed as the wet bulb integral cube could notionally be created by some mechanism other than the IMPROVER plugin/CLI. * CLI tests updated. * Style fixes and CLI test changes. * New tests. * Remove last mentions of snow where inappropriate. * Pylint improvements. * Corrected a typo and expanded unit test doc-strings to be more descriptive. Also added an additional unit test for the sleet-rain phase change. * Review typo changes. * Merged in pytest changes and deleted redundant snow_falling_level test. * Replicate BATS test changes in pytest * Added a cli to get a remainder so that we can calculate sleet in the … (#1020) * Added a cli to get a remainder so that we can calculate sleet in the future. * Deleted trailing white space * Deleted pylint message * Putting back a line I accidentally deleted. * I added a plugin and rewrote the cli and tests to be able to filter sleet from snow and rain * Fixed a codacy error. * Added warning and test for negative cube values. * Deleting an else that wasn't needed. * Changes to the unit tests to use set_up_probability_cube * Deleting whitespaces. * Fixing indentation errors. * Changes to the docstrings. * Adding comments and sorting out variable names. * Modifying docstring in calculate_sleet_prob.py * Fixing codacy problem with test_calculate_sleet_probability.py * Modifying calculate_sleet_prob.py to run with MANDATORY ATTRIBUTES * Changes needed for pytest. * Fix codacy coverage and report combined unit/acceptance coverage (#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Remove BATS tests and associated code (#1045) * Fix duplicate clize output and errors (#1041) * Removes section that printed the error message to stdout just before it gets printed to stderr. * Adds filter for "Raises" sections in doc-strings which trigger unexplained error messages. * Improve forecast times (#1027) * Changed forecast_period_coord utility to create coordinate with expected datatype and units, which are not configurable - since we have decided to fail on save if these metadata do not conform. Updated time-lagging unit tests to set up appropriate cubes. * Rearranged unit tests to cover coordinate creation separately from top level manipulation. * Fixed trailing whitespace. * Tests with Github Actions (#1030) * Create tests.yml * Create environment.yml * Delete .travis.yml because GitHub actions > * Include style/doc build tests * Update environment.yml * Keep travis for the time being * conflict * Pylint with Github Actions (#1066) * Add pylint to actions * Update environment.yml * pin astroid fix pylint output * IMPRO-1534: convert extend radar mask to use clize (#1056) * Updated radar to use clize * Updated to remove float checking * Minor code style fixes (#1079) * Removed no-else-raise issues and reworked torturous if / else tangle in wind_gust_diagnostic.py. * Removed no-else-returns. * Removed unnecessary "pass" and "try-except-raise" statements. * Reworked loop in more pythonic way. * save atomically by writing to a temporary file and then renaming (#1065) * save atomically by writing to a temporary file and then renaming * stringify filename * IMPRO-1519: extrapolate clize (#1035) * Updated to remove speed and direction * Updated for clize * Updated nowcast acc tests. * Updated to move error handling into plugin * Updated to move error into plugin * IMPRO-1517: Update Orographic enhancement to use clize. (#1032) * Updated to use clize. * Updated type * Updated for errors * IMPRO-1523: Convert optical flow to clize (#1053) * Updated to clize and pytests Co-authored-by: @MoseleyS * Fixed extrapolate CLI tests. * Removed TODO. Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: Ben Fitzpatrick <ben.fitzpatrick@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: LaurenceBeard <laurence.beard@metoffice.gov.uk> Co-authored-by: Tomek Trzeciak <TomekTrzeciak@users.noreply.github.com> * Added pysteps to environment config for GitHub actions automated testing. * Update accumulate in nowcast feature branch (#1091) * Updated nowcast accumulate CLI to use new extrapolation method. * Removed commented-out blocks. * Relocates files new to this branch to their shiny new home. * Fixes import errors and reinstates lost module. * Fixes nowcast-accumulate CLI test * Fixed paths to test data. Removed test that will not work with pysteps extrapolation code. * New cubes nowcasting (#1150) * Removed legacy file. * Changed attribute inheritance to reflect wider IMPROVER convention. * Blacked nowcast files. * Two step advection (#1262) * Draft CLI to calculate optical flow components as perturbation of an initial field. * Removed unused arguments to optical flow CLIs. Added basic test for new CLI. * Blacked new code. * Updated CLI to take full forecast and slice as needed. * Added test for forecast period argument and time mismatch error. * Updated process docstring to better reflect CLI purpose. * Fixed bug where unadjusted rates were used for optical flow. * Optical flow from winds (#1278) * Wrote CLI. Needs tests and sanity check notebook. * Blacked new code. * Bypassed hideous black formatting. * Added subtraction of orographic enhancement and commented procedure. * Added test file. Need to generate input data - resolved wind components. * Some tweaks to test paths - will need to recreate when data generated. * Tidied up tests and debugged CLI. Have come across the amusing case of a radar observation with a 1 second forecast period. This will need fixing further up the chain. * Fixed and generated KGO. * Response to first review. * Fixed unused import and typo. * Moved CLI code into separate functions. Acceptance tests pass; need to add unit tests. * Tidied up and factored out function to perturb background flow. * Added unit tests for perturbations function. * Finished unit tests for new functions. * Added test for cube sorting in order of vlaidity time. * Blacked new code. * Added unit tests for perturbation function. * Updated licence information in new files. * Black & isort. * Fixed unit tests following merge from master. * Optical flow from winds parameters (#1318) * Updated optical flow perturbation code to use different tunable parameters. Will need to update acceptance test data. * Updated checksums for new acceptance test data. * Removed unneeded interface / plotting modules. * Removed unused method (optical flow by perturbing a previous forecast). * Pointed to central acceptance test data dirs. * Updated checksums for rearranged acceptance test data. * Removed unused argument. Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: Ben Fitzpatrick <ben.fitzpatrick@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: LaurenceBeard <laurence.beard@metoffice.gov.uk> Co-authored-by: Tomek Trzeciak <TomekTrzeciak@users.noreply.github.com>
MoseleyS
pushed a commit
to MoseleyS/improver
that referenced
this pull request
Aug 22, 2024
…oppv#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage
MoseleyS
added a commit
to MoseleyS/improver
that referenced
this pull request
Aug 22, 2024
* Clize: Combine CLI (metoppv#1034) * Removes argparser section from combine CLI and updates pytests with clize argument names. * Replaces = with blank in keyword args. * Removes "if __main__" and makes acceptance tests pass without the equals symbols. * Changes to convert feels like temperature to clize. (metoppv#1043) * Changes to convert feels like temperature to clize. * Deleting trailing whitespace. * Modified a line of code that was too long. * Put flag on new line as per Neil's suggestion. * Clizify weighted blending CLI (metoppv#1037) * First attempt at refactor. Pytests fail for the moment. * Fixed CLI and pytests. * Removed accidental whitespace. * Fixed pycodestyle issue. * Improved docstring. * Removed reference to "main". * Removed unnecessary numpy dependency. * Clize applied to apply-lapse-rate CLI (metoppv#1036) * Clize applied to apply-lapse-rate CLI * Replaces = with blank in keyword args. * Removes __main__ section and = separators in pytest argument setup. * Removes unused imports. * Small improvement following review. * Clizes create-grid-with-halo (metoppv#1046) * Clizes create-grid-with-halo * Removes default statement which is repeated in the help-string anyway. * Convert create-grid-with-halo CLI to clize (metoppv#1040) * Convert create-grid-with-halo CLI to clize * Update lib/improver/cli/create_grid_with_halo.py * Clize time lagging (2) (metoppv#1047) * Clizefied time lagging CLI. * Tidied up. * Updated several docstrings with correct types. * Removed unused "cycletime" argument and unnecessary __init__ and __repr__ methods. * Changed variable name and corrected plugin docstring. * Simplified function signature and call. * Convert wind-gust-diagnostic CLI to use Clize (metoppv#1054) * Convert wind-gust-diagnostic CLI to use Clize. * Minor edit to default arguments. * Removing stray trailing "o". * Conversion of extract to use clize. (metoppv#1052) * Conversion of extract to use clize. Required changes to user inputs due to treatment of comma separation by clize. * Pep fixes. * Remove __main__ calling. * change improver-extract CLI multiple constraint handling * Slightly expanded help to describe input requirements. * Modified to use clize. (metoppv#1055) * clize changes for uv-index (metoppv#1059) * Clize phase-change-level (metoppv#1058) * Modify phase-change-level to use clize. * Update lib/improver/cli/phase_change_level.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/acceptance/test_phase_change_level.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/acceptance/test_phase_change_level.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Clize wet-bulb-temperature-integral (metoppv#1060) * Modify wet-bulb-temperature-integral to use clize. * Update lib/improver/psychrometric_calculations/psychrometric_calculations.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Removed extraneous main. * Remove pylint warning about touching data having no effect. * Clize wind direction (metoppv#1062) * Modify wind-direction CLI to use clize. * Remove long comment. Rely upon plugin doc-string to retain information. * Convert apply-emos-coefficients to clize (metoppv#1038) * Convert apply-emos-coefficients to clize * apply-emos-coefficients pytest style changes * Convert blend-adjacent-points CLI to use clize (metoppv#1039) * Convert blend-adjacent-points CLI to use clize * Update mandatory kwargs * blend-adjacent-points pytest style update * Remove newline in blend-adjacent-points CLI tests * Clize threshold (metoppv#1064) * Removed unnecessary error handling. * Tidied up some names and errors. * Clizefied CLI and fixed tests. * Tidied up threshold CLI logic and docstrings. * Fixes pycodestyle errors (metoppv#1068) * Migrates spot-extract to use Clize. (metoppv#1048) * Migrates spot-extract to use Clize. * Clize standardise (metoppv#1049) * Updated CLI not to reuse names. * Removed unneeded test and moved error handling into "process" function. * Got everything working except list of coordinates to remove. * Fixed last test. * Removed "default is" from docstrings and tidied up some descriptions. * Made target grid and source landmask positional arguments, and removed unnecessary error. * Added type to landmask vicinity argument. * Clize topographic band CLIs (metoppv#1063) * Modify topographic band related CLIs to use clize. * Review changes. * Describes behaviour if optional output file is not supplied. (metoppv#1067) * between-thresholds CLI to clize (metoppv#1074) * Use clize for generate-landmask-ancillary CLI (metoppv#1071) * Impro 1497 clize percentiles to probabilities (metoppv#1069) * Changes to clize percentiles_to_probabilities, including updating docstrings. * Update lib/improver/cli/percentiles_to_probabilities.py Added additional * argument to separate out cube arguments and non-cube arguments Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/acceptance/test_percentiles_to_probabilities.py Adding --output-diagnostic-name before the value in test call. Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * neighbour-finding CLI to clize (metoppv#1073) * neighbour-finding CLI to clize * Update lib/improver/tests/acceptance/test_neighbour_finding.py Co-Authored-By: bayliffe <benjamin.ayliffe@metoffice.gov.uk> * wxcode CLI to clize (metoppv#1076) * wxcode CLI to clize * Adds back extended help as an option to the CLI. Code to generate help moved to wxcode utilities. This approach avoids the need to import the module unless requested and keeps the CLI tidy. * Refactor wxcode CLI help * make cli.wxcode.extend_help private * Added missing blank line that was making tests fail. * wind-downscaling CLI to clize (metoppv#1077) * wind-downscaling CLI to clize * Remove unused CLI option in wind downscaling * update-grid-metadata CLI to clize (metoppv#1075) * update-grid-metadata CLI to clize * update-grid-metadata changes following review * temporal-interpolate CLI to clize (metoppv#1078) * temporal-interpolate CLI to clize * Update lib/improver/cli/temporal_interpolate.py Co-Authored-By: fionaRust <fiona.rust@metoffice.gov.uk> * merge cubes at end of temporal-interpolate CLI Co-authored-by: fionaRust <fiona.rust@metoffice.gov.uk> * temp-lapse-rate CLI to clize (metoppv#1081) * Clizefied temperature lapse rate CLI. * Clarified why the temperature cube is a required argument. * Removed trailing whitespace. * temp-lapse-rate CLI, change landmask and orography to optional positional arguments Co-authored-by: cgsandford <35029690+cgsandford@users.noreply.github.com> * IMPRO-1516: clize nbhood land sea (metoppv#1042) * Updated to use clize * Updated pytests * Updated nbhood land and sea changed how it deals with radius/radii and deals with error handling. * Updated cli/nbhood names. * Updated radius doctrings. * Updated variable name * Updated to add new radii test * Updated kgo and changed tests input order * Create CLI for generating recursive filters alphas (metoppv#1083) * Update temporal_interpolate.py Removing a space that was causing an error * Updated to remove none-assignment and assert (metoppv#1094) * IMPRO-1539: clizify nbhood (metoppv#1092) * Clizify nbhood. This change includes the removal of the recursive filter options from the nbhood CLI and all associated tests. * Review changes. * Clize nbhood iterate with mask (metoppv#1096) * Clizify nbhood-iterate-with-mask. Remove a common utility that is no longer required. * IMPRO-1539: clizify nbhood (metoppv#1092) * Clizify nbhood. This change includes the removal of the recursive filter options from the nbhood CLI and all associated tests. * Review changes. * Review changes. * update generate-orographic-alphas test to reflect most common usage (metoppv#1098) * nbhood clizification tidy-up (metoppv#1100) * Changes following an inadvertently skipped review. * Removed redundant raises doc-string entries. * Convert recursive-filter CLI to use clize (metoppv#1099) * Fix codacy coverage and report combined unit/acceptance coverage (metoppv#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Changes to recursive filter to allow it to work with clize. * Deleting whitespaces and an extra line. * Fixing indentation issues. * Simplify recursive-filter CLI Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> * estimate-emos-coefficients CLI to clize (metoppv#1095) * Update estimate-emos-coefficients * fix tests * remove duplicated line in emos plugin * review feedback * set default tolerance to its value from argparser CLI Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> * Fix style in recursive filter CLI (metoppv#1102) * Silences "interpreted-text" warning when parsing apply-emos-coefficients CLI. (metoppv#1103) * Update lib/improver/cli/threshold.py * Clize porting review (metoppv#1104) * remove explicit defaults from docstrings * remove ' or None' from docstrings * orog -> orography * temp_lapse_rate renames, improver.constants without iris * warnings_on -> check_metadata * ecc_bounds_warning -> ignore_ecc_bounds, random_ordering -> randomise, reordering -> reorder, rebadging -> rebadge, num_realizations -> realizations_count, no_of_realizations -> realizations_count, no_of_percentiles -> percentiles_count * thresholds_dict -> bands_config * sum_or_fraction -> area_sum, re_mask -> remask * wts_calc_method -> weighting_method, wts_dict -> weighting_config * attributes_dict -> new_metadata * consistent naming of land-sea mask * cubelist -> cubes * minimum_dz -> similar_altitude * site list as positional argument * wind gust renames * extend_radar_mask renames * wind_downscaling renames * various input cube renames * combine: new_metadata -> bounds_config and drop one level of dict nesting * new_metadata -> attributes_config * rebadge or reorder based on presence of the raw ensemble argument * pep8 * remove tests subcommand, use improver-tests directly instead * one line command description * remove argparser * remove legacy support in clizefy decorator * document orography argument Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: cgsandford <35029690+cgsandford@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: fionaRust <fiona.rust@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net>
MoseleyS
added a commit
to MoseleyS/improver
that referenced
this pull request
Aug 22, 2024
* Implement pysteps semi-Lagrangian advection scheme (metoppv#958) * Added wrapper for pysteps semi-Lagrangian advection method. * Added pysteps to conda path in travis, and removed hard-coded pysteps path from advection module. * IMPRO-1354 use PySteps extrapolate method in CLI (metoppv#1018) * Switches extrapolation technique to PySteps. * Fixes syntax error. * Adds option to PySteps for Orographic Enhancement data to be None if data are not precipitation. * Adds use of attributes_dict to PySteps advection class. * Suppresses stdout message when importing PySteps. * Moves new CLI KGO to unique locations relating to the feature branch so that we don't break master. * Moves argument "attributes_dict" from init to process method in PySteps Extrapolate class. * Deals with Codacy issues and comments. * Removes options for handling non-precipitation fields. * Uses contextmanager instead of creating a class to redirect stdout. * Small improvement to contextmanager * Removes unused import and adds doc-string. * Merge branch (metoppv#1085) * Minor edits to reduce Read the Docs memory consumption (metoppv#1022) * Minor edits to try to reduce Read the Docs memory consumption. * Move environment.yml to cause less confusion, as this file is specific for Read the Docs. * Fixed bug caused by unnecessary processing of percentile values. (metoppv#1023) * Enable Travis build config validation (metoppv#1004) * Update .travis.yml * Update .travis.yml * IMPRO-1490: Wind vector to UV (metoppv#1012) * Added cli/vector_to_uv Added a cli to convert speed and direction to u and v. * Updated wind-vetor-to-uv to move imports locally * Update lib/improver/cli/wind_vector_to_uv.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Updated to remove wind from cli __init__.py * Update wind_vector_to_uv.py * Added tests for create constrainted inputcube converter * Updated to remove unused import * Update lib/improver/cli/__init__.py * Removed constraints loading. * Remved unused import * Removed old changes. * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/resolve_wind_components.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * IMPRO-1195: create convert_to_realizations CLI (metoppv#959) * Added cli/to_realizations. * Update -to-realizations CLI to use clize and rename * Add BATS tests using probabilities-to-realizations and percentiles-to-realizations tests * Update lib/improver/cli/convert_to_realizations.py * Apply suggestions from code review * Mandatory attributes (metoppv#1019) * Added a function to generate default values for mandatory attributes. * Set up default title attribute and removal of grid attributes in spot extraction. * Handled and checked for mandatory attributes in cube creation utility. * Moved mandatory attribute definitions to constants module. * Added unit tests and tidied up mandatory attribute generation function. * Updated blending code to add mandatory attributes. * Used constant time types and units in forecast_times.py. * Added new title argument to StandardiseGridAndMetadata plugin, to be applied on regridding. * Added "regridded_title" argument to standardise CLI and updated tests. * Used global constant MOSG_GRID_ATTRIBUTES in standardisation and tidied up a few code style issues. * Fixed codacy issue. * Refactored forecast_period_coord to be acceptable to Codacy. * Fixed style errors. * Removed unnecessary type-cast. * Responded to review comments. * Responded to review comments. * Conversion of acceptance tests to pytest (metoppv#1026) * Squash pytest work * Update improver-tests CLI to use pytest * Specify pycodestyle error code * Fix pylint marker, mark more tests as slow * Bring back dechunking via temporary files * Remove coverage from acceptance tests * Backup skip in kgo_root function * Set time lagged ensemble basic acceptance test to xfail * Increase nccmp timeout to 5 minutes Some systems are really slow to run nccmp, as by default it reads small pieces from files and the -B option to buffer whole variables in memory is not available on older nccmp versions. * Change order of named args in cli/__init__.py * Convert acceptance tests to use clize * Fix standardise pytest after metoppv#1019 * Add blank line to quieten pylint * Pytest for 1195 (metoppv#1029) * Added pytests for convert to realizations * Deleted bats tests * Updated for clize * Update test_convert_to_realizations.py * Update test_convert_to_realizations.py * IMPRO-1476: Accumulate CLI (metoppv#1025) * Added an accumlate CLI. * Updated nowcast accumuate * Update lib/improver/cli/nowcast_accumulate.py * Update lib/improver/cli/nowcast_accumulate.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Apply suggestions from code review Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/tests/cli/test_init.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Update lib/improver/cli/nowcast_accumulate.py * Updated nowcast accumuate * Apply suggestions from code review Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Updated for second review * Update lib/improver/cli/nowcast_accumulate.py Co-Authored-By: Aaron Hopkinson <arh89@users.noreply.github.com> * Added pytest for accumulate * Added new pytest for accumulate * Updated pytests for accumulate * Removed bats test * Updated for codacy * IMPRO-1424: Making snow-falling-level more general (metoppv#1024) * Almost working apart from a mismatch for some points in the KGO. * Remove almost all mentions of snow from the psychrometric calculations. FallingSnowLevel plugin renamed to PhaseChangeLevel. * Update CLI to use PhaseChangeLevel plugin. * Most unit tests fixed. CLIs updated, CLI tests still needed. Committing prior to investigating PhaseChangeLevel process unit test failures. * Attribute tweak. * Additional check added to ensure wet bulb integral data is in descending height order within the input cube. This is no longer guaranteed as the wet bulb integral cube could notionally be created by some mechanism other than the IMPROVER plugin/CLI. * CLI tests updated. * Style fixes and CLI test changes. * New tests. * Remove last mentions of snow where inappropriate. * Pylint improvements. * Corrected a typo and expanded unit test doc-strings to be more descriptive. Also added an additional unit test for the sleet-rain phase change. * Review typo changes. * Merged in pytest changes and deleted redundant snow_falling_level test. * Replicate BATS test changes in pytest * Added a cli to get a remainder so that we can calculate sleet in the … (metoppv#1020) * Added a cli to get a remainder so that we can calculate sleet in the future. * Deleted trailing white space * Deleted pylint message * Putting back a line I accidentally deleted. * I added a plugin and rewrote the cli and tests to be able to filter sleet from snow and rain * Fixed a codacy error. * Added warning and test for negative cube values. * Deleting an else that wasn't needed. * Changes to the unit tests to use set_up_probability_cube * Deleting whitespaces. * Fixing indentation errors. * Changes to the docstrings. * Adding comments and sorting out variable names. * Modifying docstring in calculate_sleet_prob.py * Fixing codacy problem with test_calculate_sleet_probability.py * Modifying calculate_sleet_prob.py to run with MANDATORY ATTRIBUTES * Changes needed for pytest. * Fix codacy coverage and report combined unit/acceptance coverage (metoppv#1033) * Fix codacy coverage reporting bin/improver uses lib as the working directory, so the previous coverage path was wrong. * Report combined unit and acceptance test coverage * Remove BATS tests and associated code (metoppv#1045) * Fix duplicate clize output and errors (metoppv#1041) * Removes section that printed the error message to stdout just before it gets printed to stderr. * Adds filter for "Raises" sections in doc-strings which trigger unexplained error messages. * Improve forecast times (metoppv#1027) * Changed forecast_period_coord utility to create coordinate with expected datatype and units, which are not configurable - since we have decided to fail on save if these metadata do not conform. Updated time-lagging unit tests to set up appropriate cubes. * Rearranged unit tests to cover coordinate creation separately from top level manipulation. * Fixed trailing whitespace. * Tests with Github Actions (metoppv#1030) * Create tests.yml * Create environment.yml * Delete .travis.yml because GitHub actions > * Include style/doc build tests * Update environment.yml * Keep travis for the time being * conflict * Pylint with Github Actions (metoppv#1066) * Add pylint to actions * Update environment.yml * pin astroid fix pylint output * IMPRO-1534: convert extend radar mask to use clize (metoppv#1056) * Updated radar to use clize * Updated to remove float checking * Minor code style fixes (metoppv#1079) * Removed no-else-raise issues and reworked torturous if / else tangle in wind_gust_diagnostic.py. * Removed no-else-returns. * Removed unnecessary "pass" and "try-except-raise" statements. * Reworked loop in more pythonic way. * save atomically by writing to a temporary file and then renaming (metoppv#1065) * save atomically by writing to a temporary file and then renaming * stringify filename * IMPRO-1519: extrapolate clize (metoppv#1035) * Updated to remove speed and direction * Updated for clize * Updated nowcast acc tests. * Updated to move error handling into plugin * Updated to move error into plugin * IMPRO-1517: Update Orographic enhancement to use clize. (metoppv#1032) * Updated to use clize. * Updated type * Updated for errors * IMPRO-1523: Convert optical flow to clize (metoppv#1053) * Updated to clize and pytests Co-authored-by: @MoseleyS * Fixed extrapolate CLI tests. * Removed TODO. Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: Ben Fitzpatrick <ben.fitzpatrick@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: LaurenceBeard <laurence.beard@metoffice.gov.uk> Co-authored-by: Tomek Trzeciak <TomekTrzeciak@users.noreply.github.com> * Added pysteps to environment config for GitHub actions automated testing. * Update accumulate in nowcast feature branch (metoppv#1091) * Updated nowcast accumulate CLI to use new extrapolation method. * Removed commented-out blocks. * Relocates files new to this branch to their shiny new home. * Fixes import errors and reinstates lost module. * Fixes nowcast-accumulate CLI test * Fixed paths to test data. Removed test that will not work with pysteps extrapolation code. * New cubes nowcasting (metoppv#1150) * Removed legacy file. * Changed attribute inheritance to reflect wider IMPROVER convention. * Blacked nowcast files. * Two step advection (metoppv#1262) * Draft CLI to calculate optical flow components as perturbation of an initial field. * Removed unused arguments to optical flow CLIs. Added basic test for new CLI. * Blacked new code. * Updated CLI to take full forecast and slice as needed. * Added test for forecast period argument and time mismatch error. * Updated process docstring to better reflect CLI purpose. * Fixed bug where unadjusted rates were used for optical flow. * Optical flow from winds (metoppv#1278) * Wrote CLI. Needs tests and sanity check notebook. * Blacked new code. * Bypassed hideous black formatting. * Added subtraction of orographic enhancement and commented procedure. * Added test file. Need to generate input data - resolved wind components. * Some tweaks to test paths - will need to recreate when data generated. * Tidied up tests and debugged CLI. Have come across the amusing case of a radar observation with a 1 second forecast period. This will need fixing further up the chain. * Fixed and generated KGO. * Response to first review. * Fixed unused import and typo. * Moved CLI code into separate functions. Acceptance tests pass; need to add unit tests. * Tidied up and factored out function to perturb background flow. * Added unit tests for perturbations function. * Finished unit tests for new functions. * Added test for cube sorting in order of vlaidity time. * Blacked new code. * Added unit tests for perturbation function. * Updated licence information in new files. * Black & isort. * Fixed unit tests following merge from master. * Optical flow from winds parameters (metoppv#1318) * Updated optical flow perturbation code to use different tunable parameters. Will need to update acceptance test data. * Updated checksums for new acceptance test data. * Removed unneeded interface / plotting modules. * Removed unused method (optical flow by perturbing a previous forecast). * Pointed to central acceptance test data dirs. * Updated checksums for rearranged acceptance test data. * Removed unused argument. Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk> Co-authored-by: Ben Fitzpatrick <ben.fitzpatrick@metoffice.gov.uk> Co-authored-by: Neil <43375279+neilCrosswaite@users.noreply.github.com> Co-authored-by: Aaron Hopkinson <arh89@users.noreply.github.com> Co-authored-by: Tom Gale <tom@tjtg.net> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: Katie-Howard <kathryn.howard@metoffice.gov.uk> Co-authored-by: LaurenceBeard <laurence.beard@metoffice.gov.uk> Co-authored-by: Tomek Trzeciak <TomekTrzeciak@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The coverage reporting path for
pytest-covwas previously set incorrectly, causing no coverage data file to be created.This PR also applies coverage testing to the acceptance tests, so acceptance test coverage is appended to the unit test coverage and the combined coverage result gets reported to codacy and codecov. This will currently have no effect, as all the acceptance tests are skipped on Travis CI.