Skip to content

Some adjustments to simplify ingestion of data #158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mortenwh wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Some adjustments to simplify ingestion of data #158

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
5dd4868
140: exclude datasets without geographic_location
mortenwh Dec 17, 2020
e3149f3
Merge branch 'master' of github.com:mortenwh/django-geo-spaas
mortenwh Mar 26, 2021
612aa94
exception on valueerror
mortenwh Mar 26, 2021
8c8ae05
allow a simpler way of providing the platfrom and instrument metadata
mortenwh Mar 27, 2021
d012a0f
corrected app name
Apr 22, 2021
edddb69
Merge branch 'master' of github.com:mortenwh/django-geo-spaas
mortenwh Apr 22, 2021
ca18a17
organise products in subfolders based on the dataset time_coverage_start
mortenwh Feb 2, 2023
fd4c33b
do not populate all vocabularies
mortenwh Jun 28, 2023
8ba94ec
change data_center to NO/MET instead of NERSC
mortenwh Jun 28, 2023
fcd0388
adjust to MET standard
mortenwh Aug 20, 2023
d25f47e
warnings to log at debug level
mortenwh Aug 20, 2023
eb5d5fb
close db connection
mortenwh Aug 20, 2023
f6be534
Hack to adapt to changes in gcmd platform list
mortenwh Oct 2, 2023
d17cdeb
handle FileExistsError
mortenwh Jun 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
warnings to log at debug level
  • Loading branch information
@mortenwh
mortenwh committed Aug 20, 2023
commit d25f47e98d36a475e433f819e90133d8bc4157a5
8 changes: 4 additions & 4 deletions geospaas/nansat_ingestor/managers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import json
import logging
import uuid
import warnings

import pythesint as pti
from django.contrib.gis.geos import WKTReader
Expand Down Expand Up @@ -100,7 +100,7 @@ def get_or_create(self,
existing_ds = None
for name in default_char_fields:
if name not in n_metadata:
warnings.warn('%s is not provided in Nansat metadata!' % name)
logging.debug('%s is not provided in Nansat metadata!' % name)
# prevent overwriting of existing values by defaults
if existing_ds:
options[name] = existing_ds.__getattribute__(name)
Expand All @@ -123,12 +123,12 @@ def get_or_create(self,
value = default_foreign_keys[name]['value']
model = default_foreign_keys[name]['model']
if name not in n_metadata:
warnings.warn('%s is not provided in Nansat metadata!' % name)
logging.debug('%s is not provided in Nansat metadata!' % name)
else:
try:
value = json.loads(n_metadata[name])
except:
warnings.warn('%s value of %s metadata provided in Nansat is wrong!' %
logging.debug('%s value of %s metadata provided in Nansat is wrong!' %
(n_metadata[name], name))
if existing_ds:
options[name] = existing_ds.__getattribute__(name)
Expand Down