Skip to content

Fill null values with 'unknown'#33

Merged
egrace479 merged 4 commits into
mainfrom
na-fix
Jul 21, 2023
Merged

Fill null values with 'unknown'#33
egrace479 merged 4 commits into
mainfrom
na-fix

Conversation

@egrace479
Copy link
Copy Markdown
Member

Fills null values in data with "unknown" and adds two test files with null values (one for image retrieval and one without):

  • graphs.py: mapping function filters out values in lat-lon column that contain the string 'unknown', i.e., if either latitude or longitude is unknown.
  • query.py:
    • get_data: gets information about included columns (from dashboard.py), sets null values to 'unknown', returns only suggested/required columns.
    • get_filenames: filters out any entries with 'unknown' Image_filename or file_url. Will return number of matching entries that are missing these values if no other images exist. Next update may do this when there are matching that can't be displayed in addition to the returned image(s).
    • get_images: Now will note that entries exist, but no image path or filename is available.
  • Adjusted query tests to account for the changes in get_data and get_filenames functions.

Fixes Issues #29 and #30

@egrace479 egrace479 added the bug Something isn't working label Jul 20, 2023
@egrace479 egrace479 added this to the MVP ImgDP milestone Jul 20, 2023
Comment thread components/query.py Outdated
@egrace479 egrace479 requested a review from johnbradley July 20, 2023 22:24
@egrace479
Copy link
Copy Markdown
Member Author

get_filenames returns ValueError indicating there are no images matching the user's selections, further states "Unknown filename(s) or path(s)" if there are still matching entries.

Copy link
Copy Markdown
Member

@hlapp hlapp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd note that applying a search or filter to data that returns no results is in general not in error, and arguably should result in an empty result set, not an exception. It seems the picture here is complicated by the fact that based on filtering alone the result set isn't empty, but becomes empty once we remove rows that don't reference a valid image? Then the question is, is it an error to have rows without image to begin with, or is it not.

Regardless of the answer, I feel this doesn't need to be sorted out here. We're at the prototype stage, and now the function at least either returns a result of a consistent type, or it doesn't return a result (by raising an exception), which seems good enough. So perhaps open an issue with the above question to be considered down the road if and when there's a lot more continued development.

Comment thread components/query.py
Copy link
Copy Markdown
Contributor

@johnbradley johnbradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@egrace479 egrace479 mentioned this pull request Jul 21, 2023
@egrace479 egrace479 merged commit 6d95312 into main Jul 21, 2023
This was referenced Jul 21, 2023
@egrace479 egrace479 deleted the na-fix branch August 1, 2023 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants