Error parsing JSON

[thodson-usgs]

    @elbeejay it has fixed the original issue. Maybe unrelated, but when I try to get the data from  **all** the sites in Maryland, it gives me an error related to parsing the JSON. My assumption is maybe the `site` is too large and the webAPI responds with an error (?) which fails to be parsed. Is there any way to have better error messages? e.g. area is too large for fetching data?

Originally posted by @SorooshMani-NOAA in #62 (comment)

[thodson-usgs]

@SorooshMani-NOAA,
could you provide a usage example, so we can verify the issue.
In the meantime, I don't remember the exact site limit, but you should be able to work in blocks of 100 or so (also try 10, 50, 500 or 1000).

[elbeejay]

I suspect this is a minimal reproducible example of the error (for which the corresponding error message, I agree, is not very helpful):

>>> import dataretrieval.nwis as nwis
>>> sites, _ = nwis.what_sites(stateCd='MD')
>>> iv, _ = nwis.get_iv(sites=sites.site_no.values.tolist())
Traceback (most recent call last):
  File "/Users/jhariharan/mambaforge/envs/dataretrieval/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/Users/jhariharan/mambaforge/envs/dataretrieval/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/Users/jhariharan/mambaforge/envs/dataretrieval/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/jhariharan/mambaforge/envs/dataretrieval/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jhariharan/Documents/dataretrieval-1/dataretrieval/nwis.py", line 493, in get_iv
    return _iv(startDT=start, endDT=end, sites=sites,
  File "/Users/jhariharan/Documents/dataretrieval-1/dataretrieval/nwis.py", line 499, in _iv
    df = _read_json(response.json())
  File "/Users/jhariharan/mambaforge/envs/dataretrieval/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

A quick look at the API query response shows a 414 response which is related to a URL that is too long. I will be seeing if we can parse that information to return a more useful error message to the user within the dataretrieval package.

[SorooshMani-NOAA]

@thodson-usgs, sorry for the late reply, yes the comment from @elbeejay covers my use-case. Thanks!