Enhance datatype determination and improve service_ok util

[KaiVolland]

This PR contains two enhancements.

1. The datatype determination for featuretype attributes is updated to support all possible variants of the GML spec. Currently only the simplest case (<xsd:element minOccurs="0" maxOccurs="1" name="objectid" type="xsd:int"/>) is supported.

2. The service_ok util now uses requests.get with stream=True. This adds support for a known bug in ArcGIS servers where the response header of the HEAD request response with the wrong content-type. stream=True should ensure, that there is no data fetched with the request. The traffic should be almost the same as with the HEAD request.

[KaiVolland]

Hi @geographika. Maybe you can have a look at this. 👋

[geographika]

Thanks (and hi!) @KaiVolland - a very nice addition. I've been testing the branch using pip install git+https://github.com/KaiVolland/OWSLib.git@arcgis-server-support and the code below:

import json
from owslib.wfs import WebFeatureService

WFS_SERVICE_URL = 'https://geoportal.stadt-koeln.de/arcgis/services/basiskarten/adressen_stadtteil/MapServer/WFSServer?SERVICE=WFS&request=GetCapabilities'

wfs = WebFeatureService(WFS_SERVICE_URL, version="2.0.0")
schema = wfs.get_schema('adressen_stadtteil:Blumenberg')
print(json.dumps(schema, indent=4))

Output:

{
    "properties": {
        "objectid": "int",
        "Nummer": "string",
        "Adresse": "string",
        "Strassenschl\u00fcssel": "string",
        "Stra\u00dfe": "string",
        "Amtl._Stra\u00dfenname": "string",
        "Stra\u00dfe__Stadtbezirk_": "string",
        "Hausnummer": "string",
        "Geb\u00e4ude_ID_ALKIS": "string",
        "Nutzung__ALKIS_Geb\u00e4udefunktion_": "string",
        "Blockseite": "string",
        "Stadtviertel__Nr._": "string",
        "Stadtviertel": "string",
        "Statistisches_Quartier__Nr._": "string",
        "Statistisches_Quartier": "string",
        "Stadtteil__Nr._": "string",
        "Stadtteil": "string",
        "Stadtbezirk__Nr._": "string",
        "Stadtbezirk": "string",
        "Stimmbezirk": "string",
        "Postleitzahlgebiet": "string",
        "Schiedsamtsbezirk": "string",
        "Sozialraum__Nr._": "string",
        "Sozialraum": "string",
        "globalid": "string"
    },
    "required": [
        "objectid",
        "globalid"
    ],
    "geometry": "Point",
    "geometry_column": "shape"
}

If I try and run the code without your PR I'm getting errors, so I guess this also fixes errors with missing types?

>>> schema = wfs.get_schema('adressen_stadtteil:Blumenberg')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\VirtualEnvs\misc\lib\site-packages\owslib\feature\__init__.py", line 391, in get_schema
    return get_schema(self.url, typename, self.version, auth=self.auth)
  File "C:\VirtualEnvs\misc\lib\site-packages\owslib\feature\schema.py", line 62, in get_schema
    return _construct_schema(elements, nsmap)
  File "C:\VirtualEnvs\misc\lib\site-packages\owslib\feature\schema.py", line 126, in _construct_schema
    data_type = element.attrib["type"].replace(gml_key + ":", "")
  File "src\\lxml\\etree.pyx", line 2548, in lxml.etree._Attrib.__getitem__

For mocking server results an option is to use pytest_httpserver added in #978 with an example at https://github.com/geopython/OWSLib/pull/978/files#diff-6830fccba0ba34d726c392cad7487e14412cb980a123aa263f93370b95b69ba3

I'm also fine with the monkeypatch approach.
Noted re stream=True - makes sense to go to a GET request without worrying about HEAD.

It would be nice to get another sample test with a few other datatypes if you have one - most of the current example are xsd:string.

I'm +1 on merging, but will leave open for a few days to see if anyone else has feedback.

[KaiVolland]

Thanks for the feedback. I parametrized the data_type tests to cover more cases.

I kept the new test (tests/test_wfs_schema_arcgis_server.py) similiar to the existing one so i used the monkey test. 🙂

[geographika]

Merging - thanks @KaiVolland.

[KaiVolland]

Is there any chance this will be released soon? We rely on this fixes in a project but publishing a package with git+https://github.com/* does not seem to work.

[geographika]

Is there any chance this will be released soon? We rely on this fixes in a project but publishing a package with git+https://github.com/* does not seem to work.

@KaiVolland - I'll raise this at a GeoPython call tomorrow, with an aim to release by the end of the month.

[geographika]

@KaiVolland - new release available on PyPI at https://pypi.org/project/OWSLib/ (see also https://github.com/geopython/OWSLib/releases/tag/0.34.0)