Cannot validate Redfish schemas from last redfish spec

[frsauvage]

Hi,
I cannot validate my simple Chassis schema with jsonschema.
I downloaded last redfish schemas DSP8010.
I tried to validate with :

mychassis.json

{
"@odata.type": "#Chassis.v1_7_0.Chassis",
"Id": "Rack_1",
"Name": "Sequana2 Rack",
"ChassisType": "Rack",
"Model": "Sequana 2 Ancien Rack",
"WeightKg": 123,
"WidthMm": 654,
"Manufacturer": "Bull",
"SerialNumber": "3M220100SL",
"PartNumber": "",
"PowerState": "On",
"@odata.context": "/redfish/v1/$metadata#Chassis.Chassis",
"@odata.id": "/redfish/v1/Chassis/Rack_1",
"@Redfish.Copyright": "Copyright 2014-2016 Distributed Management Task Force, Inc. (DMTF). For the full DMTF copyright policy, see http://www.dmtf.org/about/policies/copyright."
}

jsonschema -i mychassis.json jsonschema/Chassis.v1_7_0.json

Error:

File "/bin/jsonschema", line 11, in <module> sys.exit(main()) File "/usr/lib/python2.7/site-packages/jsonschema/cli.py", line 73, in main sys.exit(run(arguments=parse_args(args=args))) File "/usr/lib/python2.7/site-packages/jsonschema/cli.py", line 84, in run for error in validator.iter_errors(instance): File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 323, in iter_errors for error in errors: File "/usr/lib/python2.7/site-packages/jsonschema/_validators.py", line 263, in ref for error in validator.descend(instance, resolved): File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 339, in descend for error in self.iter_errors(instance, schema): File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 323, in iter_errors for error in errors: File "/usr/lib/python2.7/site-packages/jsonschema/_validators.py", line 286, in properties schema_path=property, File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 339, in descend for error in self.iter_errors(instance, schema): File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 323, in iter_errors for error in errors: File "/usr/lib/python2.7/site-packages/jsonschema/_validators.py", line 259, in ref scope, resolved = validator.resolver.resolve(ref) File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 734, in resolve return url, self._remote_cache(url) File "/usr/lib/python2.7/site-packages/functools32/functools32.py", line 400, in wrapper result = user_function(*args, **kwds) File "/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 744, in resolve_from_url raise exceptions.RefResolutionError(exc) jsonschema.exceptions.RefResolutionError: No JSON object could be decoded
Any suggestion?

Best Regards,
Francine

[Julian]

Hi, hard to answer this without a reproducible example (your schema at least).

That traceback is telling you it hit some URL and got back something that wasn't JSON, but beyond that impossible to tell without whatever you're running there (the schema).

[frsauvage]

Hi Julian, I investigate a lot.
I git/clone jsonschema. I debug it. I put a breakpoint on jsonchema.compat.geturl (see below).
Weird.

Well, I have a proxy, i guess it could be the origin of this issue.

There is an exception just after the breakpoint with HttpError 403 Forbidden (See attachement in the Visual Code terminal on the bottom when I use a terminal)

The url is accessible from a browser on the same machine.

Any idea ? I guess I will change geturl to request library.

Regards,
Francine

[Julian]

I don't see the relationship there between what you're breaking on -- the traceback is saying the URL it hit is returning a 403. Why it is returning a 403, or what URL is being hit and why (probably because you have a $ref in your schema?) is the question -- but, and apologies for this, the issue tracker isn't really the place for these sorts of things.

It's for bug reports or feature requests -- if you need some help debugging there are other channels where that help is more than welcome, though you still will likely need to provide the reproducible example for others to help. https://sscce.org is a good site if you want some guidelines on what that looks like. And the JSON Schema slack channel is probably your first bet on where specifically I think.

[frsauvage]

I finally understand:
Redfish site does NOT allow request without User-Agent Header.
I added in compat.py

    def urlopen(*args, **kwargs):
        url=args[0]
        req = urllib2.Request(url, headers= {'User-Agent': 'Mozilla/5.0'})
        url_open = urllib2.urlopen(req)
        return contextlib.closing(url_open) 

I guess you should take care of that :

Regards,
Francine

[Julian]

Are you sure that no header is being sent at all? I haven't looked in awhile, but very likely one is being sent, but the site you're talking to rejects scripting user agents. Changing it in the way you have likely evades their "ban" on scripting user agents since you effectively now have a user agent it doesn't know about or assumes might be a browser, but that's not something that jsonschema is going to change (either the site you're talking to doesn't want to be talked to in this way, which is something you need to check with the site owner, or its somehow misconfigured).

It's true that jsonschema should likely set its own user agent that indicates requests coming specifically from it, but spoofing that to be some user agent that looks like a browser isn't going to happen, it'd just be the python-requests user agent (or whatever HTTP library we use), with the additional info that it comes from jsonschema.

But yeah if you do find that no user agent header is being set at all that certainly should be fixed.

[frsauvage]

Are you sure that no header is being sent at all?
@Julian
Yes, I swear :-) no "User-Agent" is sent. Adding it solve my issue definitely.
See https://stackoverflow.com/questions/38489386/python-requests-403-forbidden

I don't know why Redfish / DMTF site cares about it.

Thank you for your help and your reactivity,
Francine

[Julian]

jsonschema -i =(printf '""') =(printf '{"$ref": "http://localhost:8080/integer.json"}')

with a running webserver on 8080, and with requests installed, sends a user agent of: "python-requests/2.22.0" -- i.e. the default requests user agent.

Without requests (via urllib2), it sends: "Python-urllib/3.7", i.e. the default urllib user agent.

2019-10-16T16:30:00-0400 [twisted.python.log#info] "127.0.0.1" - - [16/Oct/2019:20:30:00 +0000] "GET /integer.json HTTP/1.1" 200 26 "-" "python-requests/2.22.0"
2019-10-16T16:34:36-0400 [twisted.python.log#info] "127.0.0.1" - - [16/Oct/2019:20:34:36 +0000] "GET /integer.json HTTP/1.1" 200 26 "-" "Python-urllib/3.7"

What are you seeing that tells you no User-Agent is being sent?