Robust signature decoding

[tomato42]

Ensure that the signature verification raises the same exception irrespective if the error is caused by mismatch of the hash value or because the encoding of the signature is invalid

fixes #114

[coveralls]

Coverage increased (+0.5%) to 87.009% when pulling 932b7e3 on tomato42:robust-sig-decode into 72621b1 on warner:master.

[coveralls]

Coverage increased (+1.005%) to 87.489% when pulling b96e02e on tomato42:robust-sig-decode into 72621b1 on warner:master.

[rjrelyea]

it basically looks good with one exception:
line 138 in function remove_integer. You check for nbytes < 0x80 and say 'Negative integers not supported'. I think this is checking that the length of the integer is less than 128 bytes (1024 bits), not that the first bit of the integer is on (indicating that the integer is negative). This seems wrong. Either the error message or the code is wrong. I presume the latter since in RSA signatures, we often have integer lengths longer than 128 (like 2048 bit keys).

[tomato42]

You check for nbytes < 0x80 and say 'Negative integers not supported'. I think this is checking that the length of the integer is less than 128 bytes (1024 bits),

no, nbytes is the first byte of the part of the buffer that represents the integer – it's the most significant byte – I'll rename the variable to make it more readable

[tomato42]

@rjrelyea: updated, I've also added more checks to verify if the values are correct DER encodings (not just BER), so please take a second look

[rjrelyea]

OK, that looks good. nbytes was probably from the copy of dealing with the length field,
Your checks for minimal integer and minimal length encoding are correct.

[opoplawski]

Is there an ETA for a 0.14 release? I just got CVE bugs filed against this in Fedora. Thanks.

[tomato42]

@opoplawski actually, I'm working on 0.13.3 in #124 – there has been too little testing in master to consider it 0.14-worthy