Set decimal precision

[rabidaudio]

The following is an example schema output by tap-postgres for a numeric column:

{"type":"SCHEMA","stream":"foobar","schema":{"type":"object","properties":{"id":{"type":["integer"],"minimum":-2147483648,"maximum":2147483647},"sample_numeric":{"type":["null","number"],"exclusiveMaximum":true,"maximum":100000000000000000000000000000000000000000000000000000000000000,"multipleOf":1E-38,"exclusiveMinimum":true,"minimum":-100000000000000000000000000000000000000000000000000000000000000}}},"key_properties":["id"],"bookmark_properties":[]}
{"type":"RECORD","stream":"foobar","record":{"id":1,"sample_numeric":0.000913808181253534},"version":1596488217992,"time_extracted":"2020-08-03T20:56:57.992931Z"}

This results in the following error when validating the data against the JSON Schema:

  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/bin/target-postgres", line 11, in <module>
    load_entry_point('target-postgres===meltano.1.1.6', 'console_scripts', 'target-postgres')()
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/target_postgres/__init__.py", line 151, in main
    state = persist_lines(config, input)
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/target_postgres/__init__.py", line 80, in persist_lines
    validators[stream].validate(float_to_decimal(o['record']))
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/jsonschema/validators.py", line 129, in validate
    for error in self.iter_errors(*args, **kwargs):
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/jsonschema/_validators.py", line 306, in properties_draft4
    schema_path=property,
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/Users/cjk/fixd/meltano/.meltano/loaders/target-postgres/venv/lib/python3.7/site-packages/jsonschema/_validators.py", line 129, in multipleOf
    failed = instance % dB
decimal.InvalidOperation: [<class 'decimal.DivisionImpossible'>]

The decimal.DivisionImpossible error is this issue, caused by running out of the default decimal precision when trying to run Decimal('0.000666259154868687') % Decimal('1E-38') (for verifying multipleOf).

tap-postgres is using multipleOf to effectively report the scale of the column.

This PR walks all new JSON Schemas, and uses minimum, maximum, and multipleOf to try and predict the required precision and update the decimal context appropriately.

[DouweM]

@rabidaudio This is clever!

• https://gitlab.com/meltano/target-snowflake just unconditionally sets the precision to 40: https://gitlab.com/meltano/target-snowflake/-/blob/master/target_snowflake/target_snowflake.py#L43, and
• https://github.com/transferwise/pipelinewise-target-postgres simply doesn't support higher precision than 15 decimal places: https://github.com/transferwise/pipelinewise-target-postgres/blob/master/target_postgres/__init__.py#L123-L130

[DouweM]

@rabidaudio Released version meltano.1.1.7!