[AIRFLOW-2612] Enable some Hive-related tests on CI

[sekikn]

Make sure you have checked all steps below.

JIRA

• My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/browse/AIRFLOW-2612
  • In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a JIRA issue.

Description

• Here are some details about my PR, including screenshots of any UI changes:

tests/operators/hive_operator.py has test cases
called HiveServer2Test and HivePrestoTest,
which are disabled by default.
This PR enables the former as the first step
to improve test coverage for Hive-related
hooks and operators.

HiveServer2Test had the following tests:

• test_select_conn, test_multi_statements, test_to_csv, test_select_conn_with_schema, test_get_results_with_schema, test_get_records_with_schema, test_get_pandas_df_with_schema: moved to TestHiveServer2Hook in tests/hooks/test_hive_hook.py since they were tests for HiveServer2Hook
• test_get_metastore_databases: removed since it was a test for HiveMetastoreHook and TestHiveMetastoreHook in tests/hooks/test_hive_hook.py already has a similar test called test_get_partitions
• connect_mock: removed since it was unused

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

This PR enables the tests described above.

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.

Code Quality

• Passes git diff upstream/master -u -- "*.py" | flake8 --diff

[codecov-io]

Codecov Report

Merging #3498 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3498   +/-   ##
=======================================
  Coverage   77.34%   77.34%           
=======================================
  Files         204      204           
  Lines       15159    15159           
=======================================
  Hits        11725    11725           
  Misses       3434     3434

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9cb54f...17a3839. Read the comment docs.

[Fokko]

@gglanzani PTAL

[gglanzani]

Hi @sekikn,

I am against merging this now. PR #3432 adds lots of tests, and allows tests to run on Python 2 (plus it doesn't mock them).

Let's wait until that is merge and then take a look at what can be kept from this.

/cc @Fokko

[sekikn]

@gglanzani Sure thing! I missed that PR.

[sekikn]

@gglanzani I found that #3432 covers almost all of tests implemented in tests.operators.hive_operator:HiveServer2Test. Great work!
So I just moved test_multi_statements to tests.hooks.test_hive_hook:TestHiveServer2Hook and removed others.

[gglanzani]

@Fokko +1 on this one. All the tests that were mocked are now in #3432 indeed.

The tests that have been removed, moreover, were all mocking the real thing.

And the multiple statements test is a welcome addition