AMBARI-24745. Enable encryption of sensitive data in Ambari DB using …

[ghost]

…Ambari CLI (dlysnichenko)

What changes were proposed in this pull request?

Enable encryption of sensitive data on Ambari DB using Ambari CLI. This is an update to the existing "ambari-server setup-security", option #2 - Encrypt passwords stored in ambari.properties file.

In addition to what the current script does, the following must also happen:

• Change the subtitle to "Encrypt passwords managed by Ambari."
• Authenticate an Ambari administrator user
• Ask the user if they want to encrypt sensitive service configuration data
• Add command line options for authenticating an Ambari administrator and whether sensitive service configuration data is to be encrypted
• Issue a request to Ambari to encrypt sensitive data (if the user wants this)

How was this patch tested?

Unit tests are pending
Checked encryption / re-encryption on live cluster.
Decryption is not implemented yet
Encryption without persisting master key is not possible (if we need it)

[asfgit]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4811/
Test FAILed.
Test FAILured.

[smolnar82]

retest this please

[asfgit]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4812/
Test FAILed.
Test FAILured.

[ghost]

Work still in progress.
Looks like case of a forgotten master key was present at the code, but has been broken for a long time

[asfgit]

FAILURE
5339 tests run, 72 skipped, 1 failed.
Test FAILed.
Test FAILured.

[ghost]

Added test coverage for sensitive data encryption, and for some previously not covered (non)persistent master key cases

[asfgit]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4835/
Test FAILed.
Test FAILured.

[rlevas]

2086438 [INFO] ------------------------------------------------------------------------
2086438 [INFO] Reactor Summary:
2086438 [INFO] 
2086438 [INFO] Ambari Main 3.0.0.0-SNAPSHOT ....................... SUCCESS [  6.267 s]
2086439 [INFO] Apache Ambari Project POM 3.0.0.0-SNAPSHOT ......... SUCCESS [  0.125 s]
2086439 [INFO] Ambari Web 3.0.0.0-SNAPSHOT ........................ FAILURE [02:36 min]
2086439 [INFO] Ambari Views 3.0.0.0-SNAPSHOT ...................... SUCCESS [  3.545 s]
2086439 [INFO] Ambari Admin View 3.0.0.0-SNAPSHOT ................. SUCCESS [02:39 min]
2086440 [INFO] ambari-utility 3.0.0.0-SNAPSHOT .................... SUCCESS [  6.530 s]
2086440 [INFO] Ambari Server SPI 3.0.0.0-SNAPSHOT ................. SUCCESS [  1.990 s]
2086440 [INFO] Ambari Service Advisor 1.0.0.0-SNAPSHOT ............ SUCCESS [  0.600 s]
2086440 [INFO] Ambari Server 3.0.0.0-SNAPSHOT ..................... SUCCESS [25:14 min]
2086440 [INFO] Ambari Functional Tests 3.0.0.0-SNAPSHOT ........... SUCCESS [02:42 min]
2086441 [INFO] Ambari Agent 3.0.0.0-SNAPSHOT ...................... SUCCESS [01:12 min]
2086441 [INFO] ------------------------------------------------------------------------
2086441 [INFO] BUILD FAILURE
2086441 [INFO] ------------------------------------------------------------------------

Ambari-web builds and tests fine for me locally and the changes in this patch appear to not be relevant.

retest this please

[ghost]

Test failure is unrelated (tests of server module passed)

2086438 [INFO] ------------------------------------------------------------------------
2086438 [INFO] Reactor Summary:
2086438 [INFO] 
2086438 [INFO] Ambari Main 3.0.0.0-SNAPSHOT ....................... SUCCESS [  6.267 s]
2086439 [INFO] Apache Ambari Project POM 3.0.0.0-SNAPSHOT ......... SUCCESS [  0.125 s]
2086439 [INFO] Ambari Web 3.0.0.0-SNAPSHOT ........................ FAILURE [02:36 min]
2086439 [INFO] Ambari Views 3.0.0.0-SNAPSHOT ...................... SUCCESS [  3.545 s]
2086439 [INFO] Ambari Admin View 3.0.0.0-SNAPSHOT ................. SUCCESS [02:39 min]
2086440 [INFO] ambari-utility 3.0.0.0-SNAPSHOT .................... SUCCESS [  6.530 s]
2086440 [INFO] Ambari Server SPI 3.0.0.0-SNAPSHOT ................. SUCCESS [  1.990 s]
2086440 [INFO] Ambari Service Advisor 1.0.0.0-SNAPSHOT ............ SUCCESS [  0.600 s]
2086440 [INFO] Ambari Server 3.0.0.0-SNAPSHOT ..................... SUCCESS [25:14 min]
2086440 [INFO] Ambari Functional Tests 3.0.0.0-SNAPSHOT ........... SUCCESS [02:42 min]
2086441 [INFO] Ambari Agent 3.0.0.0-SNAPSHOT ...................... SUCCESS [01:12 min]
2086441 [INFO] ------------------------------------------------------------------------

[asfgit]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4838/
Test PASSed.