Skip to content

New Rules Must Have a prodtype#11252

Merged
jan-cerny merged 5 commits into
ComplianceAsCode:masterfrom
Mab879:new_rules_need_prodtype
Nov 10, 2023
Merged

New Rules Must Have a prodtype#11252
jan-cerny merged 5 commits into
ComplianceAsCode:masterfrom
Mab879:new_rules_need_prodtype

Conversation

@Mab879
Copy link
Copy Markdown
Member

@Mab879 Mab879 commented Nov 6, 2023

Description:

This adds a test that checks

  • New rules must a have prodtype
  • Existing rules with a prodtype must continue to have it

Rationale:

This will be helpful in the future when we move prodtypes.

Review Hints:

Remove a prodtype rule and watch the new test fail.

@Mab879 Mab879 added the Test Suite Update in Test Suite. label Nov 6, 2023
@Mab879 Mab879 added this to the 0.1.71 milestone Nov 6, 2023
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Nov 6, 2023
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Nov 6, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Nov 6, 2023

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

@jan-cerny jan-cerny self-assigned this Nov 7, 2023
Copy link
Copy Markdown
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works as expected. I have removed prodtype from 2 random rules and it detected it and failed.

1/2 Test  #9: test-rule-dir-json ....................   Passed   15.32 sec
test 39
    Start 39: test-no-new-global-applicable-rules

39: Test command: /usr/bin/env "PYTHONPATH=/home/jcerny/work/git/scap-security-guide" "/usr/bin/python3" "/home/jcerny/work/git/scap-security-guide/utils/no_new_global_applicable_rules.py" "--root" "/home/jcerny/work/git/scap-security-guide" "--json" "/home/jcerny/work/git/scap-security-guide/build/rule_dirs.json"
39: Working Directory: /home/jcerny/work/git/scap-security-guide/build/tests
39: Test timeout computed to be: 10000000
39: Rule service_timesyncd_root_distance_configured doesn't have a prodtype and it is expected to. You must add one.
39: Rule service_timesyncd_configured doesn't have a prodtype and it is expected to. You must add one.
2/2 Test #39: test-no-new-global-applicable-rules ...***Failed    0.26 sec

The following tests passed:
	test-rule-dir-json

50% tests passed, 1 tests failed out of 2

Label Time Summary:
quick    =  15.32 sec*proc (1 test)

Total Test time (real) =  15.59 sec

The following tests FAILED:
	 39 - test-no-new-global-applicable-rules (Failed)
Errors while running CTest

@Mab879 Mab879 marked this pull request as ready for review November 8, 2023 17:21
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Nov 8, 2023
@Mab879 Mab879 requested a review from jan-cerny November 9, 2023 03:04
@Mab879
Copy link
Copy Markdown
Member Author

Mab879 commented Nov 9, 2023

/packit retest-failed

Comment thread utils/no_new_global_applicable_rules.py Outdated
return expected_set


def _get_current_noprodtypes(rule_dirs):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Climate wants you to simplify this function

@qlty-cloud-legacy
Copy link
Copy Markdown

Code Climate has analyzed commit de666d5 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 58.8%.

View more on Code Climate.

@Mab879 Mab879 requested a review from jan-cerny November 9, 2023 15:02
Copy link
Copy Markdown
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jcerny@fedora ~/work/git/scap-security-guide/build (pr/11252) $ ctest --output-on-failure -R test-no-new-global-applicable-rules
Test project /home/jcerny/work/git/scap-security-guide/build
    Start  6: test-rule-dir-json
1/2 Test  #6: test-rule-dir-json ....................   Passed   15.69 sec
    Start 25: test-no-new-global-applicable-rules
2/2 Test #25: test-no-new-global-applicable-rules ...***Failed    0.27 sec
Rule service_timesyncd_configured doesn't have a prodtype and it is expected to. You must add one.


50% tests passed, 1 tests failed out of 2

Label Time Summary:
quick    =  15.69 sec*proc (1 test)

Total Test time (real) =  15.97 sec

The following tests FAILED:
	 25 - test-no-new-global-applicable-rules (Failed)
Errors while running CTest
jcerny@fedora ~/work/git/scap-security-guide/build (pr/11252) $ 

Thanks for this great improvement!

@jan-cerny jan-cerny merged commit 07c765c into ComplianceAsCode:master Nov 10, 2023
@Mab879 Mab879 deleted the new_rules_need_prodtype branch November 10, 2023 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Test Suite Update in Test Suite.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants