chore(e2e): rhidp 6618 e2e auditor log by gustavolira · Pull Request #2827 · redhat-developer/rhdh

gustavolira · 2025-04-30T16:56:00Z

Description

Please explain the changes you made here.

Which issue(s) does this PR fix

Fixes #?

PR acceptance criteria

Please make sure that the following steps are complete:

GitHub Actions are completed and successful
Unit Tests are updated and passing
E2E Tests are updated and passing
Documentation is updated if necessary (requirement for new features)
Add a screenshot if the change is UX/UI related

How to test changes / Special notes to the reviewer

gustavolira · 2025-04-30T18:20:32Z

/retest

gustavolira · 2025-04-30T18:25:11Z

/test e2e-tests

github-actions · 2025-04-30T20:14:39Z

The image is available at:

github-actions · 2025-04-30T22:37:59Z

The image is available at:

github-actions · 2025-04-30T22:57:23Z

The image is available at:

dzemanov

Thank you for refactoring the repeated test cases and fixing skip of scaffolder audit logs, looks great. I have left a couple of suggestions.

dzemanov

Thank you for the changes. I left 2 more questions and then I think we are good to go from my side. For the scaffolder tests, should we maybe update them to the new auditor version and just skip them till we can use severityLogLevelMappings, instead of deleting them? Or there is not much value in testing fetch events?

github-actions · 2025-05-02T15:48:11Z

The image is available at:

github-actions · 2025-05-02T18:29:32Z

The image is available at:

github-actions · 2025-05-03T20:03:09Z

The image is available at:

github-actions · 2025-05-04T15:11:16Z

The image is available at:

github-actions · 2025-05-04T15:14:04Z

The image is available at:

github-actions · 2025-05-04T15:26:21Z

The image is available at:

github-actions · 2025-05-04T17:49:19Z

The image is available at:

github-actions · 2025-05-04T18:07:54Z

The image is available at:

github-actions · 2025-05-04T18:45:11Z

The image is available at:

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

github-actions · 2025-05-04T19:06:09Z

The image is available at:

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

github-actions · 2025-05-04T19:45:45Z

The image is available at:

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

github-actions · 2025-05-04T19:48:05Z

The image is available at:

gustavolira · 2025-05-05T11:39:23Z

Thank you for the changes. I left 2 more questions and then I think we are good to go from my side. For the scaffolder tests, should we maybe update them to the new auditor version and just skip them till we can use severityLogLevelMappings, instead of deleting them? Or there is not much value in testing fetch events?

@dzemanov In practice these logs add very little audit value because they’re tied to a read-only action. During a typical scaffold run the backend issues dozens of entity-fetch calls, so enabling them just floods the log with low-severity noise without providing extra security or compliance insight.

For that reason I’d rather drop the test instead of keeping a skipped placeholder – we’re already validating all write-path events

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

github-actions · 2025-05-05T11:47:37Z

The image is available at:

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

github-actions · 2025-05-05T13:02:36Z

The image is available at:

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

github-actions · 2025-05-05T20:14:46Z

The image is available at:

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

dzemanov · 2025-05-06T10:52:47Z

/retest

github-actions · 2025-05-06T10:53:15Z

The image is available at:

dzemanov

Looks great! Thank you for all the changes.

dzemanov · 2025-05-06T10:55:03Z

/lgtm

albarbaro · 2025-05-06T14:03:50Z

/lgtm
/approve

openshift-ci · 2025-05-06T14:03:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: albarbaro, dzemanov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~.ibm/OWNERS~~ [albarbaro]
~~e2e-tests/OWNERS~~ [albarbaro]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

zdrapela · 2025-05-16T08:10:23Z

+sed_inplace() {
+  if [[ "$OSTYPE" == "darwin"* ]]; then
+    # macOS
+    sed -i '' "$@"
+  else
+    # Linux
+    sed -i "$@"
+  fi
+}


Hi, I suggest using GNU sed on your Mac OS machine instead of creating a new function for it. It is generally useful to use the GNU sed instead of the default one for development work, and we rely on the GNU sed all over our scripts in the RHDH repo, RHDH operator repo, etc.

We should even add it to https://github.com/redhat-developer/rhdh/blob/main/docs/e2e-tests/README.md#prerequisites

The installation is quite simple. It is just brew install gnu-sed and then add this line: export PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH" to your ~/.zshrc file.

zdrapela · 2025-05-16T08:18:00Z

+to_lowercase() {
+  if [[ "$OSTYPE" == "darwin"* ]]; then
+    # macOS - using tr
+    echo "$1" | tr '[:upper:]' '[:lower:]'
+  else
+    # Linux - using bash parameter expansion
+    echo "${1,,}"
+  fi
+}


For the MacOS (zsh) support, I'd rather simplify the condition.

From: if [[ -z "${IS_OPENSHIFT}" || "${IS_OPENSHIFT,,}" == "false" ]]; then
To: if [[ -z "${IS_OPENSHIFT}" || "${IS_OPENSHIFT}" == "false" ]]; then

As you can see at L1097, the function that sets the IS_OPENSHIFT env. variable uses lower case, and it will not change throughout the test. It will not cause any issues.

* Introduce rbac auditor tests Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Test Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Catch the logs Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Test pod logs Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Enable change of namespace Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Update pod selector Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Filter the logs more precisely Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Switch to json logs Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Fix policy-read by-query Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Enable more precise filtering Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Make permission-evaluation more concrete Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Remove testing lines Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * Increase tail number Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> * RHIDP-6618 Refactor RBAC tests for improved structure and readability Signed-off-by: Gustavo Lira <guga.java@gmail.com> * test Signed-off-by: Gustavo Lira <guga.java@gmail.com> * test Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 Update scaffold tests: refine event log validation and improve test descriptions Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 - Add shared utilities for RBAC audit-log Playwright tests Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 - Add shared utilities for RBAC audit-log Playwright tests Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 - Add shared utilities for RBAC audit-log Playwright tests Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 - Add shared utilities for RBAC audit-log Playwright tests Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 Refactor RBAC API references in tests to use RBAC_API constants Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Refactor RBAC policy update call for improved readability Signed-off-by: Gustavo Lira <guga.java@gmail.com> * RHIDP-6618 Rename scaffold.spec.ts to auditor-catalog.spec.ts and update test description for clarity Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> * Enhance audit log validation in Playwright tests by adding log method and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com> --------- Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com> Signed-off-by: Gustavo Lira <guga.java@gmail.com> Co-authored-by: Dominika Zemanovicova <dzemanov@redhat.com>

openshift-ci Bot requested review from albarbaro and zdrapela April 30, 2025 16:56

dzemanov suggested changes May 2, 2025

View reviewed changes

dzemanov mentioned this pull request May 2, 2025

chore(e2e): introduce e2e tests for RBAC Auditor log #2759

Closed

5 tasks

dzemanov suggested changes May 2, 2025

View reviewed changes

Comment thread e2e-tests/playwright/e2e/audit-log/scaffold.spec.ts Outdated

Comment thread e2e-tests/playwright/e2e/audit-log/scaffold.spec.ts Outdated

dzemanov added 11 commits May 4, 2025 15:51

Introduce rbac auditor tests

e4786c9

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Test

0d4402f

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Catch the logs

a87038d

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Test pod logs

dc99976

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Enable change of namespace

fa3bfd5

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Update pod selector

18c7449

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Filter the logs more precisely

f5733bf

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Switch to json logs

8358529

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Fix policy-read by-query

b6e7d14

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Enable more precise filtering

3a3e6b1

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Make permission-evaluation more concrete

4fc72fe

Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>

Enhance audit log validation in Playwright tests by adding log method…

130e5ab

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

Enhance audit log validation in Playwright tests by adding log method…

d9c3f23

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

Enhance audit log validation in Playwright tests by adding log method…

32544cf

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

dzemanov suggested changes May 5, 2025

View reviewed changes

Comment thread e2e-tests/playwright/e2e/audit-log/auditor-catalog.spec.ts Outdated

Comment thread e2e-tests/playwright/e2e/audit-log/auditor-catalog.spec.ts Outdated

Comment thread e2e-tests/playwright/e2e/audit-log/rbac-test-utils.ts Outdated

Enhance audit log validation in Playwright tests by adding log method…

ce832ce

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

Enhance audit log validation in Playwright tests by adding log method…

43bff52

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

Enhance audit log validation in Playwright tests by adding log method…

ef86be2

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

Enhance audit log validation in Playwright tests by adding log method…

bcbec2b

… and refactoring log utilities Signed-off-by: Gustavo Lira <guga.java@gmail.com>

dzemanov approved these changes May 6, 2025

View reviewed changes

openshift-ci Bot assigned dzemanov May 6, 2025

openshift-ci Bot added the lgtm label May 6, 2025

openshift-ci Bot assigned albarbaro May 6, 2025

openshift-ci Bot added the approved label May 6, 2025

openshift-merge-bot Bot merged commit 0fc7eec into redhat-developer:main May 6, 2025

zdrapela reviewed May 16, 2025

View reviewed changes

zdrapela mentioned this pull request Mar 23, 2026

chore(ci): fix Operator tests, artifact handling, pod selectors #4414

Merged

5 tasks

Uh oh!

Conversation

gustavolira commented Apr 30, 2025

Description

Which issue(s) does this PR fix

PR acceptance criteria

How to test changes / Special notes to the reviewer

Uh oh!

gustavolira commented Apr 30, 2025

Uh oh!

gustavolira commented Apr 30, 2025

Uh oh!

github-actions Bot commented Apr 30, 2025

Uh oh!

github-actions Bot commented Apr 30, 2025

Uh oh!

github-actions Bot commented Apr 30, 2025

Uh oh!

dzemanov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dzemanov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 2, 2025

Uh oh!

github-actions Bot commented May 2, 2025

Uh oh!

github-actions Bot commented May 3, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

github-actions Bot commented May 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gustavolira commented May 5, 2025

Uh oh!

github-actions Bot commented May 5, 2025

Uh oh!

github-actions Bot commented May 5, 2025

Uh oh!

github-actions Bot commented May 5, 2025

Uh oh!

dzemanov commented May 6, 2025

Uh oh!

github-actions Bot commented May 6, 2025

Uh oh!

dzemanov left a comment

Choose a reason for hiding this comment

Uh oh!

dzemanov commented May 6, 2025

Uh oh!

albarbaro commented May 6, 2025