CNTRLPLANE-2649: always run OTE CI profiles for kso#78299
CNTRLPLANE-2649: always run OTE CI profiles for kso#78299ropatil010 wants to merge 1 commit intoopenshift:mainfrom
Conversation
|
@ropatil010: This pull request references CNTRLPLANE-2649 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/assign @ingvagabund |
WalkthroughConsolidates three AWS OTE tests into a single combined OTE test, updates presubmit jobs to use the combined target (making the presubmit always run), and adds a new step-registry OTE script, ref, metadata, and OWNERS for cluster-kube-scheduler-operator. Changes
Sequence Diagram(s)sequenceDiagram
participant CI as CI System
participant Presubmit as Presubmit Job
participant StepRef as Step Ref (ref.yaml)
participant Script as e2e-ote-commands.sh
participant Cluster as Cluster API
participant TestRunner as openshift-tests
participant Artifacts as Artifact Storage
CI->>Presubmit: trigger presubmit (e2e-aws-operator-ote-combined)
Presubmit->>StepRef: invoke step ref
StepRef->>Script: execute e2e-ote-commands.sh
Script->>Cluster: load credentials, detect platform, wait for stability
Script->>Cluster: apply permissions / proxy configuration if needed
Script->>Artifacts: create artifact directories, record timestamps
Script->>TestRunner: run operator-serial suite
TestRunner-->>Artifacts: write console.log and junit
Script->>TestRunner: run operator-parallel suite
TestRunner-->>Artifacts: write console.log and junit
Script->>TestRunner: run preferred-host-serial suite
TestRunner-->>Artifacts: write console.log and junit
Script-->>Presubmit: return aggregated exit status
Presubmit-->>CI: report result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 12✅ Passed checks (12 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yaml (1)
103-110: Consider adding a timeout for consistency with other OTE tests.The
e2e-aws-operator-preferred-host-serial-otetest lacks atimeoutsetting, while bothe2e-aws-operator-serial-oteande2e-aws-operator-parallel-otehavetimeout: 8h0m0s. Since this is also a serial E2E test, it may benefit from the same timeout configuration to prevent unexpected job terminations.Proposed fix to add timeout
- as: e2e-aws-operator-preferred-host-serial-ote steps: cluster_profile: openshift-org-aws env: TEST_SUITE: openshift/cluster-kube-scheduler-operator/preferred-host/serial test: - ref: openshift-e2e-test workflow: ipi-aws + timeout: 8h0m0s🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yaml` around lines 103 - 110, Add a timeout to the preferred-host serial OTE test: locate the job named e2e-aws-operator-preferred-host-serial-ote (the block with TEST_SUITE: openshift/cluster-kube-scheduler-operator/preferred-host/serial) and add a timeout entry matching the other OTE jobs (e.g., timeout: 8h0m0s) under the steps section so the job uses the same maximum runtime as e2e-aws-operator-serial-ote and e2e-aws-operator-parallel-ote.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In
`@ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yaml`:
- Around line 103-110: Add a timeout to the preferred-host serial OTE test:
locate the job named e2e-aws-operator-preferred-host-serial-ote (the block with
TEST_SUITE: openshift/cluster-kube-scheduler-operator/preferred-host/serial) and
add a timeout entry matching the other OTE jobs (e.g., timeout: 8h0m0s) under
the steps section so the job uses the same maximum runtime as
e2e-aws-operator-serial-ote and e2e-aws-operator-parallel-ote.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 4cef7004-3977-4899-a2bc-32ccf183aa97
📒 Files selected for processing (2)
ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yamlci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yaml
|
@ropatil010: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@ropatil010: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
1 similar comment
|
@ropatil010: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
|
/pj-rehearse pull-ci-openshift-cluster-kube-scheduler-operator-main-e2e-aws-operator-preferred-host-serial-ote |
|
@ropatil010: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-cluster-kube-scheduler-operator-main-e2e-aws-operator-serial-ote |
|
@ropatil010: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/assign @gangwgr @sandeepknd |
|
Before moving forward with this is there a limit on how many CI jobs it is still practical to run? The current KSO test suite is quite minimal. Is it worth spinning another cluster (= more resources) for running this vs. running the KSO teset suite as part of already existing jobs? |
|
Also, s/sso/kso in the PR name. |
|
why we need 3 jobs for 4-5 cases only it will too expensive, if we have not more cases |
Consolidates 3 separate OTE test jobs into a single job that runs all test suites sequentially on one AWS cluster, reducing infrastructure costs by 66%. Changes: - Created custom step-registry component: cluster-kube-scheduler-operator-e2e-ote - Removed 3 individual jobs (e2e-aws-operator-serial-ote, e2e-aws-operator-parallel-ote, e2e-aws-operator-preferred-host-serial-ote) - Added consolidated job: e2e-aws-operator-ote-combined The new job runs all three test suites sequentially: 1. openshift/cluster-kube-scheduler-operator/operator/serial 2. openshift/cluster-kube-scheduler-operator/operator/parallel 3. openshift/cluster-kube-scheduler-operator/preferred-host/serial Each test suite maintains separate log files and JUnit directories for proper test result tracking. Benefits: - Reduces from 3 AWS clusters to 1 cluster (66% cost reduction) - Same test coverage maintained - Separate JUnit results for debugging Addresses resource efficiency concern from PR review: openshift#78299 (comment) Co-Authored-By: Rohit Patil <ropatil@redhat.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: ropatil010 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.sh`:
- Around line 148-180: The script currently aborts on the first failing
openshift-tests run because of errexit; modify the test-run blocks that call
openshift-tests run (for TEST_SUITE values like
"openshift/cluster-kube-scheduler-operator/operator/serial",
"operator/parallel", and "preferred-host/serial") so each invocation captures
its exit code without exiting the script (e.g., run the command normally but
append || true or run in a subshell and capture $?), write its logs to
"${ARTIFACT_DIR}/e2e-*.log" and junit to "${ARTIFACT_DIR}/junit-*" as before,
and record the per-suite result into a variable/array (e.g., SUITE_FAILURES or
LAST_EXIT) so you can print per-suite status immediately and, after all suites
run, exit non-zero if any recorded exit codes indicate failure; ensure you use
the existing TEST_ARGS, TEST_PROVIDER and ARTIFACT_DIR variables when invoking
openshift-tests run.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 53e91a6c-c50f-4bbf-9ea3-6ec61a830eb3
📒 Files selected for processing (5)
ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yamlci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yamlci-operator/step-registry/cluster-kube-scheduler-operator/OWNERSci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.shci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.yaml
✅ Files skipped from review due to trivial changes (2)
- ci-operator/step-registry/cluster-kube-scheduler-operator/OWNERS
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.yaml
Consolidates 3 separate OTE test jobs into a single job that runs all test suites sequentially on one AWS cluster, reducing infrastructure costs by 66%. Changes: - Created custom step-registry component: cluster-kube-scheduler-operator-e2e-ote - Includes proper error handling to run all suites even if individual tests fail - Each test suite has separate log files and JUnit directories - Supports all cloud providers (AWS, GCP, Azure, vSphere, OpenStack, etc.) - Includes cluster stability checks and retry strategy for presubmits - Removed 3 individual jobs: - e2e-aws-operator-serial-ote - e2e-aws-operator-parallel-ote - e2e-aws-operator-preferred-host-serial-ote - Added consolidated job: e2e-aws-operator-ote-combined - Runs automatically on PRs - Blocks merges if tests fail Test suites executed sequentially: 1. openshift/cluster-kube-scheduler-operator/operator/serial 2. openshift/cluster-kube-scheduler-operator/operator/parallel 3. openshift/cluster-kube-scheduler-operator/preferred-host/serial Error handling: - Initializes rc=0 before test execution - Uses if/else blocks to capture failures without aborting - All three test suites run regardless of individual failures - Exits with rc=1 if any test suite fails - Ensures all test logs and JUnit artifacts are generated OWNERS files: - Created OWNERS at both parent and subdirectory levels - Contains only valid OpenShift org members - Generated metadata file for step registry Benefits: - Reduces from 3 AWS clusters to 1 cluster (66% cost reduction) - Same test coverage maintained - Separate JUnit results for debugging - Proper artifact generation even on test failures Addresses resource efficiency concern from PR review: openshift#78299 (comment) Addresses CodeRabbit review feedback: openshift#78299 (review) Co-Authored-By: Rohit Patil <ropatil@redhat.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.sh (1)
25-29: Use a Bash array forTEST_ARGS.Line 156, Line 171, and Line 186 expand
TEST_ARGSunquoted. It works for the current single flag, but the first arg containing whitespace or shell metacharacters will break all three invocations.Suggested change
-TEST_ARGS="" +TEST_ARGS=() if [[ "${JOB_TYPE:-}" == "presubmit" && ( "${PULL_BASE_REF:-}" == "main" || "${PULL_BASE_REF:-}" == "master" ) ]]; then if openshift-tests run --help 2>/dev/null | grep -q 'retry-strategy'; then - TEST_ARGS+=" --retry-strategy=aggressive" + TEST_ARGS+=(--retry-strategy=aggressive) echo "Enabled aggressive retry strategy for presubmit" fi fi @@ -if openshift-tests run "${TEST_SUITE}" ${TEST_ARGS} \ +if openshift-tests run "${TEST_SUITE}" "${TEST_ARGS[@]}" \ @@ -if openshift-tests run "${TEST_SUITE}" ${TEST_ARGS} \ +if openshift-tests run "${TEST_SUITE}" "${TEST_ARGS[@]}" \ @@ -if openshift-tests run "${TEST_SUITE}" ${TEST_ARGS} \ +if openshift-tests run "${TEST_SUITE}" "${TEST_ARGS[@]}" \Also applies to: 156-186
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.sh` around lines 25 - 29, The script builds TEST_ARGS as a plain string which breaks when flags contain whitespace or metacharacters; change TEST_ARGS to a Bash array (use TEST_ARGS=() instead of TEST_ARGS=""), append flags with TEST_ARGS+=(--retry-strategy=aggressive) where currently TEST_ARGS+=" …", and update all invocations that expand TEST_ARGS (the openshift-tests run calls referenced around the earlier TEST_ARGS uses and the later expansions at the sites noted) to use the safe expansion "${TEST_ARGS[@]}" so each element is passed as a separate quoted argument.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.sh`:
- Around line 151-194: You’re running three suites sequentially by setting
TEST_SUITE and invoking openshift-tests run (operator/serial, operator/parallel,
preferred-host/serial) on the same cluster without any explicit cleanup or
documented isolation; fix this by adding explicit per-suite isolation and
verification steps: after each openshift-tests run (the invocations that write
to ${ARTIFACT_DIR}/e2e-*.log and --junit-dir ${ARTIFACT_DIR}/junit-*), perform a
deterministic teardown or cluster-state verification (e.g., delete test
namespaces/resources or run a cluster-sanity check) and record its result (so rc
is set on failures), or alternatively run each TEST_SUITE in an isolated
namespace/unique resource prefix and document in the job comment or README that
TEST_SUITE definitions are order-independent and self-cleaning; ensure these
changes reference the TEST_SUITE variable and the openshift-tests run calls so
reviewers can find and validate the isolation.
---
Nitpick comments:
In
`@ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.sh`:
- Around line 25-29: The script builds TEST_ARGS as a plain string which breaks
when flags contain whitespace or metacharacters; change TEST_ARGS to a Bash
array (use TEST_ARGS=() instead of TEST_ARGS=""), append flags with
TEST_ARGS+=(--retry-strategy=aggressive) where currently TEST_ARGS+=" …", and
update all invocations that expand TEST_ARGS (the openshift-tests run calls
referenced around the earlier TEST_ARGS uses and the later expansions at the
sites noted) to use the safe expansion "${TEST_ARGS[@]}" so each element is
passed as a separate quoted argument.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 530e6adf-4d01-48e8-b279-00c63c515c63
📒 Files selected for processing (7)
ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yamlci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yamlci-operator/step-registry/cluster-kube-scheduler-operator/OWNERSci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/OWNERSci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.shci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.metadata.jsonci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.yaml
✅ Files skipped from review due to trivial changes (4)
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/OWNERS
- ci-operator/step-registry/cluster-kube-scheduler-operator/OWNERS
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.metadata.json
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.yaml
Consolidates 3 separate OTE test jobs into a single job that runs all test suites sequentially on one AWS cluster, reducing infrastructure costs by 66%. Changes: - Created custom step-registry component: cluster-kube-scheduler-operator-e2e-ote - Includes proper error handling to run all suites even if individual tests fail - Each test suite has separate log files and JUnit directories - Supports all cloud providers (AWS, GCP, Azure, vSphere, OpenStack, etc.) - Includes cluster stability checks and retry strategy for presubmits - Removed 3 individual jobs: - e2e-aws-operator-serial-ote - e2e-aws-operator-parallel-ote - e2e-aws-operator-preferred-host-serial-ote - Added consolidated job: e2e-aws-operator-ote-combined - Runs automatically on PRs - Blocks merges if tests fail Test suites executed sequentially: 1. openshift/cluster-kube-scheduler-operator/operator/serial 2. openshift/cluster-kube-scheduler-operator/operator/parallel 3. openshift/cluster-kube-scheduler-operator/preferred-host/serial Error handling: - Initializes rc=0 before test execution - Uses if/else blocks to capture failures without aborting - All three test suites run regardless of individual failures - Exits with rc=1 if any test suite fails - Ensures all test logs and JUnit artifacts are generated OWNERS files: - Created OWNERS at both parent and subdirectory levels - Contains only valid OpenShift org members - Generated metadata file for step registry Benefits: - Reduces from 3 AWS clusters to 1 cluster (66% cost reduction) - Same test coverage maintained - Separate JUnit results for debugging - Proper artifact generation even on test failures Addresses resource efficiency concern from PR review: openshift#78299 (comment) Addresses CodeRabbit review feedback: openshift#78299 (review) Co-Authored-By: Rohit Patil <ropatil@redhat.com>
|
/pj-rehearse pull-ci-openshift-cluster-kube-scheduler-operator-main-e2e-aws-operator-ote-combined |
|
@ropatil010: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
ci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yaml (1)
84-108: Please confirm the cost tradeoff before making this mandatory.This turns the OTE job into an always-run presubmit while the existing non-OTE operator presubmits in this file are still kept, so every PR now pays for another AWS cluster/job instead of swapping coverage. Worth getting explicit owner sign-off on the added default load.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yaml` around lines 84 - 108, The job currently sets always_run: true for the OTE presubmit (name: pull-ci-openshift-cluster-kube-scheduler-operator-main-e2e-aws-operator-ote-combined, context: ci/prow/e2e-aws-operator-ote-combined), which forces AWS cost on every PR; change always_run to false (or remove the always_run key) so it is not mandatory, and add a note in the PR requesting explicit owner sign-off (or add an annotation/label indicating "requires-owner-approval") before reintroducing always_run:true to ensure the cost tradeoff is approved.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In
`@ci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yaml`:
- Around line 84-108: The job currently sets always_run: true for the OTE
presubmit (name:
pull-ci-openshift-cluster-kube-scheduler-operator-main-e2e-aws-operator-ote-combined,
context: ci/prow/e2e-aws-operator-ote-combined), which forces AWS cost on every
PR; change always_run to false (or remove the always_run key) so it is not
mandatory, and add a note in the PR requesting explicit owner sign-off (or add
an annotation/label indicating "requires-owner-approval") before reintroducing
always_run:true to ensure the cost tradeoff is approved.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: ef6fb2c5-fa57-4b7d-9ec3-2c012a0c616e
📒 Files selected for processing (7)
ci-operator/config/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main.yamlci-operator/jobs/openshift/cluster-kube-scheduler-operator/openshift-cluster-kube-scheduler-operator-main-presubmits.yamlci-operator/step-registry/cluster-kube-scheduler-operator/OWNERSci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/OWNERSci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-commands.shci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.metadata.jsonci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.yaml
✅ Files skipped from review due to trivial changes (4)
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/OWNERS
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.metadata.json
- ci-operator/step-registry/cluster-kube-scheduler-operator/OWNERS
- ci-operator/step-registry/cluster-kube-scheduler-operator/e2e-ote/cluster-kube-scheduler-operator-e2e-ote-ref.yaml
|
@ropatil010: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
CI Logs: Used 1 cluster to execute these test suites
|
|
Design implementation: Using 1 cluster and execution of cases on that. Here it triggers
Future wise if there are increase in cases with serial/Parallel it handles automatically in 1 cluster instead of new CI profiles. If in case we need 2 separate CI profiles we can call existing openshift-e2e-test which works fine. Visualization @ingvagabund @gangwgr PTAL on this PR. |
Hi @ingvagabund
Earlier we added OTE profiles for scheduler operator but with always_run: false, optional: true parameters.
We need to make sure both non OTE/OTE CI profile should work properly. Once OTE profiles work fine then we can remove non OTE ci profiles e2e-aws-operator, e2e-aws-operator-preferred-host.
Summary by CodeRabbit