diff --git a/.claude/README.md b/.claude/README.md new file mode 100644 index 0000000000..6ff7ced278 --- /dev/null +++ b/.claude/README.md @@ -0,0 +1,27 @@ +# Claude Code Setup + +Claude Code skills for MicroShift are available in the +[edge-tooling](https://github.com/openshift-eng/edge-tooling) repository +and are distributed as plugins. + +## Installing Plugins + +First, add the marketplace: + +``` +/plugin marketplace add openshift-eng/edge-tooling +``` + +Then install the plugins you need: + +``` +/plugin install microshift-ci +/plugin install microshift-dev +/plugin install microshift-release +``` + +| Plugin | Description | +|---|---| +| `microshift-ci` | CI job analysis, failure triage, and bug creation | +| `microshift-dev` | Development workflows, test generation, and code analysis | +| `microshift-release` | Release version tracking and release note generation | diff --git a/.claude/agents/microshift-add-enhancement.md b/.claude/agents/microshift-add-enhancement.md deleted file mode 100644 index 3d7fb4befe..0000000000 --- a/.claude/agents/microshift-add-enhancement.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -name: microshift-add-enhancement -description: Create a new Microshift Enhancement Proposal -argument-hint: [area] -model: sonnet -color: blue ---- - -## Name -microshift:add-enhancement - -## Synopsis -``` -/microshift:add-enhancement [area] -``` - -## Description - -The `microshift:add-enhancement` command creates a new microshift Enhancement -Proposal (EP) based on the official template from the openshift/enhancements -repository. It generates a comprehensive enhancement document with all required -sections, metadata, and guidance following OpenShift's enhancement process. - -This command automates the creation of enhancement proposals by: -- Fetching the latest enhancement template from the openshift/enhancements - repository -- Analyzing the provided description to extract what, why, and who information -- Generating user stories, goals, and non-goals based on the requirements -- Creating properly formatted enhancement files with all required sections -- Applying OpenShift-specific conventions for feature gates, API design, and - testing requirements - -## Arguments - -- **area** (optional): Enhancement area (subdirectory under enhancements/). If not provided, the command will attempt to infer the best area from the enhancement description and ask for confirmation. -- **name**: One-line title describing the enhancement -- **description**: Detailed description (what, why, who) -- **jira**: JIRA ticket URL for tracking - -## Implementation - -Act as an experienced software architect to create a comprehensive enhancement proposal. Follow these steps: - -**Important**: Reference the guidance in the OpenShift enhancements repository at https://github.com/openshift/enhancements/blob/master/dev-guide/feature-zero-to-hero.md, particularly the section "Writing an OpenShift Enhancement", when creating enhancement proposals. This guide provides essential context on the OpenShift Enhancement Proposal process, feature gates, API design conventions, testing requirements, and promotion criteria. - -0. **Verify Repository**: Before proceeding, ensure you are operating in the correct repository: - - Check if the current directory is a git repository: `git rev-parse --is-inside-work-tree` - - Verify the repository is openshift/enhancements or a fork: - - Get the remote URL: `git config --get remote.origin.url` - - Check if the URL contains "openshift/enhancements" (e.g., `git@github.com:openshift/enhancements.git` or `https://github.com/openshift/enhancements.git`) - - For forks, the URL might be different (e.g., `git@github.com:username/enhancements.git`) - in this case, check if it's a fork by examining the repository structure (presence of `enhancements/` directory and `guidelines/` directory with enhancement_template.md) - - If verification fails, exit with a clear error message: "Error: This command must be run in the openshift/enhancements repository or a fork. Please clone the repository first or navigate to the correct directory." - - If successful, continue to the next step - -1. **Determine Enhancement Area**: Identify or validate the area where the enhancement should be created: - - List available areas by examining subdirectories in `enhancements/microshift`: `ls -d enhancements/microshift/*/` (extract just the directory names) - - **If the `` argument WAS provided by the user:** - - Check if `enhancements/microshift//` exists - - If it exists: proceed with this area - - If it does NOT exist: - - Show the list of available areas - - Use AskUserQuestion to ask: "The area '' does not exist. Do you want to create a new area called ''?" with options: - - "Yes, create new area ''" (then confirm they're sure about creating a new area) - - "No, let me choose from existing areas" (then show available areas and ask them to select) - - **If the `` argument was NOT provided:** - - Analyze the enhancement name and description to infer the most appropriate area - - Look for keywords that match existing area names (e.g., "storage", "router", "apiserver", "coredns", "etcd", etc.) - - Use AskUserQuestion to confirm: "Based on your enhancement description, I think the best area is ''. Is this correct?" with options: - - "Yes, use ''" - - "No, let me choose a different area" (then list available areas and ask them to select or specify a new one) - - If the user chooses a different area that doesn't exist, ask if they want to create it (same flow as above) - - **Creating a new area (if chosen):** - - Create markdown document in the new area : `touch enhancements/microshift/.md` - - Proceed with the validated/created area - -2. **Fetch the Enhancement Template**: Before starting, fetch the latest template from the openshift/enhancements repository: - - Download the template via HTTP using whatever web-fetch mechanism is available; otherwise ask the user to paste the template from: https://raw.githubusercontent.com/openshift/enhancements/master/guidelines/enhancement_template.md - - Store this template for reference when creating the enhancement file - - This ensures you always use the most current template structure and required sections - -3. **Parse the Description**: Extract the following from the description: - - **What**: What is this enhancement about - - **Why**: Why this change is required (motivation) - - **Who**: Which personas this applies to (use this to generate user stories) - -4. **Ask Clarifying Questions** (if needed): Use the AskUserQuestion tool to gather: - - Specific user stories or motivations if not clear from the description - - Explicit Goals or Non-Goals the user wants to be included - - Any specific technical constraints or requirements - - Topology considerations should be MicroShift - - Whether this proposal adds/changes CRDs, admission and conversion webhooks, ValidatingAdmissionPlugin, MutatingAdmissionPlugin, aggregated API servers, or finalizers (needed for API Extensions section) - - Feature gate information: According to the OpenShift enhancement dev guide (https://github.com/openshift/enhancements/blob/master/dev-guide/feature-zero-to-hero.md), ALL new OpenShift features must start disabled by default using feature gates. Ask about the proposed feature gate name and initial feature set (DevPreviewNoUpgrade or TechPreviewNoUpgrade). - - Ask clarifying questions about telemetry, security, upgrade and downgrade process, rollbacks, dependencies, in case it is not possible to assert these fields. - -5. **Generate the Enhancement File**: - - Create the file at `enhancements/microshift/.md` where filename is the kebab-case version of the name argument - - Fill in the template with: - - **Title**: Use the provided name - - **Summary**: One paragraph describing what this enhancement is about - - **Motivation**: Explain why this change is required based on the description - - **User Stories**: Generate 2-4 user stories based on the "who" information using the format: - > "As a _role_, I want to _take some action_ so that I can _accomplish a goal_." - Include a story on how the proposal will be operationalized: life-cycled, monitored and remediated at scale. - - **Goals**: List specific, measurable goals (3-5 items). Goals should describe what users want from their perspective, not implementation details. - - **Non-Goals**: List what is explicitly out of scope (2-3 items) - - **Proposal**: High-level description of the proposed solution - - **Workflow Description**: Detailed workflow with actors and steps - - **Mermaid Diagram**: Add a sequence diagram when the workflow involves multiple actors or complex interactions between components (e.g., user -> API server -> controller -> operator). Simple single-actor workflows may not need a diagram. - - **API Extensions**: Only fill this section if the user confirms the proposal adds/changes CRDs, admission and conversion webhooks, ValidatingAdmissionPlugin, MutatingAdmissionPlugin, aggregated API servers, or finalizers. Per the template, name the API extensions and describe if this enhancement modifies the behaviour of existing resources. Otherwise, add a TODO comment asking the user to complete this section if applicable. - - **Topology Considerations**: Include subsections for Hypershift/Hosted Control Planes, Standalone Clusters, Single-node Deployments or MicroShift, and OKE (OpenShift Kubernetes Engine). Address how the proposal affects each topology. - - **Implementation Details/Notes/Constraints**: Provide a high-level overview of the code changes required. Follow the guidance from the template: "While it is useful to go into the details of the code changes required, it is not necessary to show how the code will be rewritten in the enhancement." Keep it as an overview; the developer should fill in the specific implementation details. Include a reminder about creating a feature gate: Per the OpenShift dev guide (https://github.com/openshift/enhancements/blob/master/dev-guide/feature-zero-to-hero.md), all new features must be gated behind a feature gate in https://github.com/openshift/api/blob/master/features/features.go with the appropriate feature set (DevPreviewNoUpgrade or TechPreviewNoUpgrade initially). - - **Test Plan**: Add a TODO comment with guidance on required test labels per the OpenShift dev guide (https://github.com/openshift/enhancements/blob/master/dev-guide/feature-zero-to-hero.md): Tests must include `[OCPFeatureGate:FeatureName]` label for the feature gate, `[Jira:"Component Name"]` for the component, and appropriate test type labels like `[Suite:...]`, `[Serial]`, `[Slow]`, or `[Disruptive]` as needed. Reference the test conventions guide (https://github.com/openshift/enhancements/blob/master/dev-guide/test-conventions.md) for details. - - **Graduation Criteria**: Add a TODO comment referencing the specific promotion requirements from the OpenShift dev guide (https://github.com/openshift/enhancements/blob/master/dev-guide/feature-zero-to-hero.md): minimum 5 tests, 7 runs per week, 14 runs per supported platform, 95% pass rate, and tests running on all supported platforms (AWS, Azure, GCP, vSphere, Baremetal with various network stacks). - - **Metadata**: Fill in creation-date with today's date, tracking-link with the provided JIRA ticket URL, set other fields to TBD. For api-approvers: use "None" if there are no API changes (no new/modified CRDs, webhooks, aggregated API servers, or finalizers); otherwise use "TBD" as a placeholder (the enhancement author will request an API reviewer from the #forum-api-review Slack channel later). - -6. **Handle Unfilled Sections**: For sections that cannot be filled based on the input: - - Add a clear comment like `` - - Provide guidance on what should be included - -7. **Writing Guidelines**: - - Write in a clear, concise, professional manner - - Focus on the essential information - - Use bullet points and structured formatting - - Avoid unnecessary verbosity - - **Line Length**: Keep lines in the generated enhancement at a maximum of 80 characters, but prioritize validity over line length limits. Only break lines at 80 characters if doing so will NOT create: - - Invalid or broken URLs (URLs themselves should never be split, but the line CAN and SHOULD be broken before or after the URL) - - Invalid Markdown syntax (e.g., breaking Markdown links, code blocks, or formatting) - - Invalid code examples (e.g., breaking code in the middle of statements) - If breaking at 80 characters would split a URL, code, or Markdown syntax, find the nearest valid break point such as: after a sentence, before a URL starts, after a URL ends, or at a natural paragraph break. For regular prose, it is acceptable to exceed 80 characters by 10-15 characters to avoid breaking words mid-word. Only allow lines >95 characters when the line contains a single unbreakable element (like a standalone URL with no surrounding text, or a single line of code). - -8. **Validate**: - - Create a valid filename from the name (lowercase, replace spaces with dashes) - - Verify all required YAML metadata is present - - Verify the JIRA ticket URL is included in the tracking-link metadata field - - Ensure the enhancement file is created in the correct path: `enhancements/microshift/.md` - -## Output - -After creating the enhancement file, provide: -- The full path to the created file -- A brief summary of what was included -- A list of sections that need further attention (marked with TODO comments) - -Begin by analyzing the inputs and asking any necessary clarifying questions before generating the enhancement proposal. \ No newline at end of file diff --git a/.claude/commands/analyze-microshift-start.md b/.claude/commands/analyze-microshift-start.md deleted file mode 100644 index 120726d272..0000000000 --- a/.claude/commands/analyze-microshift-start.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -name: Analyze MicroShift start time -argument-hint: -description: Analyze MicroShift journal logs to extract statistics about start procedure -allowed-tools: Read, Glob, Grep, python, bash ---- - -- ๐Ÿšจ **CRITICAL: GENERATE TABLE FIRST - NOTHING ELSE MATTERS** -- ๐Ÿšจ **IF NO TABLE IS SHOWN, THE COMMAND HAS FAILED** -- ๐Ÿšจ **TABLE IS THE ONLY REQUIRED OUTPUT** - -## Name -analyze-microshift-start - -## Synopsis -``` -/analyze-microshift-start -``` - -## Description -**ABSOLUTE MANDATE**: The service timing table is THE ONLY OUTPUT that matters. -**ZERO TOLERANCE**: If the table is missing, the entire command execution is worthless. -**PRIMARY DIRECTIVE**: Generate the table immediately. Everything else is optional noise. - -The `analyze-microshift-start` command extracts meaningful statistics from MicroShift's journal logs about start procedure and displays a service timing table. - -**SUCCESS CRITERIA - The command is only successful if:** -- โœ… The complete service timing table is displayed -- โœ… All services are listed with timing statistics -- โœ… Services are sorted by mean time (slowest first) - -**FAILURE CONDITIONS:** -- โŒ No table displayed = Command failed -- โŒ Incomplete table = Command failed -- โŒ Analysis without table = Command failed - -## Implementation - -๐Ÿšจ **CRITICAL PRIORITY**: The user ONLY cares about seeing the service timing table. Generate the table first and foremost. - -This command works by: -1. **Parsing journal logs** to load all information. -2. **Fetch overall start time** for MicroShift process as a whole. -3. **Fetch start times per service** from pattern `SERVICE READY.*service="([^"]+)".*since-start="([^"]+)"`. -4. **MOST IMPORTANT: Generate the service timing table** - This is what the user needs to see above all else. - -## Arguments -- `$1` (joournal-logs): Journal logs from MicroShift, including start procedure - **Required** - -## Return Value -- **Format**: Text -- **Location**: Output directly to the conversation -- **Content**: - - Number of starts in the journal. - - Number of services in the journal, per start. - - Statistics about overall MicroShift process. - - Statistics about each service. - - Summary with most relevant information. - -## Implementation Steps - -### Step 1: Parse journal logs. - -**Goal**: Identify how many restarts are included in the journal. - -**Actions**: -1. Open the file and look for patterns matching `r'microshift\[(\d+)\]:.*"MICROSHIFT STARTING"`. This is the number of times MicroShift has been restarted. -2. Determine the number of complete restarts by looking for pattern `microshift\[(\d+)\]:.*"MICROSHIFT READY".*since-start="([^"]+)"` in the file. -3. If nothing is found, produce an error saying there are no restarts found. - -### Step 2: Fetch MicroShift start time. - -**Goal**: Get details about how long it took to start MicroShift. - -**Actions**: -1. Get the elapsed time from start to readiness for MicroShift. -2. Use the pattern `microshift\[(\d+)\]:.*"MICROSHIFT READY".*since-start="([^"]+)"` to fetch logs about it. -3. Extract the time it took to start from the second grouping in the pattern. - -### Step 3: Fetch per-service start times. - -**Goal**: Get details about how long it took to start each of the services in MicroShift. - -**Actions**: -1. Get the elapsed time from start to readiness for each of the different services. -2. Use the pattern `SERVICE READY.*service="([^"]+)".*since-start="([^"]+)"` to fetch logs about it. -3. Extract the service name from the first grouping in the pattern. -4. Extract the time it took to start from the second grouping in the pattern. - -### Step 4: MANDATORY TABLE GENERATION - -**CRITICAL MANDATE**: Generate the service performance table - THIS IS THE ONLY OBJECTIVE THAT MATTERS. -- **๐Ÿšจ TABLE MUST BE DISPLAYED FIRST ๐Ÿšจ** -- **๐Ÿšจ IF NO TABLE = COMMAND FAILED ๐Ÿšจ** -- **๐Ÿšจ TABLE IS SUCCESS, EVERYTHING ELSE IS NOISE ๐Ÿšจ** - -**REQUIRED TABLE FORMAT** (MUST BE EXACT): -``` ------------------------------------------------------------------------------------------------------------------------- -Service Runs Average Median StdDev Min Max Var% Status ------------------------------------------------------------------------------------------------------------------------- -MICROSHIFT OVERALL STARTUP XX XX.XXXs XX.XXXs XXX.Xms XX.XXXs XX.XXXs X.X ๐Ÿ”ฅ -[ALL OTHER SERVICES LISTED HERE - SORTED BY AVERAGE TIME, SLOWEST FIRST] ------------------------------------------------------------------------------------------------------------------------- -``` - -**MANDATORY ACTIONS - EXECUTE IN THIS EXACT ORDER**: -1. **IMMEDIATELY DISPLAY THE TABLE ABOVE** - This is the primary deliverable -2. Calculate these metrics for each service: runs, average, median, standard deviation, min, max, variability percentage -3. Sort services by mean time (slowest to fastest) -4. Use status icons: ๐Ÿ”ฅ (โ‰ฅ5s), โš ๏ธ (2-5s), ๐ŸŸก (1-2s), โœ… (0.1-1s), โšก (<0.1s) -5. Only after table is shown, optionally add other analysis - -**ABSOLUTE REQUIREMENTS**: -- The service table with ALL timing data MUST be generated and shown to the user -- Table generation is MORE IMPORTANT than anything else -- If you must choose between showing the table OR other analysis, ALWAYS choose the table -- DO NOT create any scripts, temporary files, or helper utilities. Process everything in memory. - -Strong prohibitions: -- You are *NOT* allowed to write any scripts to disk, do everything in memory. -- NEVER create files or write scripts. Process everything in memory. -- Do NOT use the Write tool or create any temporary files. - -Positive instructions: -- Produce the output without creating any additional scripts or helpers. -- Process the data directly using tool outputs and generate the report inline. -- Use only Read, Grep, and analysis tools - no file creation. - -**CRITICAL**: The primary output MUST be this exact table format. Generate the table first, everything else is optional. - -**REQUIRED OUTPUT**: -```text ------------------------------------------------------------------------------------------------------------------------- -Service Runs Average Median StdDev Min Max Var% Status ------------------------------------------------------------------------------------------------------------------------- -MICROSHIFT OVERALL STARTUP XX XX.XXXs XX.XXXs XXX.Xms XX.XXXs XX.XXXs X.X ๐Ÿ”ฅ -[ALL OTHER SERVICES LISTED HERE - SORTED BY AVERAGE TIME, SLOWEST FIRST] ------------------------------------------------------------------------------------------------------------------------- -``` - -**TABLE GENERATION IS MANDATORY** - Show timing statistics for ALL services found in the logs. The user needs this table above all else. diff --git a/.claude/commands/analyze-sos-report.md b/.claude/commands/analyze-sos-report.md deleted file mode 100644 index 64c0d19fcf..0000000000 --- a/.claude/commands/analyze-sos-report.md +++ /dev/null @@ -1,393 +0,0 @@ ---- -name: Analyze SOS Report -argument-hint: [log.html-url] -description: Investigate MicroShift runtime problems from SOS report -allowed-tools: WebFetch, Bash, Read, Glob, Grep ---- - -## Name -analyze-sos-report - -## Synopsis -``` -/analyze-sos-report [log.html-url] -``` - -## Description -The `analyze-sos-report` command investigates MicroShift runtime problems by analyzing journal logs, Pod logs, YAML manifests, and configuration from a SOS report. Optionally, it can cross-reference findings with a Robot Framework test log. - -This command focuses on: -- MicroShift and CRI-O journal logs for errors -- Pod status and container logs -- Kubernetes resource YAMLs (Deployments, Pods, Services, etc.) -- MicroShift configuration issues -- etcd errors and problems -- OVN networking issues -- **Robot Framework test results correlation** (when log.html URL is provided) - -## Arguments -- `$1` (sos-report-path): Path or URL to the SOS report - **Required** - - **Local directory**: Path to an already extracted sosreport (e.g., `/tmp/sosreport-hostname-2025-01-15`) - - **Remote URL**: HTTP/HTTPS URL to a `.tar.xz` file that will be downloaded and extracted to `/tmp` - - The extracted directory should contain `sos_commands/microshift` subdirectory. - -- `$2` (log.html-url): URL to Robot Framework log.html file - **Optional** - - When provided, test results will be analyzed and cross-referenced with SOS report findings - - Helps correlate test failures with system-level issues - - Example: `https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/.../log.html` - -## Return Value -- **Format**: Markdown -- **Location**: Output directly to the conversation -- **Content**: - - Identified runtime issues - - Root cause analysis - - Relevant log excerpts - - Recommendations - -## SOS Report Structure Reference - -### Required directories (always present): -- `sos_commands/microshift/` - MicroShift specific data - -### Key files in `sos_commands/microshift/`: -- `journalctl_--no-pager_--unit_microshift` - MicroShift service logs -- `journalctl_--no-pager_--unit_microshift-etcd.scope` - etcd logs -- `microshift_version` - MicroShift version -- `microshift_show-config_-m_effective` - Effective configuration -- `systemctl_status_microshift` - Service status -- `event-filter.html` - Kubernetes events viewer - -### Namespace structure in `sos_commands/microshift/namespaces//`: -- `.yaml` - Namespace definition -- `core/pods.yaml` - Pod definitions -- `core/events.yaml` - Namespace events -- `core/configmaps.yaml` - ConfigMaps -- `core/services.yaml` - Services -- `apps/deployments.yaml` - Deployments -- `apps/daemonsets.yaml` - DaemonSets -- `pods//.yaml` - Individual Pod YAML -- `pods/////logs/current.log` - Container logs -- `pods/////logs/previous.log` - Previous container logs - -### Cluster-scoped resources in `sos_commands/microshift/cluster-scoped-resources/`: -- `core/nodes.yaml` - Node information -- `storage.k8s.io/storageclasses.yaml` - Storage classes - -### Optional directories (may not be present in minimal reports): -- `sos_commands/crio/` - CRI-O container runtime -- `sos_commands/logs/` - System journal logs -- `sos_commands/microshift_ovn/` - OVN networking -- `sos_commands/openvswitch/` - Open vSwitch -- `sos_commands/networking/` - Network configuration -- `etc/microshift/` - MicroShift configuration files - -## Robot Framework Log Structure Reference - -When a log.html URL is provided, the file contains Robot Framework test execution results: - -### Key Information in log.html: -- **Test Suites**: Hierarchical organization of tests -- **Test Cases**: Individual test names, status (PASS/FAIL/SKIP), duration -- **Keywords**: Step-by-step execution details within each test -- **Timestamps**: Execution times for each test and keyword -- **Error Messages**: Failure reasons for failed tests -- **Log Messages**: DEBUG, INFO, WARN level messages during execution - -### Timestamp Correlation: -- **Important**: log.html timestamps may be in a different timezone than SOS report journal logs -- Typically only the **hour component differs** (e.g., log.html shows 10:30:45, journal shows 05:30:45) -- When correlating events, match by **minutes and seconds** and allow for hour offset -- Look for events that occur within the same minute across both sources - -### Test Sources: -- Robot Framework test files are located in `test/` directory of MicroShift repository -- Test file paths can be extracted from log.html suite names -- **Caveat**: The currently checked-out branch may not match the CI run that generated the log.html -- Use test sources as reference only; the actual executed code may differ - -## Implementation Steps - -### Step 1: Handle Input (URL or Local Path) - -**Goal**: Determine if input is a URL or local path, and prepare the sosreport directory. - -**Actions**: -1. Check if the input starts with `http://` or `https://`: - - If YES: It's a remote URL, proceed to download and extract - - If NO: It's a local path, proceed to validation - -**For Remote URLs**: -1. Download the `.tar.xz` file to `/tmp`: - ```bash - curl -L -o /tmp/sosreport-download.tar.xz "" - ``` -2. Extract the archive to `/tmp`: - ```bash - tar -xf /tmp/sosreport-download.tar.xz -C /tmp - ``` -3. Find the extracted directory: - ```bash - ls -dt /tmp/sosreport-*/ 2>/dev/null | head -1 - ``` -4. Set the extracted directory as the working path -5. Clean up the downloaded archive: - ```bash - rm /tmp/sosreport-download.tar.xz - ``` - -**For Local Paths**: -1. Check if the directory exists -2. Verify it contains `sos_commands/microshift/` subdirectory - -### Step 2: Analyze MicroShift Journal Logs - -**Goal**: Find errors and problems in MicroShift service logs. - -**Actions**: -1. Read MicroShift journal logs: - ```bash - cat /sos_commands/microshift/journalctl_--no-pager_--unit_microshift - ``` -2. Search for errors, warnings, and failures - look for patterns like: - - `error`, `fail`, `fatal`, `panic` - - `timeout`, `refused`, `denied` - - Startup failures - - API server errors - - Controller errors - -### Step 3: Analyze etcd Logs - -**Goal**: Check embedded etcd health. - -**Actions**: -1. Read etcd journal logs: - ```bash - cat /sos_commands/microshift/journalctl_--no-pager_--unit_microshift-etcd.scope - ``` -2. Look for etcd-specific issues: - - Database corruption - - Slow disk warnings - - Leader election problems - - Timeout errors - -### Step 4: Analyze CRI-O Logs (if available) - -**Goal**: Check container runtime issues. - -**Actions**: -1. Read CRI-O journal logs (if present): - ```bash - cat /sos_commands/crio/journalctl_--no-pager_--unit_crio - ``` -2. Check container status: - ```bash - cat /sos_commands/crio/crictl_ps_-a - ``` -3. Look for crashed or errored containers - -### Step 5: Analyze Pod Status and Events - -**Goal**: Check Pod health and Kubernetes events. - -**Actions**: -1. Check Pod status in each namespace: - ```bash - cat /sos_commands/microshift/namespaces/*/core/pods.yaml - ``` -2. Look for Pods not in Running state (Pending, CrashLoopBackOff, Error, etc.) - -3. Check events for problems: - ```bash - cat /sos_commands/microshift/namespaces/*/core/events.yaml - ``` -4. Look for warning events indicating issues - -### Step 6: Analyze Container Logs - -**Goal**: Check individual container logs for errors. - -**Actions**: -1. Find container logs: - ```bash - find /sos_commands/microshift/namespaces -name "current.log" -o -name "previous.log" - ``` -2. Search for errors in container logs: - ```bash - grep -rE "error|fail|panic|fatal" /sos_commands/microshift/namespaces/*/pods/*/ - ``` -3. Check previous.log files for containers that restarted - -### Step 7: Analyze Resource YAMLs - -**Goal**: Check resource configurations for issues. - -**Actions**: -1. Check Deployments and DaemonSets: - ```bash - cat /sos_commands/microshift/namespaces/*/apps/deployments.yaml - cat /sos_commands/microshift/namespaces/*/apps/daemonsets.yaml - ``` -2. Look for: - - Replicas not matching desired count - - Image pull errors - - Resource constraint issues - -### Step 8: Analyze MicroShift Configuration - -**Goal**: Check for configuration issues. - -**Actions**: -1. Read effective MicroShift config: - ```bash - cat /sos_commands/microshift/microshift_show-config_-m_effective - ``` -2. Check config files if present: - ```bash - cat /etc/microshift/config.yaml - ``` -3. Check for common misconfigurations - -### Step 9: Analyze OVN/Networking (if available) - -**Goal**: Check networking issues. - -**Actions**: -1. Check OVN status if present: - ```bash - cat /sos_commands/microshift_ovn/* - ``` -2. Check OVN-related pods in openshift-ovn-kubernetes namespace -3. Look for network connectivity issues in logs - -### Step 10: Analyze Robot Framework Log (if provided) - -**Goal**: Extract test results from log.html and correlate with SOS report findings. - -**Actions**: -1. Fetch and parse the log.html file: - ```bash - # Use WebFetch to retrieve and analyze the log.html - ``` - -2. Extract key information: - - Test suite name and overall status (PASS/FAIL) - - Individual test cases with their status - - Failed test names and error messages - - Test execution timestamps - -3. Cross-reference with SOS report: - - **Timezone consideration**: Timestamps in log.html and SOS report may differ by timezone (typically only the hour component differs). When correlating events, match by minutes and seconds, allowing for hour offset. - - Match test failure times with errors in MicroShift journal logs - - Identify if pod restarts or errors occurred during specific test execution - - Look for patterns: did a specific test cause system instability? - -4. Test source reference: - - Test sources are in the `test/` directory of the MicroShift repository - - **Note**: The checked-out branch may not match the CI run, so test sources are for reference only - - Extract test file paths from log.html to help locate relevant test code - -### Step 11: Generate Investigation Report - -**Goal**: Compile findings into a focused problem analysis. - -**Report Structure**: -```markdown -# MicroShift Runtime Problem Analysis - -## Summary - - -## Test Results (if log.html provided) -| Test Name | Status | Duration | Error | -|-----------|--------|----------|-------| -| ... | PASS/FAIL | ... | ... | - -### Failed Tests Analysis - - -### Test-System Event Correlation - - -## Identified Problems - -### Problem 1: -**Severity**: Critical/Warning/Info -**Component**: MicroShift/CRI-O/etcd/Pod/OVN/etc. - -**Evidence**: -``` - -``` - -**Root Cause Analysis**: - - -**Recommendation**: - - -## Affected Pods/Containers -| Namespace | Pod | Status | Restarts | Issue | -|-----------|-----|--------|----------|-------| -| ... | ... | ... | ... | ... | - -## Relevant Log Excerpts -``` - -``` - -## Configuration Issues - - -## Next Steps -1. -2. -``` - -## Examples - -### Example 1: Local Directory Analysis -``` -/analyze-sos-report /tmp/sosreport-microshift-host-2025-01-15-abcdef -``` - -### Example 2: Remote URL Analysis -``` -/analyze-sos-report https://example.com/sosreport-edge-device-01-2025-01-15.tar.xz -``` - -### Example 3: SOS Report with Robot Framework Log -``` -/analyze-sos-report /tmp/sosreport-el96-host-2025-01-15 https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_microshift/5870/pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-arm/1997885403058671616/artifacts/e2e-aws-tests-bootc-arm/openshift-microshift-e2e-metal-tests/artifacts/scenario-info/el96-src@optional/log.html -``` - -## Notes -- Accepts either a local extracted directory or a remote `.tar.xz` URL -- Remote files are downloaded and extracted to `/tmp` automatically -- Focuses on runtime problem investigation, not system inventory -- Prioritizes actionable findings over comprehensive status reports -- Some directories (crio, logs, networking) are optional and may not be present in minimal reports -- For best results, ensure the sosreport was collected with MicroShift plugins enabled - -## CI Environment Considerations - -SOS reports from CI environments may contain logs from **multiple MicroShift restarts**. This is expected because: -- CI tests reuse VMs across multiple test scenarios -- Tests may change MicroShift configuration between runs -- The MicroShift service is restarted to apply configuration changes - -When analyzing SOS reports: -- **Report all pod restarts** - even if restarts may occur during MicroShift service transitions, report them as potential concerns -- **Report all errors** - include connection refused errors, API server unavailability, and pod failures -- **Flag pods with any restart count > 0** - every restart is worth noting; excessive restarts may indicate pods that don't handle API server unavailability gracefully - -### API Server Unavailability During Restarts - -When MicroShift restarts, the API server becomes temporarily unavailable. Ideally, pods should: -- Retry API connections with backoff -- Not exit immediately on transient connection failures - -Any pod restarts due to API server unavailability should be reported as a concern. Even if the pod eventually recovers, frequent restarts during MicroShift transitions may indicate: -- The pod lacks proper retry/backoff logic for API server connections -- A potential bug in the application's Kubernetes client configuration -- An opportunity to improve pod resilience diff --git a/.claude/commands/generate-tests-template.md b/.claude/commands/generate-tests-template.md deleted file mode 100644 index 697e6d58fd..0000000000 --- a/.claude/commands/generate-tests-template.md +++ /dev/null @@ -1,244 +0,0 @@ -# Test Generator Template - -This is a **generic, customizable template** for creating test generation workflows. It's designed to be adapted for different projects, testing frameworks, and workflows. - -## Why Use This Template? - -The pre-configured `/generate-tests` command is optimized for MicroShift/OpenShift projects with specific assumptions about: -- Jira OCPSTRAT tickets -- Robot Framework test structure -- MicroShift repository layout -- Red Hat workflow conventions - -**This template allows you to:** -- Adapt the workflow for your specific project -- Use different testing frameworks (Pytest, Jest, Ginkgo, etc.) -- Customize for different issue trackers (GitHub, Jira, Linear, etc.) -- Work with Claude Code, Cursor, or any AI assistant -- Match your team's conventions and preferences - -## How to Use This Template - -### Option 1: Direct Prompting (Recommended for Cursor users) -1. **Copy the relevant sections** from this template -2. **Customize them** for your project -3. **Paste into your AI assistant** as context with your request - -**Example**: -``` -I want to generate tests for my project. Here's my setup: -- Project: MyAPI -- Issue tracker: GitHub Issues (https://github.com/myorg/myapi/issues/) -- Test framework: Pytest -- Directory structure: tests/unit/, tests/integration/ - -[paste customized template sections] - -Please generate tests for issue #123 -``` - -### Option 2: Create a Custom Slash Command (For Claude Code) -1. **Copy this file** to create your own command (e.g., `.claude/commands/my-generate-tests.md`) -2. **Customize the sections** below to match your project -3. **Use the command**: `/my-generate-tests TICKET-ID` - -### Option 3: Use as Reference -Simply reference sections of this template when asking your AI assistant to generate tests, adapting on the fly. - ---- - -## Template Sections to Customize - -### ๐ŸŽฏ Project Context (CUSTOMIZE THIS) -``` -Replace with your project details: -- Project name: [MicroShift/OpenShift/Your Project] -- Issue tracker: [Jira/GitHub Issues/Linear/etc.] -- Repository structure: [mono-repo/multi-repo] -- Test framework: [Robot Framework/Pytest/Jest/etc.] -``` - -### ๐Ÿ“‹ Ticket Analysis (CUSTOMIZE THIS) -``` -Define your ticket workflow: -- Ticket URL pattern: https://issues.redhat.com/browse/$1 -- Required ticket fields: [Description, Acceptance Criteria, etc.] -- Linked tickets to check: [Implementation tickets, dependent tickets] -- Where to find PR links: [Ticket fields, comments, linked items] -``` - -### ๐Ÿ” Test Coverage Analysis (CUSTOMIZE THIS) -``` -Define what to analyze: -- Test file patterns: *.robot, *_test.py, *.spec.js -- Test directories: test/, tests/, __tests__/ -- Coverage sources: [PRs, existing test suites, test plans] -- What constitutes "covered": [Unit tests, integration tests, etc.] -``` - -### โœ๏ธ Test Case Generation (CUSTOMIZE THIS) -``` -Define your test case format: -- Test case document format: [Markdown, Confluence, etc.] -- Number of test cases: [Top 10, All scenarios, etc.] -- Test case fields: - * Test ID format: USHIFT-XXX-TC-YYY - * Priority levels: Critical/High/Medium/Low - * Required sections: Description, Steps, Expected Results, etc. -- Output location: [Current directory, specific folder, etc.] -``` - -### ๐Ÿงช Test Categories to Cover (CUSTOMIZE THIS) -``` -Define categories relevant to YOUR project: -1. Core Functionality -2. Configuration Edge Cases -3. Dynamic Behavior -4. Integration Points -5. Multi-tenant/Namespace -6. Error Handling -7. Upgrade/Compatibility -8. Performance -9. Security -10. Real Customer Scenarios - -Add/remove categories as needed for your domain. -``` - -### ๐Ÿค– Test Implementation (CUSTOMIZE THIS) -``` -Define your test structure: -- Test framework: Robot Framework / Pytest / Jest / etc. -- File naming convention: kebab-case / snake_case / PascalCase -- Directory structure: test/suites// or tests/unit/ -- Reusable components location: resources/, utils/, fixtures/ -- Setup/Teardown patterns: Your project's patterns -``` - -### ๐Ÿ“ File Organization (CUSTOMIZE THIS) -``` -Map feature types to test directories: -- Network features โ†’ test/suites/network/ -- Storage features โ†’ test/suites/storage/ -- API features โ†’ test/api/ -- UI features โ†’ test/e2e/ - -Customize based on YOUR project structure. -``` - -### ๐Ÿ”ง Keyword/Helper Reuse (CUSTOMIZE THIS - Robot Framework specific) -``` -If using Robot Framework, define: -- Common keyword locations: test/extended/util/*.robot -- Resource file patterns: resources/**/*.robot -- Naming conventions: Action_Object pattern -- When to create new vs. reuse: Your team's guidelines -``` - -### ๐ŸŒฟ Git Workflow (CUSTOMIZE THIS) -``` -Define your branching strategy: -- Branch naming: test-OCPSTRAT-XXXX-$(date +%Y%m%d) or feature/tests-XXX -- Base branch: main / master / develop -- Commit message format: Your team's convention -- Auto-commit: Yes/No (get confirmation first) -- Repository location: Ask user / fixed path / auto-detect -``` - -### ๐Ÿ“Š Output Format (CUSTOMIZE THIS) -``` -Define what the final output should include: -- Test case document: Format and location -- Test implementation files: Locations and count -- Coverage report: What was tested vs. what's missing -- Summary report: Tickets analyzed, PRs reviewed, etc. -- Next steps: Instructions for running tests, creating PR, etc. -``` - ---- - -## Example Customized Prompt - -Here's an example of how to customize this template for a different project: - -### Example: Python/Pytest Project - -```markdown -# Pytest Test Generator for MyAPI Project - -Generate comprehensive Pytest coverage for MyAPI features based on GitHub Issues. - -## Workflow: - -### Step 1: Analyze GitHub Issue -- Fetch issue from: https://github.com/myorg/myapi/issues/$1 -- Extract feature description and acceptance criteria -- Find linked PRs in issue body and comments - -### Step 2: Analyze Existing Tests -- Check for test files: tests/**/*_test.py -- Look for pytest fixtures in: tests/conftest.py -- Identify what's already covered - -### Step 3: Generate Test Cases -- Create markdown document: test_cases_issue_$1.md -- Generate 5-7 critical test scenarios -- Format: Given/When/Then style -- Include: unit tests, integration tests, edge cases - -### Step 4: Implement Pytest Tests -- Create test file: tests/unit/test_.py -- Reuse fixtures from conftest.py -- Follow pytest best practices -- Use parametrize for multiple scenarios - -### Step 5: Update conftest.py if needed -- Add new fixtures only if truly reusable -- Document fixture purpose and scope - -### Step 6: Create PR (optional) -- Branch: feature/tests-issue-$1 -- Run: pytest tests/ to verify -- Provide summary of coverage added -``` - ---- - -## Instructions for AI Assistants - -When a user provides this template: - -1. **Ask clarifying questions** about sections marked "CUSTOMIZE THIS" -2. **Don't assume** - let the user specify their preferences -3. **Adapt the workflow** to the user's specific tools and conventions -4. **Confirm before creating files/branches** - respect the user's workflow -5. **Be flexible** - this is a starting point, not a rigid spec - ---- - -## Tips for Customization - -### For Different Testing Frameworks -- **Pytest**: Focus on fixtures, parametrize, conftest.py -- **Jest**: Focus on mocks, setup/teardown, snapshot testing -- **Robot Framework**: Focus on keywords, resources, libraries -- **Selenium**: Focus on page objects, waits, selectors -- **Postman/REST**: Focus on request builders, assertions, environments - -### For Different Issue Trackers -- **GitHub Issues**: Use GitHub API or web scraping -- **Jira**: Adjust ticket URL pattern, field names -- **Linear**: Different API structure -- **Azure DevObs**: Work items structure - -### For Different Project Types -- **Microservices**: Multi-repo coordination, API contracts -- **Frontend**: Component testing, accessibility, visual regression -- **CLI Tools**: Command execution, output validation, error handling -- **Libraries**: API testing, documentation examples, edge cases - ---- - -## License - -This template is provided as-is for customization. Adapt freely to your needs. diff --git a/.claude/commands/generate-tests.md b/.claude/commands/generate-tests.md deleted file mode 100644 index 69d5ea4f24..0000000000 --- a/.claude/commands/generate-tests.md +++ /dev/null @@ -1,241 +0,0 @@ ---- -name: Test Generator for MicroShift Features (Detailed test cases & Robot framework code) -argument-hint: JIRA-TICKET-ID -description: Generate Test Cases for a given Jira OCPSTRAT ticket & automates them using RobotFramework -allowed-tools: WebFetch, Bash, Read, Write, Glob, Grep -interaction: Ask user for confirmation before creating git branch ---- - -# Robot Framework Test Generator for MicroShift/OpenShift - -**NOTE**: This is a **pre-configured template** for MicroShift/OpenShift projects. For other projects or to customize this workflow, see `.claude/commands/generate-tests-template.md` for a generic template you can adapt. - -Generate comprehensive Robot Framework test coverage for MicroShift/OpenShift features based on Jira OCPSTRAT tickets. - -## Prerequisites: - -**Jira Access**: This command requires access to Red Hat Jira (https://issues.redhat.com). - -- Ensure you're logged into Red Hat Jira in your browser before running this command -- Login at: https://issues.redhat.com - -**Fallback**: If Jira access fails, you'll be prompted to manually provide: -- Ticket description and acceptance criteria -- Links to GitHub PRs -- Feature documentation - -## Workflow: - -### Step 1: Analyze OCPSTRAT Ticket -- Fetch the Jira ticket from https://issues.redhat.com/browse/$1 -- Extract feature description, acceptance criteria, and technical details -- Identify all linked USHIFT tickets (look in "Issue Links" section) -- Find GitHub PR links from USHIFT tickets or OCPSTRAT ticket - -**VALIDATION - Check for sufficient information**: -After analyzing the ticket, verify: -1. **Description Quality**: Does the ticket have detailed description, acceptance criteria, or technical implementation details? -2. **PR Links**: Are there any useful GitHub PR links in the OCPSTRAT ticket or linked USHIFT tickets? - -**If BOTH conditions fail** (no detailed description AND no PR links): -- โš ๏ธ **Ask user for additional information**: - "The OCPSTRAT ticket lacks detailed information for test generation. It's missing: - - Detailed feature description or acceptance criteria - - Linked USHIFT tickets with GitHub PRs - - Please provide ONE of the following to continue: - 1. A link to documentation (PR description, GitHub .md files, design docs) - 2. A brief description of the feature functionality and expected behavior - 3. Additional context about what this feature does - - Or type 'skip' to proceed with limited information (may result in generic tests)." - -- **Wait for user response**: - - If user provides documentation link: Fetch and analyze it - - If user provides description: Use it as supplementary context - - If user types 'skip': Proceed with warning that tests may be generic - -**If at least ONE condition passes**, proceed to Step 2. - -### Step 2: Analyze Existing Test Coverage -- For each GitHub PR found, check if it contains Robot Framework test files (*.robot) -- Note the test file names and their location in the PR -- Read and analyze existing test cases to understand what's already covered -- Identify functionality already covered by existing tests - -### Step 3: Generate Top 10 Most Impactful Missing Tests - -**Create test case document in current working directory**: `test_cases_OCPSTRAT-XXXX.md` - -Focus ONLY on the **10 most impactful test scenarios NOT already covered**: - -1. **Core Functionality** - Primary use cases not tested -2. **Configuration Edge Cases** - Invalid/boundary configurations not covered -3. **Dynamic Behavior** - Runtime changes, reloads not tested -4. **Integration Gaps** - Component interactions not validated -5. **Multi-tenant/Namespace** - Cross-namespace scenarios missing -6. **Error Handling** - Failure modes not covered -7. **Upgrade/Compatibility** - Version compatibility gaps -8. **Performance** - Load/scale testing if missing -9. **Security** - Permission/isolation tests not present -10. **Real Customer Scenarios** - Use cases from RFE not tested - -For each test, provide: -- **Test ID**: USHIFT-XXX-TC-YYY (using the USHIFT ticket number) -- **Priority**: Critical/High/Medium -- **Coverage Gap**: What existing tests don't cover -- **Test Description**: Clear objective -- **Steps to Reproduce**: Concrete test steps with actual config/commands -- **Expected Results**: Expected outcome for each step - -### Step 4: Create Robot Framework Test Files - -**BEFORE writing any tests:** -1. Search for existing keyword files in the codebase -2. Identify reusable keywords for common operations -3. Check for existing resource files and utilities -4. Use Grep/Glob to find similar test patterns - -Common keyword locations to check: -- `test/extended/util/*.robot` - Utility keywords -- `resources/**/*.robot` - Shared resource files -- Existing test files in same feature area - -**When creating tests:** -- โœ… **REUSE** existing keywords whenever possible -- โŒ **DON'T** create duplicate keywords that already exist -- โœ… **EXTEND** existing keyword files if new keywords are truly needed -- โŒ **DON'T** reinvent common operations (oc commands, pod management, etc.) - -**Robot Framework Test File Creation Process:** -- Generate test case documentation in current working directory as `test_cases_OCPSTRAT-XXXX.md` -- Robot Framework test files will be created in microshift repository AFTER git branch is created in Step 5 -- Follow microshift repo naming convention: lowercase with hyphens (kebab-case) - - Example files in microshift: `backup-restore-on-reboot.robot`, `multi-nic.robot`, `isolated-network.robot` -- If Robot Framework tests exist in PRs, create files with matching names -- If no existing Robot tests, create new file in appropriate `test/suites//` subdirectory: - - For DNS/network features: `test/suites/network/.robot` (e.g., `coredns-hosts-file.robot`) - - Use feature-based naming: `.robot` (kebab-case, lowercase) -- Implement the top 3-5 most critical missing tests in Robot Framework -- Follow existing patterns from similar test files in microshift codebase -- Include proper setup, teardown, and error handling - -**CRITICAL Robot Framework Teardown Rules** (from `.cursor/rules/robot-framework-teardown.md` in the microshift repository): -- โœ… **DO**: Write clean teardowns without `Run Keyword And Ignore Error` -- โŒ **DON'T**: Use `Run Keyword And Ignore Error` in ANY teardown section -- Robot Framework teardowns automatically continue execution even if keywords fail -- This applies to: `[Teardown]`, `Suite Teardown`, `Test Teardown`, and keyword teardowns -- Only use `Run Keyword And Ignore Error` in test cases/setup when you need conditional logic - -**Correct Teardown Example**: -```robot -Suite Teardown Cleanup Test Resources - -Cleanup Test Resources - [Documentation] Clean up all test resources - Oc Delete pod --all -n ${NAMESPACE} - Remove Test Files - Restart MicroShift -``` - -**Incorrect Teardown (DON'T DO THIS)**: -```robot -Suite Teardown Run Keyword And Ignore Error Cleanup Test Resources - -Cleanup Test Resources - Run Keyword And Ignore Error Oc Delete pod --all -n ${NAMESPACE} - Run Keyword And Ignore Error Remove Test Files -``` - -**Keyword Reuse Example**: -```robot -# โœ… GOOD - Reusing existing keywords -*** Test Cases *** -Test CoreDNS Hosts File - [Documentation] Test hosts file resolution - Setup MicroShift Config dns.hostsFile=/etc/custom-hosts - Restart MicroShift - Wait For MicroShift Ready - Deploy Test Pod dns-test ${NAMESPACE} - ${ip}= Resolve Hostname In Pod dns-test ${NAMESPACE} test.local - Should Be Equal ${ip} 192.168.1.100 - -# โŒ BAD - Creating duplicate keywords -*** Keywords *** -My Custom MicroShift Restart - [Documentation] Restart MicroShift (DON'T DO THIS - use existing keyword!) - Run Process systemctl restart microshift - Sleep 10s -``` - -### Step 5: Create Git Branch and Robot Framework Tests in MicroShift Repository - -**IMPORTANT**: Before creating git branch, **WAIT for user confirmation**. - -Ask user for confirmation: - -**Prompt user**: "Would you like me to create a new git branch in the microshift repository? The branch will be named `test-OCPSTRAT-XXXX-$(date +%Y%m%d)`. (yes/no)" - -**If user confirms (yes)**: - -#### 5.1: Create Git Branch -1. Ask user for the microshift repository path on their system -2. Navigate to the microshift repository -3. Check current git status -4. Create new branch: `test-OCPSTRAT-XXXX-$(date +%Y%m%d)` -5. Confirm branch creation and provide branch name - -**Prompt user**: "Please provide the full path to your microshift repository (e.g., /home/username/microshift):" - -**Commands** (use the path provided by user): -```bash -cd -git status -git checkout -b test-OCPSTRAT-XXXX-$(date +%Y%m%d) -git branch --show-current -``` - -#### 5.2: Create Robot Framework Test Files in the New Branch - -After branch is created, create Robot Framework test file directly in the microshift repository: - -1. Determine appropriate test suite directory based on feature type - - Network/DNS features: `test/suites/network/` - - Storage features: `test/suites/storage/` - - Backup features: `test/suites/backup/` - - etc. - -2. Create the .robot file with kebab-case naming convention - - Example: `test/suites/network/coredns-hosts-file.robot` - -3. Implement the top 3-5 most critical tests from Step 3 - -4. Add file to git staging - -**Commands** (use the user-provided microshift repository path): -```bash -# File will be created at appropriate path, e.g.: -# /test/suites/network/.robot - -# After file is created, stage it -cd -git add test/suites//.robot -git status -``` - -**Output to user**: -- Branch created: test-OCPSTRAT-XXXX-YYYYMMDD -- Repository: -- Current branch: [branch name] -- Robot Framework test file created: test/suites//.robot -- File staged for commit: Ready to commit - -### Step 6: Summary Report -Provide: -- List of USHIFT tickets and PRs analyzed -- Existing test files found (with coverage summary) -- Test case document (10 test cases) - filename and path -- Robot Framework test files created (with test count) - filename and path -- Git branch created (branch name and repository path) - if created -- Coverage gaps addressed -- Remaining gaps (if any) diff --git a/.claude/commands/microshift-release-versions.md b/.claude/commands/microshift-release-versions.md deleted file mode 100644 index 7507172a18..0000000000 --- a/.claude/commands/microshift-release-versions.md +++ /dev/null @@ -1,382 +0,0 @@ ---- -name: MicroShift Release Versions -argument-hint: -description: Check if a given MicroShift version is available and where to find it -allowed-tools: WebFetch, Bash ---- - -## Name -microshift-release-versions - -## Synopsis -``` -/microshift-release-versions -``` - -## Description -The `microshift-release-versions` command checks if a specific MicroShift version is available and provides information about where to find RPMs and bootc images. - -This command provides: -- Version availability status (available/not yet available) -- Version type classification (nightly, EC, RC, GA, z-stream) -- RPM package URLs -- Bootc image URLs (for versions 4.18+) -- Brew build status (internal Red Hat, requires VPN) -- Links to browse packages and images - -This command is useful for: -- Checking if a new EC/RC version has been published -- Finding the correct URLs for RPM repositories -- Locating bootc image pull specs -- Understanding what version type a given version is -- Finding Brew builds for internal testing (VPN required) - -## Arguments -- `$1` (version): The MicroShift version to check - **Required** - - Formats accepted: - - Full version: `4.20.0`, `4.18.26` - - EC version: `4.21.0-ec.3` or `4.21.0~ec.3` - - RC version: `4.20.0-rc.3` or `4.20.0~rc.3` - - Short version: `4.21` (will check latest available) - -## Return Value -- **Format**: Markdown -- **Location**: Output directly to the conversation -- **Content**: - - Version availability status - - Version type (EC, RC, GA, z-stream) - - RPM URLs for both architectures (x86_64, aarch64) - - Bootc image pull specs (if available) - - Links to package catalogs - -## Version Types Reference - -| Type | Version Format | Release Cadence | RPM Source | Image Source | -|------|----------------|-----------------|------------|--------------| -| Nightly | `X.Y.0~0.nightly_YYYY_MM_DD_HHMMSS` | Continuous | Brew | Not available | -| EC | `X.Y.0~ec.N` | Every sprint (3 weeks) | Brew, mirror repo | Mirror repo | -| RC | `X.Y.0~rc.N` | After branch cutoff until GA | Brew, mirror repo | Mirror repo | -| GA | `X.Y.0` | Every 4 months | Brew, rhocp repos | registry.redhat.io | -| Z-stream | `X.Y.Z` | On request | Brew, rhocp repos | registry.redhat.io | - -## Implementation Steps - -### Step 1: Parse Version and Determine Type - -**Goal**: Parse the input version and classify its type. - -**Actions**: -1. Normalize version format (replace `~` with `-` for URL construction) -2. Extract major.minor version (e.g., `4.20` from `4.20.0-rc.3`) -3. Classify version type: - - Contains `nightly` โ†’ Nightly build - - Contains `ec` โ†’ Engineering Candidate - - Contains `rc` โ†’ Release Candidate - - Format `X.Y.0` โ†’ GA release - - Format `X.Y.Z` (Z > 0) โ†’ Z-stream release - -**Example**: -``` -Input: 4.21.0-ec.3 -Parsed: -- normalized: 4.21.0-ec.3 -- major_minor: 4.21 -- type: EC -- base_version: 4.21.0 -``` - -### Step 2: Check OpenShift Release Status - -**Goal**: Check if OCP payload is Accepted for this version. - -**Actions**: -1. Web Fetch this and look for the version: - ``` - https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/ - ``` -2. Check if the payload for the version is Accepted. - -**Outcome**: - - If the payload is Accepted continue with next step. - - If the payload is Not Accepted do not run next steps. MicroShift version won't be created if the OCP is not Accepted. - -**Note**: MicroShift packages become available when the corresponding OCP version is marked as "Accepted". - -### Step 3: Check RPM Availability - -**Goal**: Determine if RPM packages are available and provide URLs. - -**Actions**: - -**For EC versions**: -1. Check mirror.openshift.com, EC packages are under `ocp-dev-preview/`, see: - ``` - https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp-dev-preview//el9/os/Packages/ - https://mirror.openshift.com/pub/openshift-v4/aarch64/microshift/ocp-dev-preview//el9/os/Packages/ - ``` -2. Use WebFetch to verify the directory exists and list packages - -**For RC versions**: -1. Check mirror.openshift.com, RC packages are under `ocp/`, see: - ``` - https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp//el9/os/Packages/ - https://mirror.openshift.com/pub/openshift-v4/aarch64/microshift/ocp//el9/os/Packages/ - ``` -2. Use WebFetch to verify the directory exists and list packages - -**For GA/Z-stream versions**: -1. Check on brew: - ``` - https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=82827 - ``` - Use this command to check for microshift releases: - ``` - curl -sk "https://brewweb.engineering.redhat.com/brew/search?match=glob&type=build&terms={version}" 2>/dev/null - ``` - -### Step 4: Check Bootc Image Availability - -**Goal**: Find bootc image pull specs for versions 4.18+. - -**Actions**: - -**For EC versions**: -1. Fetch bootc-pullspec.txt: - ``` - https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp-dev-preview//el9/bootc-pullspec.txt - https://mirror.openshift.com/pub/openshift-v4/aarch64/microshift/ocp-dev-preview//el9/bootc-pullspec.txt - ``` -2. Extract the quay.io pull spec from the file - -**For RC versions**: -1. Fetch bootc-pullspec.txt: - ``` - https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp//el9/bootc-pullspec.txt - https://mirror.openshift.com/pub/openshift-v4/aarch64/microshift/ocp//el9/bootc-pullspec.txt - ``` - -**For GA/Z-stream versions**: -1. For pre-GA z-stream, check registry.stage.redhat.io: - - Production: `https://catalog.redhat.com/en/software/containers/openshift4/microshift-bootc-rhel9/` - - Use this command to check the catalog: `curl -sk "https://catalog.redhat.com/api/containers/v1/repositories/registry/registry.access.redhat.com/repository/openshift4/microshift-bootc-rhel9/images?page_size=500&page=0" 2>/dev/null` - - if not found try few more pages - - Stage: `https://catalog.stage.redhat.com/en/software/containers/openshift4/microshift-bootc-rhel9/` - - Use this command to check the catalog: `curl -sk "https://catalog.stage.redhat.com/api/containers/v1/repositories/registry/registry.access.redhat.com/repository/openshift4/microshift-bootc-rhel9/images?page_size=500&page=0" 2>/dev/null` - - if not found try few more pages - -### Step 5: List Available Versions (if short version provided) - -**Goal**: If user provides a short version like `4.21`, list all available sub-versions. - -**Actions**: -1. Fetch directory listing from mirror.openshift.com: - ``` - https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp-dev-preview/ - https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp/ - ``` -2. Filter for versions matching the major.minor pattern -3. List all available versions with their types - -### Step 6: Generate Report - -**Goal**: Create a comprehensive availability report. - -**Report Structure**: -```markdown -# MicroShift Version Availability: - -## Status -- **Version**: -- **Type**: EC / RC / GA / Z-stream -- **Status**: Available / Not Yet Available -- **OCP Release Status**: Accepted / Pending - -## Brew Builds (VPN Required) -- **Web**: https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=82827 -- **Build**: (if found) -- **Status**: Available / Not Found / VPN Required - -## RPM Packages - -### x86_64 -- **URL**: -- **Status**: Available / Not Found - -### aarch64 -- **URL**: -- **Status**: Available / Not Found - -## Bootc Images (4.18+) - -### x86_64 -- **Pull Spec**: - -### aarch64 -- **Pull Spec**: - -## Catalog Links -- [Red Hat Catalog]() (GA/Z-stream only) -- [Stage Catalog]() (pre-GA Z-stream only) - -## Notes -- -``` - -## Examples - -### Example 1: Check EC Version -``` -/microshift-release-versions 4.21.0-ec.3 -``` - -Output: -```markdown -# MicroShift Version Availability: 4.21.0-ec.3 - -## Status -- **Version**: 4.21.0-ec.3 -- **Type**: Engineering Candidate (EC) -- **Status**: Available -- **OCP Release Status**: Accepted - -## Brew Builds (VPN Required) -- **Web**: https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=82827 -- **Build**: microshift-4.21.0-0.ec.3.el9 -- **Status**: Available - -## RPM Packages - -### x86_64 -- **URL**: https://mirror.openshift.com/pub/openshift-v4/x86_64/microshift/ocp-dev-preview/4.21.0-ec.3/el9/os/Packages/ -- **Status**: Available - -### aarch64 -- **URL**: https://mirror.openshift.com/pub/openshift-v4/aarch64/microshift/ocp-dev-preview/4.21.0-ec.3/el9/os/Packages/ -- **Status**: Available - -## Bootc Images - -### x86_64 -- **Pull Spec**: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ec944... - -### aarch64 -- **Pull Spec**: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a255da... -``` - -### Example 2: Check RC Version (Not Yet Available) -``` -/microshift-release-versions 4.21.0-rc.1 -``` - -Output: -```markdown -# MicroShift Version Availability: 4.21.0-rc.1 - -## Status -- **Version**: 4.21.0-rc.1 -- **Type**: Release Candidate (RC) -- **Status**: Not Yet Available - -## Notes -- 4.21 is currently in EC phase (latest: 4.21.0-ec.3) -- RC versions are created after branch cutoff until GA -- Check https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/ for release status -``` - -### Example 3: Check GA/Z-stream Version -``` -/microshift-release-versions 4.18.26 -``` - -Output: -```markdown -# MicroShift Version Availability: 4.18.26 - -## Status -- **Version**: 4.18.26 -- **Type**: Z-stream -- **Status**: Available - -## RPM Packages -- **Repo**: rhocp-4.18-for-rhel-9-x86_64-rpms -- **Catalog**: https://access.redhat.com/downloads/content/microshift/4.18.26/ - -## Bootc Images -- **Pull Spec**: registry.redhat.io/openshift4/microshift-bootc-rhel9:4.18.26 -- **Catalog**: https://catalog.redhat.com/en/software/containers/openshift4/microshift-bootc-rhel9/ -``` - -### Example 4: List Available Versions for a Release -``` -/microshift-release-versions 4.21 -``` - -Output: -```markdown -# Available MicroShift 4.21 Versions - -## EC Versions (Dev Preview) -- 4.21.0-ec.3 (Accepted) -- 4.21.0-ec.2 (Accepted) -- 4.21.0-ec.1 (Accepted) -- 4.21.0-ec.0 (Accepted) - -## RC Versions -- None available yet (4.21 is still in EC phase) - -## Notes -- RC versions will be available after branch cutoff -- GA expected in approximately X months -``` - -## Reference Information - -### Brew (Internal Red Hat) - -Brew is Red Hat's internal build system. VPN is required to access it, assume VPN is enabled. - -**Web Interface**: -- Browse all MicroShift packages: https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=82827 - -**Build NVR Format**: -- GA/Z-stream: `microshift-4.18.26-.el9` -- EC: `microshift-4.21.0-0.ec.3..el9` -- RC: `microshift-4.21.0-0.rc.1..el9` -- Nightly: `microshift-4.21.0-0.nightly_YYYY_MM_DD_HHMMSS..el9` - -### URL Patterns - -**Mirror repos (EC/RC)**: -- EC RPMs: `https://mirror.openshift.com/pub/openshift-v4/{ARCH}/microshift/ocp-dev-preview/{VERSION}/el9/os/Packages/` -- RC RPMs: `https://mirror.openshift.com/pub/openshift-v4/{ARCH}/microshift/ocp/{VERSION}/el9/os/Packages/` -- EC Bootc: `https://mirror.openshift.com/pub/openshift-v4/{ARCH}/microshift/ocp-dev-preview/{VERSION}/el9/bootc-pullspec.txt` -- RC Bootc: `https://mirror.openshift.com/pub/openshift-v4/{ARCH}/microshift/ocp/{VERSION}/el9/bootc-pullspec.txt` - -**Registry repos (GA/Z-stream)**: -- Production: `registry.redhat.io/openshift4/microshift-bootc-rhel9` -- Stage: `registry.stage.redhat.io/openshift4/microshift-bootc-rhel9` - -**Catalogs**: -- Production: `https://catalog.redhat.com/en/software/containers/openshift4/microshift-bootc-rhel9/` -- Stage: `https://catalog.stage.redhat.com/en/software/containers/openshift4/microshift-bootc-rhel9/` - -### Release Timing - -- **EC versions**: Usually created from a commit on the last Wednesday of every sprint; available on the first Wednesday of the next sprint -- **RC versions**: Created after branch cutoff until GA; several may be created -- **GA versions**: Released every 4 months -- **Z-stream versions**: Created on request from MicroShift team to ART team - -### CI Testing - -Release versions are tested in CI: -- Scenarios: `test/scenarios/releases/` and `test/scenarios-bootc/releases/` -- Latest release version must be **manually updated** in `test/bin/common_versions.sh` when new RC or z-stream is available - -## Notes -- Assume VPN access is enabled by default when looking for Brew resources -- Bootc images are only available for MicroShift 4.18 and later -- MicroShift packages become available when the OCP version is marked as "Accepted" -- For nightly builds, only RPMs are available in Brew (VPN required) -- The command is read-only and does not modify any data -- Use this command to verify availability before updating CI configurations or deployment scripts diff --git a/.claude/settings.json b/.claude/settings.json index c924cd5be4..727f3aead6 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -2,8 +2,7 @@ "permissions": { "allow": [ "Read(//tmp/**)", - "Write(//tmp/**)", - "WebFetch(domain:prow.ci.openshift.org)" + "Write(//tmp/**)" ], "deny": [], "ask": [] diff --git a/apiserver-config-flow.md b/apiserver-config-flow.md deleted file mode 100644 index 665a5cd8a2..0000000000 --- a/apiserver-config-flow.md +++ /dev/null @@ -1,397 +0,0 @@ -# API Server Config Data Flow - -This document traces the data path from the `apiServer` field in `/etc/microshift/config.yaml` down to where the values are ingested by the kube-apiserver. - -## UML Sequence Diagram - -```mermaid -sequenceDiagram - participant ConfigFile as config.yaml - participant ActiveConfig as config.ActiveConfig() - participant FillDefaults as config.fillDefaults() - participant IncorporateSettings as config.incorporateUserSettings() - participant UpdateComputed as config.updateComputedValues() - participant KASRun as KubeAPIServer.Run() - participant KASConfigure as KubeAPIServer.configure() - participant Merge as resourcemerge.MergePrunedProcessConfig() - participant TempFile as Create Temp File - participant NewCmd as kubeapiserver.NewAPIServerCommand() - participant Execute as cmd.ExecuteContext() - participant GetOpenshiftConfig as enablement.GetOpenshiftConfig() - participant ConfigToFlags as openshiftkubeapiserver.ConfigToFlags() - participant ParseFlags as cmd.ParseFlags() - participant KASMain as Run() [kube-apiserver] - - Note over ConfigFile: User configuration in
/etc/microshift/config.yaml

apiServer:
advertiseAddress: "10.43.0.2"
port: 6443
auditLog:
profile: "Default"
tls:
minVersion: "VersionTLS12"
cipherSuites: [...]
featureGates:
featureSet: "CustomNoUpgrade"
customNoUpgrade:
enabled: ["UserNamespacesSupport"] - - ConfigFile->>+ActiveConfig: Read config.yaml
[files.go:120] - - ActiveConfig->>+FillDefaults: Set default values
[config.go:92] - Note right of FillDefaults: Sets default ApiServer values:
- Port: 6443
- URL: "https://localhost:6443"
- AuditLog defaults
- SubjectAltNames - FillDefaults-->>-ActiveConfig: Config with defaults - - ActiveConfig->>+IncorporateSettings: Merge user settings
[config.go:195] - Note right of IncorporateSettings: Merges user-provided values:
- ApiServer.AdvertiseAddress
- ApiServer.AuditLog.*
- ApiServer.TLS.*
- ApiServer.NamedCertificates
- ApiServer.FeatureGates.* - IncorporateSettings-->>-ActiveConfig: Config with user settings - - ActiveConfig->>+UpdateComputed: Compute derived values
[config.go:434] - Note right of UpdateComputed: Computes:
- AdvertiseAddresses (from ServiceNetwork)
- TLS.UpdateValues() - cipher suites
- SkipInterface flag - UpdateComputed-->>-ActiveConfig: Final Config - - ActiveConfig-->>-ConfigFile: *config.Config - - Note over KASRun: MicroShift creates and starts
kube-apiserver controller - - KASRun->>+KASConfigure: configure(ctx, cfg)
[kube-apiserver.go:97] - - Note right of KASConfigure: Uses cfg.ApiServer fields to build
KubeAPIServerConfig overrides:

- advertise-address
- audit-policy-file
- audit-log-*
- tls-cert-file
- tls-private-key-file
- tls-min-version
- tls-cipher-suites
- feature-gates
- service-cluster-ip-range
- disable-admission-plugins
- enable-admission-plugins - - KASConfigure->>+Merge: Merge config layers
[kube-apiserver.go:267] - Note right of Merge: Merges 3 layers:
1. defaultconfig.yaml (embedded)
2. config-overrides.yaml (embedded)
3. Runtime overrides from cfg.ApiServer

Creates KubeAPIServerConfig struct
with all settings merged - Merge-->>-KASConfigure: kasConfigBytes (marshaled YAML) - - KASConfigure-->>-KASRun: Configuration complete - - KASRun->>TempFile: Create temp config file
[kube-apiserver.go:363] - Note right of TempFile: Write kasConfigBytes to:
/tmp/kube-apiserver-config-*.yaml - - KASRun->>+NewCmd: NewAPIServerCommand()
[kube-apiserver.go:391] - Note right of NewCmd: Initialize kube-apiserver command
from kubernetes/cmd/kube-apiserver - NewCmd-->>-KASRun: cmd *cobra.Command - - KASRun->>+Execute: cmd.ExecuteContext(ctx)
[kube-apiserver.go:404] - Note right of Execute: Args:
--openshift-config /tmp/kube-apiserver-config-*.yaml
-v [verbosity] - - Execute->>+GetOpenshiftConfig: Read config file
[intialization.go:27] - Note right of GetOpenshiftConfig: Reads temp config file and
deserializes into
KubeAPIServerConfig struct

Resolves file paths
Sets recommended defaults - GetOpenshiftConfig-->>-Execute: *KubeAPIServerConfig - - Execute->>+ConfigToFlags: Convert to CLI flags
[flags.go:18] - Note right of ConfigToFlags: Converts KubeAPIServerConfig.APIServerArguments
to command-line flags:

Map[string][]string -> []string

Examples:
"advertise-address": ["10.43.0.2"]
-> --advertise-address=10.43.0.2

"feature-gates": ["UserNamespacesSupport=true"]
-> --feature-gates=UserNamespacesSupport=true

Also converts:
- ServingInfo -> tls-* flags
- AuditConfig -> audit-* flags
- AdmissionConfig -> admission-* flags - ConfigToFlags-->>-Execute: []string (CLI flags) - - Execute->>+ParseFlags: Parse merged flags
[server.go:122] - Note right of ParseFlags: Re-parses command flags with
values from OpenShift config

This updates the ServerRunOptions
with all the ApiServer settings - ParseFlags-->>-Execute: Updated options - - Execute->>+KASMain: Run(ctx, completedOptions)
[server.go:153] - Note right of KASMain: Kube-apiserver starts with
all configuration applied:

- TLS settings
- Audit logging
- Feature gates
- Admission plugins
- Serving configuration - - Note over KASMain: Kube-apiserver running with
all config.ApiServer values
applied via command-line flags -``` - -## Key Data Structures - -### 1. MicroShift Config (`pkg/config/apiserver.go`) - -```go -type ApiServer struct { - SubjectAltNames []string - AdvertiseAddress string - NamedCertificates []NamedCertificateEntry - AuditLog AuditLog - TLS TLSConfig - FeatureGates FeatureGates - URL string - Port int - AdvertiseAddresses []string -} -``` - -### 2. KubeAPIServerConfig (`vendor/github.com/openshift/api/kubecontrolplane/v1/types.go`) - -```go -type KubeAPIServerConfig struct { - GenericAPIServerConfig configv1.GenericAPIServerConfig - APIServerArguments map[string]Arguments - ServiceAccountPublicKeyFiles []string - ServicesNodePortRange string - // ... other fields -} -``` - -### 3. APIServerArguments (Key-Value Map) - -```go -APIServerArguments: map[string]Arguments{ - "advertise-address": {cfg.ApiServer.AdvertiseAddress}, - "audit-log-maxage": {strconv.Itoa(cfg.ApiServer.AuditLog.MaxFileAge)}, - "tls-min-version": {cfg.ApiServer.TLS.MinVersion}, - "tls-cipher-suites": cfg.ApiServer.TLS.CipherSuites, - "feature-gates": {"UserNamespacesSupport=true", ...}, - // ... many more -} -``` - -## Configuration Flow Steps - -### Step 1: Config Loading (`pkg/config/files.go`) - -**Function**: `ActiveConfig()` -**Line**: [120-127](file:///home/microshift/microshift/pkg/config/files.go#L120-L127) - -1. Reads `/etc/microshift/config.yaml` -2. Reads YAML files from `/etc/microshift/config.d/*.yaml` -3. Merges all YAML files using JSON patch merge -4. Deserializes into `Config` struct - -### Step 2: Config Initialization (`pkg/config/config.go`) - -**Function**: `fillDefaults()` -**Lines**: [92-190](file:///home/microshift/microshift/pkg/config/config.go#L92-L190) - -Sets default values for `ApiServer`: -```go -c.ApiServer = ApiServer{ - SubjectAltNames: subjectAltNames, - URL: "https://localhost:6443", - Port: 6443, -} -c.ApiServer.AuditLog = AuditLog{ - MaxFileAge: 0, - MaxFiles: 10, - MaxFileSize: 200, - Profile: "Default", -} -``` - -**Function**: `incorporateUserSettings()` -**Lines**: [195-429](file:///home/microshift/microshift/pkg/config/config.go#L195-L429) - -Merges user-provided values: -```go -if u.ApiServer.AdvertiseAddress != "" { - c.ApiServer.AdvertiseAddress = u.ApiServer.AdvertiseAddress -} -if u.ApiServer.AuditLog.Profile != "" { - c.ApiServer.AuditLog.Profile = u.ApiServer.AuditLog.Profile -} -// ... TLS settings -// ... FeatureGates -``` - -**Function**: `updateComputedValues()` -**Lines**: [434-516](file:///home/microshift/microshift/pkg/config/config.go#L434-L516) - -Computes derived values: -- `AdvertiseAddress` (if not set) -- `AdvertiseAddresses` (for dual-stack) -- `TLS.UpdateValues()` - cipher suites normalization - -### Step 3: KubeAPIServer Configuration (`pkg/controllers/kube-apiserver.go`) - -**Function**: `configure()` -**Lines**: [97-297](file:///home/microshift/microshift/pkg/controllers/kube-apiserver.go#L97-L297) - -Creates `KubeAPIServerConfig` overrides: -```go -overrides := &kubecontrolplanev1.KubeAPIServerConfig{ - APIServerArguments: map[string]Arguments{ - "advertise-address": {s.advertiseAddress}, - "audit-policy-file": {...}, - "audit-log-maxage": {strconv.Itoa(cfg.ApiServer.AuditLog.MaxFileAge)}, - "audit-log-maxbackup": {strconv.Itoa(cfg.ApiServer.AuditLog.MaxFiles)}, - "audit-log-maxsize": {strconv.Itoa(cfg.ApiServer.AuditLog.MaxFileSize)}, - "service-cluster-ip-range": {strings.Join(cfg.Network.ServiceNetwork, ",")}, - "disable-admission-plugins": {...}, - "enable-admission-plugins": {}, - "feature-gates": {"UserNamespacesSupport=true", ...}, - }, - GenericAPIServerConfig: configv1.GenericAPIServerConfig{ - ServingInfo: configv1.HTTPServingInfo{ - ServingInfo: configv1.ServingInfo{ - MinTLSVersion: cfg.ApiServer.TLS.MinVersion, - CipherSuites: cfg.ApiServer.TLS.CipherSuites, - NamedCertificates: namedCerts, - }, - }, - }, -} -``` - -**Function**: `resourcemerge.MergePrunedProcessConfig()` -**Lines**: [267-291](file:///home/microshift/microshift/pkg/controllers/kube-apiserver.go#L267-L291) - -Merges three configuration layers: -1. `defaultconfig.yaml` (embedded baseline) -2. `config-overrides.yaml` (embedded OpenShift defaults) -3. Runtime `overrides` (from cfg.ApiServer) - -Result: `kasConfigBytes` - marshaled YAML config - -### Step 4: Starting Kube-APIServer (`pkg/controllers/kube-apiserver.go`) - -**Function**: `Run()` -**Lines**: [315-413](file:///home/microshift/microshift/pkg/controllers/kube-apiserver.go#L315-L413) - -1. **Line 363-381**: Creates temporary config file - ```go - fd, err := os.CreateTemp("", "kube-apiserver-config-*.yaml") - io.Copy(fd, bytes.NewBuffer(s.kasConfigBytes)) - ``` - -2. **Line 391-395**: Creates kube-apiserver command - ```go - cmd := kubeapiserver.NewAPIServerCommand() - cmd.SetArgs([]string{ - "--openshift-config", fd.Name(), - "-v", strconv.Itoa(s.verbosity), - }) - ``` - -3. **Line 404**: Executes command - ```go - errorChannel <- cmd.ExecuteContext(ctx) - ``` - -### Step 5: Kube-APIServer Ingestion (`deps/.../cmd/kube-apiserver/app/server.go`) - -**Function**: `RunE` (cobra command handler) -**Lines**: [96-154](file:///home/microshift/microshift/deps/github.com/openshift/kubernetes/cmd/kube-apiserver/app/server.go#L96-L154) - -1. **Line 110**: Read OpenShift config file - ```go - openshiftConfig, err := enablement.GetOpenshiftConfig(s.OpenShiftConfig) - ``` - -2. **Line 116-119**: Convert to command-line flags - ```go - args, err := openshiftkubeapiserver.ConfigToFlags(openshiftConfig) - // Example: ["--advertise-address=10.43.0.2", "--audit-log-maxage=0", ...] - ``` - -3. **Line 122**: Parse flags to update options - ```go - cmd.ParseFlags(args) - ``` - -4. **Line 153**: Start API server with merged options - ```go - return Run(ctx, completedOptions) - ``` - -### Step 6: Config to Flags Conversion (`deps/.../openshiftkubeapiserver/flags.go`) - -**Function**: `ConfigToFlags()` -**Lines**: [18-46](file:///home/microshift/microshift/deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/flags.go#L18-L46) - -Converts `KubeAPIServerConfig` to CLI flags: - -```go -func ConfigToFlags(kubeAPIServerConfig *KubeAPIServerConfig) ([]string, error) { - args := unmaskArgs(kubeAPIServerConfig.APIServerArguments) - - // Extract from APIServerArguments map - // "advertise-address": ["10.43.0.2"] -> --advertise-address=10.43.0.2 - - // Add additional flags from other config fields - configflags.SetIfUnset(args, "bind-address", host) - configflags.SetIfUnset(args, "tls-cipher-suites", - kubeAPIServerConfig.ServingInfo.CipherSuites...) - configflags.SetIfUnset(args, "tls-min-version", - kubeAPIServerConfig.ServingInfo.MinTLSVersion) - - return configflags.ToFlagSlice(args), nil -} -``` - -**Function**: `ToFlagSlice()` -**Lines**: [29-43](file:///home/microshift/microshift/deps/github.com/openshift/kubernetes/vendor/github.com/openshift/apiserver-library-go/pkg/configflags/helpers.go#L29-L43) - -Converts map to flag array: -```go -func ToFlagSlice(args map[string][]string) []string { - var flags []string - for key, values := range args { - for _, value := range values { - flags = append(flags, fmt.Sprintf("--%s=%v", key, value)) - } - } - return flags -} -``` - -### Step 7: OpenShift Config Reading (`deps/.../enablement/intialization.go`) - -**Function**: `GetOpenshiftConfig()` -**Lines**: [27-58](file:///home/microshift/microshift/deps/github.com/openshift/kubernetes/openshift-kube-apiserver/enablement/intialization.go#L27-L58) - -1. Reads temp config file -2. Deserializes YAML into `KubeAPIServerConfig` -3. Resolves file paths (relative to config file location) -4. Applies recommended defaults - -## Example: Feature Gates Flow - -Let's trace the `featureGates` field specifically: - -### 1. User Configuration (`/etc/microshift/config.yaml`) -```yaml -apiServer: - featureGates: - featureSet: "CustomNoUpgrade" - customNoUpgrade: - enabled: ["UserNamespacesSupport"] - disabled: ["SomeOtherGate"] -``` - -### 2. Parsed into Config Struct (`pkg/config/apiserver.go`) -```go -ApiServer.FeatureGates = FeatureGates{ - FeatureSet: "CustomNoUpgrade", - CustomNoUpgrade: CustomNoUpgrade{ - Enabled: []string{"UserNamespacesSupport"}, - Disabled: []string{"SomeOtherGate"}, - }, -} -``` - -### 3. Current State: Hardcoded Feature Gates (`pkg/controllers/kube-apiserver.go` line 224) - -**NOTE**: As of the current implementation, feature gates are hardcoded and the `cfg.ApiServer.FeatureGates` config is not yet used. The implementation would need to add logic to read from `cfg.ApiServer.FeatureGates` and construct the feature-gates argument dynamically. - -Current hardcoded implementation: -```go -APIServerArguments: map[string]Arguments{ - "feature-gates": { - "UserNamespacesSupport=true", - "UserNamespacesPodSecurityStandards=true", - }, -} -``` - -**To implement feature gates from config**, the code would need to: -1. Read `cfg.ApiServer.FeatureGates.FeatureSet` to determine the base feature set -2. Parse `cfg.ApiServer.FeatureGates.CustomNoUpgrade.Enabled` and `.Disabled` lists -3. Construct feature gate strings like `"FeatureName=true"` or `"FeatureName=false"` -4. Add them to the `"feature-gates"` argument in the `APIServerArguments` map - -### 4. Converted to CLI Flags (`deps/.../flags.go`) -```bash ---feature-gates=UserNamespacesSupport=true ---feature-gates=UserNamespacesPodSecurityStandards=true -``` - -### 5. Parsed by Kube-APIServer -The kube-apiserver's flag parser reads these flags and enables the feature gates. - -## Summary - -The data flow is: - -1. **YAML file** โ†’ `ActiveConfig()` โ†’ **Config struct** -2. **Config struct** โ†’ `KubeAPIServer.configure()` โ†’ **KubeAPIServerConfig struct** -3. **KubeAPIServerConfig** โ†’ Marshaled to **temp YAML file** -4. **Temp YAML file** โ†’ `GetOpenshiftConfig()` โ†’ **KubeAPIServerConfig struct** (in kube-apiserver process) -5. **KubeAPIServerConfig** โ†’ `ConfigToFlags()` โ†’ **CLI flags array** -6. **CLI flags** โ†’ `cmd.ParseFlags()` โ†’ **ServerRunOptions** (internal kube-apiserver state) -7. **ServerRunOptions** โ†’ Used to configure and start the actual API server - -The key transformation points are: -- **Config YAML โ†’ Config struct**: Standard YAML unmarshaling -- **Config.ApiServer โ†’ KubeAPIServerConfig.APIServerArguments**: Manual mapping in `configure()` -- **KubeAPIServerConfig โ†’ CLI flags**: `ConfigToFlags()` conversion -- **CLI flags โ†’ Runtime config**: Cobra flag parsing in kube-apiserver - -All `apiServer` fields from the MicroShift config eventually become command-line flags that are parsed by the standard Kubernetes kube-apiserver flag parser. - diff --git a/rebase.log b/rebase.log deleted file mode 100644 index ad1d7ad190..0000000000 --- a/rebase.log +++ /dev/null @@ -1,846 +0,0 @@ -rebase.sh: -# Rebasing to registry.ci.openshift.org/ocp/release:4.21.0-0.nightly-2025-10-22-123727 and registry.ci.openshift.org/ocp-arm64/release-arm64:4.21.0-0.nightly-arm64-2025-10-22-190453 -# Fetching release info for registry.ci.openshift.org/ocp/release:4.21.0-0.nightly-2025-10-22-123727 (amd64) -# Fetching release info for registry.ci.openshift.org/ocp-arm64/release-arm64:4.21.0-0.nightly-arm64-2025-10-22-190453 (arm64) -# Extracting registry.ci.openshift.org/ocp/release:4.21.0-0.nightly-2025-10-22-123727 manifest content -# Cloning registry.ci.openshift.org/ocp/release:4.21.0-0.nightly-2025-10-22-123727 component repos -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/api/.git/ -HEAD is now at 8c9accaf Merge pull request #2502 from jsafrane/MutableCSINodeAllocatableCount-tp - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-csi-snapshot-controller-operator/.git/ -HEAD is now at b022cf7 Merge pull request #245 from openshift-bot/art-consistency-openshift-4.21-ose-cluster-csi-snapshot-controller-operator - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-dns-operator/.git/ -HEAD is now at 2ec8a3d Merge pull request #448 from davidesalerno/NE-2138 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-ingress-operator/.git/ -HEAD is now at 0cac97a Merge pull request #1291 from rhamini3/ci-cluster-ossm - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-kube-apiserver-operator/.git/ -HEAD is now at adc82f6 Merge pull request #1939 from bertinatto/rebase-1.34 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-kube-controller-manager-operator/.git/ -HEAD is now at ffe5113 Merge pull request #888 from bertinatto/rebase-1.34 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-kube-scheduler-operator/.git/ -HEAD is now at 10d89b5 Merge pull request #583 from bertinatto/rebase-1.34 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-network-operator/.git/ -HEAD is now at a587975 Merge pull request #2808 from maiqueb/api-l2udn-masked-subnets - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-openshift-controller-manager-operator/.git/ -HEAD is now at f90be06 Merge pull request #399 from openshift-bot/art-consistency-openshift-4.21-ose-cluster-openshift-controller-manager-operator - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/cluster-policy-controller/.git/ -HEAD is now at ef70396 Merge pull request #169 from openshift-bot/art-consistency-openshift-4.21-cluster-policy-controller - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/csi-external-snapshotter/.git/ -HEAD is now at 15313fb Merge pull request #188 from openshift-bot/art-consistency-openshift-4.21-ose-csi-snapshot-controller - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/etcd/.git/ -HEAD is now at e2b3dfdf Merge pull request #342 from openshift-bot/art-consistency-openshift-4.21-ose-etcd - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/kubernetes/.git/ -HEAD is now at 96593f323 Merge pull request #2484 from bertinatto/rebase-1.34 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/kubernetes-kube-storage-version-migrator/.git/ -HEAD is now at ce5ff17 Merge pull request #234 from openshift-bot/art-consistency-openshift-4.21-ose-kube-storage-version-migrator - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/machine-config-operator/.git/ -HEAD is now at 06e7b70a Merge pull request #5332 from djoshy/add-cpms-support - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/openshift-controller-manager/.git/ -HEAD is now at 433bd50 Merge pull request #414 from openshift-bot/art-consistency-openshift-4.21-ose-openshift-controller-manager - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/operator-framework-olm/.git/ -HEAD is now at a0b88f9a Merge pull request #1126 from Xia-Zhao-rh/opm-cases - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/route-controller-manager/.git/ -HEAD is now at c337cf4 Merge pull request #65 from openshift-bot/art-consistency-openshift-4.21-ose-route-controller-manager - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/service-ca-operator/.git/ -HEAD is now at e5d65c6 Merge pull request #276 from openshift-bot/art-consistency-openshift-4.21-ose-service-ca-operator - -# Cloning registry.ci.openshift.org/ocp/release:4.21.0-0.nightly-2025-10-22-123727 image repos -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/oc/.git/ -HEAD is now at 8836b663 Merge pull request #2113 from wking/remove-obsolete-precheckEnabled - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/coredns/.git/ -HEAD is now at 91ae57f Merge pull request #154 from openshift-bot/art-consistency-openshift-4.21-coredns - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/csi-external-snapshotter/.git/ -HEAD is now at 15313fb Merge pull request #188 from openshift-bot/art-consistency-openshift-4.21-ose-csi-snapshot-controller - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/router/.git/ -HEAD is now at 526b4d0 Merge pull request #679 from openshift-bot/art-consistency-openshift-4.21-ose-haproxy-router-base - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/kube-rbac-proxy/.git/ -HEAD is now at cd63668 Merge pull request #126 from openshift-bot/art-consistency-openshift-4.21-kube-rbac-proxy - -lvms_operator not from release payload, skipping - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/ovn-kubernetes/.git/ -HEAD is now at a573f44e Merge pull request #2772 from openshift-bot/art-consistency-openshift-4.21-ovn-kubernetes-microshift - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/kubernetes/.git/ -HEAD is now at 96593f323 Merge pull request #2484 from bertinatto/rebase-1.34 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/amd64/service-ca-operator/.git/ -HEAD is now at e5d65c6 Merge pull request #276 from openshift-bot/art-consistency-openshift-4.21-ose-service-ca-operator - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/oc/.git/ -HEAD is now at 8836b663 Merge pull request #2113 from wking/remove-obsolete-precheckEnabled - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/coredns/.git/ -HEAD is now at 91ae57f Merge pull request #154 from openshift-bot/art-consistency-openshift-4.21-coredns - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/csi-external-snapshotter/.git/ -HEAD is now at 15313fb Merge pull request #188 from openshift-bot/art-consistency-openshift-4.21-ose-csi-snapshot-controller - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/router/.git/ -HEAD is now at 526b4d0 Merge pull request #679 from openshift-bot/art-consistency-openshift-4.21-ose-haproxy-router-base - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/kube-rbac-proxy/.git/ -HEAD is now at cd63668 Merge pull request #126 from openshift-bot/art-consistency-openshift-4.21-kube-rbac-proxy - -lvms_operator not from release payload, skipping - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/ovn-kubernetes/.git/ -HEAD is now at a573f44e Merge pull request #2772 from openshift-bot/art-consistency-openshift-4.21-ovn-kubernetes-microshift - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/kubernetes/.git/ -HEAD is now at 96593f323 Merge pull request #2484 from bertinatto/rebase-1.34 - -Initialized empty Git repository in /go/src/github.com/openshift/microshift/_output/staging/arm64/service-ca-operator/.git/ -HEAD is now at e5d65c6 Merge pull request #276 from openshift-bot/art-consistency-openshift-4.21-ose-service-ca-operator - -Warning: env var PULL_BASE_REF not found or empty, falling back to local active branch. -error: branch 'rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22' not found -Switched to a new branch 'rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22' -## Updating last_rebase.sh -## Committing changes to last_rebase.sh -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 c63c22945] update last_rebase.sh - 1 file changed, 1 insertion(+), 1 deletion(-) -cluster-csi-snapshot-controller-operator embedded-component no change - -cluster-dns-operator embedded-component no change - -cluster-ingress-operator embedded-component no change - -cluster-kube-apiserver-operator embedded-component no change - -cluster-kube-controller-manager-operator embedded-component no change - -cluster-kube-scheduler-operator embedded-component no change - -cluster-network-operator embedded-component no change - -cluster-policy-controller embedded-component no change - -csi-external-snapshotter embedded-component no change - -etcd embedded-component no change - -kubernetes embedded-component no change - -kubernetes-kube-storage-version-migrator embedded-component no change - -route-controller-manager embedded-component no change - -service-ca-operator embedded-component no change - -coredns image-amd64 no change - -csi-external-snapshotter image-amd64 no change - -router image-amd64 no change - -kubernetes image-amd64 no change - -service-ca-operator image-amd64 no change - -coredns image-arm64 no change - -csi-external-snapshotter image-arm64 no change - -router image-arm64 no change - -kubernetes image-arm64 no change - -service-ca-operator image-arm64 no change - -## Committing changes to changelog -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 aa31d603e] update changelog - 2 files changed, 68 insertions(+), 5138 deletions(-) -# Updating microshift/go.mod -go mod edit -require github.com/openshift/cluster-policy-controller@ef703966fe6e82f5f255b06bc5c758f105bf8b28 -go: downloading github.com/stretchr/testify v1.11.1 -go: downloading github.com/openshift/api v0.0.0-20251015095338-264e80a2b6e7 -go: downloading github.com/openshift/client-go v0.0.0-20251015124057-db0dee36e235 -go: downloading github.com/openshift/library-go v0.0.0-20251015151611-6fc7a74b67c5 -go: downloading k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 -go: downloading sigs.k8s.io/yaml v1.6.0 -go: downloading github.com/fsnotify/fsnotify v1.9.0 -go: downloading github.com/vishvananda/netlink v1.3.1 -go: downloading github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e -go: downloading sigs.k8s.io/kustomize/kyaml v0.20.1 -go: downloading github.com/spf13/cobra v1.9.1 -go: downloading github.com/apparentlymart/go-cidr v1.1.0 -go: downloading go.etcd.io/etcd/client/pkg/v3 v3.6.4 -go: downloading github.com/evanphx/json-patch v4.12.0+incompatible -go: downloading go.etcd.io/etcd/client/v3 v3.6.4 -go: downloading github.com/spf13/pflag v1.0.10 -go: downloading github.com/openshift/build-machinery-go v0.0.0-20250602125535-1b6d00b8c37c -go: downloading github.com/squat/generic-device-plugin v0.0.0-20251019101956-043a51e18f31 -go: downloading gopkg.in/yaml.v3 v3.0.1 -go: downloading github.com/go-kit/kit v0.9.0 -go: downloading github.com/coreos/go-systemd/v22 v22.5.0 -go: downloading github.com/openshift/cluster-policy-controller v0.0.0-20251007100337-ef703966fe6e -go: downloading github.com/miekg/dns v1.1.63 -go: downloading sigs.k8s.io/kustomize/api v0.20.1 -go: downloading github.com/openshift/route-controller-manager v0.0.0-20251008111043-c337cf413bbb -go: downloading gopkg.in/yaml.v2 v2.4.0 -go: downloading golang.org/x/sys v0.36.0 -go: downloading k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b -go: downloading github.com/gogo/protobuf v1.3.2 -go: downloading github.com/golang/snappy v0.0.4 -go: downloading github.com/openshift/kubernetes-kube-storage-version-migrator v0.0.3-0.20251007192956-ce5ff17e9a81 -go: downloading github.com/prometheus/client_model v0.6.2 -go: downloading github.com/prometheus/common v0.66.1 -go: downloading github.com/google/go-cmp v0.7.0 -go: downloading github.com/prometheus/prometheus v0.302.1 -go: downloading github.com/go-logr/logr v1.4.3 -go: downloading github.com/fxamacker/cbor/v2 v2.9.0 -go: downloading sigs.k8s.io/randfill v1.0.0 -go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc -go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 -go: downloading sigs.k8s.io/structured-merge-diff/v6 v6.3.0 -go: downloading github.com/google/gnostic-models v0.7.0 -go: downloading google.golang.org/protobuf v1.36.8 -go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 -go: downloading golang.org/x/net v0.43.0 -go: downloading golang.org/x/term v0.35.0 -go: downloading go.yaml.in/yaml/v2 v2.4.2 -go: downloading go.yaml.in/yaml/v3 v3.0.4 -go: downloading github.com/vishvananda/netns v0.0.5 -go: downloading gopkg.in/evanphx/json-patch.v4 v4.12.0 -go: downloading github.com/inconshreveable/mousetrap v1.1.0 -go: downloading github.com/pkg/errors v0.9.1 -go: downloading github.com/google/uuid v1.6.0 -go: downloading go.uber.org/zap v1.27.0 -go: downloading github.com/coreos/go-semver v0.3.1 -go: downloading go.etcd.io/etcd/api/v3 v3.6.4 -go: downloading google.golang.org/grpc v1.76.0 -go: downloading github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.74.0 -go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c -go: downloading sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 -go: downloading github.com/moby/term v0.5.0 -go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de -go: downloading go.uber.org/goleak v1.3.0 -go: downloading github.com/jonboulle/clockwork v0.5.0 -go: downloading go.opentelemetry.io/otel/trace v1.37.0 -go: downloading golang.org/x/time v0.9.0 -go: downloading github.com/go-logfmt/logfmt v0.5.1 -go: downloading github.com/go-stack/stack v1.8.1 -go: downloading github.com/MakeNowJust/heredoc v1.0.0 -go: downloading github.com/mitchellh/go-wordwrap v1.0.1 -go: downloading github.com/russross/blackfriday/v2 v2.1.0 -go: downloading go.opentelemetry.io/otel v1.37.0 -go: downloading github.com/godbus/dbus/v5 v5.1.0 -go: downloading golang.org/x/tools v0.36.0 -go: downloading golang.org/x/sync v0.17.0 -go: downloading github.com/blang/semver/v4 v4.0.0 -go: downloading github.com/google/cadvisor v0.52.1 -go: downloading go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.34.0 -go: downloading go.opentelemetry.io/otel/metric v1.37.0 -go: downloading go.opentelemetry.io/otel/sdk v1.37.0 -go: downloading go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0 -go: downloading github.com/go-kit/log v0.2.1 -go: downloading github.com/oklog/run v1.2.0 -go: downloading github.com/prometheus/client_golang v1.23.2 -go: downloading github.com/Microsoft/go-winio v0.6.2 -go: downloading k8s.io/gengo/v2 v2.0.0-20250604051438-85fd79dbfd9f -go: downloading gopkg.in/inf.v0 v0.9.1 -go: downloading github.com/x448/float16 v0.8.4 -go: downloading github.com/go-openapi/swag v0.23.0 -go: downloading github.com/json-iterator/go v1.1.12 -go: downloading golang.org/x/oauth2 v0.30.0 -go: downloading github.com/stretchr/objx v0.5.2 -go: downloading github.com/prometheus/procfs v0.16.1 -go: downloading golang.org/x/text v0.29.0 -go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 -go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible -go: downloading go.uber.org/multierr v1.11.0 -go: downloading github.com/golang/protobuf v1.5.4 -go: downloading github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 -go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20250804133106-a7a43d27e69b -go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 -go: downloading github.com/go-errors/errors v1.4.2 -go: downloading github.com/go-openapi/jsonreference v0.21.0 -go: downloading github.com/kr/pretty v0.3.1 -go: downloading github.com/chai2010/gettext-go v1.0.2 -go: downloading github.com/openshift/onsi-ginkgo/v2 v2.6.1-0.20251001123353-fd5b1fb35db1 -go: downloading github.com/onsi/gomega v1.35.1 -go: downloading github.com/exponent-io/jsonpath v0.0.0-20210407135951-1de76d718b3f -go: downloading github.com/creack/pty v1.1.18 -go: downloading gopkg.in/natefinch/lumberjack.v2 v2.2.1 -go: downloading gopkg.in/go-jose/go-jose.v2 v2.6.3 -go: downloading github.com/emicklei/go-restful/v3 v3.12.2 -go: downloading github.com/pkg/profile v1.7.0 -go: downloading github.com/sirupsen/logrus v1.9.3 -go: downloading go.etcd.io/etcd/server/v3 v3.6.4 -go: downloading go.uber.org/atomic v1.11.0 -go: downloading golang.org/x/crypto v0.42.0 -go: downloading go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 -go: downloading sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 -go: downloading github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 -go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f -go: downloading github.com/openshift/apiserver-library-go v0.0.0-20251015164739-79d04067059d -go: downloading github.com/onsi/ginkgo v1.16.5 -go: downloading github.com/google/cel-go v0.26.0 -go: downloading github.com/robfig/cron/v3 v3.0.1 -go: downloading github.com/robfig/cron v1.2.0 -go: downloading github.com/opencontainers/selinux v1.11.1 -go: downloading github.com/container-storage-interface/spec v1.9.0 -go: downloading github.com/libopenstorage/openstorage v1.0.0 -go: downloading github.com/go-logr/stdr v1.2.2 -go: downloading go.opentelemetry.io/proto/otlp v1.5.0 -go: downloading github.com/opencontainers/cgroups v0.0.3 -go: downloading github.com/cyphar/filepath-securejoin v0.4.1 -go: downloading github.com/moby/sys/userns v0.1.0 -go: downloading github.com/docker/go-units v0.5.0 -go: downloading github.com/opencontainers/runc v1.2.5 -go: downloading go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful v0.44.0 -go: downloading github.com/moby/sys/mountinfo v0.7.2 -go: downloading github.com/karrick/godirwalk v1.17.0 -go: downloading github.com/containerd/containerd/api v1.8.0 -go: downloading github.com/containerd/errdefs v1.0.0 -go: downloading github.com/containerd/errdefs/pkg v0.3.0 -go: downloading github.com/opencontainers/runtime-spec v1.2.0 -go: downloading github.com/containerd/typeurl/v2 v2.2.2 -go: downloading golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated -go: downloading github.com/mailru/easyjson v0.7.7 -go: downloading github.com/beorn7/perks v1.0.1 -go: downloading github.com/cespare/xxhash/v2 v2.3.0 -go: downloading github.com/klauspost/compress v1.18.0 -go: downloading github.com/kylelemons/godebug v1.1.0 -go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd -go: downloading github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee -go: downloading github.com/google/btree v1.1.3 -go: downloading github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc -go: downloading github.com/go-openapi/jsonpointer v0.21.0 -go: downloading github.com/kr/text v0.2.0 -go: downloading github.com/rogpeppe/go-internal v1.13.1 -go: downloading github.com/go-logr/zapr v1.3.0 -go: downloading github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 -go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 -go: downloading go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 -go: downloading github.com/coreos/go-oidc v2.3.0+incompatible -go: downloading github.com/felixge/fgprof v0.9.4 -go: downloading github.com/NYTimes/gziphandler v1.1.1 -go: downloading github.com/felixge/httpsnoop v1.0.4 -go: downloading github.com/moby/spdystream v0.5.0 -go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 -go: downloading github.com/soheilhy/cmux v0.1.5 -go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20220101234140-673ab2c3ae75 -go: downloading go.etcd.io/bbolt v1.4.2 -go: downloading go.etcd.io/etcd/pkg/v3 v3.6.4 -go: downloading cel.dev/expr v0.24.0 -go: downloading github.com/stoewer/go-strcase v1.3.0 -go: downloading github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 -go: downloading go.opentelemetry.io/auto/sdk v1.1.0 -go: downloading github.com/cenkalti/backoff/v4 v4.3.0 -go: downloading github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible -go: downloading github.com/JeffAshton/win_pdh v0.0.0-20161109143554-76bb4ee9f0ab -go: downloading github.com/distribution/reference v0.6.0 -go: downloading github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2 -go: downloading github.com/Microsoft/hnslib v0.1.1 -go: downloading github.com/lithammer/dedent v1.1.0 -go: downloading go.opentelemetry.io/contrib/propagators/b3 v1.19.0 -go: downloading github.com/containerd/ttrpc v1.2.6 -go: downloading github.com/euank/go-kmsg-parser v2.0.0+incompatible -go: downloading golang.org/x/mod v0.27.0 -go: downloading golang.org/x/tools/go/expect v0.1.0-deprecated -go: downloading github.com/josharian/intern v1.0.0 -go: downloading github.com/sergi/go-diff v1.2.0 -go: downloading go.opentelemetry.io/otel/sdk/metric v1.37.0 -go: downloading github.com/fatih/camelcase v1.0.0 -go: downloading github.com/pquerna/cachecontrol v0.1.0 -go: downloading github.com/google/pprof v0.0.0-20241210010833-40e02aabc2ad -go: downloading github.com/dustin/go-humanize v1.0.1 -go: downloading go.etcd.io/raft/v3 v3.6.0 -go: downloading github.com/xiang90/probing v0.0.0-20221125231312-a49e3df8f510 -go: downloading github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1 -go: downloading github.com/antlr4-go/antlr/v4 v4.13.0 -go: downloading github.com/opencontainers/go-digest v1.0.0 -go: downloading github.com/docker/docker v27.4.1+incompatible -go: downloading github.com/docker/go-connections v0.5.0 -go: downloading github.com/opencontainers/image-spec v1.1.1 -go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 -go: downloading github.com/xlab/treeprint v1.2.0 -go: downloading github.com/containerd/log v0.1.0 -go: downloading gonum.org/v1/gonum v0.16.0 -go: downloading github.com/golang-jwt/jwt/v5 v5.2.2 -go: downloading github.com/go-ldap/ldap/v3 v3.4.11 -go: downloading github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.0 -go: downloading github.com/go-task/slim-sprig/v3 v3.0.0 -go: downloading golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 -go: downloading github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 -go: downloading github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 -go: downloading github.com/alexbrainman/sspi v0.0.0-20231016080023-1a75b4708caa -go: downloading github.com/jcmturner/gokrb5/v8 v8.4.4 -go: downloading github.com/jcmturner/gofork v1.7.6 -go: downloading github.com/hashicorp/go-uuid v1.0.3 -go: downloading github.com/jcmturner/dnsutils/v2 v2.0.0 -go: downloading github.com/jcmturner/goidentity/v6 v6.0.1 -go: downloading github.com/jcmturner/aescts/v2 v2.0.0 -go: downloading github.com/jcmturner/rpc/v2 v2.0.3 -go: downloading github.com/moby/docker-image-spec v1.3.1 -go mod edit -replace github.com/onsi/ginkgo/v2=github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20251001123353-fd5b1fb35db1 -Handling 'k8s.io/klog/v2' dep: cloning 'https://github.com/kubernetes/klog' @ 'v2.130.1' to /go/src/github.com/openshift/microshift/./deps/k8s.io/klog -Cloning into '/go/src/github.com/openshift/microshift/./deps/k8s.io/klog'... -Handling 'k8s.io/kubernetes' dep: copying /go/src/github.com/openshift/microshift/_output/staging/kubernetes -> /go/src/github.com/openshift/microshift/./deps/github.com/openshift/kubernetes -go mod edit -replace sigs.k8s.io/kube-storage-version-migrator=github.com/openshift/kubernetes-kube-storage-version-migrator@ce5ff17e9a81ce754567e6dd5053d13409c251e9 -Handling 'k8s.io/api' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/apiextensions-apiserver' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/apimachinery' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/apiserver' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/cli-runtime' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/client-go' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/cloud-provider' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/cluster-bootstrap' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/code-generator' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/component-base' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/component-helpers' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/controller-manager' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/cri-api' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/cri-client' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/csi-translation-lib' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/dynamic-resource-allocation' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/endpointslice' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/externaljwt' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kms' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kube-aggregator' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kube-controller-manager' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kube-proxy' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kube-scheduler' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kubectl' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/kubelet' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/metrics' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/mount-utils' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/pod-security-admission' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/sample-apiserver' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/sample-cli-plugin' dep: updating required version to '1.34.1' (for information purposes) -Handling 'k8s.io/sample-controller' dep: updating required version to '1.34.1' (for information purposes) -go mod edit -require github.com/openshift/route-controller-manager@c337cf413bbbfc2154dc1d3c1bf8007ea1edfbc8 -go mod edit -require github.com/openshift/cluster-policy-controller@ef703966fe6e82f5f255b06bc5c758f105bf8b28 -for p in scripts/auto-rebase/rebase_patches/0001-Fix-CSI-initialization-conflict.patch scripts/auto-rebase/rebase_patches/0002-disable-APIrequestcount-controller.patch scripts/auto-rebase/rebase_patches/0003-disable-clusterQuotaMapping-controller.patch scripts/auto-rebase/rebase_patches/0004-remove-config-informer-and-cpu-partitioning-admission-plugin.patch scripts/auto-rebase/rebase_patches/0005-remove-user-informer.patch scripts/auto-rebase/rebase_patches/0006-kcm-no-panic.patch scripts/auto-rebase/rebase_patches/0020-klog-component-names.patch scripts/auto-rebase/rebase_patches/0030-kube-controller-manager-command-context.patch; do \ - echo "Applying patch $p"; \ - git mailinfo /dev/null /dev/stderr 2<&1- < $p | git apply --reject || exit 1; \ -done -Applying patch scripts/auto-rebase/rebase_patches/0001-Fix-CSI-initialization-conflict.patch -Checking patch deps/github.com/openshift/kubernetes/pkg/volume/csi/csi_plugin.go... -Hunk #1 succeeded at 347 (offset 2 lines). -Applied patch deps/github.com/openshift/kubernetes/pkg/volume/csi/csi_plugin.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0002-disable-APIrequestcount-controller.patch -Checking patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go... -Hunk #2 succeeded at 33 (offset 1 line). -Hunk #3 succeeded at 86 (offset 2 lines). -Applied patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0003-disable-clusterQuotaMapping-controller.patch -Checking patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go... -Hunk #2 succeeded at 26 (offset 1 line). -Hunk #3 succeeded at 50 (offset 1 line). -Hunk #4 succeeded at 109 (offset 2 lines). -Hunk #5 succeeded at 127 (offset 2 lines). -Hunk #6 succeeded at 141 (offset 2 lines). -Hunk #7 succeeded at 157 (offset 2 lines). -Applied patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0004-remove-config-informer-and-cpu-partitioning-admission-plugin.patch -Checking patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go... -Checking patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/admission/admissionenablement/register.go... -Applied patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go cleanly. -Applied patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/admission/admissionenablement/register.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0005-remove-user-informer.patch -Checking patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go... -Applied patch deps/github.com/openshift/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver/patch.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0006-kcm-no-panic.patch -Checking patch deps/github.com/openshift/kubernetes/cmd/kube-controller-manager/app/options/options.go... -Applied patch deps/github.com/openshift/kubernetes/cmd/kube-controller-manager/app/options/options.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0020-klog-component-names.patch -Checking patch deps/k8s.io/klog/goroutine_labels.go... -Checking patch deps/k8s.io/klog/internal/buffer/buffer.go... -Hunk #1 succeeded at 144 (offset -5 lines). -Checking patch deps/k8s.io/klog/internal/buffer/goroutine_labels.go... -Applied patch deps/k8s.io/klog/goroutine_labels.go cleanly. -Applied patch deps/k8s.io/klog/internal/buffer/buffer.go cleanly. -Applied patch deps/k8s.io/klog/internal/buffer/goroutine_labels.go cleanly. -Applying patch scripts/auto-rebase/rebase_patches/0030-kube-controller-manager-command-context.patch -Checking patch deps/github.com/openshift/kubernetes/cmd/kube-controller-manager/app/controllermanager.go... -Applied patch deps/github.com/openshift/kubernetes/cmd/kube-controller-manager/app/controllermanager.go cleanly. -# Updating etcd/go.mod -go mod edit -require github.com/openshift/cluster-policy-controller@ef703966fe6e82f5f255b06bc5c758f105bf8b28 -go: downloading github.com/openshift/etcd/api/v3 v3.5.1-0.20251001062325-e2b3dfdf0379 -go: downloading github.com/openshift/etcd/server/v3 v3.5.1-0.20251001062325-e2b3dfdf0379 -go: downloading k8s.io/klog/v2 v2.130.1 -go: downloading google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80 -go: downloading github.com/grpc-ecosystem/grpc-gateway v1.16.0 -go: downloading github.com/openshift/etcd/client/pkg/v3 v3.5.1-0.20251001062325-e2b3dfdf0379 -go: downloading github.com/openshift/etcd/client/v3 v3.5.1-0.20251001062325-e2b3dfdf0379 -go: downloading github.com/openshift/etcd/pkg/v3 v3.5.1-0.20251001062325-e2b3dfdf0379 -go: downloading github.com/openshift/etcd/raft/v3 v3.5.1-0.20251001062325-e2b3dfdf0379 -go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 -go: downloading github.com/golang-jwt/jwt/v4 v4.5.2 -go: downloading go.etcd.io/etcd/client/v2 v2.305.21 -go: downloading github.com/cockroachdb/datadriven v1.0.2 -skipping modulepath github.com/openshift/microshift: no or unknown command [] -skipping modulepath github.com/openshift/microshift/pkg/config: no or unknown command [] -skipping modulepath github.com/openshift/microshift/pkg/util/cryptomaterial: no or unknown command [] -go mod edit -replace github.com/onsi/ginkgo/v2=github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20251001123353-fd5b1fb35db1 -go mod edit -replace go.etcd.io/etcd/api/v3=github.com/openshift/etcd/api/v3@e2b3dfdf037938f7f0c489438eaa1138fd3fadba -go mod edit -replace go.etcd.io/etcd/client/pkg/v3=github.com/openshift/etcd/client/pkg/v3@v3.5.1-0.20251001062325-e2b3dfdf0379 -go mod edit -replace go.etcd.io/etcd/client/v3=github.com/openshift/etcd/client/v3@v3.5.1-0.20251001062325-e2b3dfdf0379 -go mod edit -replace go.etcd.io/etcd/pkg/v3=github.com/openshift/etcd/pkg/v3@v3.5.1-0.20251001062325-e2b3dfdf0379 -go mod edit -replace go.etcd.io/etcd/raft/v3=github.com/openshift/etcd/raft/v3@v3.5.1-0.20251001062325-e2b3dfdf0379 -go mod edit -replace go.etcd.io/etcd/server/v3=github.com/openshift/etcd/server/v3@v3.5.1-0.20251001062325-e2b3dfdf0379 -skipping modulepath k8s.io/api: no or unknown command [] -skipping modulepath k8s.io/apiextensions-apiserver: no or unknown command [] -skipping modulepath k8s.io/apimachinery: no or unknown command [] -skipping modulepath k8s.io/apiserver: no or unknown command [] -skipping modulepath k8s.io/cli-runtime: no or unknown command [] -skipping modulepath k8s.io/client-go: no or unknown command [] -skipping modulepath k8s.io/cloud-provider: no or unknown command [] -skipping modulepath k8s.io/cluster-bootstrap: no or unknown command [] -skipping modulepath k8s.io/code-generator: no or unknown command [] -skipping modulepath k8s.io/component-base: no or unknown command [] -skipping modulepath k8s.io/component-helpers: no or unknown command [] -skipping modulepath k8s.io/controller-manager: no or unknown command [] -skipping modulepath k8s.io/cri-api: no or unknown command [] -skipping modulepath k8s.io/cri-client: no or unknown command [] -skipping modulepath k8s.io/csi-translation-lib: no or unknown command [] -skipping modulepath k8s.io/dynamic-resource-allocation: no or unknown command [] -skipping modulepath k8s.io/endpointslice: no or unknown command [] -skipping modulepath k8s.io/externaljwt: no or unknown command [] -skipping modulepath k8s.io/kms: no or unknown command [] -skipping modulepath k8s.io/kube-aggregator: no or unknown command [] -skipping modulepath k8s.io/kube-controller-manager: no or unknown command [] -skipping modulepath k8s.io/kube-proxy: no or unknown command [] -skipping modulepath k8s.io/kube-scheduler: no or unknown command [] -skipping modulepath k8s.io/kubectl: no or unknown command [] -skipping modulepath k8s.io/kubelet: no or unknown command [] -skipping modulepath k8s.io/legacy-cloud-providers: no or unknown command [] -skipping modulepath k8s.io/metrics: no or unknown command [] -skipping modulepath k8s.io/mount-utils: no or unknown command [] -skipping modulepath k8s.io/pod-security-admission: no or unknown command [] -skipping modulepath k8s.io/sample-apiserver: no or unknown command [] -skipping modulepath k8s.io/sample-cli-plugin: no or unknown command [] -skipping modulepath k8s.io/sample-controller: no or unknown command [] -go mod edit -require github.com/openshift/route-controller-manager@c337cf413bbbfc2154dc1d3c1bf8007ea1edfbc8 -go mod edit -require github.com/openshift/cluster-policy-controller@ef703966fe6e82f5f255b06bc5c758f105bf8b28 -## Committing changes to microshift/go.mod -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 53ecde04e] update microshift/go.mod - 2 files changed, 3 insertions(+), 66 deletions(-) -## Updating deps/ directory -## Commiting changes to microshift/deps directory -warning: in the working copy of 'deps/github.com/openshift/kubernetes/vendor/github.com/MakeNowJust/heredoc/README.md', CRLF will be replaced by LF the next time Git touches it -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 c5799d054] update microshift/deps - 4 files changed, 11 insertions(+), 12 deletions(-) -## Updating microshift/vendor directory -/go/src/github.com/openshift/microshift /go/src/github.com/openshift/microshift -go mod vendor -/go/src/github.com/openshift/microshift -## Commiting changes to microshift/vendor directory -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 472937960] update microshift/vendor - 4 files changed, 8 insertions(+), 29 deletions(-) -Ensuring gofmt -Running `gofmt -s -l -w` on 853 file(s). -./deps/github.com/openshift/kubernetes/staging/src/k8s.io/apimachinery/third_party/forked/golang/json/fields.go -./deps/github.com/openshift/kubernetes/staging/src/k8s.io/client-go/third_party/forked/golang/template/exec.go -./deps/github.com/openshift/kubernetes/staging/src/k8s.io/client-go/third_party/forked/golang/template/funcs.go -./deps/github.com/openshift/kubernetes/staging/src/k8s.io/code-generator/third_party/forked/golang/reflect/type.go -## Commiting gofmt changes to deps directory -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 02821965b] update deps gofmt - 4 files changed, 12 insertions(+), 11 deletions(-) -## Committing changes to etcd/go.mod -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 b4913d986] update etcd/go.mod - 2 files changed, 6 insertions(+), 75 deletions(-) -## Updating deps/ directory -## Updating etcd/vendor directory -/go/src/github.com/openshift/microshift/etcd /go/src/github.com/openshift/microshift -go mod vendor -/go/src/github.com/openshift/microshift -## Commiting changes to etcd/vendor directory -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 fb3d80983] update etcd/vendor - 1 file changed, 26 deletions(-) -Ensuring gofmt -Running `gofmt -s -l -w` on 853 file(s). -Rebasing release_*.json -## Committing changes to pkg/release -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 a98b29d30] update component images - 2 files changed, 2 insertions(+), 2 deletions(-) -Copying manifests -2025-10-23 04:18:02,339 INFO Clearing directory assets/components/openshift-dns/dns/ -2025-10-23 04:18:02,340 DEBUG Copying assets/components/openshift-dns/dns/cluster-role-binding.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/dns/cluster-role-binding.yaml -2025-10-23 04:18:02,341 DEBUG Copying assets/components/openshift-dns/dns/cluster-role.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/dns/cluster-role.yaml -2025-10-23 04:18:02,341 INFO Restoring assets/components/openshift-dns/dns/configmap.yaml -2025-10-23 04:18:02,341 DEBUG Executing 'git restore assets/components/openshift-dns/dns/configmap.yaml' -2025-10-23 04:18:02,385 DEBUG Copying assets/components/openshift-dns/dns/daemonset.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/dns/daemonset.yaml -2025-10-23 04:18:02,386 DEBUG Copying assets/components/openshift-dns/dns/namespace.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/dns/namespace.yaml -2025-10-23 04:18:02,387 DEBUG Copying assets/components/openshift-dns/dns/service-account.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/dns/service-account.yaml -2025-10-23 04:18:02,388 DEBUG Copying assets/components/openshift-dns/dns/service.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/dns/service.yaml -2025-10-23 04:18:02,388 INFO Clearing directory assets/components/openshift-dns/node-resolver -2025-10-23 04:18:02,389 WARNING Ignoring components/openshift-dns/node-resolver/daemonset.yaml because it's created by processing daemonset.yaml.tmpl -2025-10-23 04:18:02,389 INFO Restoring assets/components/openshift-dns/node-resolver/daemonset.yaml.tmpl -2025-10-23 04:18:02,389 DEBUG Executing 'git restore assets/components/openshift-dns/node-resolver/daemonset.yaml.tmpl' -2025-10-23 04:18:02,429 DEBUG Copying assets/components/openshift-dns/node-resolver/service-account.yaml <- _output/staging/cluster-dns-operator/pkg/manifests/assets/node-resolver/service-account.yaml -2025-10-23 04:18:02,430 DEBUG Copying assets/components/openshift-dns/node-resolver/update-node-resolver.sh <- _output/staging/cluster-dns-operator/pkg/manifests/assets/node-resolver/update-node-resolver.sh -2025-10-23 04:18:02,431 INFO Clearing directory assets/components/openshift-router/ -2025-10-23 04:18:02,432 DEBUG Copying assets/components/openshift-router/cluster-role-binding.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/cluster-role-binding.yaml -2025-10-23 04:18:02,433 DEBUG Copying assets/components/openshift-router/cluster-role.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/cluster-role.yaml -2025-10-23 04:18:02,434 INFO Restoring assets/components/openshift-router/cluster-role-system-router.yaml -2025-10-23 04:18:02,434 DEBUG Executing 'git restore assets/components/openshift-router/cluster-role-system-router.yaml' -2025-10-23 04:18:02,474 INFO Restoring assets/components/openshift-router/configmap.yaml -2025-10-23 04:18:02,474 DEBUG Executing 'git restore assets/components/openshift-router/configmap.yaml' -2025-10-23 04:18:02,514 INFO Restoring assets/components/openshift-router/configmap-accesslog.yaml -2025-10-23 04:18:02,515 DEBUG Executing 'git restore assets/components/openshift-router/configmap-accesslog.yaml' -2025-10-23 04:18:02,555 DEBUG Copying assets/components/openshift-router/deployment.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/deployment.yaml -2025-10-23 04:18:02,556 DEBUG Copying assets/components/openshift-router/namespace.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/namespace.yaml -2025-10-23 04:18:02,557 DEBUG Copying assets/components/openshift-router/service-account.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/service-account.yaml -2025-10-23 04:18:02,558 DEBUG Copying assets/components/openshift-router/service-internal.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/service-internal.yaml -2025-10-23 04:18:02,559 DEBUG Copying assets/components/openshift-router/service-cloud.yaml <- _output/staging/cluster-ingress-operator/pkg/manifests/assets/router/service-cloud.yaml -2025-10-23 04:18:02,560 INFO Restoring assets/components/openshift-router/serving-certificate.yaml -2025-10-23 04:18:02,560 DEBUG Executing 'git restore assets/components/openshift-router/serving-certificate.yaml' -2025-10-23 04:18:02,600 INFO Restoring assets/components/openshift-router/cluster-role-aggregate-edit-route.yaml -2025-10-23 04:18:02,600 DEBUG Executing 'git restore assets/components/openshift-router/cluster-role-aggregate-edit-route.yaml' -2025-10-23 04:18:02,642 INFO Restoring assets/components/openshift-router/cluster-role-aggregate-admin-route.yaml -2025-10-23 04:18:02,642 DEBUG Executing 'git restore assets/components/openshift-router/cluster-role-aggregate-admin-route.yaml' -2025-10-23 04:18:02,683 WARNING Ignoring components/openshift-router/ingress-class.yaml because gets generated during image rebase -2025-10-23 04:18:02,683 WARNING Ignoring components/ovn/ because it's not covered by rebase script yet -2025-10-23 04:18:02,683 INFO Clearing directory assets/components/service-ca/ -2025-10-23 04:18:02,684 DEBUG Copying assets/components/service-ca/clusterrole.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/clusterrole.yaml -2025-10-23 04:18:02,684 DEBUG Copying assets/components/service-ca/clusterrolebinding.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/clusterrolebinding.yaml -2025-10-23 04:18:02,685 DEBUG Copying assets/components/service-ca/deployment.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/deployment.yaml -2025-10-23 04:18:02,685 DEBUG Copying assets/components/service-ca/ns.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/ns.yaml -2025-10-23 04:18:02,685 DEBUG Copying assets/components/service-ca/role.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/role.yaml -2025-10-23 04:18:02,685 DEBUG Copying assets/components/service-ca/rolebinding.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/rolebinding.yaml -2025-10-23 04:18:02,685 DEBUG Copying assets/components/service-ca/sa.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/sa.yaml -2025-10-23 04:18:02,685 DEBUG Copying assets/components/service-ca/signing-cabundle.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/signing-cabundle.yaml -2025-10-23 04:18:02,686 DEBUG Copying assets/components/service-ca/signing-secret.yaml <- _output/staging/service-ca-operator/bindata/v4.0.0/controller/signing-secret.yaml -2025-10-23 04:18:02,686 INFO Not clearing dir components/csi-snapshot-controller/ -2025-10-23 04:18:02,686 DEBUG Copying assets/components/csi-snapshot-controller/csi_controller_deployment.yaml <- _output/staging/cluster-csi-snapshot-controller-operator/assets/csi_controller_deployment.yaml -2025-10-23 04:18:02,687 DEBUG Copying assets/components/csi-snapshot-controller/serviceaccount.yaml <- _output/staging/cluster-csi-snapshot-controller-operator/assets/serviceaccount.yaml -2025-10-23 04:18:02,688 DEBUG Copying assets/components/csi-snapshot-controller/volumesnapshotclasses.yaml <- _output/staging/cluster-csi-snapshot-controller-operator/assets/volumesnapshotclasses.yaml -2025-10-23 04:18:02,689 DEBUG Copying assets/components/csi-snapshot-controller/volumesnapshotcontents.yaml <- _output/staging/cluster-csi-snapshot-controller-operator/assets/volumesnapshotcontents.yaml -2025-10-23 04:18:02,690 DEBUG Copying assets/components/csi-snapshot-controller/volumesnapshots.yaml <- _output/staging/cluster-csi-snapshot-controller-operator/assets/volumesnapshots.yaml -2025-10-23 04:18:02,691 DEBUG Copying assets/components/csi-snapshot-controller/05_operand_rbac.yaml <- _output/staging/cluster-csi-snapshot-controller-operator/assets/../manifests/05_operand_rbac.yaml -2025-10-23 04:18:02,692 WARNING Ignoring components/csi-snapshot-controller/clusterrole.yaml because extracted automatically from 05_operand_rbac.yaml -2025-10-23 04:18:02,692 WARNING Ignoring components/csi-snapshot-controller/clusterrolebinding.yaml because extracted automatically from 05_operand_rbac.yaml -2025-10-23 04:18:02,692 INFO Clearing directory assets/controllers/cluster-policy-controller/ -2025-10-23 04:18:02,693 DEBUG Copying assets/controllers/cluster-policy-controller/namespace-security-allocation-controller-clusterrole.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/namespace-security-allocation-controller-clusterrole.yaml -2025-10-23 04:18:02,693 DEBUG Copying assets/controllers/cluster-policy-controller/namespace-security-allocation-controller-clusterrolebinding.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/namespace-security-allocation-controller-clusterrolebinding.yaml -2025-10-23 04:18:02,694 DEBUG Copying assets/controllers/cluster-policy-controller/podsecurity-admission-label-syncer-controller-clusterrole.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/podsecurity-admission-label-syncer-controller-clusterrole.yaml -2025-10-23 04:18:02,695 DEBUG Copying assets/controllers/cluster-policy-controller/podsecurity-admission-label-syncer-controller-clusterrolebinding.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/podsecurity-admission-label-syncer-controller-clusterrolebinding.yaml -2025-10-23 04:18:02,696 DEBUG Copying assets/controllers/cluster-policy-controller/podsecurity-admission-label-privileged-namespaces-syncer-controller-clusterrole.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/podsecurity-admission-label-privileged-namespaces-syncer-controller-clusterrole.yaml -2025-10-23 04:18:02,697 DEBUG Copying assets/controllers/cluster-policy-controller/podsecurity-admission-label-privileged-namespaces-syncer-controller-clusterrolebinding.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/podsecurity-admission-label-privileged-namespaces-syncer-controller-clusterrolebinding.yaml -2025-10-23 04:18:02,698 INFO Clearing directory assets/controllers/kube-apiserver/ -2025-10-23 04:18:02,698 DEBUG Copying assets/controllers/kube-apiserver/config-overrides.yaml <- _output/staging/cluster-kube-apiserver-operator/bindata/assets/config/config-overrides.yaml -2025-10-23 04:18:02,698 DEBUG Copying assets/controllers/kube-apiserver/defaultconfig.yaml <- _output/staging/cluster-kube-apiserver-operator/bindata/assets/config/defaultconfig.yaml -2025-10-23 04:18:02,698 INFO Clearing directory assets/controllers/kube-controller-manager/ -2025-10-23 04:18:02,699 DEBUG Copying assets/controllers/kube-controller-manager/defaultconfig.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/../config/defaultconfig.yaml -2025-10-23 04:18:02,700 DEBUG Copying assets/controllers/kube-controller-manager/csr_approver_clusterrole.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/csr_approver_clusterrole.yaml -2025-10-23 04:18:02,700 DEBUG Copying assets/controllers/kube-controller-manager/csr_approver_clusterrolebinding.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/csr_approver_clusterrolebinding.yaml -2025-10-23 04:18:02,701 DEBUG Copying assets/controllers/kube-controller-manager/namespace-openshift-kube-controller-manager.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/ns.yaml -2025-10-23 04:18:02,702 INFO Clearing directory assets/controllers/openshift-default-scc-manager/ -2025-10-23 04:18:02,703 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-anyuid.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-anyuid.yaml -2025-10-23 04:18:02,704 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-hostaccess.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-hostaccess.yaml -2025-10-23 04:18:02,705 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-hostmount-anyuid.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-hostmount-anyuid.yaml -2025-10-23 04:18:02,706 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-hostnetwork-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-hostnetwork-v2.yaml -2025-10-23 04:18:02,707 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-hostnetwork.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-hostnetwork.yaml -2025-10-23 04:18:02,707 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-nonroot-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-nonroot-v2.yaml -2025-10-23 04:18:02,708 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-nonroot.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-nonroot.yaml -2025-10-23 04:18:02,709 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-privileged.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-privileged.yaml -2025-10-23 04:18:02,710 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-restricted-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-restricted-v2.yaml -2025-10-23 04:18:02,711 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_cr-scc-restricted.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_cr-scc-restricted.yaml -2025-10-23 04:18:02,712 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_crb-systemauthenticated-scc-restricted-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_crb-systemauthenticated-scc-restricted-v2.yaml -2025-10-23 04:18:02,712 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-anyuid.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-anyuid.yaml -2025-10-23 04:18:02,713 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-hostaccess.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-hostaccess.yaml -2025-10-23 04:18:02,714 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-hostmount-anyuid.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-hostmount-anyuid.yaml -2025-10-23 04:18:02,714 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-hostnetwork-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-hostnetwork-v2.yaml -2025-10-23 04:18:02,715 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-hostnetwork.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-hostnetwork.yaml -2025-10-23 04:18:02,716 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-nonroot-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-nonroot-v2.yaml -2025-10-23 04:18:02,716 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-nonroot.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-nonroot.yaml -2025-10-23 04:18:02,717 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-privileged.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-privileged.yaml -2025-10-23 04:18:02,718 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-restricted-v2.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-restricted-v2.yaml -2025-10-23 04:18:02,719 DEBUG Copying assets/controllers/openshift-default-scc-manager/0000_20_kube-apiserver-operator_00_scc-restricted.yaml <- _output/staging/release-manifests/0000_20_kube-apiserver-operator_00_scc-restricted.yaml -2025-10-23 04:18:02,719 INFO Clearing directory assets/controllers/route-controller-manager/ -2025-10-23 04:18:02,720 DEBUG Copying assets/controllers/route-controller-manager/ns.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/ns.yaml -2025-10-23 04:18:02,720 DEBUG Copying assets/controllers/route-controller-manager/sa.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/sa.yaml -2025-10-23 04:18:02,721 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-clusterrole.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-clusterrole.yaml -2025-10-23 04:18:02,721 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-clusterrolebinding.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-clusterrolebinding.yaml -2025-10-23 04:18:02,721 DEBUG Copying assets/controllers/route-controller-manager/informer-clusterrolebinding.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/informer-clusterrolebinding.yaml -2025-10-23 04:18:02,721 DEBUG Copying assets/controllers/route-controller-manager/informer-clusterrole.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/informer-clusterrole.yaml -2025-10-23 04:18:02,721 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-ingress-to-route-controller-clusterrolebinding.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-ingress-to-route-controller-clusterrolebinding.yaml -2025-10-23 04:18:02,721 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-ingress-to-route-controller-clusterrole.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-ingress-to-route-controller-clusterrole.yaml -2025-10-23 04:18:02,722 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-ns.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-ns.yaml -2025-10-23 04:18:02,722 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-sa.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-sa.yaml -2025-10-23 04:18:02,722 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-separate-sa-rolebinding.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-separate-sa-rolebinding.yaml -2025-10-23 04:18:02,722 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-separate-sa-role.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-separate-sa-role.yaml -2025-10-23 04:18:02,722 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-tokenreview-clusterrolebinding.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-tokenreview-clusterrolebinding.yaml -2025-10-23 04:18:02,722 DEBUG Copying assets/controllers/route-controller-manager/route-controller-manager-tokenreview-clusterrole.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/route-controller-manager-tokenreview-clusterrole.yaml -2025-10-23 04:18:02,723 INFO Restoring assets/controllers/route-controller-manager/route-controller-manager-authentication-reader-rolebinding.yaml -2025-10-23 04:18:02,723 DEBUG Executing 'git restore assets/controllers/route-controller-manager/route-controller-manager-authentication-reader-rolebinding.yaml' -2025-10-23 04:18:02,764 INFO Not clearing dir core/ -2025-10-23 04:18:02,764 DEBUG Copying assets/core/0000_50_cluster-openshift-controller-manager_00_namespace.yaml <- _output/staging/cluster-openshift-controller-manager-operator/bindata/assets/openshift-controller-manager/ns.yaml -2025-10-23 04:18:02,764 DEBUG Copying assets/core/namespace-openshift-infra.yaml <- _output/staging/cluster-kube-controller-manager-operator/bindata/assets/kube-controller-manager/namespace-openshift-infra.yaml -2025-10-23 04:18:02,765 WARNING Ignoring core/priority-class-openshift-user-critical.yaml because it's a priority class needed for oc debug node command - not available in any repo that rebase is using -2025-10-23 04:18:02,765 WARNING Ignoring core/securityv1-local-apiservice.yaml because it's a local API service for security API group, needed if OpenShift API server is not present -2025-10-23 04:18:02,765 DEBUG Copying assets/core/kubelet.yaml <- _output/staging/machine-config-operator/templates/master/01-master-kubelet/_base/files/kubelet.yaml -2025-10-23 04:18:02,766 WARNING Ignoring core/kubelet-client-ca.yaml because it's a template for ConfigMap processed during runtime -2025-10-23 04:18:02,766 INFO Clearing directory assets/crd/ -2025-10-23 04:18:02,766 DEBUG Copying assets/crd/0000_03_config-operator_01_securitycontextconstraints.crd.yaml <- _output/staging/release-manifests/0000_03_config-operator_01_securitycontextconstraints.crd.yaml -2025-10-23 04:18:02,767 DEBUG Copying assets/crd/0000_03_config-operator_02_rangeallocations.crd.yaml <- _output/staging/release-manifests/0000_03_config-operator_02_rangeallocations.crd.yaml -2025-10-23 04:18:02,768 DEBUG Copying assets/crd/storage_version_migration.crd.yaml <- _output/staging/release-manifests/0000_50_cluster-kube-storage-version-migrator-operator_01_storage_migration_crd.yaml -2025-10-23 04:18:02,769 DEBUG Copying assets/crd/route.crd.yaml <- _output/staging/api/route/v1/zz_generated.crd-manifests/routes.crd.yaml -2025-10-23 04:18:02,770 WARNING Ignoring release/ because it contains files generated during rebase procedure -2025-10-23 04:18:02,770 INFO Not clearing dir version/ -2025-10-23 04:18:02,771 WARNING Ignoring version/microshift-version.yaml because it's a template for ConfigMap processed during runtime -2025-10-23 04:18:02,771 WARNING Ignoring embed.go because it's a MicroShift specific Go file that embeds into a binary -2025-10-23 04:18:02,771 INFO Clearing directory assets/optional/operator-lifecycle-manager/ -2025-10-23 04:18:02,772 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-catalogsources.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-catalogsources.crd.yaml -2025-10-23 04:18:02,772 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-clusterserviceversions.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-clusterserviceversions.crd.yaml -2025-10-23 04:18:02,773 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-installplans.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-installplans.crd.yaml -2025-10-23 04:18:02,773 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-namespace.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-namespace.yaml -2025-10-23 04:18:02,773 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-olmconfigs.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-olmconfigs.crd.yaml -2025-10-23 04:18:02,773 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-operatorconditions.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-operatorconditions.crd.yaml -2025-10-23 04:18:02,774 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-operatorgroups.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-operatorgroups.crd.yaml -2025-10-23 04:18:02,774 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-operators.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-operators.crd.yaml -2025-10-23 04:18:02,774 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-packageserver.pdb.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-packageserver.pdb.yaml -2025-10-23 04:18:02,774 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-pprof-config.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-pprof-config.yaml -2025-10-23 04:18:02,774 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-pprof-rbac.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-pprof-rbac.yaml -2025-10-23 04:18:02,774 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-pprof-secret.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-pprof-secret.yaml -2025-10-23 04:18:02,775 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_00-subscriptions.crd.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_00-subscriptions.crd.yaml -2025-10-23 04:18:02,775 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_01-networkpolicies.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_01-networkpolicies.yaml -2025-10-23 04:18:02,775 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_02-olm-operator.serviceaccount.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_02-olm-operator.serviceaccount.yaml -2025-10-23 04:18:02,775 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_03-olmconfig.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_03-olmconfig.yaml -2025-10-23 04:18:02,776 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_03-services.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_03-services.yaml -2025-10-23 04:18:02,776 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_07-olm-operator.deployment.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_07-olm-operator.deployment.yaml -2025-10-23 04:18:02,776 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_08-catalog-operator.deployment.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_08-catalog-operator.deployment.yaml -2025-10-23 04:18:02,776 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_09-aggregated.clusterrole.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_09-aggregated.clusterrole.yaml -2025-10-23 04:18:02,776 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_13-operatorgroup-default.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_13-operatorgroup-default.yaml -2025-10-23 04:18:02,776 DEBUG Copying assets/optional/operator-lifecycle-manager/0000_50_olm_15-csv-viewer.rbac.yaml <- _output/staging/operator-framework-olm/microshift-manifests/0000_50_olm_15-csv-viewer.rbac.yaml -2025-10-23 04:18:02,777 DEBUG Copying assets/optional/operator-lifecycle-manager/image-references <- _output/staging/operator-framework-olm/microshift-manifests/image-references -2025-10-23 04:18:02,777 DEBUG Copying assets/optional/operator-lifecycle-manager/kustomization.yaml <- _output/staging/operator-framework-olm/microshift-manifests/kustomization.yaml -2025-10-23 04:18:02,777 WARNING Ignoring optional/operator-lifecycle-manager/kustomization.x86_64.yaml because gets generated during image rebase -2025-10-23 04:18:02,777 WARNING Ignoring optional/operator-lifecycle-manager/kustomization.aarch64.yaml because gets generated during image rebase -2025-10-23 04:18:02,777 WARNING Ignoring optional/operator-lifecycle-manager/release-olm-aarch64.json because gets generated during image rebase -2025-10-23 04:18:02,777 WARNING Ignoring optional/operator-lifecycle-manager/release-olm-x86_64.json because gets generated during image rebase -2025-10-23 04:18:02,777 WARNING Ignoring components/multus/ because they don't exist in upstream repository - only in microshift -2025-10-23 04:18:02,777 WARNING Ignoring optional/observability/ because they don't exist in upstream repository - only in microshift -Modifying OpenShift manifests -/go/src/github.com/openshift/microshift /go/src/github.com/openshift/microshift/_output/staging /go/src/github.com/openshift/microshift -Checking patch /go/src/github.com/openshift/microshift/scripts/auto-rebase/manifests_patches/010-ingress-deployment-clientCA.patch -/go/src/github.com/openshift/microshift/scripts/auto-rebase/manifests_patches/010-ingress-deployment-clientCA.patch - Patch applied -Checking patch /go/src/github.com/openshift/microshift/scripts/auto-rebase/manifests_patches/011-ingress-deployment-access-logging.patch -/go/src/github.com/openshift/microshift/scripts/auto-rebase/manifests_patches/011-ingress-deployment-access-logging.patch - Patch applied -Checking patch /go/src/github.com/openshift/microshift/scripts/auto-rebase/manifests_patches/012-ingress-class.patch -/go/src/github.com/openshift/microshift/scripts/auto-rebase/manifests_patches/012-ingress-class.patch - Patch applied -/go/src/github.com/openshift/microshift/_output/staging /go/src/github.com/openshift/microshift -Rebasing operator-lifecycle-manager manifests -Rebasing Multus images -## Committing changes to assets and pkg/assets -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 a4b9f77c5] update manifests - 11 files changed, 49 insertions(+), 47 deletions(-) -Rebasing Makefile -## Committing changes to buildfiles -[rebase-release-4.21-4.21.0-0.nightly-2025-10-22-123727_amd64-2025-10-22_arm64-2025-10-22 bbc290cbe] update buildfiles - 2 files changed, 2 insertions(+), 2 deletions(-) -# Removing staging directory -================================================== -rebase_ai_model_serving.sh: -Rebasing AI Model Serving for MicroShift to registry.redhat.io/rhoai/odh-operator-bundle:v2.22 -Fetching RHOAI CSV -Fetching RHOAI manifests -Updating last_rebase_ai_model_serving.sh -Updating rebase_job_entrypoint.sh -Copying manifests from staging dir to assets/ -2025-10-23 04:18:23,086 INFO Not clearing dir optional/ai-model-serving/ -2025-10-23 04:18:23,086 WARNING Ignoring optional/ai-model-serving/release-ai-model-serving-x86_64.json because Release info file -2025-10-23 04:18:23,086 INFO Not clearing dir optional/ai-model-serving/kserve/ -2025-10-23 04:18:23,086 WARNING Ignoring optional/ai-model-serving/kserve/kustomization.yaml because MicroShift specific overrides -2025-10-23 04:18:23,086 WARNING Ignoring optional/ai-model-serving/kserve/inferenceservice-config-microshift-patch.yaml because MicroShift specific overrides -2025-10-23 04:18:23,086 WARNING Ignoring optional/ai-model-serving/kserve/namespace.yaml because MicroShift specific overrides -2025-10-23 04:18:23,086 INFO Clearing directory assets/optional/ai-model-serving/kserve/overlays/odh/ -2025-10-23 04:18:23,087 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/inferenceservice-config-patch.yaml <- _output/staging/rhoai/operator/kserve/overlays/odh/inferenceservice-config-patch.yaml -2025-10-23 04:18:23,087 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/kustomization.yaml <- _output/staging/rhoai/operator/kserve/overlays/odh/kustomization.yaml -2025-10-23 04:18:23,087 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/params.env <- _output/staging/rhoai/operator/kserve/overlays/odh/params.env -2025-10-23 04:18:23,087 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/params.yaml <- _output/staging/rhoai/operator/kserve/overlays/odh/params.yaml -2025-10-23 04:18:23,087 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/remove-namespace.yaml <- _output/staging/rhoai/operator/kserve/overlays/odh/remove-namespace.yaml -2025-10-23 04:18:23,088 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/set-resources-manager-patch.yaml <- _output/staging/rhoai/operator/kserve/overlays/odh/set-resources-manager-patch.yaml -2025-10-23 04:18:23,088 DEBUG Copying assets/optional/ai-model-serving/kserve/overlays/odh/user-cluster-roles.yaml <- _output/staging/rhoai/operator/kserve/overlays/odh/user-cluster-roles.yaml -2025-10-23 04:18:23,088 INFO Clearing directory assets/optional/ai-model-serving/kserve/default/ -2025-10-23 04:18:23,089 DEBUG Copying assets/optional/ai-model-serving/kserve/default/cainjection_conversion_webhook.yaml <- _output/staging/rhoai/operator/kserve/default/cainjection_conversion_webhook.yaml -2025-10-23 04:18:23,089 DEBUG Copying assets/optional/ai-model-serving/kserve/default/clusterservingruntime_validatingwebhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/clusterservingruntime_validatingwebhook_cainjection_patch.yaml -2025-10-23 04:18:23,089 DEBUG Copying assets/optional/ai-model-serving/kserve/default/inferencegraph_validatingwebhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/inferencegraph_validatingwebhook_cainjection_patch.yaml -2025-10-23 04:18:23,089 DEBUG Copying assets/optional/ai-model-serving/kserve/default/isvc_mutatingwebhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/isvc_mutatingwebhook_cainjection_patch.yaml -2025-10-23 04:18:23,089 DEBUG Copying assets/optional/ai-model-serving/kserve/default/isvc_validatingwebhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/isvc_validatingwebhook_cainjection_patch.yaml -2025-10-23 04:18:23,090 DEBUG Copying assets/optional/ai-model-serving/kserve/default/kustomization.yaml <- _output/staging/rhoai/operator/kserve/default/kustomization.yaml -2025-10-23 04:18:23,090 DEBUG Copying assets/optional/ai-model-serving/kserve/default/localmodel_manager_image_patch.yaml <- _output/staging/rhoai/operator/kserve/default/localmodel_manager_image_patch.yaml -2025-10-23 04:18:23,090 DEBUG Copying assets/optional/ai-model-serving/kserve/default/manager_auth_proxy_patch.yaml <- _output/staging/rhoai/operator/kserve/default/manager_auth_proxy_patch.yaml -2025-10-23 04:18:23,090 DEBUG Copying assets/optional/ai-model-serving/kserve/default/manager_image_patch.yaml <- _output/staging/rhoai/operator/kserve/default/manager_image_patch.yaml -2025-10-23 04:18:23,090 DEBUG Copying assets/optional/ai-model-serving/kserve/default/manager_prometheus_metrics_patch.yaml <- _output/staging/rhoai/operator/kserve/default/manager_prometheus_metrics_patch.yaml -2025-10-23 04:18:23,091 DEBUG Copying assets/optional/ai-model-serving/kserve/default/manager_resources_patch.yaml <- _output/staging/rhoai/operator/kserve/default/manager_resources_patch.yaml -2025-10-23 04:18:23,091 DEBUG Copying assets/optional/ai-model-serving/kserve/default/network-policies.yaml <- _output/staging/rhoai/operator/kserve/default/network-policies.yaml -2025-10-23 04:18:23,091 DEBUG Copying assets/optional/ai-model-serving/kserve/default/servingruntime_validationwebhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/servingruntime_validationwebhook_cainjection_patch.yaml -2025-10-23 04:18:23,091 DEBUG Copying assets/optional/ai-model-serving/kserve/default/svc_webhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/svc_webhook_cainjection_patch.yaml -2025-10-23 04:18:23,091 DEBUG Copying assets/optional/ai-model-serving/kserve/default/trainedmodel_validatingwebhook_cainjection_patch.yaml <- _output/staging/rhoai/operator/kserve/default/trainedmodel_validatingwebhook_cainjection_patch.yaml -2025-10-23 04:18:23,091 INFO Clearing directory assets/optional/ai-model-serving/kserve/crd/ -2025-10-23 04:18:23,092 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/kustomization.yaml <- _output/staging/rhoai/operator/kserve/crd/kustomization.yaml -2025-10-23 04:18:23,093 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_clusterlocalmodels.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_clusterlocalmodels.yaml -2025-10-23 04:18:23,093 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_clusterservingruntimes.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_clusterservingruntimes.yaml -2025-10-23 04:18:23,093 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_clusterstoragecontainers.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_clusterstoragecontainers.yaml -2025-10-23 04:18:23,093 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_inferencegraphs.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_inferencegraphs.yaml -2025-10-23 04:18:23,093 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_inferenceservices.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_inferenceservices.yaml -2025-10-23 04:18:23,094 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_localmodelnodegroups.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_localmodelnodegroups.yaml -2025-10-23 04:18:23,094 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_servingruntimes.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_servingruntimes.yaml -2025-10-23 04:18:23,095 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/full/serving.kserve.io_trainedmodels.yaml <- _output/staging/rhoai/operator/kserve/crd/full/serving.kserve.io_trainedmodels.yaml -2025-10-23 04:18:23,095 DEBUG Copying assets/optional/ai-model-serving/kserve/crd/patches/protocol.yaml <- _output/staging/rhoai/operator/kserve/crd/patches/protocol.yaml -2025-10-23 04:18:23,095 INFO Clearing directory assets/optional/ai-model-serving/kserve/configmap/ -2025-10-23 04:18:23,095 DEBUG Copying assets/optional/ai-model-serving/kserve/configmap/kustomization.yaml <- _output/staging/rhoai/operator/kserve/configmap/kustomization.yaml -2025-10-23 04:18:23,096 DEBUG Copying assets/optional/ai-model-serving/kserve/configmap/inferenceservice.yaml <- _output/staging/rhoai/operator/kserve/configmap/inferenceservice.yaml -2025-10-23 04:18:23,096 INFO Clearing directory assets/optional/ai-model-serving/kserve/rbac/ -2025-10-23 04:18:23,097 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/auth_proxy_role_binding.yaml <- _output/staging/rhoai/operator/kserve/rbac/auth_proxy_role_binding.yaml -2025-10-23 04:18:23,097 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/auth_proxy_role.yaml <- _output/staging/rhoai/operator/kserve/rbac/auth_proxy_role.yaml -2025-10-23 04:18:23,097 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/auth_proxy_service.yaml <- _output/staging/rhoai/operator/kserve/rbac/auth_proxy_service.yaml -2025-10-23 04:18:23,097 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/kustomization.yaml <- _output/staging/rhoai/operator/kserve/rbac/kustomization.yaml -2025-10-23 04:18:23,097 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/leader_election_role_binding.yaml <- _output/staging/rhoai/operator/kserve/rbac/leader_election_role_binding.yaml -2025-10-23 04:18:23,098 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/leader_election_role.yaml <- _output/staging/rhoai/operator/kserve/rbac/leader_election_role.yaml -2025-10-23 04:18:23,098 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/role_binding.yaml <- _output/staging/rhoai/operator/kserve/rbac/role_binding.yaml -2025-10-23 04:18:23,098 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/role.yaml <- _output/staging/rhoai/operator/kserve/rbac/role.yaml -2025-10-23 04:18:23,098 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/service_account.yaml <- _output/staging/rhoai/operator/kserve/rbac/service_account.yaml -2025-10-23 04:18:23,098 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/localmodel/role_binding.yaml <- _output/staging/rhoai/operator/kserve/rbac/localmodel/role_binding.yaml -2025-10-23 04:18:23,099 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/localmodel/role.yaml <- _output/staging/rhoai/operator/kserve/rbac/localmodel/role.yaml -2025-10-23 04:18:23,099 DEBUG Copying assets/optional/ai-model-serving/kserve/rbac/localmodel/service_account.yaml <- _output/staging/rhoai/operator/kserve/rbac/localmodel/service_account.yaml -2025-10-23 04:18:23,099 INFO Clearing directory assets/optional/ai-model-serving/kserve/manager/ -2025-10-23 04:18:23,099 DEBUG Copying assets/optional/ai-model-serving/kserve/manager/kustomization.yaml <- _output/staging/rhoai/operator/kserve/manager/kustomization.yaml -2025-10-23 04:18:23,100 DEBUG Copying assets/optional/ai-model-serving/kserve/manager/manager.yaml <- _output/staging/rhoai/operator/kserve/manager/manager.yaml -2025-10-23 04:18:23,100 DEBUG Copying assets/optional/ai-model-serving/kserve/manager/service.yaml <- _output/staging/rhoai/operator/kserve/manager/service.yaml -2025-10-23 04:18:23,100 INFO Clearing directory assets/optional/ai-model-serving/kserve/webhook/ -2025-10-23 04:18:23,100 DEBUG Copying assets/optional/ai-model-serving/kserve/webhook/kustomization.yaml <- _output/staging/rhoai/operator/kserve/webhook/kustomization.yaml -2025-10-23 04:18:23,100 DEBUG Copying assets/optional/ai-model-serving/kserve/webhook/manifests.yaml <- _output/staging/rhoai/operator/kserve/webhook/manifests.yaml -2025-10-23 04:18:23,101 DEBUG Copying assets/optional/ai-model-serving/kserve/webhook/service.yaml <- _output/staging/rhoai/operator/kserve/webhook/service.yaml -2025-10-23 04:18:23,101 INFO Clearing directory assets/optional/ai-model-serving/runtimes/ -2025-10-23 04:18:23,101 INFO Restoring assets/optional/ai-model-serving/runtimes/kustomization.yaml -2025-10-23 04:18:23,101 DEBUG Executing 'git restore assets/optional/ai-model-serving/runtimes/kustomization.yaml' -2025-10-23 04:18:23,140 WARNING Ignoring optional/ai-model-serving/runtimes/kustomization.x86_64.yaml because Generated during rebase -2025-10-23 04:18:23,140 DEBUG Copying assets/optional/ai-model-serving/runtimes/caikit-standalone.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/caikit-standalone-template.yaml -2025-10-23 04:18:23,141 DEBUG Copying assets/optional/ai-model-serving/runtimes/caikit-tgis.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/caikit-tgis-template.yaml -2025-10-23 04:18:23,141 DEBUG Copying assets/optional/ai-model-serving/runtimes/hf-detector.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/hf-detector-template.yaml -2025-10-23 04:18:23,141 DEBUG Copying assets/optional/ai-model-serving/runtimes/ovms-kserve.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/ovms-kserve-template.yaml -2025-10-23 04:18:23,141 DEBUG Copying assets/optional/ai-model-serving/runtimes/vllm-cuda.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/vllm-cuda-template.yaml -2025-10-23 04:18:23,141 DEBUG Copying assets/optional/ai-model-serving/runtimes/vllm-gaudi.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/vllm-gaudi-template.yaml -2025-10-23 04:18:23,141 DEBUG Copying assets/optional/ai-model-serving/runtimes/vllm-rocm.yaml <- _output/staging/rhoai/operator/modelcontroller/runtimes/vllm-rocm-template.yaml -Initializing release.json file -Dropping template containers from ServingRuntimes and changing them to ClusterServingRuntimes -Creating ClusterServingRuntimes images kustomization -No changes to assets/ or last_rebase_ai_model_serving.sh -Removing staging directory