Redesign editor UI around mode-based workflows and contextual panels

[fernandotonon]

Problem Statement

QtMeshEditor has grown a dense, always-on editor UI. The current layout exposes many controls at the same time: object transforms, edit tools, viewport toggles, mesh info, normals, animation tools, material presets, validation, undo history, scene hierarchy, and utility docks all compete for attention.

This makes the editor feel bloated even when the user is doing a focused task such as positioning an object, editing mesh topology, previewing an animation, or validating an import. The impact is visible in several areas:

• The top toolbar mixes core editing commands with specialized actions such as material editing, animation merging, and viewport overlays.
• View toggles such as grid, normals, mesh info, and view cube are always present even though they are secondary display options.
• The right inspector stacks too many sections in one vertical flow, making users scan past unrelated controls.
• Edit mode, scene hierarchy, transforms, animation controls, material presets, validation, LOD controls, and undo history can all appear in the same panel.
• The left/top toolbars expose many icons without enough workflow separation.
• The dark theme and green accent are usable, but the spacing, grouping, and control density make the product feel less modern than comparable professional tools.

The result is higher cognitive load, slower discovery of task-specific tools, and a less professional first impression for a 3D editor.

Reference direction: the UI should move toward the cleaner mode-based layout shown in the provided mock/reference image, while keeping QtMeshEditor's dark theme and green accent as a recognizable identity.

Proposed Solution

Redesign the editor around progressive disclosure and mode-based workflows, similar in spirit to Blender, Unreal Editor, Godot, and Figma.

Introduce a visible mode switcher with these modes:

• Object Mode
• Edit Mode
• Animation Mode
• Material Mode
• Validation / Scan Mode

Each mode should control:

• Which toolbar actions are visible.
• Which right-inspector sections are shown by default.
• What appears in the bottom context panel.
• Which viewport overlays are enabled by default.

The redesign should preserve existing functionality and shortcuts, but move secondary controls into contextual panels, menus, or viewport overlay popovers.

High-level layout changes:

• Keep the top menu simple and stable.
• Add a clear mode bar below the menu.
• Convert the left toolbar into a contextual tool rail driven by the active mode.
• Make the right panel tabbed and collapsible instead of one long mixed inspector.
• Use a bottom panel for contextual information such as mesh stats, animation timeline, validation results, logs, or scan reports.
• Move common viewport display toggles into a viewport overlay menu or View menu instead of always-visible top toolbar buttons.

Detailed Layout Proposal

Top Menu

Keep the top menu focused on global commands:

• File: open, import, save, export, recent files, resource locations.
• Edit: undo, redo, duplicate, group, ungroup, preferences.
• View: viewport layout, grid, normals, mesh info, view cube, asset browser, overlays, palette/theme options.
• Tools or Editor: mode-specific utilities that do not belong in the active tool rail.
• Help: about, update check, keyboard shortcuts, crash reporting.

The top menu should remain text-first and predictable. It should not be the main surface for frequent per-mode editing.

Mode Bar

Add a horizontal mode switcher directly below the menu bar:

• Object
• Edit
• Animation
• Material
• Validation / Scan

Behavior:

• The active mode is visually prominent with the existing green accent.
• Switching modes updates left toolbar contents, right panel tabs, bottom panel content, and default viewport overlays.
• Existing Tab behavior for Object/Edit mode should be preserved.
• Shortcuts should continue to work even if the corresponding command is no longer always visible.
• Mode state should be represented in a single controller/model rather than scattered across toolbar visibility logic.

Left Toolbar

Replace the current always-visible tool set with a contextual tool rail.

Examples:

• Object Mode: select, move, rotate, scale, duplicate, group, delete, add primitive.
• Edit Mode: vertex/edge/face selection, knife, extrude, bevel, merge, delete/dissolve, subdivide, fill, vertex paint.
• Animation Mode: select animation, play/pause, loop toggle, skeleton display, weights display, merge animations, simplify, export pose.
• Material Mode: select material, assign material, eyedropper/pick material, apply preset, texture slot tools.
• Validation / Scan Mode: run validation, scan folder, export report, apply safe fixes.

The toolbar should show only tools that are meaningful in the current mode. Disabled states should be reserved for temporarily unavailable actions inside that mode, not for unrelated tools from other modes.

Viewport

Keep the viewport visually central and reduce chrome around it.

Viewport controls should include:

• A compact viewport header with camera/view mode and shading mode.
• A viewport overlay menu for grid, normals, mesh info, view cube, wireframe, skeleton, weights, and statistics.
• Optional small quick actions for the most common view toggles, grouped in one overlay popover rather than spread across the global toolbar.
• A clear mode/status indicator when the current mode changes the viewport interaction model.

Default overlays by mode:

• Object Mode: grid on, view cube on, mesh info off by default.
• Edit Mode: grid on, edit wire/selection overlay on, normals off by default unless enabled by the user.
• Animation Mode: timeline controls visible, optional skeleton/weights overlays available.
• Material Mode: material preview/shading controls emphasized.
• Validation / Scan Mode: issue markers and report overlays available only when validation results exist.

Right Inspector

The right panel should become tabbed/collapsible and context-aware. It should no longer show every possible section at once.

Proposed top-level tabs:

• Inspector
• Scene
• Mode Tools
• History

Alternative if simpler in Qt/QML:

• Keep a single QML inspector shell, but filter visible sections based on the active editor mode and add a compact tab row at the top.

Suggested behavior:

• Object Mode defaults to Transform, Scene, Material Summary, Object Stats.
• Edit Mode defaults to Selection, Soft Selection, Normals, Mesh Tools, Mesh Warnings.
• Animation Mode defaults to Animation List, Playback, Timeline, Simplify/Merge, Export Pose.
• Material Mode defaults to Material Editor, Texture Slots, Shader Parameters, Preview.
• Validation / Scan Mode defaults to Validation Results, Scan Rules, Findings, Export Report.
• Undo History should be available from a History tab or bottom panel, not always occupying inspector space.

Bottom Panel

Use the bottom panel for contextual, high-density information that benefits from horizontal space:

• Object Mode: object stats, selection summary, asset metadata, import/export info.
• Edit Mode: mesh stats, selected component counts, warnings, operation feedback.
• Animation Mode: timeline, animation clips, playback controls, current pose/export controls.
• Material Mode: material preview strip, texture slot details, shader parameter summaries.
• Validation / Scan Mode: validation findings table, scan progress, rule results, exportable report preview.

The panel should support collapse/expand and remember the user's preferred height.

Status Bar

Keep the status bar minimal and useful:

• Active mode.
• Selection summary.
• Transform space or snap status.
• Background process state, such as scan running, import/export progress, autosave, or ready state.

Do not duplicate large panels of information in the status bar.

Mode-Specific Behavior

Object Mode

Visible tools:

• Select
• Move
• Rotate
• Scale
• Duplicate
• Group / Ungroup
• Delete
• Add primitive
• Import/export selected, where appropriate

Right panel:

• Scene hierarchy
• Transform
• Material summary
• Object stats
• Primitive parameters when a primitive is selected

Bottom panel:

• Object stats
• File/import metadata
• Selection summary
• Mesh info summary

Viewport overlays:

• Grid on by default
• View cube on by default
• Mesh info off by default, available in overlay menu
• Normals off by default

Edit Mode

Visible tools:

• Vertex / Edge / Face selection
• Soft selection
• Move / Rotate / Scale for selected components
• Knife
• Extrude
• Bevel
• Merge
• Delete / Dissolve
• Subdivide
• Fill
• Vertex paint

Right panel:

• Component selection mode
• Soft selection settings
• Normals tools
• Mesh editing parameters
• Mesh warnings
• Active tool settings, such as bevel profile or paint brush settings

Bottom panel:

• Vertex / edge / face counts
• Selected component counts
• Mesh warnings and operation feedback
• Degenerate triangle warnings and fix actions

Viewport overlays:

• Grid on by default
• Edit selection overlay on by default
• Wire/mesh overlay on by default where helpful
• Normals available but off by default
• Mesh info available in overlay menu

Animation Mode

Visible tools:

• Animation list selection
• Play / pause
• Loop toggle
• Timeline scrubber
• Skeleton display
• Bone weights display
• Merge animations
• Simplify animation
• Export pose

Right panel:

• Animation list grouped by entity
• Clip properties
• Loop/enabled state
• Simplify tolerance and redundant keyframe analysis
• Skeleton / weights toggles

Bottom panel:

• Timeline
• Playback controls
• Clip duration and keyframe summary
• Current pose/export feedback

Viewport overlays:

• Skeleton overlay available and optionally enabled when an animated entity is selected
• Bone weights available but off by default
• Grid optional
• View cube available

Material Mode

Visible tools:

• Select material
• Assign material
• Pick material from mesh
• Apply material preset
• Texture slot actions
• Preview toggle

Right panel:

• Material editor
• Texture slots
• Shader parameters
• Material presets
• Live preview

Bottom panel:

• Material list
• Texture slot details
• Preview strip
• Warnings for missing textures/materials

Viewport overlays:

• Material/shaded view emphasized
• Grid optional
• Mesh info off by default
• Normals off by default

Validation / Scan Mode

Visible tools:

• Run mesh validation
• Run asset scan
• Scan selected asset/folder
• Apply safe fixes
• Export report
• Open report location

Right panel:

• Mesh validation results
• Missing materials
• Empty meshes
• Vertex count warnings
• Animation naming rules
• Redundant keyframes
• Rule configuration and scan scope

Bottom panel:

• Findings table
• Scan progress
• Report preview
• Exportable scan report controls
• Filters by severity/type/file

Viewport overlays:

• Issue markers available when possible
• Mesh warnings overlay available
• Grid optional
• Non-validation overlays off by default

Validation / Scan checks should include:

• Missing materials
• Empty meshes
• Vertex count warnings
• Animation naming rules
• Redundant keyframes
• Exportable scan report

Acceptance Criteria

• UI has a visible mode switcher with Object, Edit, Animation, Material, and Validation / Scan modes.
• Switching modes changes the visible left toolbar actions.
• Switching modes changes the primary right inspector sections.
• Switching modes changes the bottom context panel content.
• Top toolbar has fewer always-visible actions than the current UI.
• Grid, normals, mesh info, and view cube are no longer separate always-visible toolbar buttons.
• View options are accessible through the View menu and/or a viewport overlay menu.
• Right panel no longer shows scene, transform, edit tools, animation, material presets, validation, LOD, and undo history all in one always-scannable stack.
• Each mode exposes contextual tools matching its workflow.
• Existing functionality remains accessible through mode tools, inspector tabs, menus, shortcuts, or overlay menus.
• Existing shortcuts are preserved where possible, including Object/Edit toggle, transform shortcuts, undo/redo, duplicate, grouping, and edit-mode component shortcuts.
• Loading/importing meshes still works.
• Object transform editing still works.
• Mesh edit mode still works, including vertex/edge/face selection and topology actions.
• Animation playback still works.
• Animation merge, simplify, and export pose remain accessible.
• Material editing and material presets remain accessible.
• Mesh validation remains accessible.
• Validation / Scan mode can expose or link to an exportable scan report.
• No regression in loading, editing, animation playback, or material editing.
• QA screenshots are captured for each mode in the final implementation.

Implementation Plan

Phase 1: UI Inventory and Grouping

• Inventory current menu actions, toolbar actions, inspector sections, docks, overlays, and shortcuts.
• Group each existing action into Object, Edit, Animation, Material, Validation / Scan, View, or Global.
• Identify commands that should remain globally accessible through menus or shortcuts.
• Identify commands that should move into mode-specific toolbars or panels.
• Document any current behavior that must not change.

Initial code areas to inspect before exact file/class changes:

• ui_files/mainwindow.ui for menus, toolbars, actions, and dock layout.
• src/mainwindow.cpp / src/mainwindow.h for action wiring, dock creation, viewport overlays, and shortcuts.
• qml/PropertiesPanel.qml for inspector sections and collapsible panel structure.
• src/PropertiesPanelController.* for inspector data, settings, animation controls, undo history, and scene model access.
• src/EditModeController.* for Object/Edit behavior and edit-mode tools.
• src/AnimationControlController.*, src/AnimationWidget.*, and src/AnimationMerger.* for animation workflows.
• src/MaterialEditorQML.*, qml/MaterialEditorWindow.qml, and src/MaterialPresetLibrary.* for material workflows.
• src/MeshValidator.*, src/ScanEngine.*, and src/ScanConfig.* for validation/scan workflows.
• src/ViewportGrid.*, src/NormalVisualizer.*, src/MeshInfoOverlay.*, and src/ViewCube/* for viewport overlay behavior.

Phase 2: Create Mode Switcher

• Add an editor mode enum/controller that represents Object, Edit, Animation, Material, and Validation / Scan modes.
• Preserve existing EditModeController behavior for Object/Edit mesh editing.
• Add a visible mode bar under the menu bar.
• Wire mode changes to toolbar visibility, inspector filtering, bottom panel content, and overlay defaults.
• Preserve Tab for Object/Edit toggling.

Phase 3: Refactor Right Panel Into Tabs/Collapsible Groups

• Convert the current single stacked QML inspector into mode-aware groups.
• Add top-level tabs or a segmented control for Inspector, Scene, Mode Tools, and History.
• Show only mode-relevant groups by default.
• Keep non-current sections accessible without forcing them into the default scroll path.
• Ensure section expansion state is remembered per mode where practical.

Phase 4: Move View Toggles Into View Menu / Overlay

• Remove grid, normals, mesh info, and view cube from the always-visible toolbar surface.
• Keep those actions in the View menu.
• Add a compact viewport overlay menu for common view toggles.
• Preserve action checked states and existing settings behavior.
• Ensure overlay visibility stays synchronized with menu checkmarks and controller state.

Phase 5: Add Bottom Context Panel

• Create a bottom context panel that changes content by mode.
• Move high-density information out of the right inspector where it benefits from horizontal space.
• Add collapse/expand behavior and sensible default height.
• Include mesh stats, timeline, validation findings, scan report, or material previews depending on mode.

Phase 6: Polish Styling and Spacing

• Tighten spacing, alignment, and typography across mode bar, toolbars, inspector, bottom panel, and status bar.
• Keep the dark theme and green accent, but reduce visual noise from borders, nested panels, and repeated headers.
• Standardize icon sizing, button states, hover states, and disabled states.
• Keep the viewport as the visual center of the application.
• Review responsive behavior at common desktop window sizes.

Phase 7: QA and Screenshots

• Capture screenshots for Object Mode.
• Capture screenshots for Edit Mode.
• Capture screenshots for Animation Mode.
• Capture screenshots for Material Mode.
• Capture screenshots for Validation / Scan Mode.
• Verify keyboard shortcuts still work.
• Verify importing/loading meshes still works.
• Verify object transform editing still works.
• Verify edit-mode mesh operations still work.
• Verify animation playback, merge, simplify, and export pose still work.
• Verify material editing and material presets still work.
• Verify validation and scan report workflows still work.

Technical Considerations

• QtMeshEditor is a Qt/C++ desktop application with QML-based panels embedded in the main Qt widget shell.
• Codex should inspect the existing Qt UI code before proposing exact class/file changes for implementation.
• Prefer incremental changes over a full rewrite.
• Preserve current shortcuts and menu actions where possible.
• Avoid breaking current workflows while moving controls into mode-specific surfaces.
• The existing QML PropertiesPanel.qml already uses collapsible sections; this can be evolved into mode-aware sections rather than replaced wholesale.
• The existing EditModeController already owns Object/Edit mode behavior and should either remain the source of truth for mesh edit state or be cleanly integrated into a broader editor mode controller.
• Existing viewport overlay controllers for grid, normals, mesh info, and view cube should be reused and re-hosted behind a View menu / viewport overlay menu.
• ScanEngine appears to support exportable text, JSON, and SARIF reports; Validation / Scan mode should reuse that reporting path rather than creating a separate reporting system.
• Current QQuickWidget software rendering constraints should be preserved to avoid conflicts with Ogre rendering.
• Avoid a single large PR if possible; this should be delivered as a sequence of small UI infrastructure and mode-specific follow-up PRs.

Deliverables

• Final GitHub issue body in Markdown.
• Task checklist for phased implementation.
• Proposed labels: enhancement, ui, ux, editor, qtmesh-roadmap.
• Suggested title: Redesign editor UI around mode-based workflows and contextual panels.
• Optional milestone: QtMeshEditor UI Modernization.
• Mode screenshots attached to implementation PRs.
• QA notes confirming no regression in core workflows.

[fernandotonon]

Preview Example:

[fernandotonon]

Incremental UI Modernization Slices

Tracking the mode-based editor redesign as reviewable release slices so each PR can stay deployable and testable.

• Slice 1: Mode Infrastructure + initial editor-mode surfaces — PR Modernize editor mode surfaces #432

  • Adds EditorModeController and integrates Object/Edit state with the existing EditModeController.
  • Preserves existing shortcuts/menu behavior, including Object/Edit toggle behavior.
  • Adds the first shippable mode UI surfaces, target clarity for Node vs Mesh Geometry transforms, bottom tool grouping fixes, viewport defaults, version bump, and focused tests.

• Slice 2: Mode Bar refinement

  • Make the mode bar the primary visible workflow switcher.
  • Keep current tools accessible through menus/shortcuts while mode selection updates mode state and status text.
  • Add focused tests for mode selection and status propagation.

• Slice 3: Inspector tabs and section filtering

  • Keep Inspector, Scene, Mode Tools, and History reachable.
  • Filter default section visibility by mode without removing existing functionality.
  • Add coverage that previous sections remain accessible.

• Slice 4: Contextual tool rail

  • Convert the left toolbar into mode-filtered tools using existing QActions/controllers.
  • Keep hidden actions available through menus and shortcuts.
  • Add tests for toolbar visibility by mode and shortcut preservation.

• Slice 5: Viewport overlay/menu consolidation

  • Move grid, normals, mesh info, and view cube out of always-visible toolbar chrome.
  • Keep them synchronized through the View menu and compact viewport overlay controls.
  • Add tests for checked-state sync and menu/overlay behavior.

• Slice 6: Bottom context panel completion + QA screenshots

  • Extend the bottom panel with lightweight mode summaries and existing controls only.
  • Keep bottom tools grouped/tabbed when multiple panels are visible.
  • Capture QA screenshots for Object, Edit, Animation, Material, and Validation modes.

Validation expectations for each slice:

• Build QtMeshEditor.
• Build/run focused UnitTests for changed paths.
• Run qmllint on changed QML.
• Smoke launch when a display is available.
• Verify import/load, object transform, edit-mode toggle, animation playback, material editor access, and validation access.

[fernandotonon]

Slice 2 is now in review: PR #433.

Scope:

• Mode-bar metadata now lives in EditorModeController as a testable availableModes model.
• ModeBar.qml renders controller-owned metadata instead of duplicating labels/tooltips.
• Added focused coverage for mode metadata and main-window status indicator updates when switching modes.

Validation:

• qmllint qml/ModeBar.qml
• cmake --build build_local --target UnitTests -j"$(nproc)"
• env QT_QPA_PLATFORM=offscreen build_local/bin/UnitTests --gtest_filter=EditorModeControllerTest.*:MainWindowTest.ModeBarLoadsAndSelectionUpdatesStatusIndicator
• cmake --build build_local --target QtMeshEditor -j"$(nproc)"

[fernandotonon]

Slice 3 is now in review: PR #435.

Scope:

• Inspector tab/filter policy is now explicit through EditorModeController APIs.
• Mode Tools defaults to the current editor mode while the new All filter keeps non-current sections reachable.
• PropertiesPanel.qml now uses controller tab constants and shared section visibility helpers instead of hard-coded tab/mode checks.
• Added focused tests for tab defaults, Scene/History preservation, and all-mode reachability.

Validation:

• qmllint qml/PropertiesPanel.qml
• cmake --build build_local --target UnitTests -j"$(nproc)"
• env QT_QPA_PLATFORM=offscreen build_local/bin/UnitTests --gtest_filter=EditorModeControllerTest.*:MainWindowTest.ModeBarLoadsAndModeChangeUpdatesStatusIndicator
• cmake --build build_local --target QtMeshEditor -j"$(nproc)"

[fernandotonon]

Slice 4 (Contextual Tool Rail) is now in review: #437

Implemented:

• Added mode-scoped left rail actions for Animation, Material, and Validation modes.
• Reused the existing Dope Sheet, Curve Editor, Merge Animations, Material Editor, and MeshValidator behavior instead of introducing new behavior paths.
• Kept shared menu actions reachable while only hiding their top-toolbar widgets by mode.
• Added tests for mode-filtered rail visibility, shared menu action reachability, and the animation rail button opening the bottom Dope Sheet dock.

Validation:

• cmake --build build_local --target UnitTests -j"$(nproc)"
• env QT_QPA_PLATFORM=offscreen build_local/bin/UnitTests --gtest_filter=MainWindowTest.ContextualToolRail*:MainWindowTest.ModeBarLoadsAndModeChangeUpdatesStatusIndicator
• cmake --build build_local --target QtMeshEditor -j"$(nproc)"

[fernandotonon]

Slice 4 update: PR #437 is rebased on current master, mergeable, and all GitHub checks are green.

PR: #437