tests: add UI test infrastructure for VM Wizard#7631
tests: add UI test infrastructure for VM Wizard#7631sdrabb wants to merge 3 commits intoutmapp:mainfrom
Conversation
|
Hi please confirm that you have read https://github.com/utmapp/UTM/blob/main/CONTRIBUTING.md and have followed the guidelines. Thanks! |
Add a UTMUITests target (com.apple.product-type.bundle.ui-testing) that depends on the macOS app target and wire it into the macOS scheme so that tests can be run both from Xcode and via: xcodebuild test -scheme macOS -destination 'platform=macOS' Only the minimum required project.pbxproj entries are added: PBXBuildFile, PBXContainerItemProxy, PBXFileReference, PBXFrameworks/Resources/SourcesBuildPhase, PBXGroup, PBXNativeTarget, PBXTargetDependency, XCBuildConfiguration (Debug + Release), and XCConfigurationList. No unrelated target settings are touched. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a test-utm job that runs UTMUITests against the macOS scheme
on every push/PR. The job depends on build-sysroot (same as
build-utm) so it can share the cached sysroot and run in parallel
with the other build jobs.
xcodebuild test -scheme macOS -destination 'platform=macOS' \
CODE_SIGNING_ALLOWED=NO
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
sdrabb
force-pushed
the
feature/ui-testing
branch
from
f685795 to
466f481
Compare
sdrabb
changed the title
|
Any chance to run this on the self hosted runner that we have? |
Removed CODE_SIGNING_ALLOWED=NO from test command.
|
I configure the one I have as self hosted for a run |
|
@copilot please review this PR for any errors. |
osy
left a comment
osy
left a comment
There was a problem hiding this comment.
I looked through all the changes and it's fine and is very similar to the code I have generated with my own AI agent. The reason why I didn't commit it is because the bulk of the work (99%) is actually defining the tests and implementing them. You can probably use AI to implement most of the tests (assuming you properly review it, etc) but the real work is defining all the test cases. That takes a lot of work in understanding the use cases and where the complexity lies and is not something that you can tell the AI to do without a lot of context and guidance (especially since by default these AI agents are not too good at debugging UI issues). What I want to see (in human written form) is a test plan that covers everything that is important across all the different platforms (not just macOS but it is okay to start with just macOS).
|
Ok then I'll do a proposal, thanks for the feedback |
3 participants
Summary
UTMUITeststarget wired into the macOS scheme so tests run viaXcode and
xcodebuild test -scheme macOS -destination 'platform=macOS'VMWizardViewTestwith a window close button smoke testTESTING.mdguide covering how to run and extend the test suitetest-utmCI job tobuild.ymlthat runs on every push/PR,parallel to
build-utm, sharing the same sysroot cache#2565
Testing
Tested on: Mac mini M4, macOS 26.3 (Tahoe beta)
xcodebuild test -scheme macOS -destination 'platform=macOS'passes locallyNotes
Code is AI-assisted. All commits carry
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>.I have read the AI Contribution Guidelines and can attest that I have followed each item to the best of my ability.