Skip to content

Update screenshot test with new mocks - #15055

Merged
startuptester merged 9 commits into
developfrom
update/screenshot-mocks
Oct 7, 2020
Merged

Update screenshot test with new mocks#15055
startuptester merged 9 commits into
developfrom
update/screenshot-mocks

Conversation

@rachelmcr

@rachelmcr rachelmcr commented Oct 6, 2020

Copy link
Copy Markdown
Contributor

Updates the screenshot test with new mock data for new app store screenshots.

Notable changes:

Note: We're aiming to use the new screenshots with the 15.9 release, but I'm targeting develop because we don't expect any significant UI differences between the release/15.9 and develop branches.

Screenshots

Sample iPad screenshots:

Editor Reader My Site/Stats Notifications Editor with keyboard Media
iPad Pro (12 9-inch) (2nd generation)-1-Editor-With-BlockPicker iPad Pro (12 9-inch) (2nd generation)-2-Discover iPad Pro (12 9-inch) (2nd generation)-3-Stats iPad Pro (12 9-inch) (2nd generation)-5-Notifications iPad Pro (12 9-inch) (2nd generation)-6-Editor-With-Keyboard iPad Pro (12 9-inch) (2nd generation)-6-Media

Sample iPhone screenshots:

Editor Reader Stats My Site Notifications Media
iPhone Xs Max-1-Editor iPhone Xs Max-2-Discover iPhone Xs Max-3-Stats iPhone Xs Max-4-MySite iPhone Xs Max-5-Notifications iPhone Xs Max-6-Media

To test:

  • Confirm all the regular UI tests pass (WordPressUITests) on this PR.
  • Confirm the WordPressScreenshotGeneration tests produces the expected screenshots:
    • Run rake mocks to start the mock server.
    • Open the fastlane directory with cd Scripts/fastlane.
    • Generate a set of screenshots: bundle exec fastlane screenshots for a full set in all languages (this takes a while) or e.g. bundle exec fastlane screenshots language:"es-ES" for a set of screenshots in a single non-English language to confirm it works for all targeted devices regardless of language.

Before merge

  • Update the Podfile/Podfile.lock to use to the published WordPressAuthenticator and WordPressMocks pods (once those PRs are merged).

PR submission checklist:

  • I have considered adding unit tests where possible.
  • I have considered adding accessibility improvements for my changes.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

@peril-wordpress-mobile

peril-wordpress-mobile Bot commented Oct 6, 2020

Copy link
Copy Markdown

You can trigger an installable build for these changes by visiting CircleCI here.

@peril-wordpress-mobile

peril-wordpress-mobile Bot commented Oct 6, 2020

Copy link
Copy Markdown

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

@startuptester

Copy link
Copy Markdown
Contributor

@rachelmcr, the first thing I ran across was that the image from the Editor screenshot is missing
image

@startuptester

Copy link
Copy Markdown
Contributor
  • The UI tests pass
  • Script runs and screenshots are correct except for the missing image in the dog grooming page

@rachelmcr

Copy link
Copy Markdown
Contributor Author

the image from the Editor screenshot is missing

We currently don't mock the requests for media files, so likely the file wasn't loaded fast enough during your run.

I confirmed that it loaded and appeared in the screenshots when I generated them. Since it worked well on my end I thought we wouldn't need anything extra for that screenshot, but to avoid that flakiness I've added sleep(imagesWaitTime) before both of the editor screenshots to make sure the images have time to load.

I still need to resolve the Podfile.lock conflict, but I also need to update the two pod references that this PR depends on, so I'll plan to take care of that once those PRs are merged and the new pods are available.

@rachelmcr

Copy link
Copy Markdown
Contributor Author

@startuptester Ok, pods are updated, conflicts resolved, and this is ready for another review!

@startuptester startuptester left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran script again in Spanish after merging of the two associated PRs. Ran without error and screenshots look good!
💯

@startuptester
startuptester merged commit acf9fc8 into develop Oct 7, 2020
@startuptester
startuptester deleted the update/screenshot-mocks branch October 7, 2020 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants