Compare meta and metrics individually instead of the full JSON payload#493
Merged
Conversation
spencertransier
approved these changes
Jun 15, 2023
Contributor
spencertransier
left a comment
There was a problem hiding this comment.
I thought about changing the fastlane action to sort the meta and metrics array, to make the JSON payload deterministic. But I feel like it's not ideal to add an unnecessary functionality to the library for the sole purpose of fixing unit tests. After all, it's the unit tests that made an incorrect assumption that the elements in the metric array should be in certain order. But from the API point of view, the order doesn't really matter.
I agree with that logic 👍
You can take one step further to merge this commit to the use-ruby-3.2.2 branch locally, and bundle exec rspec ./spec/android_send_app_size_metrics_spec.rb should pass too.
All the android_send_app_size_metrics_spec.rb tests pass 👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does it do?
The app metrics JSON includes two array fields: meta and metrics. Here is an example. In an unit test which checks the JSON payload generated by an app metrics fastlane action, it compares the full JSON payload with fixtures. This JSON string comparison breaks on Ruby 3.2.2.
When running with Ruby 3.2.2, the generated metrics array contains all the elements that are expected, but in an different order. The root cause is this Dir.glob call returns files in a different order, which then causes the
metricsarray elements in the JSON has a different order than expected.I thought about changing the fastlane action to sort the
metaandmetricsarray, to make the JSON payload deterministic. But I feel like it's not ideal to add an unnecessary functionality to the library for the sole purpose of fixing unit tests. After all, it's the unit tests that made an incorrect assumption that the elements in the metric array should be in certain order. But from the API point of view, the order doesn't really matter.This PR updates related unit test assertions to compare individual
metaandmetricsproperties asSets, to ignore the elements order. They assertions should pass as long as what the expected meta and metrics are there, regardless of their places in the JSON array.Test Instructions
All is good if CI passes. You can take one step further to merge this commit to the
use-ruby-3.2.2branch locally, andbundle exec rspec ./spec/android_send_app_size_metrics_spec.rbshould pass too.Checklist before requesting a review
bundle exec rubocopto test for code style violations and recommendationsspecs/*_spec.rb) if applicablebundle exec rspecto run the whole test suite and ensure all your tests passCHANGELOG.mdfile to describe your changes under the appropriate existing###subsection of the existing## Trunksection.MIGRATION.mdfile to describe how the changes will affect the migration from the previous major version and what the clients will need to change and consider.