Skip to content

Compare meta and metrics individually instead of the full JSON payload#493

Merged
crazytonyli merged 1 commit into
trunkfrom
compare-json-payload
Jun 15, 2023
Merged

Compare meta and metrics individually instead of the full JSON payload#493
crazytonyli merged 1 commit into
trunkfrom
compare-json-payload

Conversation

@crazytonyli
Copy link
Copy Markdown
Contributor

@crazytonyli crazytonyli commented Jun 13, 2023

What does it do?

The app metrics JSON includes two array fields: meta and metrics. Here is an example. In an unit test which checks the JSON payload generated by an app metrics fastlane action, it compares the full JSON payload with fixtures. This JSON string comparison breaks on Ruby 3.2.2.

When running with Ruby 3.2.2, the generated metrics array contains all the elements that are expected, but in an different order. The root cause is this Dir.glob call returns files in a different order, which then causes the metrics array elements in the JSON has a different order than expected.

I thought about changing the fastlane action to sort the meta and metrics array, to make the JSON payload deterministic. But I feel like it's not ideal to add an unnecessary functionality to the library for the sole purpose of fixing unit tests. After all, it's the unit tests that made an incorrect assumption that the elements in the metric array should be in certain order. But from the API point of view, the order doesn't really matter.

This PR updates related unit test assertions to compare individual meta and metrics properties as Sets, to ignore the elements order. They assertions should pass as long as what the expected meta and metrics are there, regardless of their places in the JSON array.

Test Instructions

All is good if CI passes. You can take one step further to merge this commit to the use-ruby-3.2.2 branch locally, and bundle exec rspec ./spec/android_send_app_size_metrics_spec.rb should pass too.

Checklist before requesting a review

  • Run bundle exec rubocop to test for code style violations and recommendations
  • Add Unit Tests (aka specs/*_spec.rb) if applicable
  • Run bundle exec rspec to run the whole test suite and ensure all your tests pass
  • Make sure you added an entry in the CHANGELOG.md file to describe your changes under the appropriate existing ### subsection of the existing ## Trunk section.
  • If applicable, add an entry in the MIGRATION.md file to describe how the changes will affect the migration from the previous major version and what the clients will need to change and consider.

@crazytonyli crazytonyli requested a review from a team June 13, 2023 08:15
Copy link
Copy Markdown
Contributor

@spencertransier spencertransier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about changing the fastlane action to sort the meta and metrics array, to make the JSON payload deterministic. But I feel like it's not ideal to add an unnecessary functionality to the library for the sole purpose of fixing unit tests. After all, it's the unit tests that made an incorrect assumption that the elements in the metric array should be in certain order. But from the API point of view, the order doesn't really matter.

I agree with that logic 👍

You can take one step further to merge this commit to the use-ruby-3.2.2 branch locally, and bundle exec rspec ./spec/android_send_app_size_metrics_spec.rb should pass too.

All the android_send_app_size_metrics_spec.rb tests pass 👍

@crazytonyli crazytonyli merged commit 2a01842 into trunk Jun 15, 2023
@crazytonyli crazytonyli deleted the compare-json-payload branch June 15, 2023 22:12
@crazytonyli crazytonyli mentioned this pull request Jul 4, 2023
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants