You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the Pigeon testing is using unit tests on the Dart and native sides; this is useful, but leaves open the potential for unit tests on each side to pass, but for the protocol being unit tested to mis-align slightly such that the end-to-end call doesn't actually work. #111083 is an example of a case that should have been trivially caught by basic integration tests, but has to be more explicitly tested correctly with unit tests based on details of the serialization format on each end. (We also don't have that unit test on the Swift side, which is a problem, but a simple test would be even better.)
I would suggest that we rework the platform_tests directory so that instead of having a combination (depending on language) of bespoke tests and per-language dummy plugins, we just have one single test_plugin plugin that supports every platform/language (or I guess, two plugins: one for ObjC and Java, and one for everything else, since they can't really co-exist with their alternate languages in the same plugin). That plugin would have:
Unit tests for each language, using exactly the same structure that native unit tests in flutter/plugins use. That's already how Windows works, and the others could be converted.
integration_test tests that would drive full end-to-end calls for a variety of core use cases.
(Eventually I would like to revisit using the plugin repo tooling to drive all of this; I temporarily abandoned that when setting up Windows because it would involve more tool changes than I had time to make, but it would make things more sustainable long term that maintaining lots of run_tests.{dart,sh} logic. Using this structure would have the added benefit of setting ourselves up for that being easier to add later.)
These are mostly redundant with macOS, but there are features like background platform channels that we don't currently support on desktop. And the added cost of having both should be low. (I don't want to do iOS-only, because for local iteration a macOS integration test is way faster.)
Add Windows integration tests.
Add Java Android integration tests, local-only (not in CI)
Add Kotlin Android integration tests, local-only (not in CI)
Migrate the handful of (Obj-C-only) tests that are currently in the e2e_tests directory to the new structure, and eliminate e2e_tests.
Enable Java and Kotlin integration tests in CI
This is currently hard, which is why it's a separate step:
We could use FTL, like we do in flutter/plugins, but that's a lot of complexity/setup to bring into the tooling (or we need to adapt the repo tooling to work with Pigeon's unusual structure), and:
I'm hopeful that we can use the new (WIP?) LUCI Android emulator support instead, but that will require us to get that part of the testing moved over to LUCI, and emulator support wired in.
Currently the Pigeon testing is using unit tests on the Dart and native sides; this is useful, but leaves open the potential for unit tests on each side to pass, but for the protocol being unit tested to mis-align slightly such that the end-to-end call doesn't actually work. #111083 is an example of a case that should have been trivially caught by basic integration tests, but has to be more explicitly tested correctly with unit tests based on details of the serialization format on each end. (We also don't have that unit test on the Swift side, which is a problem, but a simple test would be even better.)
I would suggest that we rework the
platform_testsdirectory so that instead of having a combination (depending on language) of bespoke tests and per-language dummy plugins, we just have one singletest_pluginplugin that supports every platform/language (or I guess, two plugins: one for ObjC and Java, and one for everything else, since they can't really co-exist with their alternate languages in the same plugin). That plugin would have:integration_testtests that would drive full end-to-end calls for a variety of core use cases.(Eventually I would like to revisit using the plugin repo tooling to drive all of this; I temporarily abandoned that when setting up Windows because it would involve more tool changes than I had time to make, but it would make things more sustainable long term that maintaining lots of
run_tests.{dart,sh}logic. Using this structure would have the added benefit of setting ourselves up for that being easier to add later.)/cc @gaaclarke @tarrinneal
Checklist:
e2e_testsdirectory to the new structure, and eliminatee2e_tests.