Skip to content

Commit 5c58ebd

Browse files
adamfilipow92pawelpasterzPiotr Adamczykjan-goral
authored
Fix parametrized tests (#840)
*Add Espresso Parametrized tests using @RunWith(JUnitParamsRunner::class) and @RunWith(Parameterized::class) * Fix tests with annotation @RunWith(JUnitParamsRunner::class) * Add Parameterized test * Add espresso parameterized named test * Update androidx.test libraries Co-authored-by: Adam <adam.filipowicz92@gmail.com> Co-authored-by: Pawel Pasterz <pawel.pasterz@gmail.com> Co-authored-by: Piotr Adamczyk <piotr.adamczyk@gogoapps.io> Co-authored-by: Janek Góral <jan.goral@gogoapps.io>
1 parent 77f18db commit 5c58ebd

17 files changed

Lines changed: 349 additions & 152 deletions

‎release_notes.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
## next (unreleased)
22

3-
-
3+
- - [#840](https://github.com/Flank/flank/pull/840) Fix parametrized tests. ([jan-gogo](https://github.com/jan-gogo), [adamfilipow92](https://github.com/adamfilipow92), [pawelpasterz](https://github.com/pawelpasterz), [piotradamczyk5](https://github.com/piotradamczyk5))
44
-
55
-
66

‎test_app/app/build.gradle‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ android {
1313
targetSdkVersion 29
1414
versionCode 1
1515
versionName "1.0"
16-
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
16+
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
1717
testInstrumentationRunnerArguments(clearPackageData: 'true')
1818

1919
}
@@ -51,33 +51,33 @@ android {
5151

5252
testOptions {
5353
animationsDisabled = true
54-
execution = "ANDROIDX_TEST_ORCHESTRATOR"
54+
//execution = "ANDROIDX_TEST_ORCHESTRATOR"
5555
}
5656
}
5757

5858
dependencies {
5959
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
60-
implementation 'androidx.appcompat:appcompat:1.1.0-rc01'
61-
implementation 'androidx.core:core-ktx:1.1.0-alpha03'
60+
implementation 'androidx.appcompat:appcompat:1.3.0-alpha01'
61+
implementation 'androidx.core:core-ktx:1.4.0-alpha01'
6262
implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
6363

6464
// Espresso.
6565
// https://developer.android.com/jetpack/androidx/releases/test
6666
androidTestUtil 'androidx.test:orchestrator:1.2.0'
67-
68-
androidTestImplementation("androidx.test:runner:1.3.0-alpha02")
69-
androidTestImplementation("androidx.test.ext:junit:1.1.2-alpha02")
70-
androidTestImplementation("androidx.test.ext:junit-ktx:1.1.2-alpha02")
71-
androidTestImplementation("androidx.test.ext:truth:1.3.0-alpha02")
72-
androidTestImplementation("androidx.test.espresso.idling:idling-concurrent:3.3.0-alpha02")
73-
androidTestImplementation("androidx.test.espresso.idling:idling-net:3.3.0-alpha02")
74-
androidTestImplementation("androidx.test.espresso:espresso-accessibility:3.3.0-alpha02")
75-
androidTestImplementation("androidx.test:rules:1.3.0-alpha02")
76-
androidTestImplementation("androidx.test.espresso:espresso-core:3.3.0-alpha02")
77-
androidTestImplementation("androidx.test.espresso:espresso-contrib:3.3.0-alpha02")
78-
androidTestImplementation("androidx.test.espresso:espresso-idling-resource:3.3.0-alpha02")
79-
androidTestImplementation("androidx.test.espresso:espresso-intents:3.3.0-alpha02")
80-
androidTestImplementation("androidx.test.espresso:espresso-web:3.3.0-alpha02")
67+
androidTestImplementation 'pl.pragmatists:JUnitParams:1.1.1'
68+
androidTestImplementation("androidx.test:runner:1.3.0-rc01")
69+
androidTestImplementation("androidx.test.ext:junit:1.1.2-rc01")
70+
androidTestImplementation("androidx.test.ext:junit-ktx:1.1.2-rc01")
71+
androidTestImplementation("androidx.test.ext:truth:1.3.0-rc01")
72+
androidTestImplementation("androidx.test.espresso.idling:idling-concurrent:3.3.0-rc01")
73+
androidTestImplementation("androidx.test.espresso.idling:idling-net:3.3.0-rc01")
74+
androidTestImplementation("androidx.test.espresso:espresso-accessibility:3.3.0-rc01")
75+
androidTestImplementation("androidx.test:rules:1.3.0-rc01")
76+
androidTestImplementation("androidx.test.espresso:espresso-core:3.3.0-rc01")
77+
androidTestImplementation("androidx.test.espresso:espresso-contrib:3.3.0-rc01")
78+
androidTestImplementation("androidx.test.espresso:espresso-idling-resource:3.3.0-rc01")
79+
androidTestImplementation("androidx.test.espresso:espresso-intents:3.3.0-rc01")
80+
androidTestImplementation("androidx.test.espresso:espresso-web:3.3.0-rc01")
8181
}
8282

8383
if (coverageEnabled) {

‎test_app/app/src/androidTestMultiple/java/com.example.test_app/ParametrizedTest.kt‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
package com.example.test_app
2+
3+
import junit.framework.Assert.assertTrue
4+
import org.junit.Test
5+
import org.junit.runner.RunWith
6+
import org.junit.runners.Parameterized
7+
8+
@RunWith(Parameterized::class)
9+
class ParameterizedTest(private val paramOne: Int, private val paramTwo: String) {
10+
11+
companion object {
12+
@JvmStatic
13+
@Parameterized.Parameters
14+
fun data() = listOf(
15+
arrayOf(1, "1"),
16+
arrayOf(666, "666"),
17+
arrayOf(54321, "54321")
18+
)
19+
}
20+
21+
@Test
22+
fun shouldHopefullyPass() {
23+
assertTrue(paramOne == paramTwo.toInt())
24+
}
25+
}

‎test_app/app/src/androidTestMultiple/java/com/example/test_app/parametrized/EspressoParametrizedClassParameterizedNamed.kt‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
package com.example.test_app.parametrized
2+
3+
import androidx.test.espresso.Espresso
4+
import androidx.test.espresso.assertion.ViewAssertions
5+
import androidx.test.espresso.matcher.ViewMatchers
6+
import androidx.test.rule.ActivityTestRule
7+
import com.example.test_app.MainActivity
8+
import org.junit.Rule
9+
import org.junit.Test
10+
import org.junit.runner.RunWith
11+
import org.junit.runners.Parameterized
12+
13+
@RunWith(Parameterized::class)
14+
class EspressoParametrizedClassParameterizedNamed(
15+
private val textButton: String,
16+
private val expectedText: String
17+
) {
18+
companion object {
19+
@JvmStatic
20+
@Parameterized.Parameters(name = "{index}: {0} {1}")
21+
fun data() = listOf(
22+
arrayOf("toast", "toast"),
23+
arrayOf("alert", "alert"),
24+
arrayOf("exception", "exception")
25+
)
26+
}
27+
28+
@Rule
29+
@JvmField
30+
var activityRule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java)
31+
32+
@Test
33+
fun clickRightButton() {
34+
Espresso.onView(ViewMatchers.withText(textButton))
35+
.check(ViewAssertions.matches(ViewMatchers.withText(expectedText)))
36+
}
37+
}

‎test_app/app/src/androidTestMultiple/java/com/example/test_app/parametrized/EspressoParametrizedClassTestParameterized.kt‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
package com.example.test_app.parametrized
2+
3+
import androidx.test.espresso.Espresso
4+
import androidx.test.espresso.assertion.ViewAssertions
5+
import androidx.test.espresso.matcher.ViewMatchers
6+
import androidx.test.rule.ActivityTestRule
7+
import com.example.test_app.MainActivity
8+
import org.junit.Rule
9+
import org.junit.Test
10+
import org.junit.runner.RunWith
11+
import org.junit.runners.Parameterized
12+
13+
@RunWith(Parameterized::class)
14+
class EspressoParametrizedClassTestParameterized(
15+
private val textButton: String,
16+
private val expectedText: String
17+
) {
18+
companion object {
19+
@JvmStatic
20+
@Parameterized.Parameters
21+
fun data() = listOf(
22+
arrayOf("toast", "toast"),
23+
arrayOf("alert", "alert"),
24+
arrayOf("exception", "exception")
25+
)
26+
}
27+
28+
@Rule
29+
@JvmField
30+
var activityRule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java)
31+
32+
@Test
33+
fun clickRightButton() {
34+
Espresso.onView(ViewMatchers.withText(textButton))
35+
.check(ViewAssertions.matches(ViewMatchers.withText(expectedText)))
36+
}
37+
}

‎test_app/app/src/androidTestMultiple/java/com/example/test_app/parametrized/EspressoParametrizedMethodTestJUnitParamsRunner.kt‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
package com.example.test_app.parametrized
2+
3+
import androidx.test.espresso.Espresso
4+
import androidx.test.espresso.assertion.ViewAssertions
5+
import androidx.test.espresso.matcher.ViewMatchers
6+
import androidx.test.rule.ActivityTestRule
7+
import com.example.test_app.MainActivity
8+
import junitparams.JUnitParamsRunner
9+
import junitparams.Parameters
10+
import org.junit.Rule
11+
import org.junit.Test
12+
import org.junit.runner.RunWith
13+
14+
@RunWith(JUnitParamsRunner::class)
15+
class EspressoParametrizedMethodTestJUnitParamsRunner {
16+
@Rule
17+
@JvmField
18+
var activityRule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java)
19+
20+
@Test
21+
@Parameters(method = "methodValuesProvider")
22+
fun clickRightButtonFromMethod(textButton: String, expectedText: String) {
23+
Espresso.onView(ViewMatchers.withText(textButton))
24+
.check(ViewAssertions.matches(ViewMatchers.withText(expectedText)))
25+
}
26+
27+
@Test
28+
@Parameters(value = ["toast, toast", "alert, alert", "exception, exception"])
29+
fun clickRightButtonFromAnnotation(textButton: String, expectedText: String) {
30+
Espresso.onView(ViewMatchers.withText(textButton))
31+
.check(ViewAssertions.matches(ViewMatchers.withText(expectedText)))
32+
}
33+
34+
private fun methodValuesProvider(): Array<Any>? {
35+
return arrayOf(
36+
arrayOf<Any>("toast", "toast"),
37+
arrayOf<Any>("alert", "alert"),
38+
arrayOf<Any>("exception", "exception")
39+
)
40+
}
41+
}

‎test_app/bash/test_filters.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,4 +39,4 @@ function filter_annotation_method_foo {
3939
# should exclude both
4040
function filter_notPackage_foo_notClass_bar {
4141
run_instrument -e notPackage ${PACKAGE_FOO} -e notClass ${CLASS_BAR}
42-
}
42+
}

‎test_runner/docs/ascii/flank.jar_-android-run.adoc‎

Lines changed: 49 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,13 @@ flank.jar
1414
*flank.jar
1515
android run* [*-h*] [*--async*] [*--auto-google-login*]
1616
[*--disable-sharding*] [*--dry*] [*--dump-shards*]
17-
[*--ignore-failed-tests*] [*--keep-file-path*]
18-
[*--legacy-junit-result*] [*--no-auto-google-login*]
19-
[*--no-performance-metrics*] [*--no-record-video*]
20-
[*--no-use-orchestrator*] [*--performance-metrics*]
21-
[*--record-video*] [*--smart-flank-disable-upload*]
22-
[*--use-orchestrator*] [*--app*=_<app>_] [*-c*=_<configPath>_]
17+
[*--full-junit-result*] [*--ignore-failed-tests*]
18+
[*--keep-file-path*] [*--legacy-junit-result*]
19+
[*--no-auto-google-login*] [*--no-performance-metrics*]
20+
[*--no-record-video*] [*--no-use-orchestrator*]
21+
[*--performance-metrics*] [*--record-video*]
22+
[*--smart-flank-disable-upload*] [*--use-orchestrator*]
23+
[*--app*=_<app>_] [*-c*=_<configPath>_]
2324
[*--local-result-dir*=_<localResultsDir>_]
2425
[*--max-test-shards*=_<maxTestShards>_]
2526
[*--network-profile*=_<networkProfile>_]
@@ -47,8 +48,8 @@ flank.jar
4748
[*--files-to-download*=_<filesToDownload>_[,
4849
_<filesToDownload>_...]]... [*--other-files*=_<String=String>_
4950
[,_<String=String>_...]]...
50-
[*--robo-directives*=_<roboDirectives>_[,
51-
_<roboDirectives>_...]]... [*--test-targets*=_<testTargets>_[,
51+
[*--robo-directives*=_<String=String>_[,
52+
_<String=String>_...]]... [*--test-targets*=_<testTargets>_[,
5253
_<testTargets>_...]]...
5354
[*--test-targets-always-run*=_<testTargetsAlwaysRun>_[,
5455
_<testTargetsAlwaysRun>_...]]...
@@ -68,14 +69,17 @@ Configuration is read from flank.yml
6869
// tag::picocli-generated-man-section-options[]
6970
== Options
7071

72+
*-h*, *--help*::
73+
Prints this help message
74+
7175
*--dry*::
7276
Dry run on mock server
7377

74-
*-h*, *--help*::
75-
Prints this help message
78+
*-c*, *--config*=_<configPath>_::
79+
YAML config file path
7680

77-
*--run-timeout*=_<runTimeout>_::
78-
The max time this test run can execute before it is cancelled (default: unlimited).
81+
*--device*=_<String=String>_[,_<String=String>_...]::
82+
A list of DIMENSION=VALUE pairs which specify a target device to test against. This flag may be repeated to specify multiple devices. The four device dimensions are: model, version, locale, and orientation. If any dimensions are omitted, they will use a default value. Omitting all of the preceding dimension-related flags will run tests against a single device using defaults for all four device dimensions.
7983

8084
*--results-bucket*=_<resultsBucket>_::
8185
The name of a Google Cloud Storage bucket where raw test results will be stored (default: "test-lab-<random-UUID>"). Note that the bucket must be owned by a billing-enabled project, and that using a non-default bucket will result in billing charges for the storage used.
@@ -107,15 +111,15 @@ Configuration is read from flank.yml
107111
*--num-flaky-test-attempts*=_<flakyTestAttempts>_::
108112
The number of times a TestExecution should be re-attempted if one or more of its test cases fail for any reason. The maximum number of reruns allowed is 10. Default is 0, which implies no reruns.
109113

110-
*--max-test-shards*=_<maxTestShards>_::
111-
The amount of matrices to split the tests across.
112-
113114
*--shard-time*=_<shardTime>_::
114115
The max amount of seconds each shard should run.
115116

116117
*--num-test-runs*=_<repeatTests>_::
117118
The amount of times to run the test executions.
118119

120+
*--max-test-shards*=_<maxTestShards>_::
121+
The amount of matrices to split the tests across.
122+
119123
*--smart-flank-gcs-path*=_<smartFlankGcsPath>_::
120124
Google cloud storage path to save test timing data used by smart flank.
121125

@@ -137,6 +141,12 @@ Configuration is read from flank.yml
137141
*--local-result-dir*=_<localResultsDir>_::
138142
Saves test result to this local folder. Deleted before each run.
139143

144+
*--run-timeout*=_<runTimeout>_::
145+
The max time this test run can execute before it is cancelled (default: unlimited).
146+
147+
*--full-junit-result*::
148+
Enable create additional local junit result on local storage with failure nodes on passed flaky tests.
149+
140150
*--ignore-failed-tests*::
141151
Terminate with exit code 0 when there are failed tests. Useful for Fladle and other gradle plugins that don't expect the process to have a non-zero exit code. The JUnit XML is used to determine failure. (default: false)
142152

@@ -146,12 +156,6 @@ Configuration is read from flank.yml
146156
*--output-style*=_<outputStyle>_::
147157
Output style of execution status. May be one of [verbose, multi, single]. For runs with only one test execution the default value is 'verbose', in other cases 'multi' is used as the default. The output style 'multi' is not displayed correctly on consoles which don't support ansi codes, to avoid corrupted output use `single` or `verbose`.
148158

149-
*--dump-shards*::
150-
Dumps the shards to android_shards.json for debugging
151-
152-
*-c*, *--config*=_<configPath>_::
153-
YAML config file path
154-
155159
*--app*=_<app>_::
156160
The path to the application binary file. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation.
157161

@@ -173,26 +177,6 @@ Configuration is read from flank.yml
173177
*--no-use-orchestrator*::
174178
Orchestrator is not used. See --use-orchestrator.
175179

176-
*--robo-directives*=_<roboDirectives>_[,_<roboDirectives>_...]::
177-
A comma-separated (<type>:<key>=<value>) map of robo_directives that you can use to customize the behavior of Robo test.
178-
+
179-
The type specifies the action type of the directive, which may take on values click, text or ignore.
180-
+
181-
If no type is provided, text will be used by default.
182-
+
183-
Each key should be the Android resource name of a target UI element and each value should be the text input for that element.
184-
+
185-
Values are only permitted for text type elements, so no value should be specified for click and ignore type elements.
186-
187-
*--robo-script*=_<roboScript>_::
188-
The path to a Robo Script JSON file.
189-
+
190-
The path may be in the local filesystem or in Google Cloud Storage using gs:// notation.
191-
+
192-
You can guide the Robo test to perform specific actions by recording a Robo Script in Android Studio and then specifying this argument.
193-
+
194-
Learn more at https://firebase.google.com/docs/test-lab/robo-ux-test#scripting.
195-
196180
*--environment-variables*=_<String=String>_[,_<String=String>_...]::
197181
A comma-separated, key=value map of environment variables and their desired values. --environment-variables=coverage=true,coverageFile=/sdcard/coverage.ec The environment variables are mirrored as extra options to the am instrument -e KEY1 VALUE1 … command and passed to your test runner (typically AndroidJUnitRunner)
198182

@@ -217,15 +201,35 @@ Learn more at https://firebase.google.com/docs/test-lab/robo-ux-test#scripting.
217201
*--test-targets*=_<testTargets>_[,_<testTargets>_...]::
218202
A list of one or more test target filters to apply (default: run all test targets). Each target filter must be fully qualified with the package name, class name, or test annotation desired. Any test filter supported by am instrument -e … is supported. See https://developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner for more information.
219203

220-
*--legacy-junit-result*::
221-
Fallback for legacy xml junit results parsing.
204+
*--robo-directives*=_<String=String>_[,_<String=String>_...]::
205+
A comma-separated (<type>:<key>=<value>) map of robo_directives that you can use to customize the behavior of Robo test.
206+
+
207+
The type specifies the action type of the directive, which may take on values click, text or ignore.
208+
+
209+
If no type is provided, text will be used by default.
210+
+
211+
Each key should be the Android resource name of a target UI element and each value should be the text input for that element.
212+
+
213+
Values are only permitted for text type elements, so no value should be specified for click and ignore type elements.
222214

223-
*--device*=_<String=String>_[,_<String=String>_...]::
224-
A list of DIMENSION=VALUE pairs which specify a target device to test against. This flag may be repeated to specify multiple devices. The four device dimensions are: model, version, locale, and orientation. If any dimensions are omitted, they will use a default value. Omitting all of the preceding dimension-related flags will run tests against a single device using defaults for all four device dimensions.
215+
*--robo-script*=_<roboScript>_::
216+
The path to a Robo Script JSON file.
217+
+
218+
The path may be in the local filesystem or in Google Cloud Storage using gs:// notation.
219+
+
220+
You can guide the Robo test to perform specific actions by recording a Robo Script in Android Studio and then specifying this argument.
221+
+
222+
Learn more at https://firebase.google.com/docs/test-lab/robo-ux-test#scripting.
225223

226224
*--additional-app-test-apks*=_<String=String>_[,_<String=String>_...]::
227225
A list of app & test apks to include in the run. Useful for running multiple module tests within a single Flank run.
228226

227+
*--legacy-junit-result*::
228+
Fallback for legacy xml junit results parsing.
229+
230+
*--dump-shards*::
231+
Measures test shards from given test apks and writes them into android_shards.json file instead of executing.
232+
229233
// end::picocli-generated-man-section-options[]
230234

231235
// end::picocli-generated-full-manpage[]

0 commit comments

Comments
 (0)