Conversation
denik
requested review from
andrewnester,
pietern and
shreyas-goenka
as code owners
denik
force-pushed
the
denik/acc-CloudLong
branch
from
fde8b13 to
6e02c5f
Compare
acceptance/config_test.go
Outdated
|
|
||
| // If true, this test is not run if -short is passed and cloud env is configured | ||
| // This implies Cloud = true. | ||
| CloudLong *bool |
There was a problem hiding this comment.
Do we use -short in integration runs from a PR? That is, is this flag meant to annotate tests that run only in our nighlies?
acceptance/config_test.go
Outdated
|
|
||
| // If true, this test is not run if -short is passed and cloud env is configured | ||
| // This implies Cloud = true. | ||
| CloudLong *bool |
There was a problem hiding this comment.
How about:
| CloudLong *bool | |
| Expensive *bool |
- We already have a Cloud tag that can be composed with it.
- Normally Long refers to a number, expensive will be more explicit
There was a problem hiding this comment.
We could have a separate flag for controlling short behavior, but here I'm specifically adding short scoped to cloud runs only. The reason is that all tests should be fast locally with mock server, so if a test works both as a local test and cloud test, it would only be slow on cloud.
"Long" is used because it's the opposite of "Short", "Expensive" does not have this property.
There was a problem hiding this comment.
Long is indeed confusing here, what about CloudSlow?
There was a problem hiding this comment.
CloudSlow is fine too, updated.
|
|
||
| if testing.Verbose() { | ||
| // Combination of CloudLong and -v auto-enables -tail | ||
| tailOutput = true |
There was a problem hiding this comment.
Unrelated, but tailing the output can be the default behaviour for all tests.
There was a problem hiding this comment.
I don't think it's needed, since we already record it and show a diff, it's only useful to monitor and review timestamps of long running tests.
denik
changed the title
3 participants
Changes
New bool config setting CloudSlow in acceptance tests. If set, it enables this test on Cloud but skips it in -short setting there. It does not affect local runs, "Slow" is only applied to Cloud. The reason is that we won't need to wait for cluster with mocked testserver.
Additionally, this setting enables -tail if -v is already enabled.
Why
Certain tests that use "bundle run" are too long due to starting a cluster.
Tests
Using this option in #2471