User-facing TestWorkflowRule doesn't enforce test timeouts anymore#719
Conversation
|
Could we add a unit test that validates that JUnit way to enforce timeout works with TestWorkflowRule? |
|
@mfateev wouldn't it be a unit test for junit? Ideally, junit rules should be orthogonal and independent. actually throws a timeout exception? |
|
The reason TestWorkflowRule has this timeout is that it sets the default one to 10 seconds. You removed this timeout and the default. In the majority of cases, Temporal related unit tests don't simply fail but get stuck as it is a complex multithreaded application. So after this change almost any unit test that uses TestWorkflowRule is going to add the timeout to every test or test class. |
|
@mfateev as they should if users want this behavior. I'm not sure if you were on a standup when I was discussing this change with folks, but the main trigger for it is the fact that users are actually annoyed by these 10 seconds, don't understand why we enforce it and they don't know that they can disable it. It’a especially annoying during debugging. Also, from my experience, “stuck inside” is actually a rare scenario for java-sdk. I’m not sure I got it ever during App development other then “stuck inside after several hours of run”. |
811025a to
a1d426c
Compare
|
The workflow history got printed as before if we use standard JUnit ability to enforce timeout and the test fails/times out. |
|
|
||
| globalTimeout = | ||
| !DebugModeUtils.isTemporalDebugModeOn() | ||
| ? Timeout.seconds( | ||
| builder.testTimeoutSeconds == 0 | ||
| ? DEFAULT_TEST_TIMEOUT_SECONDS | ||
| : builder.testTimeoutSeconds) | ||
| : null; |
There was a problem hiding this comment.
nit: Swap the return values here so the condition isn't negative.
| * @deprecated Temporal test rule shouldn't be responsible for enforcing test timeouts. Use | ||
| * toolchain of your test framework to enforce timeouts. |
There was a problem hiding this comment.
Should mention not only is this deprecated, it now does nothing.
There was a problem hiding this comment.
What about deprecating it for now, but setting timeout only if the value is set?
There was a problem hiding this comment.
I rolled back the removal of the actual implementation. Let's clean it out in 1.5
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| public class SDKTestWorkflowRule implements TestRule { |
There was a problem hiding this comment.
Docstring about why you would use this over normal TestWorkflowRule would be useful
There was a problem hiding this comment.
It's in internal package, it's for us (SDK devs) only
There was a problem hiding this comment.
Right, still useful for new devs taking a look at it I mean.
Sushisource
left a comment
There was a problem hiding this comment.
This makes sense to me. Ultimately it seems reasonable to depend on existing frameworks for timeouts.
a1d426c to
13cdd65
Compare
13cdd65 to
3a8c052
Compare
What was changed
TestWorkflowRule#setTestTimeoutSeconds is deprecated and transformed to noop.
Existing timeout logic is moved to SDKTestWorkflowRule and should be used only in internal Temporal tests.
Also moved handy
newWorkflowStub(Class<T> workflow)andnewUntypedWorkflowStub(String workflow)from SDK TestWorkflowRule to TestWorkflowRule, because they can be handy for our users.Why?
As described in #634, Temporal Test rule shouldn't reimplement basic unit test framework functionality. And default timeout of 10 seconds could be annoying for users in some cases because users have to find a way to disable it.