Support step context from Python Workflows SDK

[Caio-Nogueira]

We're adding support for context within step.do's callback. This means adding a new optional parameter to the decorated step method, in a way that doesn't break existing Python Workflows.

This solution introspects the signature of the callback and checks for the size of the parameters list given (without *args and **kwargs). If there are more arguments than resolved dependencies, then the context parameter is injected into the user workflow.

EDIT:

We're also making a bigger change in the sdk regarding dependency resolution. We're following pytest fixtures' approach with a name based approach for steps. This means that we implicitly resolve dependencies based on the param names. Since we're already introspecting the signature for each decorated step, this ensures a consistent approach for declarations. Everything is based on the param name, instead of relying on param positions like previously

Related types PR:

#5625

[dom96]

LGTM but should you maybe also add some tests which verify what's inside ctx?

[hoodmane]

I think determining this positionally is likely to confuse people. How are people to remember whether context is at the end or at the beginning? What if we want to add one more argument like this?

I think it would be a good idea to shift to a design where we look at the argument names, like the way that fixtures work in pytest. So then you could do:

        @step.do(concurrent=False)
        async def step_3(step_1, step_2, context):
             ...

and it could look at the names of the arguments to determine what should go there. We could also get the name of the step from func.__name__ so that it isn't necessary to pass the step name unless people want it to be different from the name of the function.

[Caio-Nogueira]

I think determining this positionally is likely to confuse people. How are people to remember whether context is at the end or at the beginning? What if we want to add one more argument like this?

We needed one way of consistently determining where the context parameter is. That was my initial solution, but then there is the possibility of this somehow breaking existing workflows:

        @step.do(concurrent=False, depends=[context])
        async def step_3(context):
            return "done"

If we search the context param by name instead of by position, then we would possibly run into these name collision edge cases. This would probably need a compat flag.

I think it would be a good idea to shift to a design where we look at the argument names, like the way that fixtures work in pytest. So then you could do:

        @step.do(concurrent=False)
        async def step_3(step_1, step_2, context):
             ...

and it could look at the names of the arguments to determine what should go there. We could also get the name of the step from func.__name__ so that it isn't necessary to pass the step name unless people want it to be different from the name of the function.

Yes this is cleaner, but then how would users get the result of step_1 and step_2, since those are just callables? We also want to avoid encouraging nested steps.

[codspeed-hq]

CodSpeed Performance Report

Merging this PR will degrade performance by 11.43%

_{Comparing caio/WOR-1049 (e81187f) with main (f111a58)}

Summary

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 68 untouched benchmarks
⏩ 129 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	jsonResponse[Response]	42.6 µs	34.9 µs	+22.2%
❌	simpleStringBody[Response]	19.2 µs	21.7 µs	-11.43%

Footnotes

1. 129 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[pombosilva]

Approved from the Workflows team 👍