Amp\disperse() function

[rela589n]

In this pull request, I want to introduce a disperse($closures) function.

It is created to asynchronously execute an array of closures when the client needs all results.

$results = \Amp\disperse([
    fn () => $httpClient->request(new Request('https://www.google.com', 'HEAD')),
    fn () => $httpClient->request(new Request('https://www.bing.com', 'HEAD')),
]);

Initially, I considered placing this function in Amp\Future namespace, but later I opted in favor of Amp, since this function doesn't have Futures in the signature.

[xpader]

What is that??? await(iterable $futures) race order version ???
I don't think we need add this to amp library.

[rela589n]

@xpader , if you don't find this useful at all, you are the contributor, and I'm a layman - it's in your authority to close.

[rela589n]

Maybe this's not the best name for such a function.
Maybe naming it some other way would bring clarity?
What's on your mind?

[rela589n]

Hi, @webpatser, do you think it's a good thing to include?

[rela589n]

Hi, guys
Can we move toward some logical conclusion about this?

[rela589n]

Hi, @xpader !
How is it going?

[xpader]

@rela589n I don't thing we shoud add this to amp library, you can use it yourself.

[rela589n]

I hope you're doing well.

Of course, I could have added this to my repo; I would not have opened the PR then.

Yet, I think you wouldn't want to bother yourself with async() and await() functions when you just want to run something "in parallel".

For example, consider a cache warmup scenario with disperse:

disperse($cacheWarmupFunctions);

Without disperse we'd have to wrap those into async/await:

$futures = array_map(async(...), $cacheWarmupFunctions);
awaitAll($futures);

Which obscures the high picture.

I don't think you'd want to write this every time if you can avoid writing this all the time.

[webpatser]

Hey @rela589n, sorry for the slow turnaround, your ping slipped past me. Cheers @xpader for chiming in.

My honest take: I don't think disperse() earns a spot in core. The wrapper is two lines today:

Future\await(array_map(async(...), $closures));

That already reads cleanly once async()/await() are familiar, and a third name for the same thing grows the API surface without unlocking new behavior. The naming is also tricky; "disperse" didn't immediately read as "run all in parallel and collect results" to me. awaitAll or parallel are closer, but both come with their own history.

The cache-warmup example is fair. That's a great fit for a small helper in app code or a userland package, where you can name it whatever fits your team.

Not a hard no. If the maintainers want it in, I won't push back. But my vote leans toward closing and keeping the helper in user land.

[kelunik]

I think there's value in having such a function, you don't want to write that array_map repeatedly. I'm however not sure on the name:

Maybe this's not the best name for such a function.
Maybe naming it some other way would bring clarity?

There likely is a better name. And we should be really sure it's a good name if we add it to the core library, because we can't just release another major version of it if we're wrong.

I'm totally happy to have this sit around for a while, if someone comes good with naming suggestions and it's still good after a while, we can proceed and merge this.

[rela589n]

What do you think about "scatter"?

It means: (of a group of people or animals) separate and move off quickly in different directions.

[kelunik]

I think it needs to somehow cover the dispersion and then collecting the results again, thought about forkJoin (but don't like it because fork usually refers to processes) and runConcurrently so far.

[rela589n]

If we consider forkJoin, at first, I didn't really think of "fork" as of a verb, but of a noun.

cover the dispersion and then collecting the results again

TBH, it reminds me of Scatter-Gather (Not that I ever used it, but it clicked in me) enterprise pattern:

broadcasts a message to multiple recipients and re-aggregates the responses back into a single message.

In our case, it is:

$results = \Amp\scatterGather([
    fn () => $httpClient->request(new Request('https://www.google.com', 'HEAD')),
    fn () => $httpClient->request(new Request('https://www.bing.com', 'HEAD')),
]);

[webpatser]

Hey @rela589n, @kelunik, on naming.

scatterGather reads as enterprise service bus to me. Accurate to the pattern, but it sets the wrong expectation for a small concurrent-execute helper.

My vote is runConcurrently. Says what it does, no metaphor. Amp\runConcurrently([...]) reads fine at the call site. If the length grates, concurrent as a shorter alternative.

[rela589n]

IMO concurrent is better compared to runConcurrently

$results = \Amp\concurrent([
    fn () => $httpClient->request(new Request('https://www.google.com', 'HEAD')),
    fn () => $httpClient->request(new Request('https://www.bing.com', 'HEAD')),
]);

[rela589n]

Another option I think of is reduce with the idea that we accept the array of nullary closures, and evaluate them:

$results = \Amp\reduce([
    fn () => $httpClient->request(new Request('https://www.google.com', 'HEAD')),
    fn () => $httpClient->request(new Request('https://www.bing.com', 'HEAD')),
]);

Or apply:

$results = \Amp\apply([
    fn () => $httpClient->request(new Request('https://www.google.com', 'HEAD')),
    fn () => $httpClient->request(new Request('https://www.bing.com', 'HEAD')),
]);

Or, talking about evaluation, I would even think about collapse (or something like this):

$results = \Amp\collapse([
    fn () => $httpClient->request(new Request('https://www.google.com', 'HEAD')),
    fn () => $httpClient->request(new Request('https://www.bing.com', 'HEAD')),
]);

In such a case, the concurrency is the internal detail. We know that Amp is all about concurrency in the first place, and collapse would mean to collapse (or splash, squash) concurrency.

[xpader]

Why are we only talk about name? I still don't get why just not useawait([Futures..]), what is your use scenario?

[webpatser]

@xpader The use case is when you start from closures rather than Futures. Cache warmup, fan-out HTTP heads, batch job dispatch. Today that's:

Future\await(array_map(async(...), $closures))

The helper collapses the array_map(async(...)) step. That's the whole delta.

For the record I'm not sold it earns a core slot either, said as much earlier in the thread. But the "why not just await([Futures..])" framing misses that you don't have Futures yet, you have callables.

[xpader]

@webpatser I always thought there were more complicated scenarios that I couldn’t understand. So it's just turn closures into Futures and await them? Just because don't want write that code like array_map()? It's mean name like awaitClosures??

[rela589n]

@xpader , do you want me to close it up and shut down?