Make Lambda generic in effect type

[armanbilge]

Maybe just me, but building all this stuff in terms of straight IO was bugging me. This takes a step away from this to make the effect type generic.

I started by defining a trait Feral[F[_]] which encapsulates:

• an Async[F]
• a memoized F[Setup] for a Resource[F, Setup]
• a Dispatcher[F]

The idea here is not so much to provide a "real" Dispatcher (i.e. that supervises its fibers) but as a convenience abstraction for the capability to unsafely run effects. Technically this is an abuse, but here we are setting up a Resource that we have no intention of closing, so yolo.

Thus we can provide a convenience IOFeral that implements the Dispatcher[IO] directly in terms of IO's unsafe methods and the IORuntime.

From here it is straightforward to make everything else generic in F. IOLambda becomes just Lambda, and IOLambda is provided as a convenience mix-in.

Looking forward to thoughts on this.

[bpholt]

On its face, I think this is a good idea. I almost always write code in an abstract F[_] so working in concrete IO for the CloudFormation custom resource felt weird to me too.

I pulled this in locally and rebased against main so I could try it with the CFN custom resource. Without actually deploying it, it seems to work fine, especially after I added a

abstract class IOCloudFormationCustomResourceHandler[Input : Decoder, Output: Encoder]
  extends CloudFormationCustomResourceHandler[IO, Input, Output]
    with IOFeral

to make it a little more ergonomic when implementing the actual Lambda class.

[armanbilge]

Yes, I'm all for defining those convenience versions 👍 btw feel free to push your branch here, save me the rebase 😉

[bpholt]

Oops, didn't realize you had merged scalafmt. Fixing

[armanbilge]

Sorry! I think we are missing a formatting check from CI so your PR slipped through undetected 😆

[djspiewak]

I actually don't like this! :-) Here's my reasoning…

As a practical matter, you need to have a concrete effect at some point in order to run things. We see this with IOApp, which is where you make a final decision about what effect to use. Lambda is basically analogous to IOApp, and so it makes sense to me that it would be fully concrete. But with that said, there's no reason why you can't be abstract on everything else. The way I tend to write application is I'll use IO within IOApp, but then everything else is F[_]. I was envisioning something similar for Lambda.

The problem with allowing arbitrary F[_] here is it starts to become a little weird to talk about where ancillary effects "go". Like, if it's an EitherT, what happens when it's a Left? That kind of problem. The restriction of "it has to be IO" is actually really helpful because we can be comprehensive about the exit cases (i.e. Outcome[Id, *, *]).

[armanbilge]

Thanks for your thoughts! That mostly* makes sense, so I feel like we've messed up the abstraction somehow. For example, with http4s people build their HttpApp in F but then run it inside IOApp. Could we have a similar abstraction, e.g. Lambda and IOLambdaRuntime?

*Mostly, because what I'm trying to understand is: how does your above reasoning apply to Dispatcher[EitherT[F, _]]? Or is that just as problematic, and if not, what makes it different?

[bpholt]

I was thinking this would be used more to support other runtimes and to keep our specializations (e.g. CloudFormationCustomResourceHandler) abstract, not for end-users.

The way I tend to write application is I'll use IO within IOApp, but then everything else is F[_]. I was envisioning something similar for Lambda.

Yes, agreed, but the IOApp here is not necessarily Lambda: it's likely to be IOCloudFormationCustomResourceHandler or IOApiGatewayProxyHttp4sLambda.

[armanbilge]

Newer ideas in #45.