Write RFC for gRPC transport

[bajtos]

Write a RFC (request for comment) document describing our proposed design for gRPC transport.

Acceptance criteria

The document should describe the intended gRPC implementation from the point of view of app developers (people building applications using gRPC transport), extension developers (people that want to contribute e.g. custom authorization/authentication sequence action for gRPC) and core developers (people contributing changes to loopback-next and the gRPC extension).

• Supported flavors of service method handlers - Controller methods vs. handler functions
• API-first (top-down) vs. code-first (bottom-up) approach, how to provide protobuf metadata - think of @api and @get decorators for REST
• Sequence contract, what are the built-in (default) sequence actions and what is their API
• How to handle different flavours of gRPC methods (unary call, server streaming, client streaming, bidirectional streaming), especially in respect to Sequence API
• How to avoid duplication of message definitions between .proto and .ts files and keep a single source of truth
• How can controller methods/handler functions access additional request metadata and provide additional response metadata (trailer)?
• Static loading of proto specs, dynamic loading or both? What is our reasoning for this decision?
• How to control which transports/servers are started by app.start

Additional information

• The initial spike story contains a lot of useful content: Spike: gRPC support for controllers #521
• The spike pull request has a partial PoC implementation of gRPC: [DO-NOT-MERGE] spike - gRPC support #571
• The pull request moving REST from core to a standalone package needs to address some of the challenges faced by gRPC too, we should stay consistent: Migrate rest functionality from core #576

[jonathan-casarrubias]

Static loading of proto specs, dynamic loading or both? What is our reasoning for this decision?

Short Answer: Both

Long Answer: I want to contribute by saying that most of the times when extending the framework to build something on top, I usually need everything to be dynamic.

I would like to be able to dynamically create (When possible like in Models) or sometimes simply load other components not only protos, but also repositories, sequences and maybe just maybe controllers, this last one is not a must, but it might be required in a future.

I -as a platform developer- would like to be able to use the LoopBack Core to dynamically build and load entire Apps.

This way if I create some sort of extension/component that provides an interface, like a client interface, it might be able to build and load a gRPC API from a client application without even touching the backend, not even needing to restart the server.

Does that make sense? I hope so, I understand that most of the times I can use bindings, I just want to make sure these can be binded not only during boot time but in runtime.

This is very important because it was a downside on previous versions of LoopBack, If I can load and unload protos (or other components) on the fly, there should be no need to restart the server.

I hope I'm not adding an insane request here.

Cheers
Jon

[VMois]

How to control which transports/servers are started by app.start

1. We can use an additional optional parameter to control starting. Key is matching server name. I'm not sure if it's the best solution. Example:

Start only grpc server:

app.start({
  grpc: true
});

2. Also, we can use our bindings, when we connect servers, then manually start or stop them. But I need to think about this solution more.

[VMois]

How to avoid duplication of message definitions between .proto and .ts files and keep a single source of truth

Interesting project for working with proto files in JS -> https://github.com/dcodeIO/ProtoBuf.js/#usage-with-typescript . Has TypeScript support.
This project can help eliminate creating .proto and .ts files, and keep only .proto files as a single source of truth. But we need to investigate this project and solution more to be sure.

What do you think about it @jonathan-casarrubias @bajtos @raymondfeng ?

[jonathan-casarrubias]

@VMois that looks promising, I'm taking a look... also I'm starting playing around right now, I want to setup an environment to look and feel the new loopback, I think that protobuf.js is really promising.

Also, once I have an environment running, I'll try to share some diagrams from ideas I might have, before we actually go and put code in there, I think we all want to make sure we take the right choices. So, I'm still thinking the best way to interface the gRPC to App Developers.

[bajtos]

Closing this issue as no longer relevant, the work on gRPC is already in progress in https://github.com/strongloop/loopback4-extension-grpc