Stream as an accepts/returns argument

[raymondfeng]

I'm trying to implement a 'download' remote method that will pipe a stream into http response. For example,

function download(..., cb) {
var in = fs.createReadStream('1.jpg', ...);
in.pipe(res);
}

There are a few options:

1. Have the download function to callback with a ReadStream so that strong-remoting will pipe it into the response.
2. Invent a way to declare req/res parameters so that strong-remoting can inject them for the method invocation.
3. Make HTTP context accessible by calling Context.getCurrent().

[bajtos]

In my opinion, only the first option is a viable solution, as it is the only one that keeps the "download" method usable without HTTP/remoting.

The other two options introduce very strong coupling between the method and the REST transport. An important corollary is that such method most likely won't work with a different transport (websocket, etc.)

[raymondfeng]

Unfortunately, the download function is tied to HTTP by nature. Option 1 doesn't doesn't give us the ability to set up Content-Type, for example.

[bajtos]

Option 1 doesn't doesn't give us the ability to set up Content-Type, for example.

The callback can take two parameters: headers and body stream. Or we can create a new structure that holds both stream and metadata.

function download(cb) {
  var img = fs.createReadStream('1.jpg', ...);
  // option 1, last arg is optional
  cb(null, img, { 'content-type': 'image/jpeg' });
  // option 2
  cb(null, { body: img, meta: { 'content-type': 'image/jpeg' } });
  // option 3
  img.contentType = 'image/jpeg'
  cb(null, img);
}

Unfortunately, the download function is tied to HTTP by nature.

Let's look at the problem from a wider point of view. We want to expose a method that returns a stream with some metadata. Such concept is not limited to downloads only and LB users may want to call methods returning streams directly from their Node.js code too.

It is also easier to write unit-tests for a method that is not coupled with http request/response.

Taking this whole thing to the next level - think about composing a multi-part response. You can have two methods: "main" returns e-mail content, "attachment" returns e-mail attachment, each can be called on it's own. Now you want to introduce "bundle" method, that combines streams from "main" and "attachment" into a single response. When the remotable methods don't use http objects, it's easier to compose them to build something bigger.

[ritch]

Decoupling the implementation from HTTP is really the driving opinion of strong-remoting. This is entirely possible for streams. Its just a bit more complex.

The same applies for content type. The simplest way that you can accomplish this now is in a handler.

remotes.before('fileService.download', function(ctx, inst, next) {
  ctx.res.setHeader('Content-Type', mimeForReq(ctx.req));
  next();
});

If you need to pass the mime from the function itself... I suggest using the same approach as we do for err.statusCode.

function download(id, cb) {
  files.get(id, function(err, meta, stream) {
    stream.mime = meta.mime;
    cb(null, stream);
  });
}

Then in an after hook:

remotes.after('fileService.download', function(ctx, inst, next) {
  ctx.res.setHeader('Content-Type', ctx.result.mime);
  ctx.result.pipe(ctx.res);
  next();
});

That being said, I think it would make sense to implement generic http handling functions for cases where it is easiest to just use http directly.

function handle(req, res, next) {
  res.send('hello world');
}

handle.http = {
  isHandler: true
};

[raymondfeng]

Following what Ritchie described, I think there are a few things/concerns involved to implement a remote method:

1. Define a method and implement the business logic in Node.js
2. Decorate the method with metadata to describe the operation (accepts, returns)
3. Declare the remote protocol binding for the method (http.source)
4. Optionally provide logic to handle the protocol-specific mapping (handler/hook)

Technically, the before/after hooks can do the job for 4 even though hooks are mostly for cross-cutting functions, such as logging, auditing, authentication, or rate limiting. I like the idea to extend the method remoting metadata, for example:

myRemoteMethod.remoting = {
before: function(ctx, inst, next) {
...
},
after: function(ctx, inst, next) {
...
}
}

The after handler should be able to access the arguments of callback too.

This way, the remote method consists of the 'pure' business logic and protocol binding (optional in 90% cases).

[bajtos]

Great discussion, I like the direction where we are going.

Ad extending method remoting metadata with before/after hooks: We already have fn.rest.before and fn.rest.after. If that's not enough, I would add fn.http.before and fn.http.after.

The after handler should be able to access the arguments of callback too.

IMO both request parameters and callback arguments should be stored in the ctx object that is available to before/after handlers.

[ritch]

IMO both request parameters and callback arguments should be stored in the ctx object that is available to before/after handlers.

Agreed. ctx should have references to all input and output of an invocation. Right now ctx.args has the built args for invoking the remote method. Adding the callback args should be trivial.