[rust client] Feature: Generic requests–agnostic of transportation or tokio

[colelawrence]

Description

For my own application, I just finished updating the rust-client implementation from using hyper 0.11 to using hyper 0.12 (see diff of conversion)

I thought that I had to do this in order to be able to use the client generator with my generated rust-server, because all of the current examples are suited for hyper 0.12 or newer versions of tokio/futures.

openapi-generator version

3.1.0 accretion (no regression)

Suggest a fix/enhancement

This is where my idea comes in. I would like to be able to choose my own transport and not worry about conflicting threads or things like Send and Sync as a beginner.
I would like to be able to leverage a client generator for constructing requests against the correct endpoint with the correct headers, but remove it's dependence on specific async runtimes.

I would like the ::apis::Request struct to implement a generic request which I could impl Into<hyper::Request> for event_service_http::apis::Request {} for whichever version of hyper, reqwest, actix-* http library the library is using.

We could enable hyper specific client using cargo features, I think.

What are your thoughts? @bjgill @euank @wing328

[colelawrence]

I think we could do something like a trait producing http::Request and mapping http::Response into U for each endpoint. Would http 0.1.13 be a good library to allow other crates to depend on? or map from?

[bjgill]

rust-server gives a model of how you might want to go about this. We autogenerate an API trait (called by the client and implemented by the server) that hides away all mention of specific web frameworks - it even abstracts away the fact that we're using HTTP. See https://github.com/OpenAPITools/openapi-generator/blob/master/samples/server/petstore/rust-server/output/petstore-with-fake-endpoints-models-for-testing/src/lib.rs#L270 for an example.

This is designed so that we could swap out HTTP for something else in future (gRPC?). Equally, I suspect this pattern could be used to be agnostic as to the web framework.

However, I suspect being agnostic to the async framework (i.e. tokio version) would be a lot harder - I fear you'd either end up creating a new custom async framework or having to drop async altogether.

(P.S. despite the name, rust-server generates both a server and client)

[colelawrence]

@bjgill, I realized later on how much more comprehensive the client in rust-server is compared to the client in rust
For reference, I was able to adjust the existing api to generate something like this

request/mod.rs

extern crate http;
/// API endpoints create an ApiRequest which can be consumed by an http executor
pub trait ApiRequest<U>
where
    U: Sized,
{
    fn request(self) -> (http::Request<Vec<u8>>, fn(http::Response<Vec<u8>>) -> Result<U, Error<serde_json::Value>>);
    fn new(request: http::Request<Vec<u8>>) -> Self;
}

apis/calendars_api.rs

pub struct CalendarsApiCalendarsDelete(http::Request<Vec<u8>>);

impl ApiRequest<()> for CalendarsApiCalendarsDelete {
    fn request(self) -> (http::Request<Vec<u8>>, fn(http::Response<Vec<u8>>) -> Result<(), Error<serde_json::Value>>) {
        use serde::de::Error;
        (self.0, |response| {
            print!("CalendarsApiCalendarsDelete -> {:?}", response);
            Err(serde_json::Error::custom("").into())
        })
    }

    fn new(req: http::Request<Vec<u8>>) -> CalendarsApiCalendarsDelete {
        CalendarsApiCalendarsDelete(req)
    }
}

// ...

impl CalendarsApi<CalendarsApiCalendarsDelete> for CalendarsApiClient {
    fn calendar_calendars_delete(&self, calendar_id: &str, alt: &str, fields: &str, key: &str, oauth_token: &str, pretty_print: bool, quota_user: &str, user_ip: &str) -> CalendarsApiCalendarsDelete {
        CalendarsApiCalendarsDelete(
            __internal_request::Request::new(http::Method::DELETE, "/calendars/{calendarId}".to_string())
                .with_auth(__internal_request::Auth::Oauth)
                .with_query_param("alt".to_string(), alt.to_string())
                .with_query_param("fields".to_string(), fields.to_string())
                .with_query_param("key".to_string(), key.to_string())
                .with_query_param("oauth_token".to_string(), oauth_token.to_string())
                .with_query_param("prettyPrint".to_string(), pretty_print.to_string())
                .with_query_param("quotaUser".to_string(), quota_user.to_string())
                .with_query_param("userIp".to_string(), user_ip.to_string())
                .with_path_param("calendarId".to_string(), calendar_id.to_string())
                .returns_nothing()
                .into_request(self.configuration.borrow())
        )
    }
    // ...

This way, we can write a wrapper around ApiRequest<U> to actually execute it.

    fn send() -> Box<Future<Item = (), Error = ()> + Send> {
        let https = hyper_tls::HttpsConnector::new(4).unwrap();
        let client = hyper::Client::builder().build::<_, hyper::Body>(https);
        let conf = apis::configuration::Configuration::new();
        let configuration: Rc<apis::configuration::Configuration> = Rc::new(conf);

        let cal_api_client = apis::CalendarsApiClient::new(configuration.clone());
        println!("Getting calendars start");

        let res = cal_api_client.calendar_calendars_delete("cmVha3Rvci5maV9jaWNzcTlrZm8yNHU1Y2pvamwycDVqcnNxNEBncm91cC5jYWxlbmRhci5nb29nbGUuY29t", "", "", "", "", true, "", "");

        let (req, res_f) = res.request();
        let (req_parts, body_bytes) = req.into_parts();
        let req = hyper::Request::from_parts(req_parts, hyper::Body::from(body_bytes));

        Box::new(
            client
                .request(req)
                .and_then(|resp| {
                    let (res_parts, res_body) = resp.into_parts();
                    res_body.concat2().map(move |body: hyper::Chunk| {
                        let resp: http::Response<Vec<u8>> =
                            http::Response::from_parts(res_parts, body.into_bytes().to_vec());
                        resp
                    })
                }).map(res_f)
                .map_err(|err| {
                    println!("Getting calendars error {:?}", err);
                }).map(|ret| {
                    println!("Getting calendars success {:?}", ret);
                }),
        )
    }

    #[test]
    fn make_send_error() {
        hyper::rt::run(send());
    }

And this could be reused as such

    /// Put any ApiRequest through hyper
    fn send<U: Sized + Send + 'static, R: ApiRequest<U>>(
        client: Rc<hyper::Client<hyper_tls::HttpsConnector<hyper::client::HttpConnector>>>,
        res: R,
    ) -> Box<Future<Item = U, Error = Error<serde_json::Value>> + Send> {
        let (req, res_f) = res.request();
        let (req_parts, body_bytes) = req.into_parts();
        let req = hyper::Request::from_parts(req_parts, hyper::Body::from(body_bytes));

        Box::new(
            client
                .request(req)
                .and_then(|resp| {
                    let (res_parts, res_body) = resp.into_parts();
                    res_body.concat2().map(move |body: hyper::Chunk| {
                        let resp: http::Response<Vec<u8>> =
                            http::Response::from_parts(res_parts, body.into_bytes().to_vec());
                        resp
                    })
                }).map_err(|hyper_error: hyper::Error| {
                    if hyper_error.is_user() {
                        Error::ApiError(apis::ApiError {
                            code: http::status::StatusCode::from_u16(400u16).unwrap(),
                            content: None,
                        })
                    } else {
                        Error::ApiError(apis::ApiError {
                            code: http::status::StatusCode::from_u16(500u16).unwrap(),
                            content: None,
                        })
                    }
                }).and_then(res_f),
        )
    }

    #[test]
    fn make_send_cal() {
        let https = hyper_tls::HttpsConnector::new(4).unwrap();
        let client = Rc::new(hyper::Client::builder().build::<_, hyper::Body>(https));
        let conf = apis::configuration::Configuration::new();
        let configuration: Rc<apis::configuration::Configuration> = Rc::new(conf);

        let cal_api_client = apis::CalendarsApiClient::new(configuration.clone());
        println!("Delete calendar start");

        let cal_delete = cal_api_client.calendar_calendars_delete(
            "cmVha3Rvci5maV9jaWNzcTlrZm8yNHU1Y2pvamwycDVqcnNxNEBncm91cC5jYWxlbmRhci5nb29nbGUuY29t",
            "",
            "",
            "",
            "",
            true,
            "",
            "",
        );

        hyper::rt::run(
            send(client.clone(), cal_delete)
                .map_err(|err| {
                    println!("Delete calendar error {:?}", err);
                }).map(|ret| {
                    println!("Delete calendar success {:?}", ret);
                }),
        );
    }

[colelawrence]

Just sharing my findings above. I have more to learn about how the rust-server does its piece.

[colelawrence]

@bjgill my biggest hurdle is trying to figure out how I could make a request from inside what the current rust-server generates.
How could I make a generated client available to this generated endpoint method?
https://github.com/OpenAPITools/openapi-generator/blob/master/samples/server/petstore/rust-server/output/rust-server-test/examples/server_lib/server.rs#L32-L36

Do I make an Arc of somekind for the tokio core here in lib/mod.rs or in main.rs or somewhere else like using tokio::current (which I don't think is available to the used version of tokio)?

[bjgill]

I think it's simpler than that. You should be able to do something like:

struct Server {
    client: an_api::Client,
}

impl<C> Api<C> for Server<C> where C: Has<XSpanIdString>{
    fn dummy_get(&self, context: &C) -> Box<Future<Item=DummyGetResponse, Error=ApiError>> {
        debug!("dummy_get() - X-Span-ID: {:?}", context.get().0.clone());

        Box::new(
            self.client.dummy_put("sample_input", context)
            .map(|DummyPutResponse(resp)| {
                DummyGetResponse::Success(resp)
            }).map_err(|e| {
                warn!("Dummy get failed");
                e.into()
            })
        )
    }
}

This does look like a gap in the rust-server docs/example, though. I'll raise an issue.

[colelawrence]

@bjgill how do I create that client-from those example locations I linked? That's what's tripping me up. For my use case, I'm generating a client to interact with the Google Calendar API v3, using a swagger file I found for it online.
Do I create a separate tokio_core::reactor::Core at my top level? how would I specify an ./path-to/ca.pem for Google Calendar (I don't know a lot about certificates, I would guess this could be looked up from the Google's CA?)

[bjgill]

That's a rather gaping hole in the example. You need to use the same tokio core for both client and server, but tokio_proto::TcpServer doesn't allow you to pass in or get back out a handle to the tokio core.

I think you'd need to use TcpListener instead. Maybe something like the following?

let listener = TcpListener::bind("127.0.0.1:12345".parse().unwrap(), &handle).unwrap();
let http = Http::<Chunk>::new();
handle.clone().spawn(
    listener
        .incoming()
        .for_each({
            move |(socket, peer)| {
                let service = new_service.new_service().unwrap();
                http.bind_connection(&handle, socket, peer, service);
                Ok(())
            }
        }).unwrap(),
);

Alternatively, if performance is not a major problem, a hacky workaround would be to create a new client for each request as needed.

As to the CA, this is again rather ugly. Ideally, you could just use the system's built-in CA certificate store. We don't support that out of the box, however. You'd need to either work out how to create your own HTTPS connector and pass it in to try_new_with_connector, or download a copy of the Google CA certificate, and point to its location.

Actions on me:

• raise an issue for lack of support for native CA store

[colelawrence]

@bjgill, thank you very much for the guidance!

[colelawrence]

Notice that current rust-server surfaces this error for https connections
Metaswitch/swagger-rs#43
Metaswitch/swagger-rs@6488ec3

[bjgill]

Ah, yes - I and I suspect many other users of rust-server are linux-only. I've just raised #1221 to make this limitation more visible.