Find an elegant way to issue 300 Multiple Choices

[alganet]

Maybe the user may need to provide a closure for that, we can pass him the choices.

[ramsey]

The Content-Type of 300 Multiple Choices is not defined by HTTP, so it's really up to the developers of each application to return a Content-Type they think is suitable for their audience. I think the user providing a closure is the best way to go, for this reason.

[ramsey]

That said, RFC 2616 states "If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field". Perhaps we assume that the first URL in the array of choices is the preferred one and automatically set it in the Location header for the user (provided we give them the ability to override this default behavior somehow)?

[alganet]

I like it. I've put some thought on this.

Currently, if the content-negotiation headers doesn't match, we issue a 406 Not Acceptable. I believe this behavior is right, but it somehow conflicts with 300 Multiple Choices.

We could issue 300 statuses when the Accept header includes text/* and we provide text/html and text/xml, listing the options in the message body and sending a Location header to the first declared, as you said. Same for */* Accept headers.

What do you think @ramsey @augustohp?

[alganet]

I'm also thinking on mapping extensions to mime-types in a router level. Something like:

$router->extensionMap = array(
    'xml' => 'text/xml',
    'json' => 'application/json',
    'foobar' => 'text/xml'
);

Since the user can map any extensions to the same mime type, a 300 Multiple Choices could also help on this. The spec says "The requested resource corresponds to any one of a set of representations, each with its own specific location", so a different URI for each representation is needed for 300 Multiple Choices to work. The spec also says "Unless it was a HEAD request, the response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content- Type header field.", so we need to provide a way to let the user display how the multiple choices are presented.

A nice use case for this is a download page for a report. User can be sent to /reports/2012 and get a page listing all the available formats for downloading. An API could also use this for discovering "rel=alternate" hipermedia connections.

[nickl-]

Ahh the great hypermedia debacle I can't believe i just stumbled on this now.

Even though 2616 never went into much detail about the links and how they should be presented there has at least been some progress in this regard in the 12 odd years since. Although it is not widely implemented as yet aside from stylesheets the web links specification RFC5988 and in particular the link relations it defines IANA Link Relations is of the more desirable approaches as is evident by attempts to continue in that fashion.

What 5988 calls for are simple header representations of links with their relation attributes:

 Link: <http://example.com/TheBook/chapter2>; rel="previous";
         title="previous chapter"

or urlencoded

Link: </TheBook/chapter2>;
         rel="previous"; title*=UTF-8'de'letztes%20Kapitel,
         </TheBook/chapter4>;
         rel="next"; title*=UTF-8'de'n%c3%a4chstes%20Kapitel

These are exactly what the link html header tags are based on:

<link rel="stylesheet" type="text/css" href="style.css" />

Look more familiar now? =)

What it does really well is help the search engine spiders make sense of the content, is completely reliable for retrieving stylesheets but what it doesn't do at all is provide us with clickable links or a solution to the hypermedia predicament.

[nickl-]

What 2616 does make very clear is that this should be a list for the user to choose from. Which will not be the case if we go use location as the browser will simply follow it's nose to the next location treating this like any other 3xx response to follow. In either way either we will be making the choice or the browser will and this was not the intent, right?

But if the links that we are capable of presenting does not solve the problem then what does? My focus has mainly been in trying to solve the problem where it relates to application/json Content/Type in general. Don't worry everybody is having the same problems has anyone seen Xlink work yet? The Hypertext Application Language or HAL has recently filed for specification and they have their own style again which I wont give my opinion about now... what they are doing right is they insist on defining the content type (being application/hal+json) which can drive the understanding and in so the adoption of the practice.

This is what HAL looks like:

{
   "_links": {
     "self": { "href": "/orders" },
     "next": { "href": "/orders?page=2" },
     "find": { "href": "/orders{?id}", "templated": true }
   },
   "_embedded": {
     "orders": [{
         "_links": {
           "self": { "href": "/orders/123" },
           "basket": { "href": "/baskets/98712" },
           "customer": { "href": "/customers/7809" }
         },
         "total": 30.00,
         "currency": "USD",
         "status": "shipped",
       },{
         "_links": {
           "self": { "href": "/orders/124" },
           "basket": { "href": "/baskets/97213" },
           "customer": { "href": "/customers/12369" }
         },
         "total": 20.00,
         "currency": "USD",
         "status": "processing"
     }]
   },
   currentlyProcessing: 14,
   shippedToday: 20
 }
}

And what it translates to is = instead of defining their keywords, Again this limits us to having knowledge about the type to be able to use it. There are others like JSON Ref, JSON-LD, JSON-SCHEMA has quite a complex but would be brilliant solution but this spec is not even on the table anymore at time of this writing. If you want to read more about these this is a good article on the topic. I am still leaning towards the following as my first choice as maps directly to links and hyperlinks and with even a limited amount of insight you should be able to figure out, with out much difficulty, what the intent is.

{
  "movie" : {
    "link" : {
       "rel" : "self",
       "href" : "http://example.org/movie/11",
     },
    "title" : "Fiddler on the Roof",
    "year" : "1971",
    "duration" : 181,
    "genre" : ["Drama", "Musical"],
    "reviews" : {
       "links" : [
         {
           "rel": "related review",
           "href" : "http://example.org/review/1001",
         },
         {
           "rel": "related review",
           "href" : "http://example.org/review/1002",
         }
       ]
    }
  }
}

href: URI of the link.
rel: Link relation
type (optional): Media type of the representation identified by the URI
hreflang (optional): Content language of the representation identified by the URI
title (optional): Human readable information about the link
length (optional): Content length of the representation identified by the URI

[nickl-]

Approaching again from another angle I can at least show that we know what hyperlinks are as a matter of fact so that is not the issue here. A fellow by the name of Mike Amundsen has specked out what he calls the H-factor to help us identify the hypermedia types in great detail.

What we have you see is two categories to consider, data control and link support, they are broken dow as follows:

1. Link Support
   • [LE]Embedding links
   • [LO]Outbound links
   • [LT]Templated queries
   • [LN]Non-Idempotent updates
   • [LI]Idempotent updates
2. Control Data Support
   • [CR]Control data for read requests
   • [CU]Control data for update requests
   • [CM]Control data for interface methods
   • [CL]Control data for links

So if you thought we were down and out with nothing to choose from then no that is not the case mr. Amundsem lists no less than 15 media types to pick and choose from:

• X/HTML
• Atom
• VoiceXML
• CCXML
• SMIL
• SVG
• Turtle
• Notation 3
• TriX
• TriG
• RDF/XML
• SensorML
• CSS
• text/uri-list
• Sitemap XML

I just got side tracked again, yes 15 and this list is longer than when I looked at it before. The focus of my distraction now was what seemed like quite the little gem which I must've overlooked, well I guess I am not alone. RFC2483 yes back when and has been a sworn standard since January 1999 I'll have you know. It's plane it's simple it's detailed to the max, covers any and all possible translations between uri urn url etc that you might ever want or need. It's beautiful and looks like this:
This is an i2ls or URI to URLs and the original is listed first quoted:

# urn:isbn:0-201-08372-8
http://www.huh.org/books/foo.html
http://www.huh.org/books/foo.pdf
ftp://ftp.foo.org/books/foo.txt

What more do you want, right? It has everything you need, has a content type attached been a standard for 13 years already folks, and raise your hands how many people have had issues with anything that even slightly resembles that form in plain ascii I might add when word processors insist that they are links no matter what persuasions you employ to try and convince them otherwise. So these must work right?

I tried it out and guess what? Yip you guessed it, our state of the art webkit deluxe html5 glorified browser spat it right out at the first sign of text/uri-list as if it was something that has gone off and wouldn't even display it. Guess there are more important specifications like let me think yes the popular coffee pot protocol perhaps that has seen more implementation than the official text/uri-list has.

I rest...

[ramsey]

See my comment above. RFC 2616 states "If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field; user agents MAY use the Location field value for automatic redirection."

I think 300 Multiple Choices is one of those cases where a framework needs to leave much of the decision up to the developer as to how it should work. Anything we do really assumes too much. Also, I've never see 300 Multiple Choices used in practice anywhere, so it's difficult to come up with good use cases for how this should behave.

Nevertheless, I'll try to contrive an example of how I would personally implement this in my own applications, and we might be able to use this as a discussion point...

Suppose I have a book resource located at /books/1234. Also, suppose I want to allow agent-driven negotiation and provide the user agent with a list of representations that I'm providing so that they can choose the appropriate one. While the user agent can tell me what Content-Types it prefers (with the Accept header), there's not a good way for me to tell the user agent what Content-Types I provide, but I can use 300 Multiple Choices to do this.

Here's what that might look like:

The user agent makes a request of /books/1234, but they explicitly do not send the Accept header:

GET /books/1234 HTTP/1.1
Host: example.org
User-Agent: HTTPie/0.2.0

Since there's no Accept header, I might decide to show the user agent what I have available for them to get:

HTTP/1.1 300 Multiple Choices
Date: Thu, 21 Jun 2012 21:30:42 GMT
Server: Apache/2.2.22 (Ubuntu)
X-Powered-By: PHP/5.3.10-1ubuntu3.2
Link: <http://example.org/books/1234>; rel="alternate"; type="application/hal+json", <http://example.org/books/1234>; rel="alternate"; type="application/hal+xml", <http://example.org/books/1234>; rel="alternate"; type="application/atom+xml;type=entry"
Content-Length: 236
Content-Type: application/hal+json

{
    "_links": {
        "alternate": [
            {
                "href": "/books/1234", 
                "type": "application/hal+json"
            }, 
            {
                "href": "/books/1234", 
                "type": "application/hal+xml"
            }, 
            {
                "href": "/books/1234", 
                "type": "application/atom+xml;type=entry"
            }
        ], 
        "self": {
            "href": "/books/1234"
        }
    }
}

Now, the user agent can use either the Link header or the entity body to select an appropriate address and type that it may request, using the Accept header. Note that these are all the same URLs, so only the Accept header will determine which media type to return. Optionally, I could have used a different URI (file suffix, etc.) for each type.

I'm not saying that this is what we do, but if we decide to provide convenience wrappers for 300 Multiple Choices, it will need to work something like this:

• Developer designates that there are multiple choices and returns an array of URLs that represent the multiple choices
• Optionally, the developer specifies a renderer that may be used to transform the array into his/her preferred media type
• If the developer does not specify a renderer, then we must decide on a standard media type to use
• In addition to rendering the array of URLs in a standard media type, I recommend we also follow the Web Linking spec (http://tools.ietf.org/html/rfc5988) to provide a Link header

Of course, this all depends on the developer providing URLs, each with a rel and possibly other attributes needed (type, title, hreflang, etc.).

Addendum: I wrote this while @nickl- was writing his last reply. RFC2483 looks like it might be a good way to provide links in the entity body without having to decide on a media type, but it does break down in the example I've given, since it assumes that each URI will be unique and has no way to specify the media type (so there's no indication to the user agent what type will be returned at a URI and negotiation can't occur).

[nickl-]

I don't know anymore! I was all geared up, several hours ago I might add to tell you don't fear! What would Apache do?

It is still reflected in their documentation but all I seem to find is the DOS vulnerability they had with both the 300 and the 406 not Acceptable. These two responses serve the same purpose their only variations are:

• The server has more than what you the user is asking for and cannot decide 300
• The server does not have what the user is asking for but has something else 406

Try as I might I cannot get Apache to give me a 300, which is usually possible through Accept-Language since we can actually have multiple en files, remember Apache mod-negotiate works on file extensions and cannot otherwise tell what the mime type or language of text files are. You can have multiple english documents that is because in addition to your funky-article.html.en you can also have funky-article.html.en-US as well as ...html.en.UK etc. This is the perfect scenario where 300 should be implemented especially if I am asking for en-US for example but the server only has en-UK and en-NZ for example.

But nope she would much rather end in 404 then budge I tried everything. Ask for a language that she doesn't have and no problems you may have a 406, with the ones listed that she "can't" find. Broken I tell you.

This is Apache's version of a 406 response:
Based on language, there are no french files.

GET /static/baz.html HTTP/1.1
User-Agent: Shuber/Curl/1.0 (cURL/7.19.7 PHP/5.3.8 (Darwin) Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 .3.8 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5)
Host: localhost
Accept: text/html
Accept-Charset: ISO-8859-1,utf-8;q=0.7,
Accept-Language: fr
Connection: keep-alive

HTTP/1.1 406 Not Acceptable
Date: Fri, 22 Jun 2012 04:11:50 GMT
Server: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.8
Alternates: {"baz.html.en-UK" 1 {type text/html} {language en} {length 200}}, {"baz.html.en-ZA" 1 {type text/html} {language en} {length 201}}
Vary: negotiate
TCN: list
Content-Length: 463
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>406 Not Acceptable</title>
</head><body>
<h1>Not Acceptable</h1>
<p>An appropriate representation of the requested resource /static/baz.html could not be found on this server.</p>
Available variants:
<ul>
<li><a href="baz.html.en-UK">baz.html.en-UK</a> , type text/html, language en</li>
<li><a href="baz.html.en-ZA">baz.html.en-ZA</a> , type text/html, language en</li>
</ul>
</body></html>

@ramsey Don't worry I purposefully avoided responding to you, I feel very strongly about 2616 compliance are you positive you don't want to amend anything you have said? Let me just get over this Apache disappointment and calm down, but we have a date! =)

[nickl-]

So hows this for rubbing salt in open wounds what I first thought was rather cool is this Alternates: header field which apache returns on 406:

Alternates: {"baz.html.en-UK" 1 {type text/html} {language en} {length 200}}, {"baz.html.en-ZA" 1 {type text/html} {language en} {length 201}}

When I went looking for it I notice 2616 just short from the last line:

The Alternates, Content-Version, Derived-From, Link, URI, Public and Content-Base header fields were defined in previous versions of this specification, but not commonly implemented. See RFC 2068 see 19.6.2.1

Obviously no one told Apache and visa versa because back then Apache was stranger than ever so how would that constitute "not commonly implemented"? It looks like there was another document at the time which 2616 was probably meant to consume draft-ietf-http-alternates and by the looks of it implemented to the T but that which we require, which is approved spec... ag whatever. =) What else can I say?

I can't help but feel maybe it's about time someone does it right.

[ramsey]

I think this is a case where we're not going to be able to get it right, and maybe that's why no one else has yet, either. I think this is because the use cases vary too widely, the media types vary too widely, and what the user agent should do with the response is largely undefined. There is no one-size-fits-all solution. I think this is something we'll need to leave up to the implementer.

[nickl-]

@ramsey I did not coin you as a naysayer!?!? :p Come on it's not rocket science we're attempting.

Firstly, allow me to set the record straight, you cannot take one sentence of the spec, rip it totally out of context and then justify your design on that. This is not the bible... nuf said.

Here is the section you were quoting and I count 9 statements which all have to be true as they are/were when it was written.

10.3.1 300 Multiple Choices

a. The requested resource corresponds to any one of a set of representations, each with its own specific location, and agent-driven negotiation information (Content Negotiation) is being provided
b. so that the user (or user agent) can select a preferred representation and redirect its request to that location.
c. Unless it was a HEAD request, the response SHOULD include an entity containing a list of resource characteristics and location(s)
d. from which the user or user agent can choose the one most appropriate.
e. The entity format is specified by the media type given in the Content-Type header field.
f. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
g. If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field;
h. user agents MAY use the Location field value for automatic redirection.
i. This response is cacheable unless indicated otherwise.

As part of keeping it in context you have to remember that this was written 15 years ago, today we know for a fact that if we populate the Location field the user agent WILL redirect no matter what your status code is. There are several other 3xx requests designated for redirection why don't you just use one of them instead then. If the server knew what it wanted you to see it would've provided that at point f. won't you think? So not only does the use of Location disregard all the other factors but in todays time it would never even occur, unless @ramsey reads the spec. hehehehe just kidding.

Some things you are welcome to rip out and quote on their own like what should we do if the use doesn't send an Accept header?

If no Accept header field is present, then it is assumed that the client accepts all media types.

So what if I want to send something that is not acceptable?

Note: HTTP/1.1 servers are allowed to return responses which are not acceptable according to the accept headers sent in the request. In some cases, this may even be preferable to sending a 406 response. User agents are encouraged to inspect the headers of an incoming response to determine if it is acceptable.

Note that this will be a valid 200 response and I would think that this would be accommodated with a Vary: * header kinda like a "who knows? I just closed my eyes and picked one.". The point is that these are all valid. The spec allows for so many different leniencies that it really isn't necessary to make up your own rules.

So to conclude the 300 response, referring to a), is specifically intended for content negotiation when the server has different locations all satisfying the the request. It might be debatable what a location entails but I am not very confident that we will be able to justify much outside the scope of what the url identifies. i.o.w if you support the location af application/json at a .json suffix and application/xml at a .xml suffix then and only then will the notion be true that there are items at different locations. The fact of the matter is that you need to make everything mentioned in a) true before you can gain entry and use 300 for the purpose of Multiple Choices, if you want to claim compliancy.

In retrospect Apache actually has a valid point in excluding this response, they will go through the motions until h and provide you with what they've got. If they don't have what you are looking for it's a 404 and if they do but it's Not Acceptable you get a 404. Especially if the process of negotiating past h) is of such an extend that it makes you vulnerable when script kiddies use some exploit they found somewhere to bring your application to it's knees I say go for it. I am not saying there might actually be valid use cases for this and personally would see it implemented purely for completion. You would see me brew coffee over http too if I get the chance.

I am not claiming to know it all, not by a long shot and I reserve the right to change my mind too. Don't let that stop you from pointing out what you disagree with. It's all part of getting to the answers we seek

Thank for your contribution! How is HAL turning out, can we display those links in a client yet?

[alganet]

I believe we should pick one strategy and implement it as a separate routine. Something like:

$r3->any('/images/*/metadata', 'ImageMetadataFetcher')
     ->accept(
         'text/html' => new HtmlFormatter,
         'application/rdf+xml' => new RdfFormatter);

The sample above would serve metadata for images in a illustrative server. When content negotiation fails, it should issue a 406.

Then, the developer can:

$r3->any('/images/*/metadata', 'ImageMetadataFetcher')
     ->accept(
         'text/html' => new HtmlFormatter,
         'application/rdf+xml' => new RdfFormatter);
     ->multipleChoicesList(
         'text/html' => new HtmlFormatter,
         'application/rdf+xml' => new RdfFormatter);

This instructs the route to not issue a 406 and issue a 300 instead. In this particular routine, 300 behaves like a list of links in the entity body. The user must set his own formatters for the multiple choices entity body.

Alternatively, we could develop more strategies so a developer could choose, for example, to use the alternates draft:

$r3->any('/images/*/metadata', 'ImageMetadataFetcher')
     ->accept(
         'text/html' => new HtmlFormatter,
         'application/rdf+xml' => new RdfFormatter);
     ->multipleChoicesAlternates(
         'text/html',
         'application/rdf+xml');

Or even instruct the route to use the Location header:

$r3->any('/images/*/metadata', 'ImageMetadataFetcher')
     ->accept(
         'text/html' => new HtmlFormatter,
         'application/rdf+xml' => new RdfFormatter);
     ->multipleChoicesLocation(
         'text/html',
         'application/rdf+xml');

And maybe multiple combinations of these routines:

$r3->any(/***/)->accept(/***/)->multipleChoicesList(/***/)->multipleChoicesLocation(/***/);
$r3->any(/***/)->accept(/***/)->multipleChoicesList(/***/)->multipleChoicesAlternates(/***/);

What do you think?

[alganet]

By the way, I'm having a blast reading you guys discuss about these long time HTTP pitfalls. Very inspiring!

I believe we should treat HTTP and REST as separate things. HTTP is the protocol, REST is the architecture style. Two distinct systems can be RESTful while implementing different HTTP workflows. We should provide basic HTTP building blocks, to the developer can be RESTful in the way he wants.