Defining /http

[victorb]

Currently, libraries are implementing /http while it hasn't been defined 1 2. We should catch up with the implementations and define it, then implement the behaviour.

As far as I know:

• Can be be wrapped within /tls to support https.
• Is a path protocol (as /unix), meaning no further multiaddrs can follow specifying /http/a/b/c as we don't know when it ends. This is a general problem for path protocols (currently just /unix)
• Unsure of how to deal with complex strings that contains ?, / and other characters that might confuse multiaddr. Encode them maybe?

Examples:

• https://1.2.3.4:5001/api => /ip4/1.2.3.4/tcp/5001/tls/http/api
• http://1.2.3.4/api => /ip4/1.2.3.4/tcp/80/http/api
• http://1.2.3.4 => /ip4/1.2.3.4/tcp/80/http
• http://1.2.3.4/api/ => /ip4/1.2.3.4/tcp/80/http/api/ (keeping trailing slash)

Issues mentioning /http and /tls/http (https)

• Is HTTP a path protocol? go-multiaddr#37
• HTTPS? js-mafmt#23
• What is a valid HTTP multiaddr? js-multiaddr#53
• Encoding urls with http/https go-multiaddr#28

The open questions to me are:

• Encapsulation in a path protocol
• Host header
• Query string, fragment, authentication
• Proxying

Unsure of how to deal with complex strings that contains ?, / and other characters that might confuse multiaddr. Encode them maybe?

Question mark is fine, slash could be also fine if libraries stopped to just split on every slash. Instead, string multiaddrs should probably be parsed step by step, with every protocol reading as much of the remaining bytes as it thinks is neccessary.

Examples:

These look good to me for a start

Encapsulation in a path protocol

Is a path protocol (as /unix), meaning no further multiaddrs can follow specifying /http/a/b/c as we don't know when it ends. This is a general problem for path protocols (currently just /unix)

Yep -- we'll still want to anticipate a way to encapsulate stuff within a path protocol though, e.g. a websocket at specific http path or /ipfs over a unix socket. In the binary form it's fine (everything is length-prefixed), but in the string/human-readable form it's tricky since we rely on forward-slashes to delimit protocols and values. There's several options:

1. Length-prefix the path, e.g. /http/9some/path (bad, humans would need to count characters)
2. Escape slashes in the path, e.g. /http/some\/path (okay, easier for machines to parse (split-on-slash))
3. Put a delimiter around paths, e.g. /http/$/some/path$ or /http/(/some/path) (okay, easier for humans to read and write)

We should split this particular question into a separate issue.

Host header

There is one more thing to consider which is the Host header. All other http headers I can think of don't matter for addressing the endpoint since they only deal with representations and encodings, but the Host header is actually crucial for addressing.

The most elegant option seems to be /http/example.com/some/path, but that way we'd have to make the host part mandatory because we can't safely distinguish a hostname from a simple path element (domain regexpes are far from enough).

Query string, fragment, authentication

Two more things that matter for addressing are query strings and fragments. I have a hunch they can just be represented as they are in a URL, e.g. /http/some/path?key=value#fragment.

I think authentication (user[:pass]@host) should not be part of the address.

Proxying

This can likely be as simple as a /socks5 multiaddr encapsulating the /http multiaddr.

[vasco-santos]

Hey, I think that we should definitely advance with this topic.

The initial examples provided by @victorbjelkholm are good, and we can add other examples that we intend to support, in order to find a uniform solution for all of them.

Regarding @lgierth notes, I agree in every aspect, except the Encapsulation in a path protocol. Imho, think that adding delimiters to a path will be really odd for someone using multiaddr, and it could be error prone. This way, I belive that we should create a specific issue for this discussion, as you suggested. I will think in other options meanwhile.

adding delimiters to a path will be really odd for someone using multiaddr, and it could be error prone

That's fine, it can be a separate multiaddr protocol, so people can use /http without special syntax in almost all cases, and in the special case of encapsulating something within /http, they can use the special protocol, e.g. /httpx/(/path)/ws. That's why it'd be easy to take care of it later.

[vasco-santos]

I see... Adding a special protocol for this special case seems fine for me then!

[bobheadxi]

Has there been any progress with this?

[ntninja]

Bikeshed + more detailed spec proposal:
I'd think using curly braces would more aesthetically pleasing than parenthesis: /http/{/path}/ws
There should also be less cases where this conflicts with something within the path, then with parenths, but in case it does… Can we standardize the backslash for escaping stuff in paths in general? The following should then be equivalent:

1. /http/{/path\}}
2. /http/path\}

…and also this:

1. /http/{/host\{/path}
2. /http/host\{/path

…while these should be also invalid (ie: not equivalent to any of the above!):

• /http/host{/path
• /http/path}

As mentioned above the extra braces around HTTP's value should always be allowed, but only required if there is some other protocol wrapped within. Such as:

• /http/{host\{/path}/wss

The canonical representation should always be the one with the least number of braces – reducing the number of bytes required and improving readability. Libraries should always emit the canonical representation and must accept both forms. All of this of course also applies for Unix sockets and whatever else we find in terms of path-like protocols.

Sounds good? If you disagree on by bikeshed's color please also reply so that we can sort this out quickly. 🙂

[Stebalien]

I have two small concerns with that:

1. Being able to concatenate both string and binary multiaddrs is really nice (and we use this everywhere). If {} is optional, this no longer applies to string multiaddrs (/http/path + /ws -> /http/path/ws instead of /http/{path}/ws).
2. /http/{}/foobar looks really weird.

So, what if we just made it mandatory? That is, /http/foobar has no path, /http/{/path/to/thing} has one?

[Stebalien]

@vasco-santos you objected to the encapsulation. Mind taking a look at my comment?