unixfs: add file (extended) attributes, metadata

[mildred]

I think we should have the ability to add metadata to unixfs files. This could be served as (extended) attributes on fuse mounts and as headers (such as Content-Type) for the HTTP server. Tis would be an extension in unixfs.proto. We should discuss what form they take : string key, string value or more complex.

[whyrusleeping]

The way we have discussed to accomplish this is to make a 'metadata object' that wraps the file you wish to add metadata to

example (in json):

{
    "filetype":"mp3",
    "mp3 headers": {...},
    "data" : "<hash of actual file>",
}

[mildred]

But how do you know that you are dealing with a metadata object, and not a unixfs entry. and what's the advantage of introducing another level of indirection, this will only make it slower to fetch a file, don't you think?

[jbenet]

But how do you know that you are dealing with a metadata object, and not a unixfs entry

The metadata object can have its own type identifier (i.e. letting you know -- for example -- that it's a MIME Type Object, which embeds a link and a MIME type.) This is very tricky to get right but the important thing here is to ensure that the model remains extremely flexible to application use cases, and easy to use in the general case (i.e. being explicit with types and including lots of commonly used data structures with standard ipfs distros). Note that the "type" of an object can itself be a link to an object in ipfs defining a grammar (or something equivalent). The point is to make it possible for objects to be truly self-describing.

this will only make it slower to fetch a file, don't you think?

Yeah, but we can actually do a lot speeding up how we replicate files.

[wking]

On Fri, Jan 23, 2015 at 08:18:42AM -0800, Juan Batiz-Benet wrote:

But how do you know that you are dealing with a metadata object,
and not a unixfs entry

The metadata object can have its own type identifier…

More discussion on “how do you know what type of object you're looking
at?” in ipfs/ipfs#36.

[zignig]

+1 mime type.

[joamaki]

As a +1, I would be interested in seeing symbolic links and file permissions supported. This would allow using IPFS as a root filesystem and would be quite neat, for example for mounting a snapshot /nix/store in NixOS for doing immutable deployments. If it makes sense running root filesystem off of a FUSE mount is another subject...

[whyrusleeping]

@joamaki i'd be interested in finding someone with experience using fuse in a 'mission critical' scenario (such as using it for a rootfs).

Permissions and symlinks are definitely one the TODO list. permissions just waiting on implementation, and symlinks waiting on a bit of discussion for how to implement them.

[jbenet]

I have wanted this for so long!!! yes, let's get these supported asap. @whyrusleeping let's talk about this tomorrow.

[whyrusleeping]

@jbenet depends on your definition of tomorrow, but lets still talk about it

[xiaods]

any update?

[kevina]

@xiaods Please see https://github.com/ipfs/ipld-unixfs for the next version of unixfs, which will eventually have support for metadata.