[community contribution] Remove `UnixfsNode` from the trickle builder

[schomatis]

Community contribution opportunity

Area: IPFS files.
Estimated time: 10-20 hours (depends on previous knowledge of the Files API).
Difficulty: Moderate, this is not a good first issue, but the problem has already been solved in another part of the code base, so it's not an open-ended question, it has a clear template to apply.
What you can learn: interaction with files in IPFS, how is a DAG constructed, how is a file formatted inside the DAG, DAG and UnixFS layers.
Mentor: @schomatis (meaning I commit to guide the contributor throughout the entire resolution of this issue answering any question he/she may have).

[schomatis]

The UnixfsNode structure is being removed from the balanced builder in #5118 (see #5106 for background). The same task can be done for the trickle DAG builder to be able to completely eliminate the UnixfsNode structure from the code base.

This isn't a trivial task but has already been solved for the balanced case, and most of the necessary functions and structures have already been created, they just need to be applied in the trickle case replacing the analogous UnixfsNode-related functions. The most challenging part is understanding the implementation differences between the balanced and trickle builders (conceptually they are very similar) to check if there is a need to include some more functions to completely replace UnixfsNode, especially in the append family of functions which are not present in the balanced case.

---

I'm not taking this on myself because it does take time and careful analysis to get this right and there's much more to be done in the Files API milestone; the trickle builder is used in only a minor part of the code, so there's no rush here. Also not adding it to the milestone as its main objective is to clarify how Unix files work in IPFS and that is accomplished through the balanced case, the trickle won't add much more new information to the user. Personally I've learned (and suffered) a lot from studying this part of the code base so I think this is a great opportunity for a (relatively) new IPFS developer who wants to dig deeper into the code (and hopefully the suffering part will be diminished with the work done on the milestone).

[kevina]

I have not been following this discussion 100% but this is something I could probably do if it wanted.

@whyrusleeping what are your thoughts on this (if they are expressed elsewhere I apologize, like I said I have not really been following this discussion very carefully).

[schomatis]

Hey @kevina, thanks for volunteering but I created this issue as an opportunity to provide some training to a new IPFS developer (you actually have much more experience than me on this part of the code) so if you don't mind I'd rather have someone more inexperienced take this.

[kevina]

@schomatis okay, I am always looking for straightforward tasks to do that don't require a lot of back and forth with other developers and this looks like it was a good fit which is why I volunteered, but I can leave it for someone else.

[schomatis]

I am always looking for straightforward tasks to do that don't require a lot of back and forth with other developers

Good to know, I could use your help with some of the issues of the Files API milestone that actually aren't beginner friendly and would fit someone with your experience.

[kevina]

@schomatis most of those issues are labeled as documentation, I can help with writing what I know, but I am not the ultimate authority on it for that I think only @whyrusleeping and possible Juan or one of the other earlier developers that worked on that code.

[bjzhang]

Hi @schomatis I read the above comments and patches. And I am interested in this work.

[schomatis]

Hey @bjzhang, that's great! The first steps would be to:

1. Take a look at the balanced builder to understand the format of the file DAG.
2. Take a look at the previous implementation of the builder to understand how was the UnixfsNode used there (which is the structure that should be replaced in the trickle builder).
3. Study the trickle builder to first understand the topology differences of a trickle file DAG compared to the balanced one.
4. Draft a plan/issue/PR of how we could use the importer: remove UnixfsNode from the balanced builder #5118 PR to apply a similar mechanism in the trickle builder, especially with the append functions that we're not present in the balanced builder. (Be warned, I am going to bother you about documenting the code as much as possible :))

(As you said you may have already performed some of these steps.) Use this issue to ask any questions about the code (that will be a great feedback on how we could make it more accessible to the new IPFS developer) or feel free to open new issues/PRs if you think the complexity of the issue deserves it.

[bjzhang]

Hi, @schomatis Thanks for above guidelines. I read the balanced builder, #5118 and #5106. And I am reading the code in trickle builder.

IIUC, I should make use of db. NewFSNodeOverDag() and db.NewLeafDataNode() to avoid use UnixfsNode. And fillTrickleRec and others functions need to rewrite to replace UnixfsNode with FSNodeOverDag.

Given that there are several functions need to be rewrite. I will start from Layout and fillTrickleRec.