Remove multi-stage build for libraries when building from source

[dseefeld]

#29701 added a process for using GenAPI to generate non-supported assemblies in source-build. Before 5.0, the source-build process was to build the entire repo first and then use the result to generate a tarball, providing a 2 stage build. This worked well to use GenAPI (with prebuilts) in the initial stage to generate the assembly source to be included in the tarball. Then, in the tarball build, GenAPI was not required, and therefore neither were the prebuilts that it required to build.

In 5.0, the goal for source-build is to build a tarball in a single stage. This will eliminate the first stage build and the ability to generate these sources. Another method will need to be found to generate these assembly sources.

/cc: @ericstj

[ericstj]

Some initial brainstorming on this:

1. Be able to use the reference source directly (with some if-def or something) to build not-supported assemblies.
2. Make GenAPI work without pre-builts by porting to S.R.M, Roslyn, or MetadataLoadContext.
3. Make a source-rewriter that based on Roslyn that doesn't need to do the whole workload of GenAPI, but can just consume reference source and turn it into not-supported source.

[ViktorHofer]

@dseefeld do we need this for 5.0? If yes we should prioritize this accordingly.

[dseefeld]

@ViktorHofer Yes, this is needed for 5.0. What can I do to help get this prioritized?

[ViktorHofer]

Some initial brainstorming on this:

We (@ericstj @safern @joperezr @Anipik) discussed yet another possible solution which is leveraging Roslyn's refout feature and use the linker to trim everything unwanted out and throw PNSE. We were in favor of this approach as it removes the necessity of GenAPI and uses the linker which is a shipping product. Unsure if we have enough time to follow this approach.

[ViktorHofer]

What can I do to help get this prioritized?

Will discuss with @ericstj in our 1:1 later today.

[ericstj]

Not sure /refout will help here since PNSE needs to be the primary output of compile and it feels wrong to use some other configuration as the source of a PNSE. I bet the quickest way to solution is to leverage the checked in reference source + transformation to build PNSEs today. Transformation could happen pre-compile (some task), through ifdef, or post compile (linker). I'd prefer pre-compile or ifdef since it doesn't involve rewriting compiler output.

[ViktorHofer]

@dseefeld I talked with @ericstj about this and he will assign someone to start working on this soon.