From 80b9ddfbeacd47c79c0394f28c6247adc0c8f2ef Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 17:08:36 -0700 Subject: [PATCH 01/34] Create 0000-nested-publish.md --- text/0000-nested-publish.md | 172 ++++++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 text/0000-nested-publish.md diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md new file mode 100644 index 00000000000..a9c579da83d --- /dev/null +++ b/text/0000-nested-publish.md @@ -0,0 +1,172 @@ +- Feature Name: `nested_publish` +- Start Date: 2023-06-30 +- RFC PR: ... +- Rust Issue: ... + +# Summary +[summary]: #summary + +Allow Cargo packages to be bundled within other Cargo packages when they are published (not just in unpublished workspaces). + +# Motivation +[motivation]: #motivation + +There are a number of reasons why a Rust developer currently may feel the need to create multiple library crates, and therefore multiple Cargo packages (since one package contains at most one library crate). These multiple libraries could be: + +* A trait declaration and a corresponding derive macro (which must be defined in a separate proc-macro library). +* A library that uses a build script that uses another library or binary (e.g. for precomputation or bindings generation). +* A logically singular library broken into multiple parts to speed up compilation. + +Currently, developers must publish these packages separately. This has several disadvantages (see the [Rationale](#rationale-and-alternatives) section for further details): + +* Clutters the public view of the registry with packages not intended to be usable on their own, and which may even become obsolete as internal architecture changes. +* Requires multiple `cargo publish` operations (this could be fixed with bulk publication) and writing public metadata for each package. +* Can result in semver violations and thus compilation failures, due to the developer not thinking about semver compatibility within the group. + +This RFC will allow developers to avoid all of these inconveniences and hazards by publishing a single package. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +By default (and always, prior to this RFC's implementation): + +* If your package contains any sub-packages, Cargo [excludes](https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields) them from the `.crate` archive file produced by `cargo package` and `cargo publish`. +* If your package contains any non-`dev` dependencies which do not give a `version = "..."`, it cannot be published to `crates.io`. + +(By “sub-package” we mean a package (directory with `Cargo.toml`) which is a subdirectory of another package. We shall call the outermost such package, the package being published, the “parent package”.) + +You can change this default by placing in the manifest (`Cargo.toml`) of a sub-package: + +```toml +[package] +publish = "nested" +``` + +If this is done, Cargo's behavior changes as follows: + +* If you publish the parent package, the sub-package is included in the `.crate` file (unless overridden by explicit `exclude`/`include`) and will be available whenever the parent package is downloaded and compiled. +* The parent package may have a `path =` dependency upon the sub-package. (This dependency may not have a `version =` specified.) +* You cannot `cargo publish` the sub-package, just as if it had `publish = false`. (This is a safety measure against accidentally publishing the sub-package separately when this is not intended.) + +Nested sub-packages may be freely placed within other nested sub-packages. + +When a group of packages is published in this way, and depended on, this has a number of useful effects (which are not things that Cargo explicitly implements, just consequences of the system): + +* The packages are versioned in lockstep; there is no way for a version mismatch to arise since all the code was published together. Version resolution does not apply (in the same way that it does not for any other `path =` dependency). +* The sub-package is effectively “private”: it cannot be named by any other package on `crates.io`, only by its parent package and sibling sub-packages. + +## Example: trait and derive macro + +Suppose we want to declare a trait-and-derive-macro package. We can do this as follows. The parent package would have this manifest `foo/Cargo.toml`: + +```toml +[package] +name = "foo" +version = "0.1.0" +edition = "2021" +publish = true + +[dependencies] +foo-macros = { path = "macros" } # newly permitted +``` + +The sub-package manifest `foo/macros/Cargo.toml`: + +```toml +[package] +name = "macros" # this name need not be claimed on crates.io +version = "0.1.0" # this version is not used for dependency resolution +edition = "2021" +publish = "nested" # new syntax + +[lib] +proc-macro = true +``` + +Then you can `cargo publish` from within the parent `foo` directory, and this will create a single `foo` package on `crates.io`, with no `macros` (or `foo-macros`) package visible except when inspecting the source code or in compilation progress messages. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +The following changes must be made across Cargo and `crates.io`: + +* **Manifest schema** + * The Cargo manifest now allows `"nested"` as a value for the `package.publish` key. +* **`cargo package` & `cargo publish`** + * Should refuse to publish a package if that package (not its sub-packages) has `publish = "nested"`. + * Exclude/include rules should, upon finding a sub-package, check if it is `publish = "nested"` and not automatically exclude it. Instead, they should treat it like any other subdirectory; in particular, it should be affected by explicitly specified exclude/include rules. +* **`crates.io`** + * Should allow `path` dependencies that were previously prohibited, at least provided that the named package in fact exists in the `.crate` archive file. The path must not contain any upward traversal (`../`) or other hazardous or non-portable components. +* **Build process** + * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. + +# Drawbacks +[drawbacks]: #drawbacks + +* This increases the number of differences between “Cargo package (on disk)” from “Cargo package (that may be published in a registry, or downloaded as a unit)” in a way which may be confusing; it would be good if we have different words for these two entities, but we don't. +* If Cargo were to add support for multiple libraries per package, that would be largely redundant with this feature. +* It is not possible to publish a bug fix to a sub-package without republishing the entire parent package. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +The reason for doing anything at all in this area is that publishing multiple packages is often a bad solution to the problems that motivate it; in particular: + +* Non-lockstep versioning risk: If you publish `foo 1.0.0` and `foo-macros 1.0.0`, then later publish `foo 1.1.0` and `foo-macros 1.1.0`, then it is _possible_ for users' `Cargo.lock`s to get into a state where they select `foo-macros 1.1.0` and `foo 1.0.0`, and this then breaks because `foo-macros` assumed that items from `foo 1.0.0` would be present. Arguably, this is a deficiency in the proc-macro system (`foo-macros` has a _de facto_ dependency on `foo` but does not declare it), but not one that is likely to be corrected any time soon. This can be worked around by having `foo` specify an exact dependency `foo-macros = "=1.0.0"`, but this is a subtlety that library authors do not automatically think of; semver is easy to get wrong silently. +* The crates.io registry may be cluttered with many packages that are not relevant to users browsing packages. (Of course, there are many other reasons why such clutter will be found.) +* When packages are implementation details, it makes a permanent mark on the `crates.io` registry even if the implementation of the parent package stops needing that particular subdivision. By allowing sub-packages we can allow package authors to create whatever sub-packages they imagine might be useful, and delete them in later versions with no consequences. +* It is possible to depend on a published package that is intended as an implementation detail. Ideally, library authors would document this clearly and library users would obey the documentation, but that doesn't always happen. By allowing nested packages, we introduce a simple “visibility” system that is useful in the same way that `pub` and `pub(crate)` are useful within Rust crates. + +The alternative to nested packages that I have heard of as a possibility would be to support multiple library targets per package. That would be arguably cleaner, but has these disadvantages: + +* It would require new manifest syntax, not just for declaring the multiple libraries, but for referring to them, and for making per-target dependencies (e.g. only a proc-macro lib should depend on `proc-macro2`+`quote`+`syn`, not the rest of the libraries in the package). +* It would require many new mechanisms in Cargo. +* It might have unforeseen problems; by contrast, nested packages are compiled exactly the same way `path` dependencies currently are, and the only new element is the ability to publish them, so the risk of surprises is lower. + +Also, nested packages enables nesting *anything* that Cargo packages can express now and in the future; it is composable with other Cargo functionality. + +We could also do nothing, except for warning the authors of paired macro crates that they should use exact version dependencies. The consequence of this will be continued hassle for developers; it might even be that useful proc-macro features might not be written simply because the author does not want to manage a second package. + +## Details within this proposal + +Instead of introducing a new value for the `publish` key, we could simply allow sub-packages to be published when they would previously be errors. However, this would be problematic when an existing package has a dev-dependency on a sub-package; either that sub-package would suddenly start being published as nested, or there would be no way to specify the sub-package *should* be published. + +We could also introduce an explicit `[subpackages]` table in the manifest. However, I believe `publish = "nested"` has the elegant and worthwhile property that it simultaneously enables nested publication and prohibits accidental un-nested publication of the sub-package. + +# Prior art +[prior-art]: #prior-art + +I am not aware of other package systems that have a relevant similar concept, but I am not broadly informed about package systems. I have designed this proposal to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +I see no specific unclear design choices, but we might want to incorporate one or more of the below _Future possibilities_ into the current RFC, particularly omitting version numbers. + +# Future possibilities +[future-possibilities]: #future-possibilities + +## Omit version numbers + +Nested packages don't really have any use for version numbers; arguably, they should be omitted and even prohibited, since they may mislead a reader into thinking that the version numbers are used for some kind of version resolution. However, this is a further change to Cargo that is not strictly necessary to solve the original problem, and it disagrees with the precedent of how local `path` dependencies currently work (local packages must have version numbers even though they are not used). + +## Nested packages with public binary targets + +One common reason to publish multiple packages is in order to have a library and an accompanying tool binary, without causing the library to have all of the dependencies that the binary does. Examples: `wasm-bindgen` (`wasm-bindgen-cli`), `criterion` (`cargo-criterion`), `rerun` (`rerun-cli`). + +This RFC currently does not address that — if nothing is done, then `cargo install` will ignore binaries in sub-packages. It would be easy to make a change which supports that; for example, `cargo install` could traverse sub-packages and install all found binaries — but that would also install binaries which are intended as testing or (once [artifact dependencies] are implemented) code-generation helpers, which is undesirable. Thus, additional design work is needed to support `cargo install`ing from subpackages: + +* Should there be an additional manifest key which declares the binary target “public”? +* Should targets be explicitly “re-exported” from the parent package? +* Should there be an additional option to `cargo install` which picks subpackages? (This would cancel out the user-facing benefit from having a single package name.) + +## Nested packages with public library targets + +Allowing nested libraries to be named and used from outside the package would allow use cases which are currently handled by Cargo `features` and conditional compilation (optional functionality with nontrivial costs in dependencies or compilation time) to be instead handled by defining additional public libraries within one package. + +This would allow library authors to avoid writing fragile and hard-to-test conditional compilation, and allow library users to avoid accidentally depending on a feature being enabled despite not having enabled it explicitly. It would also allow compiling the optional functionality and its dependencies with maximum parallelism, by not introducing a single `feature`-ful library crate which acts as a single node in the dependency graph. + +However, it requires additional syntax and semantics, and these use cases might be better served by [#3243 packages as namespaces] or some other namespacing proposal, which would allow the libraries to be published independently. (I can also imagine a world in which both of these exist, and the library implementer can transparently use whichever publication strategy best serves their current needs.) + +[artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 +[#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 From 126b2b43ab625383e8143bb8c3fd010755f30256 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 17:34:32 -0700 Subject: [PATCH 02/34] Clarify some terms and discuss workspaces and normalization --- text/0000-nested-publish.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index a9c579da83d..1ebac63344f 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -33,7 +33,7 @@ By default (and always, prior to this RFC's implementation): * If your package contains any sub-packages, Cargo [excludes](https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields) them from the `.crate` archive file produced by `cargo package` and `cargo publish`. * If your package contains any non-`dev` dependencies which do not give a `version = "..."`, it cannot be published to `crates.io`. -(By “sub-package” we mean a package (directory with `Cargo.toml`) which is a subdirectory of another package. We shall call the outermost such package, the package being published, the “parent package”.) +(By “**sub-package**” we mean a package (directory with `Cargo.toml`) which is a subdirectory of another package. We shall call the outermost such package, the package being published, the “**parent package**”.) You can change this default by placing in the manifest (`Cargo.toml`) of a sub-package: @@ -44,7 +44,7 @@ publish = "nested" If this is done, Cargo's behavior changes as follows: -* If you publish the parent package, the sub-package is included in the `.crate` file (unless overridden by explicit `exclude`/`include`) and will be available whenever the parent package is downloaded and compiled. +* If you publish the parent package, the sub-package is included in the `.crate` file (unless overridden by explicit `exclude`/`include`) and will be available to the parent package whenever the parent package is downloaded and compiled. * The parent package may have a `path =` dependency upon the sub-package. (This dependency may not have a `version =` specified.) * You cannot `cargo publish` the sub-package, just as if it had `publish = false`. (This is a safety measure against accidentally publishing the sub-package separately when this is not intended.) @@ -95,11 +95,16 @@ The following changes must be made across Cargo and `crates.io`: * **`cargo package` & `cargo publish`** * Should refuse to publish a package if that package (not its sub-packages) has `publish = "nested"`. * Exclude/include rules should, upon finding a sub-package, check if it is `publish = "nested"` and not automatically exclude it. Instead, they should treat it like any other subdirectory; in particular, it should be affected by explicitly specified exclude/include rules. + * Nested `Cargo.toml`s should be normalized in the same way the root `Cargo.toml` is, if they declare `publish = "nested"`, and not if they do not. + * This avoids modifying the publication behavior for existing packages, even if they contain project templates or invoke `cargo` to compile sub-packages to probe the behavior of the compiler. + * If the nested `Cargo.toml` has a syntax error such that its `package.publish` value cannot be determined, then if it is depended upon, emit an error; if it is not, emit a warning and do not normalize it. * **`crates.io`** * Should allow `path` dependencies that were previously prohibited, at least provided that the named package in fact exists in the `.crate` archive file. The path must not contain any upward traversal (`../`) or other hazardous or non-portable components. * **Build process** * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. +The presence or absence of a `[workspace]` has no effect on the new behavior, just as it has no effect on existing package publication. + # Drawbacks [drawbacks]: #drawbacks From ef9a157cfbc55444bf5d3daa7d920f9c2c4ac6cf Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 17:49:12 -0700 Subject: [PATCH 03/34] Discuss trait coherence relaxation --- text/0000-nested-publish.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 1ebac63344f..8cbce9c66cd 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -175,3 +175,9 @@ However, it requires additional syntax and semantics, and these use cases might [artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 [#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 + +## Additional privileges between crates + +Since nested packages are versioned as a unit, we could relax the trait coherence rules and allow implementations that would otherwise be prohibited. + +This would be particularly useful when implementing traits from large optional libraries; for example, package `foo` with subpackages `foo_core` and `foo_tokio` could have `foo_tokio` write `impl tokio::io::AsyncRead for foo_core::DataSource`. This would improve the dependency graph compared to `foo_core` having a dependency on `tokio` (which is the only way to do this currently), though not have the maximum possible benefit unless we also added public library targets as above, since the package as a whole still only exports one library and thus one dependency graph node. From a1d4425bec78e1e4d75b63e1895589dca22e7dcd Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 17:50:00 -0700 Subject: [PATCH 04/34] Reposition footnote-links --- text/0000-nested-publish.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 8cbce9c66cd..2bd6a176edb 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -173,11 +173,11 @@ This would allow library authors to avoid writing fragile and hard-to-test condi However, it requires additional syntax and semantics, and these use cases might be better served by [#3243 packages as namespaces] or some other namespacing proposal, which would allow the libraries to be published independently. (I can also imagine a world in which both of these exist, and the library implementer can transparently use whichever publication strategy best serves their current needs.) -[artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 -[#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 - ## Additional privileges between crates Since nested packages are versioned as a unit, we could relax the trait coherence rules and allow implementations that would otherwise be prohibited. This would be particularly useful when implementing traits from large optional libraries; for example, package `foo` with subpackages `foo_core` and `foo_tokio` could have `foo_tokio` write `impl tokio::io::AsyncRead for foo_core::DataSource`. This would improve the dependency graph compared to `foo_core` having a dependency on `tokio` (which is the only way to do this currently), though not have the maximum possible benefit unless we also added public library targets as above, since the package as a whole still only exports one library and thus one dependency graph node. + +[artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 +[#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 From 35184e02f052a13dfaca2c4824ff8e8cfdfd82e6 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 17:50:51 -0700 Subject: [PATCH 05/34] Add PR number --- text/0000-nested-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 2bd6a176edb..65e66984659 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -1,6 +1,6 @@ - Feature Name: `nested_publish` - Start Date: 2023-06-30 -- RFC PR: ... +- RFC PR: [rust-lang/rfcs#3452](https://github.com/rust-lang/rfcs/pull/3452) - Rust Issue: ... # Summary From c4dd3725d0262000b4c46d8f0c7720b8b8d4a6e7 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 19:08:44 -0700 Subject: [PATCH 06/34] Clarify `path` behavior and existing dev-dependency behavior --- text/0000-nested-publish.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 65e66984659..3d54f61a140 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -32,6 +32,7 @@ By default (and always, prior to this RFC's implementation): * If your package contains any sub-packages, Cargo [excludes](https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields) them from the `.crate` archive file produced by `cargo package` and `cargo publish`. * If your package contains any non-`dev` dependencies which do not give a `version = "..."`, it cannot be published to `crates.io`. +* If your package contains `[dev-dependencies]` which do not give a `version = "..."`, they are stripped out on publication. (By “**sub-package**” we mean a package (directory with `Cargo.toml`) which is a subdirectory of another package. We shall call the outermost such package, the package being published, the “**parent package**”.) @@ -45,7 +46,7 @@ publish = "nested" If this is done, Cargo's behavior changes as follows: * If you publish the parent package, the sub-package is included in the `.crate` file (unless overridden by explicit `exclude`/`include`) and will be available to the parent package whenever the parent package is downloaded and compiled. -* The parent package may have a `path =` dependency upon the sub-package. (This dependency may not have a `version =` specified.) +* The parent package (and other sub-packages) may have `path =` dependencies upon the sub-package. (Such dependencies must not have a `version =` or `git =`; that is, the `path` must be the _only_ source for the dependency.) * You cannot `cargo publish` the sub-package, just as if it had `publish = false`. (This is a safety measure against accidentally publishing the sub-package separately when this is not intended.) Nested sub-packages may be freely placed within other nested sub-packages. From e71a9538ad584c37a317b4fc0b8ef05aec0538db Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Fri, 30 Jun 2023 19:22:58 -0700 Subject: [PATCH 07/34] Avoid "lockstep" and discuss a version duplication hazard --- text/0000-nested-publish.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 3d54f61a140..5d639c43fe5 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -53,7 +53,7 @@ Nested sub-packages may be freely placed within other nested sub-packages. When a group of packages is published in this way, and depended on, this has a number of useful effects (which are not things that Cargo explicitly implements, just consequences of the system): -* The packages are versioned in lockstep; there is no way for a version mismatch to arise since all the code was published together. Version resolution does not apply (in the same way that it does not for any other `path =` dependency). +* The packages are a single unit for all versioning purposes; there is no way for a version mismatch to arise since all the code was published together. Version resolution does not apply (in the same way that it does not for any other `path =` dependency). * The sub-package is effectively “private”: it cannot be named by any other package on `crates.io`, only by its parent package and sibling sub-packages. ## Example: trait and derive macro @@ -112,6 +112,8 @@ The presence or absence of a `[workspace]` has no effect on the new behavior, ju * This increases the number of differences between “Cargo package (on disk)” from “Cargo package (that may be published in a registry, or downloaded as a unit)” in a way which may be confusing; it would be good if we have different words for these two entities, but we don't. * If Cargo were to add support for multiple libraries per package, that would be largely redundant with this feature. * It is not possible to publish a bug fix to a sub-package without republishing the entire parent package. +* Suppose `foo` has a sub-package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publication may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. + * If this situation comes up, it can be recovered from by newly publishing `foo-core` separately (as would have been done if nested publishing were not used) and using the [semver trick](https://github.com/dtolnay/semver-trick) to maintain compatibility. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From 3da9eeb73845825061e65f50d65f124f4636f5d0 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Thu, 6 Jul 2023 12:11:07 -0700 Subject: [PATCH 08/34] Add Definitions section. --- text/0000-nested-publish.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 5d639c43fe5..88a1bb84790 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -25,6 +25,13 @@ Currently, developers must publish these packages separately. This has several d This RFC will allow developers to avoid all of these inconveniences and hazards by publishing a single package. +# Definitions +[definitions]: #definitions + +* (existing) A “**package**” is a directory with a `Cargo.toml` file, where that `Cargo.toml` file contains `[package]` metadata. (Note that valid `Cargo.toml` files can also declare `[workspace]`s without being packages; such files are irrelevant to this RFC.) +* (existing) A “**sub-package**” is a package (directory with `Cargo.toml`) which is located in a subdirectory of another package. (This is an existing term in Cargo documentation, though only once.) +* (new) A “**parent package**”, in the context of this RFC, is a package which is being or has been published, and which may contain sub-packages. + # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -34,8 +41,6 @@ By default (and always, prior to this RFC's implementation): * If your package contains any non-`dev` dependencies which do not give a `version = "..."`, it cannot be published to `crates.io`. * If your package contains `[dev-dependencies]` which do not give a `version = "..."`, they are stripped out on publication. -(By “**sub-package**” we mean a package (directory with `Cargo.toml`) which is a subdirectory of another package. We shall call the outermost such package, the package being published, the “**parent package**”.) - You can change this default by placing in the manifest (`Cargo.toml`) of a sub-package: ```toml From 7f9c09f38f69c6f77ef1c8a4ff8d3587418080d2 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Thu, 6 Jul 2023 12:31:41 -0700 Subject: [PATCH 09/34] Discuss more alternatives for marking packages; define "nested package" --- text/0000-nested-publish.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 88a1bb84790..586d9018e1b 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -31,6 +31,7 @@ This RFC will allow developers to avoid all of these inconveniences and hazards * (existing) A “**package**” is a directory with a `Cargo.toml` file, where that `Cargo.toml` file contains `[package]` metadata. (Note that valid `Cargo.toml` files can also declare `[workspace]`s without being packages; such files are irrelevant to this RFC.) * (existing) A “**sub-package**” is a package (directory with `Cargo.toml`) which is located in a subdirectory of another package. (This is an existing term in Cargo documentation, though only once.) * (new) A “**parent package**”, in the context of this RFC, is a package which is being or has been published, and which may contain sub-packages. +* (new) A “**nested package**” is a package which is published as part of some parent package rather than independently. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -142,9 +143,15 @@ We could also do nothing, except for warning the authors of paired macro crates ## Details within this proposal -Instead of introducing a new value for the `publish` key, we could simply allow sub-packages to be published when they would previously be errors. However, this would be problematic when an existing package has a dev-dependency on a sub-package; either that sub-package would suddenly start being published as nested, or there would be no way to specify the sub-package *should* be published. +There are several ways we could mark packages for nested publication, rather than using the `package.publish` key: -We could also introduce an explicit `[subpackages]` table in the manifest. However, I believe `publish = "nested"` has the elegant and worthwhile property that it simultaneously enables nested publication and prohibits accidental un-nested publication of the sub-package. +* Instead of introducing a new value for the `publish` key, we could simply allow sub-packages to be published when they would previously be errors. However, this would be problematic when an existing package has a dev-dependency on a sub-package; either that sub-package would suddenly start being published as nested, or there would be no way to specify the sub-package *should* be published. + +* We could introduce an explicit `[subpackages]` table in the manifest. However, I believe `publish = "nested"` has the elegant and worthwhile property that it simultaneously enables nested publication and prohibits accidental un-nested publication of the sub-package. + + * We could reuse `workspace.members` to also describe nested packages somehow; this might usefully avoid redundancy, but not every workspace wants to be published as a single parent package. + +* Similar to the previous, we could allow marking specific `[dependencies]` as “include this for publication”. However, this would make it impossible to publish a nested package which is not a dependency of the parent package, which blocks [the future possibility of][future-possibilities] public targets in nested packages (particularly, installable binary targets in sub-packages, which will frequently depend on the parent and not vice versa). # Prior art [prior-art]: #prior-art From 130ceb021cae5ff082ebd27a9241d4b012ecd449 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Thu, 6 Jul 2023 12:38:13 -0700 Subject: [PATCH 10/34] =?UTF-8?q?Clarify=20=E2=80=9Cis=20private=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- text/0000-nested-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 586d9018e1b..5589c44c99b 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -60,7 +60,7 @@ Nested sub-packages may be freely placed within other nested sub-packages. When a group of packages is published in this way, and depended on, this has a number of useful effects (which are not things that Cargo explicitly implements, just consequences of the system): * The packages are a single unit for all versioning purposes; there is no way for a version mismatch to arise since all the code was published together. Version resolution does not apply (in the same way that it does not for any other `path =` dependency). -* The sub-package is effectively “private”: it cannot be named by any other package on `crates.io`, only by its parent package and sibling sub-packages. +* The sub-package is effectively “private” (in a sense like the Rust language's visibility system): it cannot be named as a dependency by any other package on `crates.io`, only by its parent package and sibling sub-packages. The parent package may still re-export items from it, or even the entire crate, in the same ways as it could do with a dependency on a normally published package. ## Example: trait and derive macro From 42bdb613c91924e3f9eacd08cf98d0b626d65faa Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 10:52:58 -0800 Subject: [PATCH 11/34] Require dependencies to be explicitly nested. --- text/0000-nested-publish.md | 120 +++++++++++++++++++++++------------- 1 file changed, 78 insertions(+), 42 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 5589c44c99b..6a3c1b4e068 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -6,7 +6,7 @@ # Summary [summary]: #summary -Allow Cargo packages to be bundled within other Cargo packages when they are published (not just in unpublished workspaces). +Allow published Cargo packages to depend on other packages stored within themselves, as is currently possible in unpublished packages. # Motivation [motivation]: #motivation @@ -19,48 +19,63 @@ There are a number of reasons why a Rust developer currently may feel the need t Currently, developers must publish these packages separately. This has several disadvantages (see the [Rationale](#rationale-and-alternatives) section for further details): -* Clutters the public view of the registry with packages not intended to be usable on their own, and which may even become obsolete as internal architecture changes. -* Requires multiple `cargo publish` operations (this could be fixed with bulk publication) and writing public metadata for each package. * Can result in semver violations and thus compilation failures, due to the developer not thinking about semver compatibility within the group. +* Requires multiple `cargo publish` operations (though this could be fixed with a bulk publication feature) and writing public metadata for each package. +* Clutters the public view of the registry with packages not intended to be usable on their own, and which may even become obsolete as internal architecture changes. +* Requires each package to have a [full set of metadata](https://doc.rust-lang.org/cargo/reference/publishing.html#before-publishing-a-new-crate). This RFC will allow developers to avoid all of these inconveniences and hazards by publishing a single package. +Additionally, it may sometimes be desirable to share a small amount of code between some published packages, without making the shared code a separately published library with an appropriate public API. + # Definitions [definitions]: #definitions * (existing) A “**package**” is a directory with a `Cargo.toml` file, where that `Cargo.toml` file contains `[package]` metadata. (Note that valid `Cargo.toml` files can also declare `[workspace]`s without being packages; such files are irrelevant to this RFC.) * (existing) A “**sub-package**” is a package (directory with `Cargo.toml`) which is located in a subdirectory of another package. (This is an existing term in Cargo documentation, though only once.) * (new) A “**parent package**”, in the context of this RFC, is a package which is being or has been published, and which may contain sub-packages. -* (new) A “**nested package**” is a package which is published as part of some parent package rather than independently. +* (new) A “**nested package**” is a package which is published as part of some parent package, using this mechanism to allow dependencies, rather than independently. + * A published package may contain sub-packages that are not nested packages, as a simple file inclusion for the package's own build-time purposes. + * Note that a nested package is not necessarily a direct dependency of the parent package, though that will be the typical case. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation By default (and always, prior to this RFC's implementation): -* If your package contains any sub-packages, Cargo [excludes](https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields) them from the `.crate` archive file produced by `cargo package` and `cargo publish`. +* If your package contains any sub-packages in its directory structure, Cargo [excludes](https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields) them from the `.crate` archive file produced by `cargo package` and `cargo publish`. * If your package contains any non-`dev` dependencies which do not give a `version = "..."`, it cannot be published to `crates.io`. * If your package contains `[dev-dependencies]` which do not give a `version = "..."`, they are stripped out on publication. -You can change this default by placing in the manifest (`Cargo.toml`) of a sub-package: +You can change this default in your manifests. First, in the manifest (`Cargo.toml`) of a sub-package, add `publish = "nested"`: ```toml [package] +name = "foo-macros" publish = "nested" ``` -If this is done, Cargo's behavior changes as follows: +Then, in the manifest of the parent package, declare the dependency as `publish = "nested"`: + +```toml +[dependencies] +foo-macros = { path = "macros", publish = "nested" } +``` + +If both of these steps are done, the `foo-macros` package is considered a **nested package**, and Cargo's behavior changes as follows: -* If you publish the parent package, the sub-package is included in the `.crate` file (unless overridden by explicit `exclude`/`include`) and will be available to the parent package whenever the parent package is downloaded and compiled. -* The parent package (and other sub-packages) may have `path =` dependencies upon the sub-package. (Such dependencies must not have a `version =` or `git =`; that is, the `path` must be the _only_ source for the dependency.) -* You cannot `cargo publish` the sub-package, just as if it had `publish = false`. (This is a safety measure against accidentally publishing the sub-package separately when this is not intended.) +* If you publish the parent package, the sub-package is included in the `.crate` archive file and will be available to the parent package whenever the parent package is downloaded and compiled. If the dependency `path` does not lead to a subdirectory, then the sub-package will be automatically copied into a location under the `.cargo` directory inside the top level of the `.crate` archive file, and the `path` value will be rewritten to match. +* The parent package (and other sub-packages) may have `path =` dependencies upon the sub-package. (Such dependencies must not have a `version =` or `git =`; that is, the `path` must be the _only_ source for the dependency. They do not need to also declare `publish = "nested"`.) +* You cannot `cargo publish` the sub-package, just as if it had `publish = false`. (This is a safety measure against accidentally publishing the sub-package separately _in addition_ to its nested copies; the presumption here is that nested packages are not designed to present public API themselves.) -Nested sub-packages may be freely placed within other nested sub-packages. +Nested packages may contain other dependencies on nested packages, and these too are included in the published package. -When a group of packages is published in this way, and depended on, this has a number of useful effects (which are not things that Cargo explicitly implements, just consequences of the system): +When a group of packages is published in this way, this has a number of useful effects for its dependents (which are not things that Cargo explicitly implements, just consequences of the system): -* The packages are a single unit for all versioning purposes; there is no way for a version mismatch to arise since all the code was published together. Version resolution does not apply (in the same way that it does not for any other `path =` dependency). -* The sub-package is effectively “private” (in a sense like the Rust language's visibility system): it cannot be named as a dependency by any other package on `crates.io`, only by its parent package and sibling sub-packages. The parent package may still re-export items from it, or even the entire crate, in the same ways as it could do with a dependency on a normally published package. +* The packages are a single unit for all versioning purposes; there is no way for a version mismatch to arise among them since all the code was published together. Version resolution does not apply (in the same way that it does not for any other `path =` dependency). + + This allows package authors to avoid needing to think about SemVer correctness for their nested packages. +* The sub-package is effectively “private” (in a sense like the Rust language's visibility system): it cannot be named as a dependency by any other package on `crates.io`, only by its parent package and sibling sub-packages. The parent package may still re-export items from it, or even the entire crate, in the same ways as it could do with a dependency on a normally published package. ## Example: trait and derive macro @@ -74,23 +89,23 @@ edition = "2021" publish = true [dependencies] -foo-macros = { path = "macros" } # newly permitted +foo-macros = { path = "macros", publish = "nested" } # new syntax ``` The sub-package manifest `foo/macros/Cargo.toml`: ```toml [package] -name = "macros" # this name need not be claimed on crates.io -version = "0.1.0" # this version is not used for dependency resolution +name = "macros" # this name need not be claimed on crates.io +# version = "0.0.0" # version number is not used and may be omitted edition = "2021" -publish = "nested" # new syntax +publish = "nested" # new syntax [lib] proc-macro = true ``` -Then you can `cargo publish` from within the parent `foo` directory, and this will create a single `foo` package on `crates.io`, with no `macros` (or `foo-macros`) package visible except when inspecting the source code or in compilation progress messages. +Then you can `cargo publish` from within the parent directory `foo/`, and this will create a single `foo` package on `crates.io`, with no `macros` (or `foo-macros`) package visible except when inspecting the source code or in compilation progress messages. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -98,27 +113,35 @@ Then you can `cargo publish` from within the parent `foo` directory, and this wi The following changes must be made across Cargo and `crates.io`: * **Manifest schema** - * The Cargo manifest now allows `"nested"` as a value for the `package.publish` key. + * The `package.publish` key allows `"nested"` as a value, in addition to existing `false` and `true`. + * The `dependencies.*.publish` key may be specified with a value of `"nested"`. No other valid values are currently defined. (If desired, `publish = false` could be used to explicitly document an intent not to nest.) + * If `dependencies.foo.publish = "nested"`, but in `foo`'s manifest `package.publish` is not `"nested"`, then that is an error. + * We might want to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. (Versions are already optional as of Cargo 1.75, but this is merely equivalent to `version = "0.0.0"`.) * **`cargo package` & `cargo publish`** * Should refuse to publish a package if that package (not its sub-packages) has `publish = "nested"`. - * Exclude/include rules should, upon finding a sub-package, check if it is `publish = "nested"` and not automatically exclude it. Instead, they should treat it like any other subdirectory; in particular, it should be affected by explicitly specified exclude/include rules. - * Nested `Cargo.toml`s should be normalized in the same way the root `Cargo.toml` is, if they declare `publish = "nested"`, and not if they do not. + * The `include`/`exclude` rules should, upon finding a sub-package, instead of excluding it automatically, check if it is declared as a nested dependency by the parent package or any other nested package. If it is, it should follow the *other* include/exclude rules normally. + * Nested packages' `Cargo.toml`s should be normalized in the same way the root `Cargo.toml` is. Sub-packages explicitly `include`d but which are not declared as nested should not be. * This avoids modifying the publication behavior for existing packages, even if they contain project templates or invoke `cargo` to compile sub-packages to probe the behavior of the compiler. - * If the nested `Cargo.toml` has a syntax error such that its `package.publish` value cannot be determined, then if it is depended upon, emit an error; if it is not, emit a warning and do not normalize it. * **`crates.io`** - * Should allow `path` dependencies that were previously prohibited, at least provided that the named package in fact exists in the `.crate` archive file. The path must not contain any upward traversal (`../`) or other hazardous or non-portable components. + * Should allow `path` dependencies that were previously prohibited, provided that + + * the named package in fact exists in the `.crate` archive file and has a valid `Cargo.toml`, and + * the named package declares `package.publish = "nested"`. + + The path must not contain any upward traversal (`../`) or other hazardous or non-portable components. + * No changes are needed to the `crates.io` index, because nested packages are an implementation detail of their parent package. * **Build process** * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. -The presence or absence of a `[workspace]` has no effect on the new behavior, just as it has no effect on existing package publication. +The presence or absence of a `[workspace]` has no effect on the new behavior, just as it has no effect on existing package publication. Nested packages may use workspace inheritance. # Drawbacks [drawbacks]: #drawbacks * This increases the number of differences between “Cargo package (on disk)” from “Cargo package (that may be published in a registry, or downloaded as a unit)” in a way which may be confusing; it would be good if we have different words for these two entities, but we don't. -* If Cargo were to add support for multiple libraries per package, that would be largely redundant with this feature. -* It is not possible to publish a bug fix to a sub-package without republishing the entire parent package. -* Suppose `foo` has a sub-package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publication may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. +* If Cargo were to add support for multiple library targets per package, that would be largely redundant with this feature. +* It is not possible to publish a bug fix to a nested package without republishing the entire parent package; this is the cost we pay for the benefit of not needing to take care with versioning for nested packages. +* Suppose `foo` has a nested package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publication may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. * If this situation comes up, it can be recovered from by newly publishing `foo-core` separately (as would have been done if nested publishing were not used) and using the [semver trick](https://github.com/dtolnay/semver-trick) to maintain compatibility. # Rationale and alternatives @@ -143,42 +166,48 @@ We could also do nothing, except for warning the authors of paired macro crates ## Details within this proposal -There are several ways we could mark packages for nested publication, rather than using the `package.publish` key: +There are several ways we could mark packages for nested publication, rather than using the `package.publish` and `dependencies.*.publish` keys: + +* Instead of declaring each _dependency_ as being nested, we could only use `package.publish = "nested"` to make the determination. This would be problematic when a workspace has a root package, because that root package cannot avoid publishing all its `nested` workspace members except by writing `include`/`exclude` rules. + +* Instead of introducing `package.publish = "nested"`, we could only require that dependencies be declared as nested. The disadvantages of this are: + * May unintentionally duplicate published code between a standalone published package and a nested package + * Does not make both ends of the relationship explicit to readers of the code. -* Instead of introducing a new value for the `publish` key, we could simply allow sub-packages to be published when they would previously be errors. However, this would be problematic when an existing package has a dev-dependency on a sub-package; either that sub-package would suddenly start being published as nested, or there would be no way to specify the sub-package *should* be published. +* We could not permit nested packages that are not sub-packages (not in subdirectories of the parent package). This would avoid needing to define a place to copy the nested packages, but would make it impossible for two published packages to both nest the same package, which is useful for managing utility libraries too small and specific to be worth making public. -* We could introduce an explicit `[subpackages]` table in the manifest. However, I believe `publish = "nested"` has the elegant and worthwhile property that it simultaneously enables nested publication and prohibits accidental un-nested publication of the sub-package. +* Instead of declaring anything, we could simply allow sub-packages to be published when they would previously be errors. This would be problematic when an existing package has a dev-dependency on a sub-package; either that sub-package would suddenly start being published as nested, or there would be no way to specify the sub-package *should* be published. - * We could reuse `workspace.members` to also describe nested packages somehow; this might usefully avoid redundancy, but not every workspace wants to be published as a single parent package. +* We could introduce an explicit `[subpackages]` table in the manifest, instead of `dependencies.*.publish`. This is just a syntactic distinction, but I think it would be more cumbersome to use; forgetting to remove an entry would result in publishing dead code, and forgetting to add one would not be detected until `cargo publish` time. + * However, we might want to add something like this for [the future possibility of][future-possibilities] public targets in nested packages (particularly, installable binary targets in sub-packages, which will frequently depend on the parent and not vice versa). + +* We could reuse `workspace.members` to also describe nested packages somehow; this constrains packages to be published to be structured similar to workspaces. -* Similar to the previous, we could allow marking specific `[dependencies]` as “include this for publication”. However, this would make it impossible to publish a nested package which is not a dependency of the parent package, which blocks [the future possibility of][future-possibilities] public targets in nested packages (particularly, installable binary targets in sub-packages, which will frequently depend on the parent and not vice versa). # Prior art [prior-art]: #prior-art I am not aware of other package systems that have a relevant similar concept, but I am not broadly informed about package systems. I have designed this proposal to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. +TODO: Discuss the various prior proposals for Cargo specifically. + # Unresolved questions [unresolved-questions]: #unresolved-questions -I see no specific unclear design choices, but we might want to incorporate one or more of the below _Future possibilities_ into the current RFC, particularly omitting version numbers. +None currently known. # Future possibilities [future-possibilities]: #future-possibilities -## Omit version numbers - -Nested packages don't really have any use for version numbers; arguably, they should be omitted and even prohibited, since they may mislead a reader into thinking that the version numbers are used for some kind of version resolution. However, this is a further change to Cargo that is not strictly necessary to solve the original problem, and it disagrees with the precedent of how local `path` dependencies currently work (local packages must have version numbers even though they are not used). - ## Nested packages with public binary targets One common reason to publish multiple packages is in order to have a library and an accompanying tool binary, without causing the library to have all of the dependencies that the binary does. Examples: `wasm-bindgen` (`wasm-bindgen-cli`), `criterion` (`cargo-criterion`), `rerun` (`rerun-cli`). -This RFC currently does not address that — if nothing is done, then `cargo install` will ignore binaries in sub-packages. It would be easy to make a change which supports that; for example, `cargo install` could traverse sub-packages and install all found binaries — but that would also install binaries which are intended as testing or (once [artifact dependencies] are implemented) code-generation helpers, which is undesirable. Thus, additional design work is needed to support `cargo install`ing from subpackages: +This RFC currently does not address that — if nothing is done, then `cargo install` will ignore binaries in nested packages, and it is unlikely that those packages would actually be in the nested package dependency graph anyway. It would be easy to make a change which supports that; for example, `cargo install` could traverse nested packages and install all found binaries — but that would also install binaries which are intended as testing or (once [artifact dependencies] are implemented) code-generation helpers, which is undesirable. Thus, additional design work is needed to support `cargo install`ing from subpackages: -* Should there be an additional manifest key which declares the binary target “public”? -* Should targets be explicitly “re-exported” from the parent package? -* Should there be an additional option to `cargo install` which picks subpackages? (This would cancel out the user-facing benefit from having a single package name.) +* There must be a way to, in the parent package manifest, declare nested packages to be published even though they are not dependencies of the parent package (but are likely *dependents* instead). This could also serve as the means to declare the binary target, or the nested package, “public”. +* Should individual targets be explicitly “re-exported” from the parent package? +* Should there be an additional option to `cargo install` which picks nested packages? (This would cancel out the benefit to the `cargo install` user from having a single package name.) ## Nested packages with public library targets @@ -194,5 +223,12 @@ Since nested packages are versioned as a unit, we could relax the trait coherenc This would be particularly useful when implementing traits from large optional libraries; for example, package `foo` with subpackages `foo_core` and `foo_tokio` could have `foo_tokio` write `impl tokio::io::AsyncRead for foo_core::DataSource`. This would improve the dependency graph compared to `foo_core` having a dependency on `tokio` (which is the only way to do this currently), though not have the maximum possible benefit unless we also added public library targets as above, since the package as a whole still only exports one library and thus one dependency graph node. +## Git dependencies + +This RFC does not propose implementing a dependency declared as `{ git = "...", publish = "nested" }`. The obvious meaning is to copy the files from the target Git repository into the package, similarly to a Git submodule checkout. However, there might be things that need further consideration: + +* Exactly what set of files should be copied? +* It would become possible to depend on (and thus copy during publication) a nested package someone else wrote for their own packages' use, which creates hazards for versioning and for non-compliance with source code licenses; while these are already possible, now Cargo would be doing it invisibly for you, which seems risky. + [artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 [#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 From 25d86b106c946308b495b4e954ed8002faf9cfd2 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 11:15:59 -0800 Subject: [PATCH 12/34] More motivation and drawbacks. --- text/0000-nested-publish.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 6a3c1b4e068..1885fe0c275 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -26,7 +26,10 @@ Currently, developers must publish these packages separately. This has several d This RFC will allow developers to avoid all of these inconveniences and hazards by publishing a single package. -Additionally, it may sometimes be desirable to share a small amount of code between some published packages, without making the shared code a separately published library with an appropriate public API. +There are also some uses which are not strictly cases of one library package versus multiple library packages: + +* It may sometimes be desirable to share a small amount of code between some published packages, without making the shared code a separately published library with an appropriate public API subject to semver. +* A package intended to distribute a binary or binaries may have a library target for internal purposes (such as sharing modules between multiple binaries, or testing), but not intend for that library to be usable by other packages as a dependency. # Definitions [definitions]: #definitions @@ -129,6 +132,7 @@ The following changes must be made across Cargo and `crates.io`: * the named package declares `package.publish = "nested"`. The path must not contain any upward traversal (`../`) or other hazardous or non-portable components. + * The package index does not explicitly represent nested packages; instead, nested packages' dependencies are flattened into the dependencies of the parent package. This accurately reflects what can be expected when using the parent package. * No changes are needed to the `crates.io` index, because nested packages are an implementation detail of their parent package. * **Build process** * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. @@ -139,10 +143,13 @@ The presence or absence of a `[workspace]` has no effect on the new behavior, ju [drawbacks]: #drawbacks * This increases the number of differences between “Cargo package (on disk)” from “Cargo package (that may be published in a registry, or downloaded as a unit)” in a way which may be confusing; it would be good if we have different words for these two entities, but we don't. -* If Cargo were to add support for multiple library targets per package, that would be largely redundant with this feature. * It is not possible to publish a bug fix to a nested package without republishing the entire parent package; this is the cost we pay for the benefit of not needing to take care with versioning for nested packages. * Suppose `foo` has a nested package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publication may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. * If this situation comes up, it can be recovered from by newly publishing `foo-core` separately (as would have been done if nested publishing were not used) and using the [semver trick](https://github.com/dtolnay/semver-trick) to maintain compatibility. +* Support for duplicative nested publication (that is, nested packages that are nested within more than one parent package) has the following consequences: + * May increase the amount of source code duplicated between different published packages, increasing download sizes and compilation time. It's currently possible to duplicate code into multiple packages via symlinks, but this would make it an “official feature”. + * If packages A and B are separately published with nested package C, and A also depends on B, then A may see two copies of C's items, one direct and one transitive. This may cause a set of packages to fail to compile due to type/trait mismatches when published. [RFC 3516 public/private dependencies](https://rust-lang.github.io/rfcs/3516-public-private-dependencies.html) may be able to reduce problems of this type if we encourage, by documentation and lint, authors to think twice before allowing a multiply-used nested dependency to also be a RFC 3516 public dependency. +* Build and packaging systems that replace or wrap Cargo (e.g. mapping Cargo packages into Linux distribution packages) may have 1 library:1 package assumptions that are broken by this change. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From 743d53193a52e4dfee8f1b7690a9bf011925c4a0 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 11:19:41 -0800 Subject: [PATCH 13/34] =?UTF-8?q?Define=20=E2=80=9Cnested=20publishing?= =?UTF-8?q?=E2=80=9D.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- text/0000-nested-publish.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 1885fe0c275..5be34e13a84 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -40,6 +40,7 @@ There are also some uses which are not strictly cases of one library package ver * (new) A “**nested package**” is a package which is published as part of some parent package, using this mechanism to allow dependencies, rather than independently. * A published package may contain sub-packages that are not nested packages, as a simple file inclusion for the package's own build-time purposes. * Note that a nested package is not necessarily a direct dependency of the parent package, though that will be the typical case. +* (not for documentation but for discussion in this RFC) “**nested publishing**” means a `cargo publish` operation that includes one or more nested packages, or the act of actually making use of the fact that some packages are marked as nested packages. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -144,9 +145,9 @@ The presence or absence of a `[workspace]` has no effect on the new behavior, ju * This increases the number of differences between “Cargo package (on disk)” from “Cargo package (that may be published in a registry, or downloaded as a unit)” in a way which may be confusing; it would be good if we have different words for these two entities, but we don't. * It is not possible to publish a bug fix to a nested package without republishing the entire parent package; this is the cost we pay for the benefit of not needing to take care with versioning for nested packages. -* Suppose `foo` has a nested package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publication may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. +* Suppose `foo` has a nested package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publishing may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. * If this situation comes up, it can be recovered from by newly publishing `foo-core` separately (as would have been done if nested publishing were not used) and using the [semver trick](https://github.com/dtolnay/semver-trick) to maintain compatibility. -* Support for duplicative nested publication (that is, nested packages that are nested within more than one parent package) has the following consequences: +* Support for duplicative nested publishing (that is, nested packages that are nested within more than one parent package) has the following consequences: * May increase the amount of source code duplicated between different published packages, increasing download sizes and compilation time. It's currently possible to duplicate code into multiple packages via symlinks, but this would make it an “official feature”. * If packages A and B are separately published with nested package C, and A also depends on B, then A may see two copies of C's items, one direct and one transitive. This may cause a set of packages to fail to compile due to type/trait mismatches when published. [RFC 3516 public/private dependencies](https://rust-lang.github.io/rfcs/3516-public-private-dependencies.html) may be able to reduce problems of this type if we encourage, by documentation and lint, authors to think twice before allowing a multiply-used nested dependency to also be a RFC 3516 public dependency. * Build and packaging systems that replace or wrap Cargo (e.g. mapping Cargo packages into Linux distribution packages) may have 1 library:1 package assumptions that are broken by this change. @@ -173,7 +174,7 @@ We could also do nothing, except for warning the authors of paired macro crates ## Details within this proposal -There are several ways we could mark packages for nested publication, rather than using the `package.publish` and `dependencies.*.publish` keys: +There are several ways we could mark packages for nested publishing, rather than using the `package.publish` and `dependencies.*.publish` keys: * Instead of declaring each _dependency_ as being nested, we could only use `package.publish = "nested"` to make the determination. This would be problematic when a workspace has a root package, because that root package cannot avoid publishing all its `nested` workspace members except by writing `include`/`exclude` rules. From 34a19dc5544f8325fdb3deac0e041c054f6b73ab Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 11:47:10 -0800 Subject: [PATCH 14/34] Discuss postponed RFC 2224 as prior art. --- text/0000-nested-publish.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 5be34e13a84..779d72e9a6d 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -117,10 +117,12 @@ Then you can `cargo publish` from within the parent directory `foo/`, and this w The following changes must be made across Cargo and `crates.io`: * **Manifest schema** - * The `package.publish` key allows `"nested"` as a value, in addition to existing `false` and `true`. - * The `dependencies.*.publish` key may be specified with a value of `"nested"`. No other valid values are currently defined. (If desired, `publish = false` could be used to explicitly document an intent not to nest.) + * The `package.publish` key allows `"nested"` as a value, in addition to existing `false` and `true`. + * If `package.license` is specified in a nested package, the parent package's license expression must comply with the nested package's. This check is done solely in terms of the operators in the license expression. For example, if two nested packages contain licenses of `MIT` and `BSD-3-Clause`, then the parent package's expression must be `MIT AND BSD-3-Clause` or similar. + * However, if `package.license` is omitted, this is understood to mean the nested package is merely a component of the parent package with no separate claims about its licensing; it does not mean that the nested package has no license permitting its distribution. + * The `dependencies.*.publish` key may be specified with a value of `"nested"`. No other valid values are currently defined. (If desired, `publish = false` could be used to explicitly document an intent not to nest.) * If `dependencies.foo.publish = "nested"`, but in `foo`'s manifest `package.publish` is not `"nested"`, then that is an error. - * We might want to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. (Versions are already optional as of Cargo 1.75, but this is merely equivalent to `version = "0.0.0"`.) + * We might want to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. (Versions are already optional as of Cargo 1.75, but this is merely equivalent to `version = "0.0.0"`.) * **`cargo package` & `cargo publish`** * Should refuse to publish a package if that package (not its sub-packages) has `publish = "nested"`. * The `include`/`exclude` rules should, upon finding a sub-package, instead of excluding it automatically, check if it is declared as a nested dependency by the parent package or any other nested package. If it is, it should follow the *other* include/exclude rules normally. @@ -136,6 +138,7 @@ The following changes must be made across Cargo and `crates.io`: * The package index does not explicitly represent nested packages; instead, nested packages' dependencies are flattened into the dependencies of the parent package. This accurately reflects what can be expected when using the parent package. * No changes are needed to the `crates.io` index, because nested packages are an implementation detail of their parent package. * **Build process** + * The `Cargo.lock` format will need to be modified to handle entries for nested packages differently, as `path` dependencies are currently not allowed to introduce multiple packages with the same name, which could happen though different packages' nested packages. This modification could consist of omitting them entirely and using the same flattened dependency graph as the `crates.io` index will use. * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. The presence or absence of a `[workspace]` has no effect on the new behavior, just as it has no effect on existing package publication. Nested packages may use workspace inheritance. @@ -195,6 +198,8 @@ There are several ways we could mark packages for nested publishing, rather than # Prior art [prior-art]: #prior-art +* Postponed [RFC 2224](https://github.com/rust-lang/rfcs/pull/2224) is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. This RFC is more detailed and addresses the questions that were raised in discussion of 2224. + I am not aware of other package systems that have a relevant similar concept, but I am not broadly informed about package systems. I have designed this proposal to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. TODO: Discuss the various prior proposals for Cargo specifically. From b75a5197f1600c2a89dfcfb63fb9cca95f94547d Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 11:53:26 -0800 Subject: [PATCH 15/34] Mention vendoring. --- text/0000-nested-publish.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 779d72e9a6d..b83aaae3aba 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -147,14 +147,21 @@ The presence or absence of a `[workspace]` has no effect on the new behavior, ju [drawbacks]: #drawbacks * This increases the number of differences between “Cargo package (on disk)” from “Cargo package (that may be published in a registry, or downloaded as a unit)” in a way which may be confusing; it would be good if we have different words for these two entities, but we don't. + * It is not possible to publish a bug fix to a nested package without republishing the entire parent package; this is the cost we pay for the benefit of not needing to take care with versioning for nested packages. + * Suppose `foo` has a nested package `foo-core`. Multiple major versions of `foo` cannot share the same instance of `foo-core` as they could if `foo-core` were separately published and the `foo`s depended on the same version of `foo-core`. Thus, choosing nested publishing may lead to type incompatibilities (and greater compile times) that would not occur if the same libraries had been separately published. * If this situation comes up, it can be recovered from by newly publishing `foo-core` separately (as would have been done if nested publishing were not used) and using the [semver trick](https://github.com/dtolnay/semver-trick) to maintain compatibility. + * Support for duplicative nested publishing (that is, nested packages that are nested within more than one parent package) has the following consequences: * May increase the amount of source code duplicated between different published packages, increasing download sizes and compilation time. It's currently possible to duplicate code into multiple packages via symlinks, but this would make it an “official feature”. * If packages A and B are separately published with nested package C, and A also depends on B, then A may see two copies of C's items, one direct and one transitive. This may cause a set of packages to fail to compile due to type/trait mismatches when published. [RFC 3516 public/private dependencies](https://rust-lang.github.io/rfcs/3516-public-private-dependencies.html) may be able to reduce problems of this type if we encourage, by documentation and lint, authors to think twice before allowing a multiply-used nested dependency to also be a RFC 3516 public dependency. + * Build and packaging systems that replace or wrap Cargo (e.g. mapping Cargo packages into Linux distribution packages) may have 1 library:1 package assumptions that are broken by this change. +* In the discussion of [RFC 2224], it came up that a feature like this could be used for vendoring libraries with patches not yet accepted upstream, where people sometimes currently resort to publishing forks to crates.io. Using nested packages for vendoring has the advantage of not cluttering crates.io, but also results in hidden code duplication. We may wish to decide whether to encourage or discourage this use of the feature. + + # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives @@ -198,7 +205,7 @@ There are several ways we could mark packages for nested publishing, rather than # Prior art [prior-art]: #prior-art -* Postponed [RFC 2224](https://github.com/rust-lang/rfcs/pull/2224) is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. This RFC is more detailed and addresses the questions that were raised in discussion of 2224. +* Postponed [RFC 2224] is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. This RFC is more detailed and addresses the questions that were raised in discussion of 2224. I am not aware of other package systems that have a relevant similar concept, but I am not broadly informed about package systems. I have designed this proposal to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. @@ -245,3 +252,4 @@ This RFC does not propose implementing a dependency declared as `{ git = "...", [artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 [#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 +[RFC 2224]: https://github.com/rust-lang/rfcs/pull/2224 From d7e8deadc03084533099c8e5c46ef47e43501a76 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 12:08:32 -0800 Subject: [PATCH 16/34] =?UTF-8?q?Discuss=20=E2=80=9Csubcrate=20dependencie?= =?UTF-8?q?s=E2=80=9D=20in=20prior=20art.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- text/0000-nested-publish.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index b83aaae3aba..63e9c09c914 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -141,6 +141,8 @@ The following changes must be made across Cargo and `crates.io`: * The `Cargo.lock` format will need to be modified to handle entries for nested packages differently, as `path` dependencies are currently not allowed to introduce multiple packages with the same name, which could happen though different packages' nested packages. This modification could consist of omitting them entirely and using the same flattened dependency graph as the `crates.io` index will use. * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. +* Nested crates should not be documented as public crates in `rustdoc`, since they are an implementation detail of the parent package and their names are not unique. I think `rustdoc` should use its [`doc(inline)`](https://doc.rust-lang.org/rustdoc/write-documentation/the-doc-attribute.html#inline-and-no_inline) support so that any re-exported items are documented in the parent package's crates' documentation. I believe this will require a new command-line option to `rustdoc` to tell it to treat a certain dependency crate as if it were a private module (which already triggers documentation inlining automatically). + The presence or absence of a `[workspace]` has no effect on the new behavior, just as it has no effect on existing package publication. Nested packages may use workspace inheritance. # Drawbacks @@ -207,6 +209,10 @@ There are several ways we could mark packages for nested publishing, rather than * Postponed [RFC 2224] is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. This RFC is more detailed and addresses the questions that were raised in discussion of 2224. +* Blog post [Rust 2030 Christmas list: Subcrate dependencies, by Olivier Faure](https://poignardazur.github.io/2023/01/24/subcrates/) proposes a mechanism of declaring nested dependencies similar to this RFC, but instead of embedding the files in one package, the “subcrates” are packaged separately on crates.io, but published as a single command and are not usable by other packages. Thus, it is similar to a combination of the also-desired features of “publish an entire workspace” and “namespacing on crates.io”, plus the subcrates being private on crates.io. + + This alternative implementation has some advantages such as the potential of publishing updates to subcrates alone. It requires complex features such as the addition of package namespacing to crates.io, but that feature is desired itself for other applications. I consider that a worthy alternative to this RFC, and would be happy to see either one implemented. + I am not aware of other package systems that have a relevant similar concept, but I am not broadly informed about package systems. I have designed this proposal to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. TODO: Discuss the various prior proposals for Cargo specifically. From 6670428687ec07f0486e1fa95e57a3dbbc62bad3 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 12:22:02 -0800 Subject: [PATCH 17/34] =?UTF-8?q?Discuss=20=E2=80=9CInline=20crates?= =?UTF-8?q?=E2=80=9D=20in=20prior=20art.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- text/0000-nested-publish.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 63e9c09c914..6fa27eec1ee 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -207,15 +207,21 @@ There are several ways we could mark packages for nested publishing, rather than # Prior art [prior-art]: #prior-art -* Postponed [RFC 2224] is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. This RFC is more detailed and addresses the questions that were raised in discussion of 2224. +* Postponed [RFC 2224] (2017) is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. -* Blog post [Rust 2030 Christmas list: Subcrate dependencies, by Olivier Faure](https://poignardazur.github.io/2023/01/24/subcrates/) proposes a mechanism of declaring nested dependencies similar to this RFC, but instead of embedding the files in one package, the “subcrates” are packaged separately on crates.io, but published as a single command and are not usable by other packages. Thus, it is similar to a combination of the also-desired features of “publish an entire workspace” and “namespacing on crates.io”, plus the subcrates being private on crates.io. + This RFC is more detailed and addresses the questions that were raised in discussion of 2224. + +* Blog post [Inline crates, by Yoshua Wuyts (2022)](https://blog.yoshuawuyts.com/inline-crates/) proposes that additional library crates can be declared using Rust syntax in the manner `crate foo;` or `crate foo {}`, like modules. + + Compared to this proposal, its advantage is that it requires no new Cargo concepts and fits right in to the source-code-traversal-driven nature of existing ways to define Rust package and crate contents. However, it forces additional edges in the dependency graph (since the dependent's compilation must be started first to discover the inline crate, and there is no place to declare different dependencies for different crates). I expect it would also require significant changes to the Rust language and `rustc`, reaching beyond just “spawn another `rustc`” and including problems like computing the implied dependencies among inline crates that depend on other inline crates (unless inline crates are only allowed to depend on external dependencies and their own child inline crates, which is likely undesirable because it prohibits establishing common core vocabulary among a set of crates to be compiled in parallel). + +* Blog post [Rust 2030 Christmas list: Subcrate dependencies, by Olivier Faure (2023)](https://poignardazur.github.io/2023/01/24/subcrates/) proposes a mechanism of declaring nested dependencies similar to this RFC, but instead of embedding the files in one package, the “subcrates” are packaged separately on crates.io, but published as a single command and are not usable by other packages. Thus, it is similar to a combination of the also-desired features of “publish an entire workspace” and “namespacing on crates.io”, plus the subcrates being private on crates.io. This alternative implementation has some advantages such as the potential of publishing updates to subcrates alone. It requires complex features such as the addition of package namespacing to crates.io, but that feature is desired itself for other applications. I consider that a worthy alternative to this RFC, and would be happy to see either one implemented. -I am not aware of other package systems that have a relevant similar concept, but I am not broadly informed about package systems. I have designed this proposal to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. +Among this space of possibilities, I see the place of this particular RFC as attempting to be a **minimal addition to Cargo**, building on the existing concept of `path` dependencies to add lots of power with little implementation cost; not necessarily to make sense from a blank slate. -TODO: Discuss the various prior proposals for Cargo specifically. +I am not aware of other package systems that have a similar concept, but I am not broadly informed about package systems. # Unresolved questions [unresolved-questions]: #unresolved-questions From da547de44b37236865b5d1123c8911f70ba5f942 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 13:54:44 -0800 Subject: [PATCH 18/34] Rewrite reference-level explanation to focus more on effects than changes. --- text/0000-nested-publish.md | 88 ++++++++++++++++++++++++------------- 1 file changed, 58 insertions(+), 30 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 6fa27eec1ee..ee11c250d7b 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -40,6 +40,7 @@ There are also some uses which are not strictly cases of one library package ver * (new) A “**nested package**” is a package which is published as part of some parent package, using this mechanism to allow dependencies, rather than independently. * A published package may contain sub-packages that are not nested packages, as a simple file inclusion for the package's own build-time purposes. * Note that a nested package is not necessarily a direct dependency of the parent package, though that will be the typical case. +* (new) A “**nested dependency**” is an entry in `[dependencies]` that refers to a nested package, and is declared as such. * (not for documentation but for discussion in this RFC) “**nested publishing**” means a `cargo publish` operation that includes one or more nested packages, or the act of actually making use of the fact that some packages are marked as nested packages. # Guide-level explanation @@ -114,36 +115,59 @@ Then you can `cargo publish` from within the parent directory `foo/`, and this w # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -The following changes must be made across Cargo and `crates.io`: - -* **Manifest schema** - * The `package.publish` key allows `"nested"` as a value, in addition to existing `false` and `true`. - * If `package.license` is specified in a nested package, the parent package's license expression must comply with the nested package's. This check is done solely in terms of the operators in the license expression. For example, if two nested packages contain licenses of `MIT` and `BSD-3-Clause`, then the parent package's expression must be `MIT AND BSD-3-Clause` or similar. - * However, if `package.license` is omitted, this is understood to mean the nested package is merely a component of the parent package with no separate claims about its licensing; it does not mean that the nested package has no license permitting its distribution. - * The `dependencies.*.publish` key may be specified with a value of `"nested"`. No other valid values are currently defined. (If desired, `publish = false` could be used to explicitly document an intent not to nest.) - * If `dependencies.foo.publish = "nested"`, but in `foo`'s manifest `package.publish` is not `"nested"`, then that is an error. - * We might want to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. (Versions are already optional as of Cargo 1.75, but this is merely equivalent to `version = "0.0.0"`.) -* **`cargo package` & `cargo publish`** - * Should refuse to publish a package if that package (not its sub-packages) has `publish = "nested"`. - * The `include`/`exclude` rules should, upon finding a sub-package, instead of excluding it automatically, check if it is declared as a nested dependency by the parent package or any other nested package. If it is, it should follow the *other* include/exclude rules normally. - * Nested packages' `Cargo.toml`s should be normalized in the same way the root `Cargo.toml` is. Sub-packages explicitly `include`d but which are not declared as nested should not be. - * This avoids modifying the publication behavior for existing packages, even if they contain project templates or invoke `cargo` to compile sub-packages to probe the behavior of the compiler. -* **`crates.io`** - * Should allow `path` dependencies that were previously prohibited, provided that - - * the named package in fact exists in the `.crate` archive file and has a valid `Cargo.toml`, and - * the named package declares `package.publish = "nested"`. - - The path must not contain any upward traversal (`../`) or other hazardous or non-portable components. - * The package index does not explicitly represent nested packages; instead, nested packages' dependencies are flattened into the dependencies of the parent package. This accurately reflects what can be expected when using the parent package. - * No changes are needed to the `crates.io` index, because nested packages are an implementation detail of their parent package. -* **Build process** - * The `Cargo.lock` format will need to be modified to handle entries for nested packages differently, as `path` dependencies are currently not allowed to introduce multiple packages with the same name, which could happen though different packages' nested packages. This modification could consist of omitting them entirely and using the same flattened dependency graph as the `crates.io` index will use. - * Probably some messages will need to be adjusted; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”. - -* Nested crates should not be documented as public crates in `rustdoc`, since they are an implementation detail of the parent package and their names are not unique. I think `rustdoc` should use its [`doc(inline)`](https://doc.rust-lang.org/rustdoc/write-documentation/the-doc-attribute.html#inline-and-no_inline) support so that any re-exported items are documented in the parent package's crates' documentation. I believe this will require a new command-line option to `rustdoc` to tell it to treat a certain dependency crate as if it were a private module (which already triggers documentation inlining automatically). - -The presence or absence of a `[workspace]` has no effect on the new behavior, just as it has no effect on existing package publication. Nested packages may use workspace inheritance. +## Cargo: Manifest + +Two new possible values are added to the manifest. + +* The `package.publish` field allows `"nested"` as a value, in addition to existing `false` and `true`. This value affects `cargo publish` and nested dependencies as described below. +* The `dependencies.*.publish` field is newly defined, with the only currently allowed value being `"nested"`, to declare that that dependency is a nested dependency. (If desired, `publish = false` could be also be permitted to explicitly document an intent not to nest; this would be identical to the status quo and to omitting the key.) + * It is an error if a nested dependency does not have a `path` field, or if it has a `version`, `git`, or any other package source field, unless future work defines a meaning for that combination. + +When a nested dependency is present (making its referent be a nested package), the following additional requirements apply: + +* The nested package must have `package.publish = "nested"`; both `false` and `true` are errors. + +* If the nested package specifies `package.license`, the parent package's (not necessarily the dependent's) license expression must comply with the nested package's. This check is done solely in terms of the operators in the license expression. For example, if two nested packages contain licenses of `MIT` and `BSD-3-Clause`, then the parent package's expression must be `MIT AND BSD-3-Clause` or similar. + + This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing independent from the parent package's. + +* We might want to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. (`package.version` is already optional as of Cargo 1.75, but this is currently equivalent to `version = "0.0.0"`.) + +## **`cargo package` & `cargo publish`** + +If `cargo package` or `cargo package` is directed to operate on a package whose manifest specifies `package.publish = "nested"`, that is an error. (Users wishing to make crates in a nested package public should either write `package.publish = true` or should re-export their contents from another package.) + +When a valid parent package is packaged, each of its transitive nested dependencies must be included in the `.crate` archive file. This has two sub-cases: + +* The nested package may be in a subdirectory of the parent package directory. In this case, it is copied to the same location in the archive. +* Otherwise, it is copied to `.cargo/packages//` within the archive. + +`Cargo.toml` files for nested packages are rewritten in the same way as is already done for all packages, except that `path` dependencies which are nested dependencies are kept rather than stripped out or rejected as they currently are. Their path values may need to be rewritten to point to the nested paackages' new location in the archive. + +## **`crates.io`** + +`crates.io` will allow uploading of packages that contain `path` dependencies that were previously prohibited, as long as: + +* The dependency is a valid nested dependency as defined above. This includes that the the named package in fact exists in the `.crate` archive file, and has a valid `Cargo.toml` which declares `package.publish = "nested"`. +* The `path`s in all contained manifests must not contain any upward traversal outside of the parent package (`../../`) or other hazardous or non-portable components as determined to be necessary. + +The package index, and the `crates.io` user interface, does not explicitly represent nested packages; instead, nested packages' dependencies are flattened into the dependencies of the parent package when they are added to the index. This adequately reflects what can be expected when using the parent package. (Note that required dependencies of optional nested packages should become optional in the flattened form.) + +## `cargo build` and friends + +The `Cargo.lock` format will need to be modified to handle entries for nested packages differently, as `path` dependencies are currently not allowed to introduce multiple packages with the same name, which could happen though different packages' nested packages. This modification could consist of omitting them entirely and using the same flattened dependency graph as the `crates.io` index will use (which follows the logic that they are not truly versioned separate from their parent), or giving them some namespacing scheme; I leave this choice up to the Cargo team's judgement. + +Some new build message formatting would ideally be added; currently, `path` dependencies' full paths are always printed in progress messages, but they would be long noise here (`/home/alice/.cargo/registry/src/index.crates.io-6f17d22bba15001f/...`). Perhaps progress for sub-packages could look something like “`Compiling foo/macros v0.1.0`”, or “`Compiling foo v0.1.0 (crate macros)`”. + +## Documentation of nested packages' crates + +Nested packages' library crates should not be documented as full crates in `rustdoc` generated documentation, since they are an implementation detail of the parent package and their names are not unique among the set of all dependencies of the current build (e.g. many different packages could have nested libraries called just `macros`). I think `rustdoc` should use its [`doc(inline)`](https://doc.rust-lang.org/rustdoc/write-documentation/the-doc-attribute.html#inline-and-no_inline) support so that any re-exported items are documented in the parent package's crates' documentation. I believe this will require a new command-line option to `rustdoc` to tell it to treat a certain dependency crate as if it were a private module (which already triggers documentation inlining automatically). + +However, if this is not (yet) done, the documentation will still be usable; just with more implementation details visible, and a chance of name collision (which is already possible). + +## Workspaces + +The presence or absence of a `[workspace]` has no effect on nested packages, just as it has no effect on existing package publication. Nested packages may use workspace inheritance when they are workspace members. Nested packages may be in different workspaces than their parent package. # Drawbacks [drawbacks]: #drawbacks @@ -265,3 +289,7 @@ This RFC does not propose implementing a dependency declared as `{ git = "...", [artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 [#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 [RFC 2224]: https://github.com/rust-lang/rfcs/pull/2224 + +## Testing + +A noteworthy benefit of nesting over separately-published packages is that the entire package can be verified to build outside its development repository/workspace by running `cargo publish --dry-run` or `cargo package`. It might be interesting to add a flag which does not just build the package, but also test it; while this is not at all related to nested packages *per se*, it might be a particular benefit to the kind of large project which currently uses multiple packages. \ No newline at end of file From b4923aca0dae16f9dfc1c4b7d5a42e6f19a33a9a Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 5 Feb 2024 14:06:37 -0800 Subject: [PATCH 19/34] Expand alternatives and move inline-crates discussion there. --- text/0000-nested-publish.md | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index ee11c250d7b..882074b9138 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -198,15 +198,24 @@ The reason for doing anything at all in this area is that publishing multiple pa * When packages are implementation details, it makes a permanent mark on the `crates.io` registry even if the implementation of the parent package stops needing that particular subdivision. By allowing sub-packages we can allow package authors to create whatever sub-packages they imagine might be useful, and delete them in later versions with no consequences. * It is possible to depend on a published package that is intended as an implementation detail. Ideally, library authors would document this clearly and library users would obey the documentation, but that doesn't always happen. By allowing nested packages, we introduce a simple “visibility” system that is useful in the same way that `pub` and `pub(crate)` are useful within Rust crates. -The alternative to nested packages that I have heard of as a possibility would be to support multiple library targets per package. That would be arguably cleaner, but has these disadvantages: +## Alternatives to nested packages -* It would require new manifest syntax, not just for declaring the multiple libraries, but for referring to them, and for making per-target dependencies (e.g. only a proc-macro lib should depend on `proc-macro2`+`quote`+`syn`, not the rest of the libraries in the package). -* It would require many new mechanisms in Cargo. -* It might have unforeseen problems; by contrast, nested packages are compiled exactly the same way `path` dependencies currently are, and the only new element is the ability to publish them, so the risk of surprises is lower. +* Support multiple library targets per package. That would be arguably cleaner, but has these disadvantages: -Also, nested packages enables nesting *anything* that Cargo packages can express now and in the future; it is composable with other Cargo functionality. + * It would require new manifest syntax, not just for declaring the multiple libraries, but for referring to them, and for making per-target dependencies (e.g. only a proc-macro lib should depend on `proc-macro2`+`quote`+`syn`, not the rest of the libraries in the package). + * It would require many new mechanisms in Cargo. + * It might have unforeseen problems; by contrast, nested packages are compiled exactly the same way `path` dependencies currently are, and the only new element is the ability to publish them, so the risk of surprises is lower. -We could also do nothing, except for warning the authors of paired macro crates that they should use exact version dependencies. The consequence of this will be continued hassle for developers; it might even be that useful proc-macro features might not be written simply because the author does not want to manage a second package. + Also, nested packages enables nesting *anything* that Cargo packages can express now and in the future; it is composable with other Cargo functionality. + +* [Extend the language and `rustc` to support “inline crates”][inline-crates]; support triggering the compilation of child crates declared within the source code of the parent. This would be extremely convenient for proc-macros or when simply wants to cause fully parallel and independently-cached compilation of subsets of their code. However: + + * It does not permit specifying different dependencies for each crate (includes false dependency edges). + * The compilations cannot be started until the dependent crate has gotten past the macro expansion phase to discover crate declarations. + * Does not naturally contain a way to define binary crates. + * I expect it would also require significant changes to the Rust language and `rustc`, reaching beyond just “spawn another `rustc`” and including problems like computing the implied dependencies among inline crates that depend on other inline crates (unless inline crates are only allowed to depend on external dependencies and their own child inline crates, which is likely undesirable because it prohibits establishing common core vocabulary among a set of crates to be compiled in parallel). + +* Do nothing, except for warning the authors of paired macro crates that they should use exact version dependencies. The consequence of this will be continued hassle for developers; it might even be that useful proc-macro features might not be written simply because the author does not want to manage a second package. ## Details within this proposal @@ -232,12 +241,10 @@ There are several ways we could mark packages for nested publishing, rather than [prior-art]: #prior-art * Postponed [RFC 2224] (2017) is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. - This RFC is more detailed and addresses the questions that were raised in discussion of 2224. -* Blog post [Inline crates, by Yoshua Wuyts (2022)](https://blog.yoshuawuyts.com/inline-crates/) proposes that additional library crates can be declared using Rust syntax in the manner `crate foo;` or `crate foo {}`, like modules. - - Compared to this proposal, its advantage is that it requires no new Cargo concepts and fits right in to the source-code-traversal-driven nature of existing ways to define Rust package and crate contents. However, it forces additional edges in the dependency graph (since the dependent's compilation must be started first to discover the inline crate, and there is no place to declare different dependencies for different crates). I expect it would also require significant changes to the Rust language and `rustc`, reaching beyond just “spawn another `rustc`” and including problems like computing the implied dependencies among inline crates that depend on other inline crates (unless inline crates are only allowed to depend on external dependencies and their own child inline crates, which is likely undesirable because it prohibits establishing common core vocabulary among a set of crates to be compiled in parallel). +* Blog post [Inline crates, by Yoshua Wuyts (2022)][inline-crates] proposes that additional library crates can be declared using Rust syntax in the manner `crate foo;` or `crate foo {}`, like modules. + This is discussed above in the alternatives section. * Blog post [Rust 2030 Christmas list: Subcrate dependencies, by Olivier Faure (2023)](https://poignardazur.github.io/2023/01/24/subcrates/) proposes a mechanism of declaring nested dependencies similar to this RFC, but instead of embedding the files in one package, the “subcrates” are packaged separately on crates.io, but published as a single command and are not usable by other packages. Thus, it is similar to a combination of the also-desired features of “publish an entire workspace” and “namespacing on crates.io”, plus the subcrates being private on crates.io. @@ -286,10 +293,11 @@ This RFC does not propose implementing a dependency declared as `{ git = "...", * Exactly what set of files should be copied? * It would become possible to depend on (and thus copy during publication) a nested package someone else wrote for their own packages' use, which creates hazards for versioning and for non-compliance with source code licenses; while these are already possible, now Cargo would be doing it invisibly for you, which seems risky. +## Testing + +A noteworthy benefit of nesting over separately-published packages is that the entire package can be verified to build outside its development repository/workspace by running `cargo publish --dry-run` or `cargo package`. It might be interesting to add a flag which does not just build the package, but also test it; while this is not at all related to nested packages *per se*, it might be a particular benefit to the kind of large project which currently uses multiple packages. + [artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 [#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 [RFC 2224]: https://github.com/rust-lang/rfcs/pull/2224 - -## Testing - -A noteworthy benefit of nesting over separately-published packages is that the entire package can be verified to build outside its development repository/workspace by running `cargo publish --dry-run` or `cargo package`. It might be interesting to add a flag which does not just build the package, but also test it; while this is not at all related to nested packages *per se*, it might be a particular benefit to the kind of large project which currently uses multiple packages. \ No newline at end of file +[inline-crates]: https://blog.yoshuawuyts.com/inline-crates/ From 68ad6341c2370396ea09da929793dc4bc09e4d79 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sat, 10 Feb 2024 21:22:03 -0800 Subject: [PATCH 20/34] Move license and version ideas. --- text/0000-nested-publish.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 882074b9138..c9ac394bc0e 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -127,11 +127,9 @@ When a nested dependency is present (making its referent be a nested package), t * The nested package must have `package.publish = "nested"`; both `false` and `true` are errors. -* If the nested package specifies `package.license`, the parent package's (not necessarily the dependent's) license expression must comply with the nested package's. This check is done solely in terms of the operators in the license expression. For example, if two nested packages contain licenses of `MIT` and `BSD-3-Clause`, then the parent package's expression must be `MIT AND BSD-3-Clause` or similar. +* If the nested package specifies `package.license`, its value must be identical to the parent package's. - This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing independent from the parent package's. - -* We might want to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. (`package.version` is already optional as of Cargo 1.75, but this is currently equivalent to `version = "0.0.0"`.) + This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing. ## **`cargo package` & `cargo publish`** @@ -257,7 +255,7 @@ I am not aware of other package systems that have a similar concept, but I am no # Unresolved questions [unresolved-questions]: #unresolved-questions -None currently known. +* We could choose to explicitly prohibit nested packages from specifying a `package.version`, to avoid giving the misleading impression that it means anything. This would be notably stricter than the current meaning of absent `package.version` as of Cargo 1.75, which is that it is completely equivalent to `version = "0.0.0"`. It would also prohibit having a package that is both nested and published to a registry, if that is desired. # Future possibilities [future-possibilities]: #future-possibilities @@ -297,6 +295,10 @@ This RFC does not propose implementing a dependency declared as `{ git = "...", A noteworthy benefit of nesting over separately-published packages is that the entire package can be verified to build outside its development repository/workspace by running `cargo publish --dry-run` or `cargo package`. It might be interesting to add a flag which does not just build the package, but also test it; while this is not at all related to nested packages *per se*, it might be a particular benefit to the kind of large project which currently uses multiple packages. +## License compatibility checking + +The rule about nested packages' `package.license` could be made more lenient, only requiring the parent package's (not necessarily the dependent's) license expression to comply with the nested package's, in terms of the operators in the license expression. For example, if two nested packages contain licenses of `MIT` and `BSD-3-Clause`, then the parent package's expression could be `MIT AND BSD-3-Clause` or similar. + [artifact dependencies]: https://github.com/rust-lang/rfcs/pull/3028 [#3243 packages as namespaces]: https://github.com/rust-lang/rfcs/pull/3243 [RFC 2224]: https://github.com/rust-lang/rfcs/pull/2224 From 33d1c6e69b71848f864722789165f5c16050ef03 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sat, 10 Feb 2024 21:26:53 -0800 Subject: [PATCH 21/34] Specify that package names must be unique. --- text/0000-nested-publish.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index c9ac394bc0e..b805c43926d 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -131,6 +131,8 @@ When a nested dependency is present (making its referent be a nested package), t This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing. +It is an error for any two of the packages in the transitive closure of nested dependencies (including the parent package) to share a package name. This is validated by all Cargo operations that would generate or read a lockfile. + ## **`cargo package` & `cargo publish`** If `cargo package` or `cargo package` is directed to operate on a package whose manifest specifies `package.publish = "nested"`, that is an error. (Users wishing to make crates in a nested package public should either write `package.publish = true` or should re-export their contents from another package.) From 680709cccd34894226ca393d78da7239736ce155 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sat, 10 Feb 2024 21:27:13 -0800 Subject: [PATCH 22/34] Typo --- text/0000-nested-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index b805c43926d..1839aaa9949 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -142,7 +142,7 @@ When a valid parent package is packaged, each of its transitive nested dependenc * The nested package may be in a subdirectory of the parent package directory. In this case, it is copied to the same location in the archive. * Otherwise, it is copied to `.cargo/packages//` within the archive. -`Cargo.toml` files for nested packages are rewritten in the same way as is already done for all packages, except that `path` dependencies which are nested dependencies are kept rather than stripped out or rejected as they currently are. Their path values may need to be rewritten to point to the nested paackages' new location in the archive. +`Cargo.toml` files for nested packages are rewritten in the same way as is already done for all packages, except that `path` dependencies which are nested dependencies are kept rather than stripped out or rejected as they currently are. Their path values may need to be rewritten to point to the nested packages' new location in the archive. ## **`crates.io`** From 2af4921c0df8664c9d3a58c8ab30132f0d9f13e3 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sat, 10 Feb 2024 21:34:08 -0800 Subject: [PATCH 23/34] Mention feature flattening. --- text/0000-nested-publish.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 1839aaa9949..6e6b3d4f072 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -151,7 +151,13 @@ When a valid parent package is packaged, each of its transitive nested dependenc * The dependency is a valid nested dependency as defined above. This includes that the the named package in fact exists in the `.crate` archive file, and has a valid `Cargo.toml` which declares `package.publish = "nested"`. * The `path`s in all contained manifests must not contain any upward traversal outside of the parent package (`../../`) or other hazardous or non-portable components as determined to be necessary. -The package index, and the `crates.io` user interface, does not explicitly represent nested packages; instead, nested packages' dependencies are flattened into the dependencies of the parent package when they are added to the index. This adequately reflects what can be expected when using the parent package. (Note that required dependencies of optional nested packages should become optional in the flattened form.) +The package index, and the `crates.io` user interface, does not explicitly represent nested packages; the package is presented as if it were a single package: + +* Nested packages' dependencies are flattened into the dependencies of the parent package. +* The packages selected by nested packages' features must be rewritten in terms of the parent package's features. +* Note that optional dependencies might become required if the parent package always enables the relevant feature, and required dependencies might become optional if the dependency on the nested package is optional. + +All together, this will reflect what can be expected when using the parent package, without revealing, or needing to represent, the nested package implementation details of the parent package. ## `cargo build` and friends From 0deba104342f4976f79568d9ed72aa87c919ef70 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sat, 10 Feb 2024 21:36:31 -0800 Subject: [PATCH 24/34] Discuss workspace inheritance. --- text/0000-nested-publish.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 6e6b3d4f072..915209d3d63 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -119,9 +119,10 @@ Then you can `cargo publish` from within the parent directory `foo/`, and this w Two new possible values are added to the manifest. -* The `package.publish` field allows `"nested"` as a value, in addition to existing `false` and `true`. This value affects `cargo publish` and nested dependencies as described below. +* The `package.publish` or `workspace.package.publish` field allows `"nested"` as a value, in addition to existing `false` and `true`. This value affects `cargo publish` and nested dependencies as described below. * The `dependencies.*.publish` field is newly defined, with the only currently allowed value being `"nested"`, to declare that that dependency is a nested dependency. (If desired, `publish = false` could be also be permitted to explicitly document an intent not to nest; this would be identical to the status quo and to omitting the key.) * It is an error if a nested dependency does not have a `path` field, or if it has a `version`, `git`, or any other package source field, unless future work defines a meaning for that combination. + * Workspace inheritance is not permitted; `workspace.dependencies.*.publish` is an error at `cargo package`/`cargo publish` time. (Builds should ignore the field, for forward compatibility.) When a nested dependency is present (making its referent be a nested package), the following additional requirements apply: From b9999769dac40ac76a14b274513279bd1709a357 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sat, 10 Feb 2024 22:01:17 -0800 Subject: [PATCH 25/34] Move `dependencies.*.publish = false` to future possibilities. --- text/0000-nested-publish.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 915209d3d63..66ab2c71415 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -120,7 +120,7 @@ Then you can `cargo publish` from within the parent directory `foo/`, and this w Two new possible values are added to the manifest. * The `package.publish` or `workspace.package.publish` field allows `"nested"` as a value, in addition to existing `false` and `true`. This value affects `cargo publish` and nested dependencies as described below. -* The `dependencies.*.publish` field is newly defined, with the only currently allowed value being `"nested"`, to declare that that dependency is a nested dependency. (If desired, `publish = false` could be also be permitted to explicitly document an intent not to nest; this would be identical to the status quo and to omitting the key.) +* The `dependencies.*.publish` field is newly defined, with the only currently allowed value being `"nested"`, to declare that that dependency is a nested dependency. * It is an error if a nested dependency does not have a `path` field, or if it has a `version`, `git`, or any other package source field, unless future work defines a meaning for that combination. * Workspace inheritance is not permitted; `workspace.dependencies.*.publish` is an error at `cargo package`/`cargo publish` time. (Builds should ignore the field, for forward compatibility.) @@ -293,6 +293,14 @@ Since nested packages are versioned as a unit, we could relax the trait coherenc This would be particularly useful when implementing traits from large optional libraries; for example, package `foo` with subpackages `foo_core` and `foo_tokio` could have `foo_tokio` write `impl tokio::io::AsyncRead for foo_core::DataSource`. This would improve the dependency graph compared to `foo_core` having a dependency on `tokio` (which is the only way to do this currently), though not have the maximum possible benefit unless we also added public library targets as above, since the package as a whole still only exports one library and thus one dependency graph node. +## Additional dependency manipulation when publishing + +The `dependencies.*.publish` field could be given more possible values to give more control over the effects of publishing. + +* For example, currently it is an error to publish a package with a `path`-only or `git`-only dependency. `dependencies.*.publish = false` could mean to instead strip out that dependency. This might be suitable for unpublished dependencies that are only used under special testing conditions that aren't `cfg(test)` and therefore can't just be `[dev-dependencies]`, such as a feature of a library depended on by the test code, or a configuration that enables special assertions that need a support library like [`loom`](https://docs.rs/loom/)). + +* Or, there could be a value which explicitly selects the non-nested status quo behavior. + ## Git dependencies This RFC does not propose implementing a dependency declared as `{ git = "...", publish = "nested" }`. The obvious meaning is to copy the files from the target Git repository into the package, similarly to a Git submodule checkout. However, there might be things that need further consideration: From 3edb308e2f7118e12c51b4e544fe126774349215 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sun, 11 Feb 2024 11:19:49 -0800 Subject: [PATCH 26/34] Replace `package.publish = "nested"` with `package.publish.nested = true`. --- text/0000-nested-publish.md | 51 ++++++++++++++++++++++++++++--------- 1 file changed, 39 insertions(+), 12 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 66ab2c71415..ed830125dee 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -52,12 +52,12 @@ By default (and always, prior to this RFC's implementation): * If your package contains any non-`dev` dependencies which do not give a `version = "..."`, it cannot be published to `crates.io`. * If your package contains `[dev-dependencies]` which do not give a `version = "..."`, they are stripped out on publication. -You can change this default in your manifests. First, in the manifest (`Cargo.toml`) of a sub-package, add `publish = "nested"`: +You can change this default in your manifests. First, in the manifest (`Cargo.toml`) of a sub-package, add `publish.nested = true`: ```toml [package] name = "foo-macros" -publish = "nested" +publish.nested = true ``` Then, in the manifest of the parent package, declare the dependency as `publish = "nested"`: @@ -104,7 +104,7 @@ The sub-package manifest `foo/macros/Cargo.toml`: name = "macros" # this name need not be claimed on crates.io # version = "0.0.0" # version number is not used and may be omitted edition = "2021" -publish = "nested" # new syntax +publish.nested = true # new syntax [lib] proc-macro = true @@ -119,15 +119,40 @@ Then you can `cargo publish` from within the parent directory `foo/`, and this w Two new possible values are added to the manifest. -* The `package.publish` or `workspace.package.publish` field allows `"nested"` as a value, in addition to existing `false` and `true`. This value affects `cargo publish` and nested dependencies as described below. +* Packages may be specified as eligible for nested publishing using the `package.publish.nested` (or `workspace.package.publish.nested`) field, which takes the values `true` or `false`. `true` permits the package to be a nested package as defined in this RFC, affecting `cargo publish` and builds as discussed below, and `false` prohibits it as is the status quo. + + ```toml + [package] + # This package may be nested but may not be `cargo publish`ed by itself + publish.nested = true + ``` + + + To allow both regular publishing and nested publishing, the currently-allowed values for `publish` (`publish = true` or `publish = [registries...]`) may be specified under the key `package.publish.registries`. Examples:. + + ```toml + [package] + publish.registries = ["crates.io"] + publish.nested = true + ``` + + ```toml + [package] + publish.registries = true + publish.nested = true + ``` + + If `publish.registries` is absent and `publish.nested` is present, then `publish.registries` defaults to `false`, regardless of the value of `publish.nested`. + + Note: This dual-publishing-mode functionality is permitted mainly to keep the functionality composable/orthogonal. We hope that in most cases, packages are either published nested exactly once, or to a registry alone, to avoid duplicating code in the registry and compiling it redundantly. + * The `dependencies.*.publish` field is newly defined, with the only currently allowed value being `"nested"`, to declare that that dependency is a nested dependency. * It is an error if a nested dependency does not have a `path` field, or if it has a `version`, `git`, or any other package source field, unless future work defines a meaning for that combination. * Workspace inheritance is not permitted; `workspace.dependencies.*.publish` is an error at `cargo package`/`cargo publish` time. (Builds should ignore the field, for forward compatibility.) When a nested dependency is present (making its referent be a nested package), the following additional requirements apply: -* The nested package must have `package.publish = "nested"`; both `false` and `true` are errors. - +* The nested package must have `package.publish.nested = true`. * If the nested package specifies `package.license`, its value must be identical to the parent package's. This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing. @@ -136,8 +161,6 @@ It is an error for any two of the packages in the transitive closure of nested d ## **`cargo package` & `cargo publish`** -If `cargo package` or `cargo package` is directed to operate on a package whose manifest specifies `package.publish = "nested"`, that is an error. (Users wishing to make crates in a nested package public should either write `package.publish = true` or should re-export their contents from another package.) - When a valid parent package is packaged, each of its transitive nested dependencies must be included in the `.crate` archive file. This has two sub-cases: * The nested package may be in a subdirectory of the parent package directory. In this case, it is copied to the same location in the archive. @@ -149,7 +172,7 @@ When a valid parent package is packaged, each of its transitive nested dependenc `crates.io` will allow uploading of packages that contain `path` dependencies that were previously prohibited, as long as: -* The dependency is a valid nested dependency as defined above. This includes that the the named package in fact exists in the `.crate` archive file, and has a valid `Cargo.toml` which declares `package.publish = "nested"`. +* The dependency is a valid nested dependency as defined above. This includes that the the named package in fact exists in the `.crate` archive file, and has a valid `Cargo.toml` which declares `package.publish.nested = true`. * The `path`s in all contained manifests must not contain any upward traversal outside of the parent package (`../../`) or other hazardous or non-portable components as determined to be necessary. The package index, and the `crates.io` user interface, does not explicitly represent nested packages; the package is presented as if it were a single package: @@ -228,9 +251,13 @@ The reason for doing anything at all in this area is that publishing multiple pa There are several ways we could mark packages for nested publishing, rather than using the `package.publish` and `dependencies.*.publish` keys: -* Instead of declaring each _dependency_ as being nested, we could only use `package.publish = "nested"` to make the determination. This would be problematic when a workspace has a root package, because that root package cannot avoid publishing all its `nested` workspace members except by writing `include`/`exclude` rules. +* Instead of declaring each _dependency_ as being nested, we could only use `package.publish.nested = true` to make the determination. This could work two ways: + + * Dependencies are followed, but don't need to be specially marked as nested. I consider this undesirable because it could lead to unintended code duplication, if someone adds a dependency during development without thinking about its effect on publishing. + + * Dependencies are disregarded and only directory nesting affects inclusion. This would be a simpler model, keeping packaging closer to "just make an archive of this directory tree", but it means that a workspace with a root package cannot avoid publishing all its `nested` workspace members except by writing `include`/`exclude` rules, there would be no way to specify whether `dev-dependencies` should be nested or stripped, and it does not support nesting a small private dependency in several different published packages. -* Instead of introducing `package.publish = "nested"`, we could only require that dependencies be declared as nested. The disadvantages of this are: +* Instead of introducing `package.publish.nested = true`, we could only require that dependencies be declared as nested. The disadvantages of this are: * May unintentionally duplicate published code between a standalone published package and a nested package * Does not make both ends of the relationship explicit to readers of the code. @@ -247,7 +274,7 @@ There are several ways we could mark packages for nested publishing, rather than # Prior art [prior-art]: #prior-art -* Postponed [RFC 2224] (2017) is broadly similar to this RFC, and proposed using `publish = false` to mean what we mean by `publish = "nested"`. +* Postponed [RFC 2224] (2017) is broadly similar to this RFC, and proposed using `package.publish = false` to mean what we mean by `package.publish.nested = true`. This RFC is more detailed and addresses the questions that were raised in discussion of 2224. * Blog post [Inline crates, by Yoshua Wuyts (2022)][inline-crates] proposes that additional library crates can be declared using Rust syntax in the manner `crate foo;` or `crate foo {}`, like modules. From dd90f205afccc185eeadd77157f4ffbd889f4a5a Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Sun, 11 Feb 2024 11:24:11 -0800 Subject: [PATCH 27/34] Refine explanation of `package.publish` being a table. --- text/0000-nested-publish.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index ed830125dee..dbb498f2079 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -119,7 +119,7 @@ Then you can `cargo publish` from within the parent directory `foo/`, and this w Two new possible values are added to the manifest. -* Packages may be specified as eligible for nested publishing using the `package.publish.nested` (or `workspace.package.publish.nested`) field, which takes the values `true` or `false`. `true` permits the package to be a nested package as defined in this RFC, affecting `cargo publish` and builds as discussed below, and `false` prohibits it as is the status quo. +* Packages may be specified as eligible for nested publishing using the `package.publish.nested` (or `workspace.package.publish.nested`) field, which takes the values `true` or `false` and defaults to `false`. `true` permits the package to be a nested package as defined in this RFC, affecting `cargo publish` and builds as discussed below, and `false` prohibits it as is the status quo. ```toml [package] @@ -142,7 +142,7 @@ Two new possible values are added to the manifest. publish.nested = true ``` - If `publish.registries` is absent and `publish.nested` is present, then `publish.registries` defaults to `false`, regardless of the value of `publish.nested`. + If `package.publish` is a table, then `package.publish.registries` defaults to `false`, regardless of the value or presence of `package.publish.nested`. Note: This dual-publishing-mode functionality is permitted mainly to keep the functionality composable/orthogonal. We hope that in most cases, packages are either published nested exactly once, or to a registry alone, to avoid duplicating code in the registry and compiling it redundantly. From 07b058dc8328a08df8ed546ff95b2fda870a2919 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 11 Mar 2024 10:32:25 -0700 Subject: [PATCH 28/34] Rephrase name conflict rule to avoid "transitive closure". --- text/0000-nested-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index dbb498f2079..6ddb68c6804 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -157,7 +157,7 @@ When a nested dependency is present (making its referent be a nested package), t This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing. -It is an error for any two of the packages in the transitive closure of nested dependencies (including the parent package) to share a package name. This is validated by all Cargo operations that would generate or read a lockfile. +It is an error for a nested package to have the same package name as the parent package or any other nested package with the same parent package. This is validated by all Cargo operations that would generate or read a lockfile. ## **`cargo package` & `cargo publish`** From 2c220eed44f3d30e85217b6b9fa92d73f8c7ac18 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 11 Mar 2024 10:36:19 -0700 Subject: [PATCH 29/34] Always error on `workspace.dependencies.*.publish`. --- text/0000-nested-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 6ddb68c6804..5a0adb3c1bc 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -148,7 +148,7 @@ Two new possible values are added to the manifest. * The `dependencies.*.publish` field is newly defined, with the only currently allowed value being `"nested"`, to declare that that dependency is a nested dependency. * It is an error if a nested dependency does not have a `path` field, or if it has a `version`, `git`, or any other package source field, unless future work defines a meaning for that combination. - * Workspace inheritance is not permitted; `workspace.dependencies.*.publish` is an error at `cargo package`/`cargo publish` time. (Builds should ignore the field, for forward compatibility.) + * Workspace inheritance is not permitted; the presence of `workspace.dependencies.*.publish` is an error. When a nested dependency is present (making its referent be a nested package), the following additional requirements apply: From 4bc77cbde5ac3473aceee2b80b92ced545ba666b Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Mon, 11 Mar 2024 10:49:40 -0700 Subject: [PATCH 30/34] Rewrite feature flattening section. --- text/0000-nested-publish.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 5a0adb3c1bc..0c0a25aceea 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -177,9 +177,12 @@ When a valid parent package is packaged, each of its transitive nested dependenc The package index, and the `crates.io` user interface, does not explicitly represent nested packages; the package is presented as if it were a single package: -* Nested packages' dependencies are flattened into the dependencies of the parent package. -* The packages selected by nested packages' features must be rewritten in terms of the parent package's features. -* Note that optional dependencies might become required if the parent package always enables the relevant feature, and required dependencies might become optional if the dependency on the nested package is optional. +* Nested packages’ dependencies are flattened into the listed dependencies of the parent package. +* Each optional dependency of a nested package so flattened must accurately represent the conditions for its activation in terms of the parent package's features (or lack thereof). To illustrate, note that the following cases might occur: + * Optional dependencies of a nested package can become required, if the parent package always enables the relevant feature of the nested package. + * Required dependencies of a nested package can become optional, if the dependency on the nested package which has that dependency is optional. + * Optional dependencies that stay optional must be listed as activated by the relevant feature(s) of the parent package; the feature names of nested packages never appear in the index. + All together, this will reflect what can be expected when using the parent package, without revealing, or needing to represent, the nested package implementation details of the parent package. From e470314f7ce81d70ef14c2d2388451fa2f556f71 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Wed, 13 Mar 2024 07:55:58 -0700 Subject: [PATCH 31/34] Rationale for name uniqueness. --- text/0000-nested-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 0c0a25aceea..dda5dce71c4 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -157,7 +157,7 @@ When a nested dependency is present (making its referent be a nested package), t This check is intended only to prevent accidents (such as vendoring a third-party package without considering the implications of redistributing it). It is always valid to omit `package.license` from the nested package, thus making no machine-readable claims about its licensing. -It is an error for a nested package to have the same package name as the parent package or any other nested package with the same parent package. This is validated by all Cargo operations that would generate or read a lockfile. +It is an error for a nested package to have the same package name as the parent package or any other nested package with the same parent package. This is validated by all Cargo operations that would generate or read a lockfile. Rationale: This should ensure that whenever a nested package must be named, such as in an `.crate` archive, potentially in lock files, and potentially in Cargo user interface, the pair of (parent package name, nested package name) is sufficient to uniquely identify the package. ## **`cargo package` & `cargo publish`** From 2a5474ed29f4ed274f2d54376975af20f0135242 Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Wed, 13 Mar 2024 08:16:35 -0700 Subject: [PATCH 32/34] Polishing. --- text/0000-nested-publish.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index dda5dce71c4..2b995961b65 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -163,10 +163,10 @@ It is an error for a nested package to have the same package name as the parent When a valid parent package is packaged, each of its transitive nested dependencies must be included in the `.crate` archive file. This has two sub-cases: -* The nested package may be in a subdirectory of the parent package directory. In this case, it is copied to the same location in the archive. +* The nested package may be in a subdirectory of the parent package directory. In this case, it is copied to the same location in the archive, just like other packaged files. * Otherwise, it is copied to `.cargo/packages//` within the archive. -`Cargo.toml` files for nested packages are rewritten in the same way as is already done for all packages, except that `path` dependencies which are nested dependencies are kept rather than stripped out or rejected as they currently are. Their path values may need to be rewritten to point to the nested packages' new location in the archive. +`Cargo.toml` files for nested packages are rewritten in the same way as is already done for all packages, except that `path` dependencies which are nested dependencies are kept, rather than stripped out or rejected, as they currently are. Their `path` values may need to be rewritten to point to the nested packages' new location in the archive. ## **`crates.io`** @@ -175,7 +175,7 @@ When a valid parent package is packaged, each of its transitive nested dependenc * The dependency is a valid nested dependency as defined above. This includes that the the named package in fact exists in the `.crate` archive file, and has a valid `Cargo.toml` which declares `package.publish.nested = true`. * The `path`s in all contained manifests must not contain any upward traversal outside of the parent package (`../../`) or other hazardous or non-portable components as determined to be necessary. -The package index, and the `crates.io` user interface, does not explicitly represent nested packages; the package is presented as if it were a single package: +The package index, and the `crates.io` user interface, do not explicitly represent nested packages; the package is presented as if it were a single package: * Nested packages’ dependencies are flattened into the listed dependencies of the parent package. * Each optional dependency of a nested package so flattened must accurately represent the conditions for its activation in terms of the parent package's features (or lack thereof). To illustrate, note that the following cases might occur: From db9e7fa5cc62fb6d64e575814e8aa814f05cfd3d Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Wed, 13 Mar 2024 08:17:16 -0700 Subject: [PATCH 33/34] Update comparison with packages-as-namespaces given that that RFC has been accepted. --- text/0000-nested-publish.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 2b995961b65..8d9d271fb60 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -315,7 +315,12 @@ Allowing nested libraries to be named and used from outside the package would al This would allow library authors to avoid writing fragile and hard-to-test conditional compilation, and allow library users to avoid accidentally depending on a feature being enabled despite not having enabled it explicitly. It would also allow compiling the optional functionality and its dependencies with maximum parallelism, by not introducing a single `feature`-ful library crate which acts as a single node in the dependency graph. -However, it requires additional syntax and semantics, and these use cases might be better served by [#3243 packages as namespaces] or some other namespacing proposal, which would allow the libraries to be published independently. (I can also imagine a world in which both of these exist, and the library implementer can transparently use whichever publication strategy best serves their current needs.) +However, most of the same functionality will already be provided by accepted RFC [#3243 packages as namespaces]. The differences to library authors between that and this would be: + +* This RFC would allow the public nested library packages to depend on private nested library packages, allowing them to have internally shared items without making them public at all (not even `#[doc(hidden)]`). +* Packages-as-namespaces allows new versions of the namespaced packages to be published independently, and is thus more suited for “official plugins” that are loosely coupled and build on public API. + +Adding nested public libraries would have largely the same design considerations as nested public binaries, discussed in the previous section, and also need some way for a package to specify a dependency on some nested library. ## Additional privileges between crates From 4817861f08ac4dc05864941acbd460f83755c4dc Mon Sep 17 00:00:00 2001 From: Kevin Reid Date: Wed, 13 Mar 2024 08:29:42 -0700 Subject: [PATCH 34/34] Explicitly state that nested names are non-unique *outside* of the package. --- text/0000-nested-publish.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-nested-publish.md b/text/0000-nested-publish.md index 8d9d271fb60..1f3dc483750 100644 --- a/text/0000-nested-publish.md +++ b/text/0000-nested-publish.md @@ -159,6 +159,8 @@ When a nested dependency is present (making its referent be a nested package), t It is an error for a nested package to have the same package name as the parent package or any other nested package with the same parent package. This is validated by all Cargo operations that would generate or read a lockfile. Rationale: This should ensure that whenever a nested package must be named, such as in an `.crate` archive, potentially in lock files, and potentially in Cargo user interface, the pair of (parent package name, nested package name) is sufficient to uniquely identify the package. +However, it is allowed for a nested package to have the same name as a package that is _not_ a nested package with the same parent package (either because the latter has a different parent package or because it is not a nested package). This is appropriate because nested package names are an implementation detail of the package, and necessary to avoid different library packages from accidentally conflicting with each other by using the same nested package name. + ## **`cargo package` & `cargo publish`** When a valid parent package is packaged, each of its transitive nested dependencies must be included in the `.crate` archive file. This has two sub-cases: