Fix unindent in doc comments

[GuillaumeGomez]

Fixes #70732

r? @jyn514

[Nemo157]

How come this doesn't break https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/issue-42760.rs, which was explicitly testing for the old behaviour?

[GuillaumeGomez]

This test only check that there is a title, not that the indent isn't broken as far as I can tell?

[Nemo157]

If the unindent pass doesn't strip the leading space (because of the #[doc] attribute without it), then the # Example heading shouldn't be parsed as a heading because of the leading space. Skimming the code now I wonder if it's because of the special casing of the first line of a fragment, but I can't really follow it to tell what it's trying to do.

[GuillaumeGomez]

@Nemo157 Well, for me it seems like it should work because sugared doc comments should be handled a bit differently than the others (they need to have one extra whitespace after all).

@jyn514 Thanks to your remarks, I greatly improved the code. :)

[Nemo157]

@Nemo157 Well, for me it seems like it should work because sugared doc comments should be handled a bit differently than the others (they need to have one extra whitespace after all).

But they don't, you can have

///# Documentation
///
///Without leading whitespace

just fine, it's just convention to have a space between the doc-comment marker and the docs.

[GuillaumeGomez]

Well, glad I treated that correctly then. Thanks for the extra information! :)

[GuillaumeGomez]

Hum, actually I might need to make a small update in the code. :3

[jyn514]

We're discussing this on discord, but either way the following test cases we came up with should be added:

///no whitespace
#[doc = " lol"]

///    4 whitespaces!
#[doc = "something"]

(https://github.com/clap-rs/clap/blob/625c201f657f091d0eb627c84db3e68eec56d507/src/build/arg/mod.rs#L1901)

    #[cfg_attr(not(unix), doc = " ```ignore")]
    #[cfg_attr(unix, doc = " ```rust")]
    /// fn has_ampersand(v: &OsStr) -> Result<(), String> {
    ///     if v.as_bytes().iter().any(|b| *b == b'&') { return Ok(()); }
    ///     Err(String::from("The value did not contain the required & sigil"))
    /// }

Additionally, note that the fixed version of #70732 in the test case uses 4 spaces instead of 5, which differs from the original bug report. IMO this seems reasonable to match #42760.

There should also be test cases for doc(include = ...).

[GuillaumeGomez]

So I changed a bit how the unindent is handled. For example:

#[doc = "hello
         second"]

Before, we didn't take into account the first line that in this case, we could align the doc based on the next lines indent. It was very misleading and not that great as I discovered while updating this PR. I always assumed it was supposed to be:

#[doc = "hello
second"]

I definitely think it's better this way.

Apart from that change, I also added comments and extended tests to add doc includes.

[bors]

📌 Commit 87f2897 has been approved by jyn514

[bors]

⌛ Testing commit 87f2897 with merge 1d5057d91ad6ad54b899a9b93b5b6b8f1fe73ffa...

[bors]

💔 Test failed - checks-actions

[GuillaumeGomez]

Fetch error, let's retry!

@bors: retry

[bors]

⌛ Testing commit 87f2897 with merge 83a3d46759cbc915445e3f07addb85a82085b2c7...

[jyn514]

If this fails because of another zlib error, let's avoid retrying, @ehuss is working on a fix for the underlying issue in https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/pinging.20for.20zlib.20error.

[bors]

💔 Test failed - checks-actions

[GuillaumeGomez]

Seeing that other PRs seem to pass, let's give it another try.

@bors: retry