Publishing `fs2-node` module sometimes fails

[armanbilge]

2021-07-13 13:59:21.591Z error [SonatypeClient] Failed: no-traversal-paths-in-archive-file, failureMessage:Unable to process file /co/fs2/fs2-node_sjs1_2.12/3.0-39-594f530/fs2-node_sjs1_2.12-3.0-39-594f530.jar: Truncated ZIP file - (SonatypeClient.scala:375)

https://github.com/typelevel/fs2/runs/3049635964#step:12:890
https://github.com/typelevel/fs2/runs/3057389245#step:12:880
https://github.com/typelevel/fs2/runs/3057590901#step:12:885

Strangely enough, publication succeeded on one occasion.
https://github.com/typelevel/fs2/runs/3051877761#step:12:911

@oyvindberg—sorry to bug you again—but we could use some help. Have you ever experienced this problem? I've followed the directions at https://scalablytyped.org/docs/library-developer. This is a module containing only ST-generated code. Thanks!

Update: cross-filed at https://issues.sonatype.org/browse/OSSRH-71155

[oyvindberg]

Hey again.

No, I have no idea. I published loads of this kind of code to bintray, but not much to maven central yet. In any case it's all fairly normal code once generated.

I also saw this error in one of the runs:

Failed: signature-staging, failureMessage:Invalid Signature: '/co/fs2/fs2-node_sjs1_2.12/3.0-36-fff5955/fs2-node_sjs1_2.12-3.0-36-fff5955.jar.asc' is not a valid signature for 'fs2-node_sjs1_2.12-3.0-36-fff5955.jar'.  - (SonatypeClient.scala:375)

[armanbilge]

@oyvindberg thanks for getting back to us so quickly! No worries, hopefully the folks over at sonatype can help with this ... I agree though, the source-generation pipeline seems straightforward. Are you aware of any other projects publishing ST-generated facades?

Yes, I did see that signature error too. I assume it was related to the no-traversal-paths-in-archive-file error that also occurred in that run.

[oyvindberg]

hmm, Github search brought up these https://github.com/search?p=1&q=ScalablyTypedConverterGenSourcePlugin&type=Code at least

[armanbilge]

Thanks! Unfortunately Sonatype didn't have any advice for us :(

[armanbilge]

I had a look through those search results. Most relevant was wiringbits/sjs-material-ui-facade by @AlexITC which publishes a massive ~100 mb ST-generated facade to Maven via GH actions. As far as I can tell this problem never appears in their logs. At a glance I don't notice anything special about their build configuration.

So, clearly it's possible, but maybe just flaky?

[AlexITC]

My library was using an older scalablytyped version, I just upgraded it and triggered the release to see how it goes.

The other difference is that my library is not supporting scala 2.12, I'd try 2.13 only and see if it gets a better result.

[AlexITC]

The job was successful: https://github.com/wiringbits/sjs-material-ui-facade/runs/3063500602

The main difference I see is that my library relies in sbt-ci-release while fs2 is handling all the steps manually, I'd look either in the Scala version or trying sbt-ci-release.

Good luck.

[armanbilge]

@AlexITC thank you very much for chiming in and running your release workflow, really useful datapoint for us.

[armanbilge]

Brief update: I did a bit of experimenting with this by attempting to publish to my own Sonatype group ID com.armanbilge. See the commits here: https://github.com/armanbilge/fs2/commits/series/armanbilge

tl;dr: by starting from the node module and adding stuff back in, I had a very good run where I eventually was able to publish all modules at once. Then, restoring the original build failed. Then, attempting to revert also failed. So, it's non-deterministic aka flaky :(

@mpilquist not sure where you want to go from here?

[armanbilge]

@oyvindberg Thanks to @mpilquist's efforts we think we may have got a handle on this, seems like it may have been some sort of weird parallel publishing thing going on? Hopefully the fix will stick.

I'm pinging you because while working on this, we had to restore Javadoc publishing (see log). I think sonatype/maven requires this, perhaps your ST recommendations were targeted more for bintray?

[AlexITC]

I was actually the one submitting this recommendation:

    sources in (Compile, doc) := Nil,
    publishArtifact in (Compile, packageDoc) := false

Later, I discovered that sonatype does require the docs artifact (even if empty), meaning that sources in (Compile, doc) := Nil, should be enough, in fact, that's how my library is published.

I'm yet to submit a PR to update the scalablytyped docs.