Can't seem to upload any assets

[rwols]

Hey, thanks for this nice program. I can't seem to upload my assets. The command I use is

githubrelease asset rwols/Clara upload 0.2.0 path/to/some/zipfile.zip

It exits with a broken pipe error:

Traceback (most recent call last):
  File "/usr/local/bin/githubrelease", line 11, in <module>
    sys.exit(main())
  File "/Library/Python/2.7/site-packages/github_release.py", line 809, in main
    sys.argv[2:], "%s %s" % (PROG, args.command))
  File "/Library/Python/2.7/site-packages/github_release.py", line 653, in with_error_handling
    return func(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/github_release.py", line 757, in gh_asset
    return _gh_parse_arguments(ASSET_COMMANDS, argv, prog)
  File "/Library/Python/2.7/site-packages/github_release.py", line 721, in _gh_parse_arguments
    for arg_name in func.description["params"]
  File "/Library/Python/2.7/site-packages/github_release.py", line 417, in gh_asset_upload
    data=f.read())
  File "/Library/Python/2.7/site-packages/github_release.py", line 28, in _request
    return request(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))

The zipfile is rather large, around 8MB. Maybe that has something to do with it? Note that I can set the body of the release (and create releases).

[jcfr]

Thanks for the report 👍

Could you confirm you are using MacOSX ? Additionally which version of python 2.7 is installed on the system ?

[rwols]

Yes, using OSX 10.12.3, python version 2.7.10 (the system-provided version, not via homebrew). githubrelease was installed via pip.

[jcfr]

To rule out issue with the network connection, i would suggest to attempt the upload using curl.

Here are the steps:

1. Copy script below into a text editor
2. Update FILE and GITHUB_TOKEN and variables
3. Open a terminal, cd /path/with/test/archive/ # Contains test.zip
4. Copy script into terminal

FILE=test.zip
GITHUB_TOKEN=YOURTOKEN
PROJECT=rwols/Clara
RELEASE_ID=5635808  # This corresponds to release 0.2.0 on rwols/Clara

curl https://uploads.github.com/repos/$PROJECT/releases/$RELEASE_ID/assets?name=$FILE --request 'POST' \
--header "Authorization: token $GITHUB_TOKEN" \
--header "Content-Disposition: attachment; filename=\"$FILE\"" \
--header "Content-Type: application/zip; charset=binary" \
--data-binary "@$FILE" \
--progress-bar \
--verbose

Thanks for your help,
Jc

[rwols]

$ curl https://uploads.github.com/repos/$PROJECT/releases/$RELEASE_ID/assets\?name=$FILE --request 'POST' --header "Authorization: token $GITHUB_TOKEN" --header "Content-Disposition: attachment; filename=$FILE" --header 'Content-Type: application/zip; charset=binary' --data-binary "@$FILE" --progress-bar --verbose

output:

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 192.30.253.96...
* TCP_NODELAY set
* Connected to uploads.github.com (192.30.253.96) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.github.com
* Server certificate: DigiCert SHA2 High Assurance Server CA
* Server certificate: DigiCert High Assurance EV Root CA
> POST /repos/rwols/Clara/releases/5635808/assets?name=Clara-0.2.0-Darwin-x64.zip HTTP/1.1
> Host: uploads.github.com
> User-Agent: curl/7.51.0
> Accept: */*
> Authorization: token ~~~SECRET~~~
> Content-Disposition: attachment; filename=Clara-0.2.0-Darwin-x64.zip
> Content-Type: application/zip; charset=binary
> Content-Length: 8549477
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 201 Created
< Cache-Control: no-cache
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Etag: W/"c5fe753a904f0e3e4e6832a42f21da5b"
< Last-Modified: Tue, 07 Mar 2017 09:08:22 GMT
< Strict-Transport-Security: max-age=31557600
< Vary: Accept, Authorization, Cookie, X-GitHub-OTP
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.v3; format=json
< X-Oauth-Scopes: repo
< X-Xss-Protection: 1; mode=block
< Date: Tue, 07 Mar 2017 09:08:22 GMT
< Transfer-Encoding: chunked
< X-GitHub-Request-Id: C344:2DE04:3E00C9:407983:58BE787F
< 
* Curl_http_done: called premature == 0
* Connection #0 to host uploads.github.com left intact
{"url":"https://api.github.com/repos/rwols/Clara/releases/assets/3336983","id":3336983,"name":"Clara-0.2.0-Darwin-x64.zip","label":"","uploader":{"login":"rwols","id":2431823,"avatar_url":"https://avatars0.githubusercontent.com/u/2431823?v=3","gravatar_id":"","url":"https://api.github.com/users/rwols","html_url":"https://github.com/rwols","followers_url":"https://api.github.com/users/rwols/followers","following_url":"https://api.github.com/users/rwols/following{/other_user}","gists_url":"https://api.github.com/users/rwols/gists{/gist_id}","starred_url":"https://api.github.com/users/rwols/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rwols/subscriptions","organizations_url":"https://api.github.com/users/rwols/orgs","repos_url":"https://api.github.com/users/rwols/repos","events_url":"https://api.github.com/users/rwols/events{/privacy}","received_events_url":"https://api.github.com/users/rwols/received_events","type":"User","site_admin":false},"content_type":"application/zip; charset=binary","state":"uploaded","size":8549477,"download_count":0,"created_at":"2017-03-07T09:08:15Z","updated_at":"2017-03-07T09:08:22Z","browser_download_url":"https://github.com/rwols/Clara/releases/download/0.2.0/Clara-0.2.0-Darwin-x64.zip"}

After running the same command again:

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 192.30.253.97...
* TCP_NODELAY set
* Connected to uploads.github.com (192.30.253.97) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.github.com
* Server certificate: DigiCert SHA2 High Assurance Server CA
* Server certificate: DigiCert High Assurance EV Root CA
> POST /repos/rwols/Clara/releases/5635808/assets?name=Clara-0.2.0-Darwin-x64.zip HTTP/1.1
> Host: uploads.github.com
> User-Agent: curl/7.51.0
> Accept: */*
> Authorization: token ~~~SECRET~~~
> Content-Disposition: attachment; filename=Clara-0.2.0-Darwin-x64.zip
> Content-Type: application/zip; charset=binary
> Content-Length: 8549477
> Expect: 100-continue
> 
< HTTP/1.1 422 status code 422
< Cache-Control: no-cache
< Content-Length: 214
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Strict-Transport-Security: max-age=31557600
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.v3; format=json
< X-Oauth-Scopes: repo
< X-Xss-Protection: 1; mode=block
< Date: Tue, 07 Mar 2017 09:12:11 GMT
< X-GitHub-Request-Id: C380:2DE06:15E0791:169C44D:58BE796B
* HTTP error before end of send, stop sending
< 
* Curl_http_done: called premature == 0
* Closing connection 0
{"message":"Validation Failed","request_id":"C380:2DE06:15E0791:169C44D:58BE796B","documentation_url":"https://developer.github.com/v3","errors":[{"resource":"ReleaseAsset","code":"already_exists","field":"name"}]}

[jcfr]

Thanks for the report.

Out of curiosity, is the upload of asset using githubrelease consistently reporting Broken pipe ?

It could be related to network partition.

If it is the case, we will look into catching the exception and suggest the user to retry (or automatically retry)

[rwols]

I tried it again a bunch of times just now, and it still fails with the same exception 😞

I tried setting GITHUB_TOKEN in my env vars, doesn't work. I tried using a .netrc file, doesn't work.

[jcfr]

The "good" news is that it is reproducible. I am coming up with a short python snippet using only requests.

[jcfr]

tried it again a bunch of times just now, and it still fails with the same exception

The following snipped allows to get the Broken pipe exception instead of the already_exists message. See https://gist.github.com/jcfr/a6b778a1b41a01576fc55293106c1f0b#gistcomment-2020431

If you try to upload a new asset using githubrelease is it also failing ?

[rwols]

OK, after messing around with that snippet I believe I understand what the problem is (it's my problem)

I use the Fish shell. To set vars, I do

$ set GITHUB_TOKEN 1234etc

Your snippet still threw exceptions at me at that point. Same error about broken pipes. Then I printed the token to stdout, and it was none! So I logged into a sh session, did

$ export GITHUB_TOKEN=1234etc

and your snippet suddenly works. So I looked at the Fish docs and as it turns out, I need to use

$ set -x GITHUB_TOKEN 1234etc

in order to turn variables into "env" variables. D'oh. Anyway, I also have a ~/.netrc file with all the relevant info in it. I know that the requests lib can use that too. Maybe add that as a feature?

You can close this issue now. I think the take-away from all this is to add some more except clauses as hints for the user. Maybe also check if GITHUB_TOKEN is None as it was in my case.

[jcfr]

Thanks for the detailed analysis 👍

I also have a ~/.netrc file with all the relevant info in it. I know that the requests lib can use that too.
Maybe add that as a feature?

Good point. And it turns out the use of ~/.netrc is already documented 😄

See https://github.com/j0057/github-release#configuring

May be in your file, only api.github.com was specified and uploads.github.com was missing ? This could explain that creating the release and setting release note was working.

What do you think of adding a FAQ section at the end of the README where we would include your analysis ?

You can close this issue now.
I think the take-away from all this is to add some more except clauses as hints for the user.

Will do. After I sort out the cause of the broken pipe issue I managed to reproduce on my side ...

[rwols]

May be in your file, only api.github.com was specified and uploads.github.com was missing ?

That was exactly the case 😅

What do you think of adding a FAQ section at the end of the README where we would include your analysis ?

Sure