Is youtube-transcript-api blocked from working on AWS EC2 servers?

[iblind]

I've tried to run this script from 2 different accounts, with multiple instances in dif. zones, but I'm not having any luck. Is this something that other folks are experiencing?

[jdepoix]

Hi @iblind, unfortunately I don't have any hands on experience with this. But there's two things that come to my mind, however both are just speculations:

1. Multiple users have reported running into rate limits (see Inconsistent get_transcript results #60). I could imagine that requests coming from different EC2 instances are treated as coming from the same client by YouTube and therefore they share the same rate limit. So you might be running into a rate limit without ever having a sent a request yourself.
2. Do you know if the requests executed by your EC2 instance default to IPv6? In Could not retrieve a transcript for the video  #74 someone observed that they run into errors whenever the machine uses IPv6. I could not replicate this so I asked for their help to debug this. However, they unfortunately stopped replying. So maybe you could just pick up where the conversation stopped in Could not retrieve a transcript for the video  #74 and try out the stuff I described over there.

[iblind]

Thanks @jdepoix! The EC2 rate-limitting seems like such a heavy-handed policy, but I guess it makes sense — I'm sure youtube doesn't really care about the fact that those are all potentially distinct users coming from the same origin.

2. I'll take a look at Could not retrieve a transcript for the video  #74 and see if I can dig anything up; I'll keep this thread posted re: updates!

[jdepoix]

Hi @iblind,
did you manage to figure something out?

[iblind]

Hi @jdepoix,

sorry for the lateness, I hadn't received the notification (only noticed it in the new github ui)! Any GCP- or EC2-based requests I've made ended up being blanket-rejected, so unfortunately all the work I did had to be performed locally. I didn't have any luck on #74, with no differences between IPv4 and IPv6 in terms of allowed requests.

[jdepoix]

Thanks for the response @iblind.
That seems really odd, so any curl to YouTube will be blocked? What status code/response do you get? And can you ping YouTube from the GCP/EC2 instance?
And this might be a needless question, but are you sure that it isn't just all traffic to all public servers being blocked, so can you ping any other public servers?

[jdepoix]

any news on this @iblind?

[jdepoix]

@iblind in v0.4.0 the Exception TooManyRequests has been introduced which is raised when running into rate limits. This could be used to figure out if you are just running into rate limits or if there is another problem at hand. Would be great if you could try this out and provide some feedback! 😊

[jdepoix]

I will close this now due to the lack of responses. If anyone has any news on this just post it here and I might open the ticket again if it seems relevant.

[nipunapamuditha]

I'm also getting the same error on ec2 . works on my pc .