[Youtube] Got server error HTTP Error 403: Forbidden(latest master version)

[Linux2010]

Checklist

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2024.08.07(the latest master version)
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2024.08.07.local
[debug] Python 3.8.5 (CPython x86_64 64bit) - macOS-10.16-x86_64-i386-64bit - OpenSSL 1.1.1h  22 Sep 2020
[debug] exe versions: ffmpeg 7.0.1-tessus, ffprobe 6.1.1, rtmpdump 2.4
[debug] Proxy map: {'http': 'http://127.0.0.1:58309', 'https': 'http://127.0.0.1:58309'}
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: You must provide at least one URL.
Type youtube-dl --help to see a list of all options.

PASTE VERBOSE LOG HERE

Description

• The download will run normally for a short time, and then a 403 error will appear

youtube-dl 'https://www.youtube.com/watch?v=lLSkbZ3-EOs'

[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 10 of 10)...
[download] Skipping fragment 41...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 4 of 10)...

[dirkf]

Indeed this seems to be a pathological video where almost all video formats fail on the first fragment and 299 may fail later, regardless of Python 2.7/3.5/3.9 and User-Agent settings.

yt-dlp 2024.08.06 still works, apparently. It has fancy networking that we can't easily replicate: maybe punt to curl for all requests?

[tansy]

I have that with every single video I try. Curiously enough format '18' work all the time. Other formats that work are '136', '137', '248' and/or '160', but it depends on video - not always the case. Still, format '18' is the most reliable to work.

[dirkf]

So

1. poToken (I agree, this is always being detected today) randomly breaks the download rather than uniformly giving 403 as with the revised n-sig "throttling"?
2. different clients get more or less functional links for formats with the same itag?

[GregoriusT]

Can confirm that a lot of the video-only formats are just being 403-ed in the middle with their downloads, resulting in me getting files that stop after about 10-20 minutes into the video, but still have full sized audio.

By now I have written something into my scripts to just pick format 18 as long as a flag is set, because i foresee this issue happening again in the future once it is eventually fixed... >.>

[dirkf]

So has anyone tried fetching fragments in fragments of <1MB? We already had a work-around to download in fragments to avoid throttling IIRC.

Otherwise:

1. actually ignore the data with poToken and use the existing "punt to API" logic with a selected unafflicted client.
2. extend to user-specified other, or selection of, clients similar to the yt-dlp extractor logic.

[iactix]

Apparently the latest fix worked for not even a day, that doesn't bode well. Personally I keep getting "giving up after 0 fragment retries" in my python stuff.

From what I read in yesterdays thread, it seems like this will just not work out with fake JS interpretation if they try to combat this in the slightest. Like, that almost doesn't deserve the name attack vector, that's an attack landscape.

[tansy]

This change is significant. I checked old, pre quantum Firefox and videos don't work any more, when 3 days ago they did.

[dirkf]

Maybe the new player JS uses some G JS syntax extension (aka ECMA2021+) that hadn't been contemplated in those FF versions. Is there an error in the JS console?

[tansy]

It used to work as embedded or as mobile (when used mobile user agent). Now all of them display all saying error:

An error occurred. Please try again later. (Playback ID: j-bZsC_YehYVyZZ8)
Learn More (https://support.google.com/youtube/?p=player_error1&hl=en)

Loading any video at https://www.youtube.com/embed/1234567890a:

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42

Then pressing play:

Error: WebGL: Error during native OpenGL init. base.js:11283:169
Error: WebGL: WebGL creation failed. base.js:11283:169

---

ED.

If it's of any help, despite what was said before, there are some videos that work.

First - this one doesn't, and gives following console log

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42
Empty string passed to getElementById(). zVhcVoOEv7o line 2 > eval:795:28944

Error: WebGL: getParameter: parameter: invalid enum value <enum 0x9246> base.js:11283:254

This one does and, with this log:

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42
Empty string passed to getElementById(). KAR4fAX5T7Y line 2 > eval:4650:33869

So, after clicking 'play' it gives this error: Error: WebGL: getParameter: parameter: invalid enum value <enum 0x9246> base.js:11283:254.

[TheHolyRoger]

No matter what chunk size I use I'm seeing hard 403 errors at 1Meg as others have reported - I'm able to download as many fragments as I want up to 1Meg and then get a 403.

Have experimented with generating cpns (nonces) and adding them to the format fragment URLs without any luck as well as using rn (request numbers) in the URL query instead of byte ranges. Have tried sleeping between fragments to mimick video playback also without joy.

It feels like they've added a check somewhere which fails at the 1Meg mark but I haven't found anything yet where that might be.

Checking via the browser I can see that youtube is happily downloading /videoplayback fragment URLs above 1Meg without any issue...