[version] update

Created by: pukkandan :ci skip all
Release 2022.03.08
2025-12-18 03:42:23 +01:00 · 2022-03-08 19:57:15 +00:00 · 2022-03-09 01:24:27 +05:30 · 2022-03-08 11:42:00 -08:00 · 2022-03-08 23:02:38 +05:30 · 2022-03-08 22:38:06 +05:30
145 changed files with 5530 additions and 2059 deletions
--- a/.github/ISSUE_TEMPLATE/1_broken_site.yml
+++ b/.github/ISSUE_TEMPLATE/1_broken_site.yml
@@ -11,7 +11,7 @@ body:
      options:
        - label: I'm reporting a broken site
          required: true
-        - label: I've verified that I'm running yt-dlp version **2022.02.03**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
+        - label: I've verified that I'm running yt-dlp version **2022.03.08**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
          required: true
        - label: I've checked that all provided URLs are alive and playable in a browser
          required: true
@@ -51,12 +51,12 @@ body:
        [debug] Portable config file: yt-dlp.conf
        [debug] Portable config: ['-i']
        [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252
-        [debug] yt-dlp version 2022.02.03 (exe)
+        [debug] yt-dlp version 2022.03.08 (exe)
        [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0
        [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
        [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets
        [debug] Proxy map: {}
-        yt-dlp is up to date (2022.02.03)
+        yt-dlp is up to date (2022.03.08)
        <more lines>
      render: shell
    validations:
--- a/.github/ISSUE_TEMPLATE/2_site_support_request.yml
+++ b/.github/ISSUE_TEMPLATE/2_site_support_request.yml
@@ -11,7 +11,7 @@ body:
      options:
        - label: I'm reporting a new site support request
          required: true
-        - label: I've verified that I'm running yt-dlp version **2022.02.03**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
+        - label: I've verified that I'm running yt-dlp version **2022.03.08**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
          required: true
        - label: I've checked that all provided URLs are alive and playable in a browser
          required: true
@@ -62,12 +62,12 @@ body:
        [debug] Portable config file: yt-dlp.conf
        [debug] Portable config: ['-i']
        [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252
-        [debug] yt-dlp version 2022.02.03 (exe)
+        [debug] yt-dlp version 2022.03.08 (exe)
        [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0
        [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
        [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets
        [debug] Proxy map: {}
-        yt-dlp is up to date (2022.02.03)
+        yt-dlp is up to date (2022.03.08)
        <more lines>
      render: shell
    validations:
--- a/.github/ISSUE_TEMPLATE/3_site_feature_request.yml
+++ b/.github/ISSUE_TEMPLATE/3_site_feature_request.yml
@@ -11,7 +11,7 @@ body:
      options:
        - label: I'm reporting a site feature request
          required: true
-        - label: I've verified that I'm running yt-dlp version **2022.02.03**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
+        - label: I've verified that I'm running yt-dlp version **2022.03.08**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
          required: true
        - label: I've checked that all provided URLs are alive and playable in a browser
          required: true
@@ -60,12 +60,12 @@ body:
        [debug] Portable config file: yt-dlp.conf
        [debug] Portable config: ['-i']
        [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252
-        [debug] yt-dlp version 2022.02.03 (exe)
+        [debug] yt-dlp version 2022.03.08 (exe)
        [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0
        [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
        [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets
        [debug] Proxy map: {}
-        yt-dlp is up to date (2022.02.03)
+        yt-dlp is up to date (2022.03.08)
        <more lines>
      render: shell
    validations:
--- a/.github/ISSUE_TEMPLATE/4_bug_report.yml
+++ b/.github/ISSUE_TEMPLATE/4_bug_report.yml
@@ -11,7 +11,7 @@ body:
      options:
        - label: I'm reporting a bug unrelated to a specific site
          required: true
-        - label: I've verified that I'm running yt-dlp version **2022.02.03**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
+        - label: I've verified that I'm running yt-dlp version **2022.03.08**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
          required: true
        - label: I've checked that all provided URLs are alive and playable in a browser
          required: true
@@ -45,12 +45,12 @@ body:
        [debug] Portable config file: yt-dlp.conf
        [debug] Portable config: ['-i']
        [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252
-        [debug] yt-dlp version 2022.02.03 (exe)
+        [debug] yt-dlp version 2022.03.08 (exe)
        [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0
        [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
        [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets
        [debug] Proxy map: {}
-        yt-dlp is up to date (2022.02.03)
+        yt-dlp is up to date (2022.03.08)
        <more lines>
      render: shell
    validations:
--- a/.github/ISSUE_TEMPLATE/5_feature_request.yml
+++ b/.github/ISSUE_TEMPLATE/5_feature_request.yml
@@ -13,7 +13,7 @@ body:
          required: true
        - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
          required: true
-        - label: I've verified that I'm running yt-dlp version **2022.02.03**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
+        - label: I've verified that I'm running yt-dlp version **2022.03.08**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
          required: true
        - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
          required: true
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -161,11 +161,10 @@ jobs:
    steps:
    - uses: actions/checkout@v2
    # In order to create a universal2 application, the version of python3 in /usr/bin has to be used
-    # Pyinstaller is pinned to 4.5.1 because the builds are failing in 4.6, 4.7
    - name: Install Requirements
      run: |
          brew install coreutils
-          /usr/bin/python3 -m pip install -U --user pip Pyinstaller==4.5.1 -r requirements.txt
+          /usr/bin/python3 -m pip install -U --user pip Pyinstaller==4.10 -r requirements.txt
    - name: Bump version
      id: bump_version
      run: /usr/bin/python3 devscripts/update-version.py
@@ -234,7 +233,7 @@ jobs:
      # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds
      run: |
          python -m pip install --upgrade pip setuptools wheel py2exe
-          pip install "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-4.5.1-py3-none-any.whl" -r requirements.txt
+          pip install "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-4.10-py3-none-any.whl" -r requirements.txt
    - name: Bump version
      id: bump_version
      env:
@@ -321,7 +320,7 @@ jobs:
    - name: Install Requirements
      run: |
          python -m pip install --upgrade pip setuptools wheel
-          pip install "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-4.5.1-py3-none-any.whl" -r requirements.txt
+          pip install "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-4.10-py3-none-any.whl" -r requirements.txt
    - name: Bump version
      id: bump_version
      env:
--- a/.gitignore
+++ b/.gitignore
@@ -24,6 +24,7 @@ cookies

 *.3gp
 *.ape
+*.ass
 *.avi
 *.desktop
 *.flac
@@ -106,6 +107,7 @@ yt-dlp.zip
 *.iml
 .vscode
 *.sublime-*
+*.code-workspace

 # Lazy extractors
 */extractor/lazy_extractors.py
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -11,6 +11,7 @@
    - [Is anyone going to need the feature?](#is-anyone-going-to-need-the-feature)
    - [Is your question about yt-dlp?](#is-your-question-about-yt-dlp)
    - [Are you willing to share account details if needed?](#are-you-willing-to-share-account-details-if-needed)
+    - [Is the website primarily used for piracy](#is-the-website-primarily-used-for-piracy)
 - [DEVELOPER INSTRUCTIONS](#developer-instructions)
    - [Adding new feature or making overarching changes](#adding-new-feature-or-making-overarching-changes)
    - [Adding support for a new site](#adding-support-for-a-new-site)
@@ -24,6 +25,7 @@
        - [Collapse fallbacks](#collapse-fallbacks)
        - [Trailing parentheses](#trailing-parentheses)
        - [Use convenience conversion and parsing functions](#use-convenience-conversion-and-parsing-functions)
+    - [My pull request is labeled pending-fixes](#my-pull-request-is-labeled-pending-fixes)
 - [EMBEDDING YT-DLP](README.md#embedding-yt-dlp)


@@ -123,6 +125,10 @@ While these steps won't necessarily ensure that no misuse of the account takes p
 - Change the password before sharing the account to something random (use [this](https://passwordsgenerator.net/) if you don't have a random password generator).
 - Change the password after receiving the account back.

+### Is the website primarily used for piracy?
+
+We follow [youtube-dl's policy](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) to not support services that is primarily used for infringing copyright. Additionally, it has been decided to not to support porn sites that specialize in deep fake. We also cannot support any service that serves only [DRM protected content](https://en.wikipedia.org/wiki/Digital_rights_management). 
+



@@ -210,7 +216,7 @@ After you have ensured this site is distributing its content legally, you can fo
            }
    ```
 1. Add an import in [`yt_dlp/extractor/extractors.py`](yt_dlp/extractor/extractors.py).
-1. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, the tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in. You can also run all the tests in one go with `TestDownload.test_YourExtractor_all`
+1. Run `python test/test_download.py TestDownload.test_YourExtractor` (note that `YourExtractor` doesn't end with `IE`). This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, the tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in. You can also run all the tests in one go with `TestDownload.test_YourExtractor_all`
 1. Make sure you have atleast one test for your extractor. Even if all videos covered by the extractor are expected to be inaccessible for automated testing, tests should still be added with a `skip` parameter indicating why the particular test is disabled from running.
 1. Have a look at [`yt_dlp/extractor/common.py`](yt_dlp/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](yt_dlp/extractor/common.py#L91-L426). Add tests and code for as many as you want.
 1. Make sure your code follows [yt-dlp coding conventions](#yt-dlp-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart):
@@ -658,6 +664,10 @@ duration = float_or_none(video.get('durationMs'), scale=1000)
 view_count = int_or_none(video.get('views'))
 ```

+# My pull request is labeled pending-fixes
+
+The `pending-fixes` label is added when there are changes requested to a PR. When the necessary changes are made, the label should be removed. However, despite our best efforts, it may sometimes happen that the maintainer did not see the changes or forgot to remove the label. If your PR is still marked as `pending-fixes` a few days after all requested changes have been made, feel free to ping the maintainer who labeled your issue and ask them to re-review and remove the label.
+



--- a/24
+++ b/24
@@ -146,7 +146,7 @@ chio0hai
 cntrl-s
 Deer-Spangle
 DEvmIb
-Grabien
+Grabien/MaximVol
 j54vc1bk
 mpeter50
 mrpapersonic
@@ -160,7 +160,7 @@ PilzAdam
 zmousm
 iw0nderhow
 unit193
-TwoThousandHedgehogs
+TwoThousandHedgehogs/KathrynElrod
 Jertzukka
 cypheron
 Hyeeji
@@ -194,3 +194,23 @@ KiberInfinity
 tejing1
 Bricio
 lazypete365
+Aniruddh-J
+blackgear
+CplPwnies
+cyberfox1691
+FestplattenSchnitzel
+hatienl0i261299
+iphoting
+jakeogh
+lukasfink1
+lyz-code
+marieell
+mdpauley
+Mipsters
+mxmehl
+ofkz
+P-reducible
+pycabbage
+regarten
+Ronnnny
+schn0sch
--- a/Changelog.md
+++ b/Changelog.md
@@ -11,6 +11,142 @@
 -->


+### 2022.03.08
+
+* Merge youtube-dl: Upto [commit/6508688](https://github.com/ytdl-org/youtube-dl/commit/6508688e88c83bb811653083db9351702cd39a6a) (except NDR)
+* Add regex operator and quoting to format filters by [lukasfink1](https://github.com/lukasfink1)
+* Add brotli content-encoding support by [coletdjnz](https://github.com/coletdjnz)
+* Add pre-processor stage `after_filter`
+* Better error message when no `--live-from-start` format
+* Create necessary directories for `--print-to-file`
+* Fill more fields for playlists by [Lesmiscore](https://github.com/Lesmiscore)
+* Fix `-all` for `--sub-langs`
+* Fix doubling of `video_id` in `ExtractorError`
+* Fix for when stdout/stderr encoding is `None`
+* Handle negative duration from extractor
+* Implement `--add-header` without modifying `std_headers`
+* Obey `--abort-on-error` for "ffmpeg not installed"
+* Set `webpage_url_...` from `webpage_url` and not input URL
+* Tolerate failure to `--write-link` due to unknown URL
+* [aria2c] Add `--http-accept-gzip=true`
+* [build] Update pyinstaller to 4.10 by [shirt-dev](https://github.com/shirt-dev)
+* [cookies] Update MacOS12 `Cookies.binarycookies` location by [mdpauley](https://github.com/mdpauley)
+* [devscripts] Improve `prepare_manpage`
+* [downloader] Do not use aria2c for non-native `m3u8`
+* [downloader] Obey `--file-access-retries` when deleting/renaming by [ehoogeveen-medweb](https://github.com/ehoogeveen-medweb)
+* [extractor] Allow `http_headers` to be specified for `thumbnails`
+* [extractor] Extract subtitles from manifests for vimeo, globo, kaltura, svt by [fstirlitz](https://github.com/fstirlitz)
+* [extractor] Fix for manifests without period duration by [dirkf,](https://github.com/dirkf,) [pukkandan](https://github.com/pukkandan)
+* [extractor] Support `--mark-watched` without `_NETRC_MACHINE` by [coletdjnz](https://github.com/coletdjnz)
+* [FFmpegConcat] Abort on `--simulate`
+* [FormatSort] Consider `acodec`=`ogg` as `vorbis`
+* [fragment] Fix bugs around resuming with Range by [Lesmiscore](https://github.com/Lesmiscore)
+* [fragment] Improve `--live-from-start` for YouTube livestreams by [Lesmiscore](https://github.com/Lesmiscore)
+* [generic] Pass referer to extracted formats
+* [generic] Set rss `guid` as video id by [Bricio](https://github.com/Bricio)
+* [options] Better ambiguous option resolution
+* [options] Rename `--clean-infojson` to `--clean-info-json`
+* [SponsorBlock] Fixes for highlight and "full video labels" by [nihil-admirari](https://github.com/nihil-admirari)
+* [Sponsorblock] minor fixes by [nihil-admirari](https://github.com/nihil-admirari)
+* [utils] Better traceback for `ExtractorError`
+* [utils] Fix file locking for AOSP by [jakeogh](https://github.com/jakeogh)
+* [utils] Improve file locking
+* [utils] OnDemandPagedList: Do not download pages after error
+* [utils] render_table: Fix character calculation for removing extra gap by [Lesmiscore](https://github.com/Lesmiscore)
+* [utils] Use `locked_file` for `sanitize_open` by [jakeogh](https://github.com/jakeogh)
+* [utils] Validate `DateRange` input
+* [utils] WebSockets wrapper for non-async functions by [Lesmiscore](https://github.com/Lesmiscore)
+* [cleanup] Don't pass protocol to `_extract_m3u8_formats` for live videos
+* [cleanup] Remove extractors for some dead websites by [marieell](https://github.com/marieell)
+* [cleanup, docs] Misc cleanup
+* [AbemaTV] Add extractors by [Lesmiscore](https://github.com/Lesmiscore)
+* [adobepass] Add Suddenlink MSO by [CplPwnies](https://github.com/CplPwnies)
+* [ant1newsgr] Add extractor by [zmousm](https://github.com/zmousm)
+* [bigo] Add extractor by [Lesmiscore](https://github.com/Lesmiscore)
+* [Caltrans] Add extractor by [Bricio](https://github.com/Bricio)
+* [daystar] Add extractor by [hatienl0i261299](https://github.com/hatienl0i261299)
+* [fc2:live] Add extractor by [Lesmiscore](https://github.com/Lesmiscore)
+* [fptplay] Add extractor by [hatienl0i261299](https://github.com/hatienl0i261299)
+* [murrtube] Add extractor by [cyberfox1691](https://github.com/cyberfox1691)
+* [nfb] Add extractor by [ofkz](https://github.com/ofkz)
+* [niconico] Add playlist extractors and refactor by [Lesmiscore](https://github.com/Lesmiscore)
+* [peekvids] Add extractor by [schn0sch](https://github.com/schn0sch)
+* [piapro] Add extractor by [pycabbage,](https://github.com/pycabbage,) [Lesmiscore](https://github.com/Lesmiscore)
+* [rokfin] Add extractor by [P-reducible,](https://github.com/P-reducible,) [pukkandan](https://github.com/pukkandan)
+* [rokfin] Add stack and channel extractors by [P-reducible,](https://github.com/P-reducible,) [pukkandan](https://github.com/pukkandan)
+* [ruv.is] Add extractor by [iw0nderhow](https://github.com/iw0nderhow)
+* [telegram] Add extractor by [hatienl0i261299](https://github.com/hatienl0i261299)
+* [VideocampusSachsen] Add extractors by [FestplattenSchnitzel](https://github.com/FestplattenSchnitzel)
+* [xinpianchang] Add extractor by [hatienl0i261299](https://github.com/hatienl0i261299)
+* [abc] Support 1080p by [Ronnnny](https://github.com/Ronnnny)
+* [afreecatv] Support password-protected livestreams by [wlritchi](https://github.com/wlritchi)
+* [ard] Fix valid URL
+* [ATVAt] Detect geo-restriction by [marieell](https://github.com/marieell)
+* [bandcamp] Detect acodec
+* [bandcamp] Fix user URLs by [lyz-code](https://github.com/lyz-code)
+* [bbc] Fix extraction of news articles by [ajj8](https://github.com/ajj8)
+* [beeg] Fix extractor by [Bricio](https://github.com/Bricio)
+* [bigo] Fix extractor to not to use `form_params`
+* [Bilibili] Pass referer for all formats by [blackgear](https://github.com/blackgear)
+* [Biqle] Fix extractor by [Bricio](https://github.com/Bricio)
+* [ccma] Fix timestamp parsing by [nyuszika7h](https://github.com/nyuszika7h)
+* [crunchyroll] Better error reporting on login failure by [tejing1](https://github.com/tejing1)
+* [cspan] Support of C-Span congress videos by [Grabien](https://github.com/Grabien)
+* [dropbox] fix regex by [zenerdi0de](https://github.com/zenerdi0de)
+* [fc2] Fix extraction by [Lesmiscore](https://github.com/Lesmiscore)
+* [fujitv] Extract resolution for free sources by [YuenSzeHong](https://github.com/YuenSzeHong)
+* [Gettr] Add `GettrStreamingIE` by [i6t](https://github.com/i6t)
+* [Gettr] Fix formats order by [i6t](https://github.com/i6t)
+* [Gettr] Improve extractor by [i6t](https://github.com/i6t)
+* [globo] Expand valid URL by [Bricio](https://github.com/Bricio)
+* [lbry] Fix `--ignore-no-formats-error`
+* [manyvids] Extract `uploader` by [regarten](https://github.com/regarten)
+* [mildom] Fix linter
+* [mildom] Rework extractors by [Lesmiscore](https://github.com/Lesmiscore)
+* [mirrativ] Cleanup extractor code by [Lesmiscore](https://github.com/Lesmiscore)
+* [nhk] Add support for NHK for School by [Lesmiscore](https://github.com/Lesmiscore)
+* [niconico:tag] Add support for searching tags
+* [nrk] Add fallback API
+* [peekvids] Use JSON-LD by [schn0sch](https://github.com/schn0sch)
+* [peertube] Add media.fsfe.org by [mxmehl](https://github.com/mxmehl)
+* [rtvs] Fix extractor by [Bricio](https://github.com/Bricio)
+* [spiegel] Fix `_VALID_URL`
+* [ThumbnailsConvertor] Support `webp`
+* [tiktok] Fix `vm.tiktok`/`vt.tiktok` URLs
+* [tubitv] Fix/improve TV series extraction by [bbepis](https://github.com/bbepis)
+* [tumblr] Fix extractor by [foghawk](https://github.com/foghawk)
+* [twitcasting] Add fallback for finding running live by [Lesmiscore](https://github.com/Lesmiscore)
+* [TwitCasting] Check for password protection by [Lesmiscore](https://github.com/Lesmiscore)
+* [twitcasting] Fix extraction by [Lesmiscore](https://github.com/Lesmiscore)
+* [twitch] Fix field name of `view_count`
+* [twitter] Fix for private videos by [iphoting](https://github.com/iphoting)
+* [washingtonpost] Fix extractor by [Bricio](https://github.com/Bricio)
+* [youtube:tab] Add `approximate_date` extractor-arg
+* [youtube:tab] Follow redirect to regional channel  by [coletdjnz](https://github.com/coletdjnz)
+* [youtube:tab] Reject webpage data if redirected to home page
+* [youtube] De-prioritize potentially damaged formats
+* [youtube] Differentiate descriptive audio by language code
+* [youtube] Ensure subtitle urls are absolute by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Escape possible `$` in `_extract_n_function_name` regex by [Lesmiscore](https://github.com/Lesmiscore)
+* [youtube] Fix automatic captions
+* [youtube] Fix n-sig extraction for phone player JS by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [youtube] Further de-prioritize 3gp format
+* [youtube] Label original auto-subs
+* [youtube] Prefer UTC upload date for videos by [coletdjnz](https://github.com/coletdjnz)
+* [zaq1] Remove dead extractor by [marieell](https://github.com/marieell)
+* [zee5] Support web-series by [Aniruddh-J](https://github.com/Aniruddh-J)
+* [zingmp3] Fix extractor by [hatienl0i261299](https://github.com/hatienl0i261299)
+* [zoom] Add support for screen cast by [Mipsters](https://github.com/Mipsters)
+
+
+### 2022.02.04
+
+* [youtube:search] Fix extractor by [coletdjnz](https://github.com/coletdjnz)
+* [youtube:search] Add tests
+* [twitcasting] Enforce UTF-8 for POST payload by [Lesmiscore](https://github.com/Lesmiscore)
+* [mediaset] Fix extractor by [nixxo](https://github.com/nixxo)
+* [websocket] Make syntax error in `websockets` module non-fatal
+
 ### 2022.02.03

 * Merge youtube-dl: Upto [commit/78ce962](https://github.com/ytdl-org/youtube-dl/commit/78ce962f4fe020994c216dd2671546fbe58a5c67)
--- a/2
+++ b/2
@@ -16,7 +16,7 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites com
 clean-test:
 	rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \
 	*.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \
-	*.3gp *.ape *.avi *.desktop *.flac *.flv *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 \
+	*.3gp *.ape *.ass *.avi *.desktop *.flac *.flv *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 \
 	*.mp4 *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp
 clean-dist:
 	rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \
--- a/README.md
+++ b/README.md
@@ -71,7 +71,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t

 # NEW FEATURES

-* Based on **youtube-dl 2021.12.17 [commit/78ce962](https://github.com/ytdl-org/youtube-dl/commit/78ce962f4fe020994c216dd2671546fbe58a5c67)** and **youtube-dlc 2020.11.11-3 [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
+* Based on **youtube-dl 2021.12.17 [commit/5add3f4](https://github.com/ytdl-org/youtube-dl/commit/5add3f4373287e6346ca3551239edab549284db3)** and **youtube-dlc 2020.11.11-3 [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)

 * **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API

@@ -112,7 +112,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t

 * **Other new options**: Many new options have been added such as `--concat-playlist`, `--print`, `--wait-for-video`, `--sleep-requests`, `--convert-thumbnails`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc

-* **Improvements**: Regex and other operators in `--match-filter`, multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection), merge multi-video/audio, multiple `--config-locations`, `--exec` at different stages, etc
+* **Improvements**: Regex and other operators in `--format`/`--match-filter`, multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection), merge multi-video/audio, multiple `--config-locations`, `--exec` at different stages, etc

 * **Plugins**: Extractors and PostProcessors can be loaded from an external file. See [plugins](#plugins) for details

@@ -130,7 +130,7 @@ Some of yt-dlp's default options are different from that of youtube-dl and youtu
 * The default [format sorting](#sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
 * The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be preferred. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
 * Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
-* `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
+* `--no-abort-on-error` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
 * When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files
 * `--add-metadata` attaches the `infojson` to `mkv` files in addition to writing the metadata when used with `--write-info-json`. Use `--no-embed-info-json` or `--compat-options no-attach-info-json` to revert this
 * Some metadata are embedded into different fields when using `--add-metadata` as compared to youtube-dl. Most notably, `comment` field contains the `webpage_url` and `synopsis` contains the `description`. You can [use `--parse-metadata`](#modifying-metadata) to modify this to your liking or use `--compat-options embed-metadata` to revert this
@@ -267,7 +267,8 @@ While all the other dependencies are optional, `ffmpeg` and `ffprobe` are highly
 * [**pycryptodomex**](https://github.com/Legrandin/pycryptodome) - For decrypting AES-128 HLS streams and various other data. Licensed under [BSD2](https://github.com/Legrandin/pycryptodome/blob/master/LICENSE.rst)
 * [**websockets**](https://github.com/aaugustin/websockets) - For downloading over websocket. Licensed under [BSD3](https://github.com/aaugustin/websockets/blob/main/LICENSE)
 * [**secretstorage**](https://github.com/mitya57/secretstorage) - For accessing the Gnome keyring while decrypting cookies of Chromium-based browsers on Linux. Licensed under [BSD](https://github.com/mitya57/secretstorage/blob/master/LICENSE)
-* [**AtomicParsley**](https://github.com/wez/atomicparsley) - For embedding thumbnail in mp4/m4a if mutagen is not present. Licensed under [GPLv2+](https://github.com/wez/atomicparsley/blob/master/COPYING)
+* [**AtomicParsley**](https://github.com/wez/atomicparsley) - For embedding thumbnail in mp4/m4a if mutagen/ffmpeg cannot. Licensed under [GPLv2+](https://github.com/wez/atomicparsley/blob/master/COPYING)
+* [**brotli**](https://github.com/google/brotli) or [**brotlicffi**](https://github.com/python-hyper/brotlicffi) - [Brotli](https://en.wikipedia.org/wiki/Brotli) content encoding support. Both licensed under MIT <sup>[1](https://github.com/google/brotli/blob/master/LICENSE) [2](https://github.com/python-hyper/brotlicffi/blob/master/LICENSE) </sup>
 * [**rtmpdump**](http://rtmpdump.mplayerhq.hu) - For downloading `rtmp` streams. ffmpeg will be used as a fallback. Licensed under [GPLv2+](http://rtmpdump.mplayerhq.hu)
 * [**mplayer**](http://mplayerhq.hu/design7/info.html) or [**mpv**](https://mpv.io) - For downloading `rstp` streams. ffmpeg will be used as a fallback. Licensed under [GPLv2+](https://github.com/mpv-player/mpv/blob/master/Copyright)
 * [**phantomjs**](https://github.com/ariya/phantomjs) - Used in extractors where javascript needs to be run. Licensed under [BSD3](https://github.com/ariya/phantomjs/blob/master/LICENSE.BSD)
@@ -278,13 +279,14 @@ To use or redistribute the dependencies, you must agree to their respective lice

 The Windows and MacOS standalone release binaries are already built with the python interpreter, mutagen, pycryptodomex and websockets included.

+<!-- TODO: ffmpeg has merged this patch. Remove this note once there is new release -->
 **Note**: There are some regressions in newer ffmpeg versions that causes various issues when used alongside yt-dlp. Since ffmpeg is such an important dependency, we provide [custom builds](https://github.com/yt-dlp/FFmpeg-Builds#ffmpeg-static-auto-builds) with patches for these issues at [yt-dlp/FFmpeg-Builds](https://github.com/yt-dlp/FFmpeg-Builds). See [the readme](https://github.com/yt-dlp/FFmpeg-Builds#patches-applied) for details on the specific issues solved by these builds


 ## COMPILE

 **For Windows**:
-To build the Windows executable, you must have pyinstaller (and optionally mutagen, pycryptodomex, websockets). Once you have all the necessary dependencies installed, (optionally) build lazy extractors using `devscripts/make_lazy_extractors.py`, and then just run `pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it.
+To build the Windows executable, you must have pyinstaller (and any of yt-dlp's optional dependencies if needed). Once you have all the necessary dependencies installed, (optionally) build lazy extractors using `devscripts/make_lazy_extractors.py`, and then just run `pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it.

    py -m pip install -U pyinstaller -r requirements.txt
    py devscripts/make_lazy_extractors.py
@@ -605,11 +607,11 @@ You can also fork the project on github and run your fork's [build workflow](.gi
                                     --write-description etc. (default)
    --no-write-playlist-metafiles    Do not write playlist metadata when using
                                     --write-info-json, --write-description etc.
-    --clean-infojson                 Remove some private fields such as
+    --clean-info-json                Remove some private fields such as
                                     filenames from the infojson. Note that it
                                     could still contain some personal
                                     information (default)
-    --no-clean-infojson              Write all fields to the infojson
+    --no-clean-info-json             Write all fields to the infojson
    --write-comments                 Retrieve video comments to be placed in the
                                     infojson. The comments are fetched even
                                     without this option if the extraction is
@@ -737,9 +739,6 @@ You can also fork the project on github and run your fork's [build workflow](.gi
    --prefer-insecure                Use an unencrypted connection to retrieve
                                     information about the video (Currently
                                     supported only for YouTube)
-    --user-agent UA                  Specify a custom user agent
-    --referer URL                    Specify a custom referer, use if the video
-                                     access is restricted to one domain
    --add-header FIELD:VALUE         Specify a custom HTTP header and its value,
                                     separated by a colon ":". You can use this
                                     option multiple times
@@ -951,7 +950,7 @@ You can also fork the project on github and run your fork's [build workflow](.gi
                                     (currently supported: srt|vtt|ass|lrc)
                                     (Alias: --convert-subtitles)
    --convert-thumbnails FORMAT      Convert the thumbnails to another format
-                                     (currently supported: jpg|png)
+                                     (currently supported: jpg|png|webp)
    --split-chapters                 Split video into multiple files based on
                                     internal chapters. The "chapter:" prefix
                                     can be used with "--paths" and "--output"
@@ -982,15 +981,17 @@ You can also fork the project on github and run your fork's [build workflow](.gi
                                     semicolon ";" delimited list of NAME=VALUE.
                                     The "when" argument determines when the
                                     postprocessor is invoked. It can be one of
-                                     "pre_process" (after extraction),
-                                     "before_dl" (before video download),
-                                     "post_process" (after video download;
-                                     default), "after_move" (after moving file
-                                     to their final locations), "after_video"
-                                     (after downloading and processing all
-                                     formats of a video), or "playlist" (end of
-                                     playlist). This option can be used multiple
-                                     times to add different postprocessors
+                                     "pre_process" (after video extraction),
+                                     "after_filter" (after video passes filter),
+                                     "before_dl" (before each video download),
+                                     "post_process" (after each video download;
+                                     default), "after_move" (after moving video
+                                     file to it's final locations),
+                                     "after_video" (after downloading and
+                                     processing all formats of a video), or
+                                     "playlist" (at end of playlist). This
+                                     option can be used multiple times to add
+                                     different postprocessors

 ## SponsorBlock Options:
 Make chapter entries for, or remove various segments (sponsor,
@@ -1399,7 +1400,7 @@ The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `
 - `asr`: Audio sampling rate in Hertz
 - `fps`: Frame rate

-Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends with), `*=` (contains) and following string meta fields:
+Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends with), `*=` (contains), `~=` (matches regex) and following string meta fields:

 - `ext`: File extension
 - `acodec`: Name of the audio codec in use
@@ -1409,7 +1410,7 @@ Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends
 - `format_id`: A short description of the format
 - `language`: Language code

-Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain).
+Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain). The comparand of a string comparison needs to be quoted with either double or single quotes if it contains spaces or special characters other than `._-`.

 Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering.

@@ -1552,8 +1553,9 @@ $ yt-dlp -S "proto"



-# Download the best video with h264 codec, or the best video if there is no such video
-$ yt-dlp -f "(bv*[vcodec^=avc1]+ba) / (bv*+ba/b)"
+# Download the best video with either h264 or h265 codec,
+# or the best video if there is no such video
+$ yt-dlp -f "(bv*[vcodec~='^((he|a)vc|h26[45])']+ba) / (bv*+ba/b)"

 # Download the best video with best codec no better than h264,
 # or the best video with worst codec if there is no such video
@@ -1598,25 +1600,28 @@ This option also has a few special uses:
 * You can download an additional URL based on the metadata of the currently downloaded video. To do this, set the field `additional_urls` to the URL that you want to download. Eg: `--parse-metadata "description:(?P<additional_urls>https?://www\.vimeo\.com/\d+)` will download the first vimeo video found in the description
 * You can use this to change the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. For example, you can use this to set a different "description" and "synopsis". To modify the metadata of individual streams, use the `meta<n>_` prefix (Eg: `meta1_language`). Any value set to the `meta_` field will overwrite all default values.

+**Note**: Metadata modification happens before format selection, post-extraction and other post-processing operations. Some fields may be added or changed during these steps, overriding your changes.
+
 For reference, these are the fields yt-dlp adds by default to the file metadata:

-Metadata fields|From
-:---|:---
-`title`|`track` or `title`
-`date`|`upload_date`
-`description`,  `synopsis`|`description`
-`purl`, `comment`|`webpage_url`
-`track`|`track_number`
-`artist`|`artist`, `creator`, `uploader` or `uploader_id`
-`genre`|`genre`
-`album`|`album`
-`album_artist`|`album_artist`
-`disc`|`disc_number`
-`show`|`series`
-`season_number`|`season_number`
-`episode_id`|`episode` or `episode_id`
-`episode_sort`|`episode_number`
-`language` of each stream|From the format's `language`
+Metadata fields            | From
+:--------------------------|:------------------------------------------------
+`title`                    | `track` or `title`
+`date`                     | `upload_date`
+`description`,  `synopsis` | `description`
+`purl`, `comment`          | `webpage_url`
+`track`                    | `track_number`
+`artist`                   | `artist`, `creator`, `uploader` or `uploader_id`
+`genre`                    | `genre`
+`album`                    | `album`
+`album_artist`             | `album_artist`
+`disc`                     | `disc_number`
+`show`                     | `series`
+`season_number`            | `season_number`
+`episode_id`               | `episode` or `episode_id`
+`episode_sort`             | `episode_number`
+`language` of each stream  | the format's `language`
+
 **Note**: The file format may not support some of these fields


@@ -1662,6 +1667,7 @@ The following extractors use this feature:

 #### youtubetab (YouTube playlists, channels, feeds, etc.)
 * `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details)
+* `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off

 #### funimation
 * `language`: Languages to extract. Eg: `funimation:language=english,japanese`
@@ -1694,6 +1700,10 @@ The following extractors use this feature:
 * `app_version`: App version to call mobile APIs with - should be set along with `manifest_app_version`. (e.g. `20.2.1`)
 * `manifest_app_version`: Numeric app version to call mobile APIs with. (e.g. `221`)

+#### rokfinchannel
+* `tab`: Which tab to download. One of `new`, `top`, `videos`, `podcasts`, `streams`, `stacks`. (E.g. `rokfinchannel:tab=streams`)
+
+
 NOTE: These options may be changed/removed in the future without concern for backward compatibility

 <!-- MANPAGE: MOVE "INSTALLATION" SECTION HERE -->
@@ -1810,12 +1820,11 @@ ydl_opts = {
    }],
    'logger': MyLogger(),
    'progress_hooks': [my_hook],
+    # Add custom headers
+    'http_headers': {'Referer': 'https://www.google.com'}
 }


-# Add custom headers
-yt_dlp.utils.std_headers.update({'Referer': 'https://www.google.com'})
-
 # ℹ️ See the public functions in yt_dlp.YoutubeDL for for other available functions.
 # Eg: "ydl.download", "ydl.download_with_info_file"
 with yt_dlp.YoutubeDL(ydl_opts) as ydl:
@@ -1858,6 +1867,8 @@ While these options are redundant, they are still expected to be used due to the
    --reject-title REGEX             --match-filter "title !~= (?i)REGEX"
    --min-views COUNT                --match-filter "view_count >=? COUNT"
    --max-views COUNT                --match-filter "view_count <=? COUNT"
+    --user-agent UA                  --add-header "User-Agent:UA"
+    --referer URL                    --add-header "Referer:URL"


 #### Not recommended
@@ -1895,11 +1906,13 @@ These options are not intended to be used by the end-user
 These are aliases that are no longer documented for various reasons

    --avconv-location                --ffmpeg-location
+    --clean-infojson                 --clean-info-json
    --cn-verification-proxy URL      --geo-verification-proxy URL
    --dump-headers                   --print-traffic
    --dump-intermediate-pages        --dump-pages
    --force-write-download-archive   --force-write-archive
    --load-info                      --load-info-json
+    --no-clean-infojson              --no-clean-info-json
    --no-split-tracks                --no-split-chapters
    --no-write-srt                   --no-write-subs
    --prefer-unsecure                --prefer-insecure
--- a/devscripts/prepare_manpage.py
+++ b/devscripts/prepare_manpage.py
@@ -75,21 +75,21 @@ def filter_options(readme):
    section = re.search(r'(?sm)^# USAGE AND OPTIONS\n.+?(?=^# )', readme).group(0)
    options = '# OPTIONS\n'
    for line in section.split('\n')[1:]:
-        if line.lstrip().startswith('-'):
-            split = re.split(r'\s{2,}', line.lstrip())
-            # Description string may start with `-` as well. If there is
-            # only one piece then it's a description bit not an option.
-            if len(split) > 1:
-                option, description = split
-                split_option = option.split(' ')
+        mobj = re.fullmatch(r'''(?x)
+                \s{4}(?P<opt>-(?:,\s|[^\s])+)
+                (?:\s(?P<meta>(?:[^\s]|\s(?!\s))+))?
+                (\s{2,}(?P<desc>.+))?
+            ''', line)
+        if not mobj:
+            options += f'{line.lstrip()}\n'
+            continue
+        option, metavar, description = mobj.group('opt', 'meta', 'desc')

-                if not split_option[-1].startswith('-'):  # metavar
-                    option = ' '.join(split_option[:-1] + [f'*{split_option[-1]}*'])
-
-                # Pandoc's definition_lists. See http://pandoc.org/README.html
-                options += f'\n{option}\n:   {description}\n'
-                continue
-        options += line.lstrip() + '\n'
+        # Pandoc's definition_lists. See http://pandoc.org/README.html
+        option = f'{option} *{metavar}*' if metavar else option
+        description = f'{description}\n' if description else ''
+        options += f'\n{option}\n:   {description}'
+        continue

    return readme.replace(section, options, 1)

--- a/pyinst.py
+++ b/pyinst.py
@@ -74,7 +74,7 @@ def version_to_list(version):


 def dependency_options():
-    dependencies = [pycryptodome_module(), 'mutagen'] + collect_submodules('websockets')
+    dependencies = [pycryptodome_module(), 'mutagen', 'brotli'] + collect_submodules('websockets')
    excluded_modules = ['test', 'ytdlp_plugins', 'youtube-dl', 'youtube-dlc']

    yield from (f'--hidden-import={module}' for module in dependencies)
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,3 +1,5 @@
 mutagen
 pycryptodomex
 websockets
+brotli; platform_python_implementation=='CPython'
+brotlicffi; platform_python_implementation!='CPython'
--- a/setup.py
+++ b/setup.py
@@ -21,9 +21,9 @@ DESCRIPTION = 'A youtube-dl fork with additional features and patches'
 LONG_DESCRIPTION = '\n\n'.join((
    'Official repository: <https://github.com/yt-dlp/yt-dlp>',
    '**PS**: Some links in this document will not work since this is a copy of the README.md from Github',
-    open('README.md', 'r', encoding='utf-8').read()))
+    open('README.md').read()))

-REQUIREMENTS = ['mutagen', 'pycryptodomex', 'websockets']
+REQUIREMENTS = open('requirements.txt').read().splitlines()


 if sys.argv[1:2] == ['py2exe']:
--- a/supportedsites.md
+++ b/supportedsites.md
@@ -3,7 +3,6 @@
 - **17live:clip**
 - **1tv**: Первый канал
 - **20min**
- - **220.ro**
 - **23video**
 - **247sports**
 - **24video**
@@ -11,7 +10,6 @@
 - **3sat**
 - **4tube**
 - **56.com**
- - **5min**
 - **6play**
 - **7plus**
 - **8tracks**
@@ -26,6 +24,8 @@
 - **abcnews:video**
 - **abcotvs**: ABC Owned Television Stations
 - **abcotvs:clips**
+ - **AbemaTV**
+ - **AbemaTVTitle**
 - **AcademicEarth:Course**
 - **acast**
 - **acast:channel**
@@ -47,6 +47,8 @@
 - **AlJazeera**
 - **Allocine**
 - **AlphaPorno**
+ - **Alsace20TV**
+ - **Alsace20TVEmbed**
 - **Alura**
 - **AluraCourse**
 - **Amara**
@@ -60,6 +62,9 @@
 - **AnimeLab**
 - **AnimeLabShows**
 - **AnimeOnDemand**
+ - **ant1newsgr:article**: ant1news.gr articles
+ - **ant1newsgr:embed**: ant1news.gr embedded videos
+ - **ant1newsgr:watch**: ant1news.gr videos
 - **Anvato**
 - **aol.com**: Yahoo screen and movies
 - **APA**
@@ -77,6 +82,7 @@
 - **Arkena**
 - **arte.sky.it**
 - **ArteTV**
+ - **ArteTVCategory**
 - **ArteTVEmbed**
 - **ArteTVPlaylist**
 - **AsianCrush**
@@ -101,8 +107,8 @@
 - **bandaichannel**
 - **Bandcamp**
 - **Bandcamp:album**
+ - **Bandcamp:user**
 - **Bandcamp:weekly**
- - **BandcampMusic**
 - **bangumi.bilibili.com**: BiliBili番剧
 - **BannedVideo**
 - **bbc**: BBC
@@ -124,6 +130,7 @@
 - **bfmtv:live**
 - **BibelTV**
 - **Bigflix**
+ - **Bigo**
 - **Bild**: Bild.de
 - **BiliBili**
 - **Bilibili category extractor**
@@ -165,6 +172,7 @@
 - **BYUtv**
 - **CableAV**
 - **Callin**
+ - **Caltrans**
 - **CAM4**
 - **Camdemy**
 - **CamdemyFolder**
@@ -233,6 +241,8 @@
 - **Coub**
 - **CozyTV**
 - **cp24**
+ - **cpac**
+ - **cpac:playlist**
 - **Cracked**
 - **Crackle**
 - **CrooksAndLiars**
@@ -243,6 +253,7 @@
 - **crunchyroll:playlist**
 - **crunchyroll:playlist:beta**
 - **CSpan**: C-SPAN
+ - **CSpanCongress**
 - **CtsNews**: 華視新聞
 - **CTV**
 - **CTVNews**
@@ -264,6 +275,7 @@
 - **daum.net:clip**
 - **daum.net:playlist**
 - **daum.net:user**
+ - **daystar:clip**
 - **DBTV**
 - **DctpTv**
 - **DeezerAlbum**
@@ -355,6 +367,7 @@
 - **faz.net**
 - **fc2**
 - **fc2:embed**
+ - **fc2:live**
 - **Fczenit**
 - **Filmmodu**
 - **filmon**
@@ -374,6 +387,7 @@
 - **foxnews**: Fox News and Fox Business Video
 - **foxnews:article**
 - **FoxSports**
+ - **fptplay**: fptplay.vn
 - **FranceCulture**
 - **FranceInter**
 - **FranceTV**
@@ -381,7 +395,6 @@
 - **FranceTVSite**
 - **Freesound**
 - **freespeech.org**
- - **FreshLive**
 - **FrontendMasters**
 - **FrontendMastersCourse**
 - **FrontendMastersLesson**
@@ -413,6 +426,7 @@
 - **gem.cbc.ca:playlist**
 - **generic**: Generic downloader that works on some sites
 - **Gettr**
+ - **GettrStreaming**
 - **Gfycat**
 - **GiantBomb**
 - **Giga**
@@ -454,7 +468,6 @@
 - **hitbox:live**
 - **HitRecord**
 - **hketv**: 香港教育局教育電視 (HKETV) Educational Television, Hong Kong Educational Bureau
- - **HornBunny**
 - **HotNewHipHop**
 - **hotstar**
 - **hotstar:playlist**
@@ -499,7 +512,6 @@
 - **iq.com**: International version of iQiyi
 - **iq.com:album**
 - **iqiyi**: 爱奇艺
- - **Ir90Tv**
 - **ITTF**
 - **ITV**
 - **ITVBTCC**
@@ -516,7 +528,6 @@
 - **JWPlatform**
 - **Kakao**
 - **Kaltura**
- - **Kankan**
 - **Karaoketv**
 - **KarriereVideos**
 - **Katsomo**
@@ -628,8 +639,9 @@
 - **MiaoPai**
 - **microsoftstream**: Microsoft Stream
 - **mildom**: Record ongoing live by specific user in Mildom
+ - **mildom:clip**: Clip in Mildom
 - **mildom:user:vod**: Download all VODs from specific user in Mildom
- - **mildom:vod**: Download a VOD in Mildom
+ - **mildom:vod**: VOD in Mildom
 - **minds**
 - **minds:channel**
 - **minds:group**
@@ -672,6 +684,8 @@
 - **mtvservices:embedded**
 - **MTVUutisetArticle**
 - **MuenchenTV**: münchen.tv
+ - **Murrtube**
+ - **MurrtubeUser**: Murrtube user profile
 - **MuseScore**
 - **MusicdexAlbum**
 - **MusicdexArtist**
@@ -740,9 +754,13 @@
 - **NextTV**: 壹電視
 - **Nexx**
 - **NexxEmbed**
+ - **NFB**
 - **NFHSNetwork**
 - **nfl.com** (Currently broken)
 - **nfl.com:article** (Currently broken)
+ - **NhkForSchoolBangumi**
+ - **NhkForSchoolProgramList**
+ - **NhkForSchoolSubject**: Portal page for each school subjects, like Japanese (kokugo, 国語) or math (sansuu/suugaku or 算数・数学)
 - **NhkVod**
 - **NhkVodProgram**
 - **nhl.com**
@@ -752,7 +770,10 @@
 - **nickelodeonru**
 - **nicknight**
 - **niconico**: ニコニコ動画
- - **NiconicoPlaylist**
+ - **niconico:history**: NicoNico user history. Requires cookies.
+ - **niconico:playlist**
+ - **niconico:series**
+ - **niconico:tag**: NicoNico video tag URLs
 - **NiconicoUser**
 - **nicovideo:search**: Nico video search; "nicosearch:" prefix
 - **nicovideo:search:date**: Nico video search, newest first; "nicosearchdate:" prefix
@@ -851,6 +872,7 @@
 - **PatreonUser**
 - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
 - **PearVideo**
+ - **PeekVids**
 - **peer.tv**
 - **PeerTube**
 - **PeerTube:Playlist**
@@ -863,6 +885,7 @@
 - **PhilharmonieDeParis**: Philharmonie de Paris
 - **phoenix.de**
 - **Photobucket**
+ - **Piapro**
 - **Picarto**
 - **PicartoVod**
 - **Piksel**
@@ -882,6 +905,7 @@
 - **PlaysTV**
 - **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
 - **Playvid**
+ - **PlayVids**
 - **Playwire**
 - **pluralsight**
 - **pluralsight:course**
@@ -986,10 +1010,12 @@
 - **RICE**
 - **RMCDecouverte**
 - **RockstarGames**
+ - **Rokfin**
+ - **rokfin:channel**
+ - **rokfin:stack**
 - **RoosterTeeth**
 - **RoosterTeethSeries**
 - **RottenTomatoes**
- - **Roxwel**
 - **Rozhlas**
 - **RTBF**
 - **RTDocumentry**
@@ -1026,6 +1052,7 @@
 - **RUTV**: RUTV.RU
 - **Ruutu**
 - **Ruv**
+ - **ruv.is:spila**
 - **safari**: safaribooksonline.com online video
 - **safari:api**
 - **safari:course**: safaribooksonline.com online courses
@@ -1165,6 +1192,7 @@
 - **TeleBruxelles**
 - **Telecinco**: telecinco.es, cuatro.com and mediaset.es
 - **Telegraaf**
+ - **telegram:embed**
 - **TeleMB**
 - **Telemundo**
 - **TeleQuebec**
@@ -1181,7 +1209,6 @@
 - **TheIntercept**
 - **ThePlatform**
 - **ThePlatformFeed**
- - **TheScene**
 - **TheStar**
 - **TheSun**
 - **ThetaStream**
@@ -1327,6 +1354,8 @@
 - **video.google:search**: Google Video search; "gvsearch:" prefix
 - **video.sky.it**
 - **video.sky.it:live**
+ - **VideocampusSachsen**
+ - **VideocampusSachsenEmbed**
 - **VideoDetective**
 - **videofy.me**
 - **videomore**
@@ -1369,6 +1398,7 @@
 - **vlive**
 - **vlive:channel**
 - **vlive:post**
+ - **vm.tiktok**
 - **Vodlocker**
 - **VODPl**
 - **VODPlatform**
@@ -1388,7 +1418,6 @@
 - **VShare**
 - **VTM**
 - **VTXTV**
- - **vube**: Vube.com
 - **VuClip**
 - **Vupload**
 - **VVVVID**
@@ -1404,7 +1433,7 @@
 - **WatchBox**
 - **WatchIndianPorn**: Watch Indian Porn
 - **WDR**
- - **wdr:mobile**
+ - **wdr:mobile** (Currently broken)
 - **WDRElefant**
 - **WDRPage**
 - **web.archive:youtube**: web.archive.org saved youtube videos, "ytarchive:" prefix
@@ -1439,6 +1468,7 @@
 - **xiami:song**: 虾米音乐
 - **ximalaya**: 喜马拉雅FM
 - **ximalaya:album**: 喜马拉雅FM 专辑
+ - **xinpianchang**: xinpianchang.com
 - **XMinus**
 - **XNXX**
 - **Xstream**
@@ -1497,7 +1527,7 @@
 - **ZenYandex**
 - **ZenYandexChannel**
 - **Zhihu**
- - **zingmp3**: mp3.zing.vn
+ - **zingmp3**: zingmp3.vn
 - **zingmp3:album**
 - **zoom**
 - **Zype**
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -30,9 +30,7 @@ class YDL(FakeYDL):
        self.msgs = []

    def process_info(self, info_dict):
-        info_dict = info_dict.copy()
-        info_dict.pop('__original_infodict', None)
-        self.downloaded_info_dicts.append(info_dict)
+        self.downloaded_info_dicts.append(info_dict.copy())

    def to_screen(self, msg):
        self.msgs.append(msg)
@@ -898,20 +896,6 @@ class TestYoutubeDL(unittest.TestCase):
        os.unlink(filename)

    def test_match_filter(self):
-        class FilterYDL(YDL):
-            def __init__(self, *args, **kwargs):
-                super(FilterYDL, self).__init__(*args, **kwargs)
-                self.params['simulate'] = True
-
-            def process_info(self, info_dict):
-                super(YDL, self).process_info(info_dict)
-
-            def _match_entry(self, info_dict, incomplete=False):
-                res = super(FilterYDL, self)._match_entry(info_dict, incomplete)
-                if res is None:
-                    self.downloaded_info_dicts.append(info_dict.copy())
-                return res
-
        first = {
            'id': '1',
            'url': TEST_URL,
@@ -939,7 +923,7 @@ class TestYoutubeDL(unittest.TestCase):
        videos = [first, second]

        def get_videos(filter_=None):
-            ydl = FilterYDL({'match_filter': filter_})
+            ydl = YDL({'match_filter': filter_, 'simulate': True})
            for v in videos:
                ydl.process_ie_result(v, download=True)
            return [v['id'] for v in ydl.downloaded_info_dicts]
--- a/test/test_youtube_signature.py
+++ b/test/test_youtube_signature.py
@@ -90,6 +90,10 @@ _NSIG_TESTS = [
        'https://www.youtube.com/s/player/e06dea74/player_ias.vflset/en_US/base.js',
        'AiuodmaDDYw8d3y4bf', 'ankd8eza2T6Qmw',
    ),
+    (
+        'https://www.youtube.com/s/player/5dd88d1d/player-plasma-ias-phone-en_US.vflset/base.js',
+        'kSxKFLeqzv_ZyHSAt', 'n8gS8oRlHOxPFA',
+    ),
 ]


--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -32,6 +32,7 @@ from string import ascii_letters

 from .compat import (
    compat_basestring,
+    compat_brotli,
    compat_get_terminal_size,
    compat_kwargs,
    compat_numeric_types,
@@ -83,6 +84,7 @@ from .utils import (
    make_dir,
    make_HTTPS_handler,
    MaxDownloadsReached,
+    merge_headers,
    network_exceptions,
    number_of_digits,
    orderedSet,
@@ -233,6 +235,8 @@ class YoutubeDL(object):
                       See "Sorting Formats" for more details.
    format_sort_force: Force the given format_sort. see "Sorting Formats"
                       for more details.
+    prefer_free_formats: Whether to prefer video formats with free containers
+                       over non-free ones of same quality.
    allow_multiple_video_streams:   Allow multiple video streams to be merged
                       into a single file
    allow_multiple_audio_streams:   Allow multiple audio streams to be merged
@@ -332,6 +336,7 @@ class YoutubeDL(object):
    nocheckcertificate:  Do not verify SSL certificates
    prefer_insecure:   Use HTTP instead of HTTPS to retrieve information.
                       At the moment, this is only supported by YouTube.
+    http_headers:      A dictionary of custom headers to be used for all requests
    proxy:             URL of the proxy server to use
    geo_verification_proxy:  URL of the proxy to use for IP address verification
                       on geo-restricted sites.
@@ -647,6 +652,9 @@ class YoutubeDL(object):
            else self.params['format'] if callable(self.params['format'])
            else self.build_format_selector(self.params['format']))

+        # Set http_headers defaults according to std_headers
+        self.params['http_headers'] = merge_headers(std_headers, self.params.get('http_headers', {}))
+
        self._setup_opener()

        if auto_init:
@@ -888,7 +896,8 @@ class YoutubeDL(object):
    def _format_text(self, handle, allow_colors, text, f, fallback=None, *, test_encoding=False):
        if test_encoding:
            original_text = text
-            encoding = self.params.get('encoding') or getattr(handle, 'encoding', 'ascii')
+            # handle.encoding can be None. See https://github.com/yt-dlp/yt-dlp/issues/2711
+            encoding = self.params.get('encoding') or getattr(handle, 'encoding', None) or 'ascii'
            text = text.encode(encoding, 'ignore').decode(encoding)
            if fallback is not None and text != original_text:
                text = fallback
@@ -953,13 +962,13 @@ class YoutubeDL(object):
        except UnicodeEncodeError:
            self.to_screen('Deleting existing file')

-    def raise_no_formats(self, info, forced=False):
+    def raise_no_formats(self, info, forced=False, *, msg=None):
        has_drm = info.get('__has_drm')
-        msg = 'This video is DRM protected' if has_drm else 'No video formats found!'
-        expected = self.params.get('ignore_no_formats_error')
-        if forced or not expected:
+        ignored, expected = self.params.get('ignore_no_formats_error'), bool(msg)
+        msg = msg or has_drm and 'This video is DRM protected' or 'No video formats found!'
+        if forced or not ignored:
            raise ExtractorError(msg, video_id=info['id'], ie=info['extractor'],
-                                 expected=has_drm or expected)
+                                 expected=has_drm or ignored or expected)
        else:
            self.report_warning(msg)

@@ -1036,8 +1045,7 @@ class YoutubeDL(object):
    @staticmethod
    def _copy_infodict(info_dict):
        info_dict = dict(info_dict)
-        for key in ('__original_infodict', '__postprocessors'):
-            info_dict.pop(key, None)
+        info_dict.pop('__postprocessors', None)
        return info_dict

    def prepare_outtmpl(self, outtmpl, info_dict, sanitize=False):
@@ -1471,8 +1479,12 @@ class YoutubeDL(object):
            self.add_extra_info(ie_result, {
                'webpage_url': url,
                'original_url': url,
-                'webpage_url_basename': url_basename(url),
-                'webpage_url_domain': get_domain(url),
+            })
+        webpage_url = ie_result.get('webpage_url')
+        if webpage_url:
+            self.add_extra_info(ie_result, {
+                'webpage_url_basename': url_basename(webpage_url),
+                'webpage_url_domain': get_domain(webpage_url),
            })
        if ie is not None:
            self.add_extra_info(ie_result, {
@@ -1580,6 +1592,7 @@ class YoutubeDL(object):

            self._playlist_level += 1
            self._playlist_urls.add(webpage_url)
+            self._fill_common_fields(ie_result, False)
            self._sanitize_thumbnails(ie_result)
            try:
                return self.__process_playlist(ie_result, download)
@@ -1842,15 +1855,21 @@ class YoutubeDL(object):
                '^=': lambda attr, value: attr.startswith(value),
                '$=': lambda attr, value: attr.endswith(value),
                '*=': lambda attr, value: value in attr,
+                '~=': lambda attr, value: value.search(attr) is not None
            }
            str_operator_rex = re.compile(r'''(?x)\s*
                (?P<key>[a-zA-Z0-9._-]+)\s*
-                (?P<negation>!\s*)?(?P<op>%s)(?P<none_inclusive>\s*\?)?\s*
-                (?P<value>[a-zA-Z0-9._-]+)\s*
+                (?P<negation>!\s*)?(?P<op>%s)\s*(?P<none_inclusive>\?\s*)?
+                (?P<quote>["'])?
+                (?P<value>(?(quote)(?:(?!(?P=quote))[^\\]|\\.)+|[\w.-]+))
+                (?(quote)(?P=quote))\s*
                ''' % '|'.join(map(re.escape, STR_OPERATORS.keys())))
            m = str_operator_rex.fullmatch(filter_spec)
            if m:
-                comparison_value = m.group('value')
+                if m.group('op') == '~=':
+                    comparison_value = re.compile(m.group('value'))
+                else:
+                    comparison_value = re.sub(r'''\\([\\"'])''', r'\1', m.group('value'))
                str_op = STR_OPERATORS[m.group('op')]
                if m.group('negation'):
                    op = lambda attr, value: not str_op(attr, value)
@@ -2239,8 +2258,7 @@ class YoutubeDL(object):
        return _build_selector_function(parsed_selector)

    def _calc_headers(self, info_dict):
-        res = std_headers.copy()
-        res.update(info_dict.get('http_headers') or {})
+        res = merge_headers(self.params['http_headers'], info_dict.get('http_headers') or {})

        cookies = self._calc_cookies(info_dict)
        if cookies:
@@ -2298,62 +2316,17 @@ class YoutubeDL(object):
        else:
            info_dict['thumbnails'] = thumbnails

-    def process_video_result(self, info_dict, download=True):
-        assert info_dict.get('_type', 'video') == 'video'
-        self._num_videos += 1
-
-        if 'id' not in info_dict:
-            raise ExtractorError('Missing "id" field in extractor result', ie=info_dict['extractor'])
-        elif not info_dict.get('id'):
-            raise ExtractorError('Extractor failed to obtain "id"', ie=info_dict['extractor'])
-
-        info_dict['fulltitle'] = info_dict.get('title')
-        if 'title' not in info_dict:
-            raise ExtractorError('Missing "title" field in extractor result',
-                                 video_id=info_dict['id'], ie=info_dict['extractor'])
-        elif not info_dict.get('title'):
-            self.report_warning('Extractor failed to obtain "title". Creating a generic title instead')
-            info_dict['title'] = f'{info_dict["extractor"]} video #{info_dict["id"]}'
-
-        def report_force_conversion(field, field_not, conversion):
-            self.report_warning(
-                '"%s" field is not %s - forcing %s conversion, there is an error in extractor'
-                % (field, field_not, conversion))
-
-        def sanitize_string_field(info, string_field):
-            field = info.get(string_field)
-            if field is None or isinstance(field, compat_str):
-                return
-            report_force_conversion(string_field, 'a string', 'string')
-            info[string_field] = compat_str(field)
-
-        def sanitize_numeric_fields(info):
-            for numeric_field in self._NUMERIC_FIELDS:
-                field = info.get(numeric_field)
-                if field is None or isinstance(field, compat_numeric_types):
-                    continue
-                report_force_conversion(numeric_field, 'numeric', 'int')
-                info[numeric_field] = int_or_none(field)
-
-        sanitize_string_field(info_dict, 'id')
-        sanitize_numeric_fields(info_dict)
-
-        if 'playlist' not in info_dict:
-            # It isn't part of a playlist
-            info_dict['playlist'] = None
-            info_dict['playlist_index'] = None
-
-        self._sanitize_thumbnails(info_dict)
-
-        thumbnail = info_dict.get('thumbnail')
-        thumbnails = info_dict.get('thumbnails')
-        if thumbnail:
-            info_dict['thumbnail'] = sanitize_url(thumbnail)
-        elif thumbnails:
-            info_dict['thumbnail'] = thumbnails[-1]['url']
-
-        if info_dict.get('display_id') is None and 'id' in info_dict:
-            info_dict['display_id'] = info_dict['id']
+    def _fill_common_fields(self, info_dict, is_video=True):
+        # TODO: move sanitization here
+        if is_video:
+            # playlists are allowed to lack "title"
+            info_dict['fulltitle'] = info_dict.get('title')
+            if 'title' not in info_dict:
+                raise ExtractorError('Missing "title" field in extractor result',
+                                     video_id=info_dict['id'], ie=info_dict['extractor'])
+            elif not info_dict.get('title'):
+                self.report_warning('Extractor failed to obtain "title". Creating a generic title instead')
+                info_dict['title'] = f'{info_dict["extractor"]} video #{info_dict["id"]}'

        if info_dict.get('duration') is not None:
            info_dict['duration_string'] = formatSeconds(info_dict['duration'])
@@ -2395,6 +2368,59 @@ class YoutubeDL(object):
            if info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
                info_dict[field] = '%s %d' % (field.capitalize(), info_dict['%s_number' % field])

+    def process_video_result(self, info_dict, download=True):
+        assert info_dict.get('_type', 'video') == 'video'
+        self._num_videos += 1
+
+        if 'id' not in info_dict:
+            raise ExtractorError('Missing "id" field in extractor result', ie=info_dict['extractor'])
+        elif not info_dict.get('id'):
+            raise ExtractorError('Extractor failed to obtain "id"', ie=info_dict['extractor'])
+
+        def report_force_conversion(field, field_not, conversion):
+            self.report_warning(
+                '"%s" field is not %s - forcing %s conversion, there is an error in extractor'
+                % (field, field_not, conversion))
+
+        def sanitize_string_field(info, string_field):
+            field = info.get(string_field)
+            if field is None or isinstance(field, compat_str):
+                return
+            report_force_conversion(string_field, 'a string', 'string')
+            info[string_field] = compat_str(field)
+
+        def sanitize_numeric_fields(info):
+            for numeric_field in self._NUMERIC_FIELDS:
+                field = info.get(numeric_field)
+                if field is None or isinstance(field, compat_numeric_types):
+                    continue
+                report_force_conversion(numeric_field, 'numeric', 'int')
+                info[numeric_field] = int_or_none(field)
+
+        sanitize_string_field(info_dict, 'id')
+        sanitize_numeric_fields(info_dict)
+        if (info_dict.get('duration') or 0) <= 0 and info_dict.pop('duration', None):
+            self.report_warning('"duration" field is negative, there is an error in extractor')
+
+        if 'playlist' not in info_dict:
+            # It isn't part of a playlist
+            info_dict['playlist'] = None
+            info_dict['playlist_index'] = None
+
+        self._sanitize_thumbnails(info_dict)
+
+        thumbnail = info_dict.get('thumbnail')
+        thumbnails = info_dict.get('thumbnails')
+        if thumbnail:
+            info_dict['thumbnail'] = sanitize_url(thumbnail)
+        elif thumbnails:
+            info_dict['thumbnail'] = thumbnails[-1]['url']
+
+        if info_dict.get('display_id') is None and 'id' in info_dict:
+            info_dict['display_id'] = info_dict['id']
+
+        self._fill_common_fields(info_dict)
+
        for cc_kind in ('subtitles', 'automatic_captions'):
            cc = info_dict.get(cc_kind)
            if cc:
@@ -2421,11 +2447,14 @@ class YoutubeDL(object):
        if not self.params.get('allow_unplayable_formats'):
            formats = [f for f in formats if not f.get('has_drm')]

-        if info_dict.get('is_live'):
-            get_from_start = bool(self.params.get('live_from_start'))
+        get_from_start = not info_dict.get('is_live') or bool(self.params.get('live_from_start'))
+        if not get_from_start:
+            info_dict['title'] += ' ' + datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
+        if info_dict.get('is_live') and formats:
            formats = [f for f in formats if bool(f.get('is_from_start')) == get_from_start]
-            if not get_from_start:
-                info_dict['title'] += ' ' + datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
+            if get_from_start and not formats:
+                self.raise_no_formats(info_dict, msg='--live-from-start is passed, but there are no formats that can be downloaded from the start. '
+                                                     'If you want to download from the current time, pass --no-live-from-start')

        if not formats:
            self.raise_no_formats(info_dict)
@@ -2501,8 +2530,6 @@ class YoutubeDL(object):
        if '__x_forwarded_for_ip' in info_dict:
            del info_dict['__x_forwarded_for_ip']

-        # TODO Central sorting goes here
-
        if self.params.get('check_formats') is True:
            formats = LazyList(self._check_formats(formats[::-1]), reverse=True)

@@ -2515,6 +2542,12 @@ class YoutubeDL(object):

        info_dict, _ = self.pre_process(info_dict)

+        if self._match_entry(info_dict) is not None:
+            return info_dict
+
+        self.post_extract(info_dict)
+        info_dict, _ = self.pre_process(info_dict, 'after_filter')
+
        # The pre-processors may have modified the formats
        formats = info_dict.get('formats', [info_dict])

@@ -2599,15 +2632,12 @@ class YoutubeDL(object):
                    + ', '.join([f['format_id'] for f in formats_to_download]))
            max_downloads_reached = False
            for i, fmt in enumerate(formats_to_download):
-                formats_to_download[i] = new_info = dict(info_dict)
-                # Save a reference to the original info_dict so that it can be modified in process_info if needed
+                formats_to_download[i] = new_info = self._copy_infodict(info_dict)
                new_info.update(fmt)
-                new_info['__original_infodict'] = info_dict
                try:
                    self.process_info(new_info)
                except MaxDownloadsReached:
                    max_downloads_reached = True
-                new_info.pop('__original_infodict')
                # Remove copied info
                for key, val in tuple(new_info.items()):
                    if info_dict.get(key) == val:
@@ -2652,12 +2682,15 @@ class YoutubeDL(object):
            # given in subtitleslangs. See https://github.com/yt-dlp/yt-dlp/issues/1041
            requested_langs = []
            for lang_re in self.params.get('subtitleslangs'):
-                if lang_re == 'all':
-                    requested_langs.extend(all_sub_langs)
-                    continue
                discard = lang_re[0] == '-'
                if discard:
                    lang_re = lang_re[1:]
+                if lang_re == 'all':
+                    if discard:
+                        requested_langs = []
+                    else:
+                        requested_langs.extend(all_sub_langs)
+                    continue
                current_langs = filter(re.compile(lang_re + '$').match, all_sub_langs)
                if discard:
                    for lang in current_langs:
@@ -2721,8 +2754,9 @@ class YoutubeDL(object):
            filename = self.evaluate_outtmpl(file_tmpl, info_dict)
            tmpl = format_tmpl(tmpl)
            self.to_screen(f'[info] Writing {tmpl!r} to: {filename}')
-            with io.open(filename, 'a', encoding='utf-8') as f:
-                f.write(self.evaluate_outtmpl(tmpl, info_copy) + '\n')
+            if self._ensure_dir_exists(filename):
+                with io.open(filename, 'a', encoding='utf-8') as f:
+                    f.write(self.evaluate_outtmpl(tmpl, info_copy) + '\n')

    def __forced_printings(self, info_dict, filename, incomplete):
        def print_mandatory(field, actual_field=None):
@@ -2811,7 +2845,7 @@ class YoutubeDL(object):
        return None

    def process_info(self, info_dict):
-        """Process a single resolved IE result. (Modified it in-place)"""
+        """Process a single resolved IE result. (Modifies it in-place)"""

        assert info_dict.get('_type', 'video') == 'video'
        original_infodict = info_dict
@@ -2819,18 +2853,22 @@ class YoutubeDL(object):
        if 'format' not in info_dict and 'ext' in info_dict:
            info_dict['format'] = info_dict['ext']

+        # This is mostly just for backward compatibility of process_info
+        # As a side-effect, this allows for format-specific filters
        if self._match_entry(info_dict) is not None:
            info_dict['__write_download_archive'] = 'ignore'
            return

+        # Does nothing under normal operation - for backward compatibility of process_info
        self.post_extract(info_dict)
-        self._num_downloads += 1

        # info_dict['_filename'] needs to be set for backward compatibility
        info_dict['_filename'] = full_filename = self.prepare_filename(info_dict, warn=True)
        temp_filename = self.prepare_filename(info_dict, 'temp')
        files_to_move = {}

+        self._num_downloads += 1
+
        # Forced printings
        self.__forced_printings(info_dict, full_filename, incomplete=('format' not in info_dict))

@@ -2893,9 +2931,11 @@ class YoutubeDL(object):

        # Write internet shortcut files
        def _write_link_file(link_type):
-            if 'webpage_url' not in info_dict:
-                self.report_error('Cannot write internet shortcut file because the "webpage_url" field is missing in the media information')
-                return False
+            url = try_get(info_dict['webpage_url'], iri_to_uri)
+            if not url:
+                self.report_warning(
+                    f'Cannot write internet shortcut file because the actual URL of "{info_dict["webpage_url"]}" is unknown')
+                return True
            linkfn = replace_extension(self.prepare_filename(info_dict, 'link'), link_type, info_dict.get('ext'))
            if not self._ensure_dir_exists(encodeFilename(linkfn)):
                return False
@@ -2906,7 +2946,7 @@ class YoutubeDL(object):
                self.to_screen(f'[info] Writing internet shortcut (.{link_type}) to: {linkfn}')
                with io.open(encodeFilename(to_high_limit_path(linkfn)), 'w', encoding='utf-8',
                             newline='\r\n' if link_type == 'url' else '\n') as linkfile:
-                    template_vars = {'url': iri_to_uri(info_dict['webpage_url'])}
+                    template_vars = {'url': url}
                    if link_type == 'desktop':
                        template_vars['filename'] = linkfn[:-(len(link_type) + 1)]
                    linkfile.write(LINK_TEMPLATES[link_type] % template_vars)
@@ -3041,9 +3081,11 @@ class YoutubeDL(object):
                                'while also allowing unplayable formats to be downloaded. '
                                'The formats won\'t be merged to prevent data corruption.')
                        elif not merger.available:
-                            self.report_warning(
-                                'You have requested merging of multiple formats but ffmpeg is not installed. '
-                                'The formats won\'t be merged.')
+                            msg = 'You have requested merging of multiple formats but ffmpeg is not installed'
+                            if not self.params.get('ignoreerrors'):
+                                self.report_error(f'{msg}. Aborting due to --abort-on-error')
+                                return
+                            self.report_warning(f'{msg}. The formats won\'t be merged')

                        if temp_filename == '-':
                            reason = ('using a downloader other than ffmpeg' if FFmpegFD.can_merge_formats(info_dict, self.params)
@@ -3240,17 +3282,14 @@ class YoutubeDL(object):
            return info_dict
        info_dict.setdefault('epoch', int(time.time()))
        info_dict.setdefault('_type', 'video')
-        remove_keys = {'__original_infodict'}  # Always remove this since this may contain a copy of the entire dict
-        keep_keys = ['_type']  # Always keep this to facilitate load-info-json
+
        if remove_private_keys:
-            remove_keys |= {
+            reject = lambda k, v: v is None or (k.startswith('_') and k != '_type') or k in {
                'requested_downloads', 'requested_formats', 'requested_subtitles', 'requested_entries',
                'entries', 'filepath', 'infojson_filename', 'original_url', 'playlist_autonumber',
            }
-            reject = lambda k, v: k not in keep_keys and (
-                k.startswith('_') or k in remove_keys or v is None)
        else:
-            reject = lambda k, v: k in remove_keys
+            reject = lambda k, v: False

        def filter_fn(obj):
            if isinstance(obj, dict):
@@ -3277,14 +3316,8 @@ class YoutubeDL(object):
                    actual_post_extract(video_dict or {})
                return

-            post_extractor = info_dict.get('__post_extractor') or (lambda: {})
-            extra = post_extractor().items()
-            info_dict.update(extra)
-            info_dict.pop('__post_extractor', None)
-
-            original_infodict = info_dict.get('__original_infodict') or {}
-            original_infodict.update(extra)
-            original_infodict.pop('__post_extractor', None)
+            post_extractor = info_dict.pop('__post_extractor', None) or (lambda: {})
+            info_dict.update(post_extractor())

        actual_post_extract(info_dict or {})

@@ -3562,7 +3595,7 @@ class YoutubeDL(object):
            return

        def get_encoding(stream):
-            ret = getattr(stream, 'encoding', 'missing (%s)' % type(stream).__name__)
+            ret = str(getattr(stream, 'encoding', 'missing (%s)' % type(stream).__name__))
            if not supports_terminal_sequences(stream):
                from .compat import WINDOWS_VT_MODE
                ret += ' (No VT)' if WINDOWS_VT_MODE is False else ' (No ANSI)'
@@ -3645,6 +3678,7 @@ class YoutubeDL(object):
        from .cookies import SQLITE_AVAILABLE, SECRETSTORAGE_AVAILABLE

        lib_str = join_nonempty(
+            compat_brotli and compat_brotli.__name__,
            compat_pycrypto_AES and compat_pycrypto_AES.__name__.split('.')[0],
            SECRETSTORAGE_AVAILABLE and 'secretstorage',
            has_mutagen and 'mutagen',
@@ -3860,7 +3894,7 @@ class YoutubeDL(object):
            else:
                self.to_screen(f'[info] Downloading {thumb_display_id} ...')
                try:
-                    uf = self.urlopen(t['url'])
+                    uf = self.urlopen(sanitized_Request(t['url'], headers=t.get('http_headers', {})))
                    self.to_screen(f'[info] Writing {thumb_display_id} to: {thumb_filename}')
                    with open(encodeFilename(thumb_filename), 'wb') as thumbf:
                        shutil.copyfileobj(uf, thumbf)
--- a/yt_dlp/init.py
+++ b/yt_dlp/init.py
@@ -41,6 +41,7 @@ from .utils import (
    SameFileError,
    setproctitle,
    std_headers,
+    traverse_obj,
    write_string,
 )
 from .update import run_update
@@ -75,20 +76,15 @@ def _real_main(argv=None):
    parser, opts, args = parseOpts(argv)
    warnings, deprecation_warnings = [], []

-    # Set user agent
    if opts.user_agent is not None:
-        std_headers['User-Agent'] = opts.user_agent
-
-    # Set referer
+        opts.headers.setdefault('User-Agent', opts.user_agent)
    if opts.referer is not None:
-        std_headers['Referer'] = opts.referer
-
-    # Custom HTTP headers
-    std_headers.update(opts.headers)
+        opts.headers.setdefault('Referer', opts.referer)

    # Dump user agent
    if opts.dump_user_agent:
-        write_string(std_headers['User-Agent'] + '\n', out=sys.stdout)
+        ua = traverse_obj(opts.headers, 'User-Agent', casesense=False, default=std_headers['User-Agent'])
+        write_string(f'{ua}\n', out=sys.stdout)
        sys.exit(0)

    # Batch file verification
@@ -474,8 +470,8 @@ def _real_main(argv=None):
            'key': 'SponsorBlock',
            'categories': sponsorblock_query,
            'api': opts.sponsorblock_api,
-            # Run this immediately after extraction is complete
-            'when': 'pre_process'
+            # Run this after filtering videos
+            'when': 'after_filter'
        })
    if opts.parse_metadata:
        postprocessors.append({
@@ -767,6 +763,7 @@ def _real_main(argv=None):
        'legacyserverconnect': opts.legacy_server_connect,
        'nocheckcertificate': opts.no_check_certificate,
        'prefer_insecure': opts.prefer_insecure,
+        'http_headers': opts.headers,
        'proxy': opts.proxy,
        'socket_timeout': opts.socket_timeout,
        'bidi_workaround': opts.bidi_workaround,
--- a/yt_dlp/compat.py
+++ b/yt_dlp/compat.py
@@ -134,6 +134,16 @@ except AttributeError:
    asyncio.run = compat_asyncio_run


+try:  # >= 3.7
+    asyncio.tasks.all_tasks
+except AttributeError:
+    asyncio.tasks.all_tasks = asyncio.tasks.Task.all_tasks
+
+try:
+    import websockets as compat_websockets
+except ImportError:
+    compat_websockets = None
+
 # Python 3.8+ does not honor %HOME% on windows, but this breaks compatibility with youtube-dl
 # See https://github.com/yt-dlp/yt-dlp/issues/792
 # https://docs.python.org/3/library/os.path.html#os.path.expanduser
@@ -160,6 +170,13 @@ except ImportError:
    except ImportError:
        compat_pycrypto_AES = None

+try:
+    import brotlicffi as compat_brotli
+except ImportError:
+    try:
+        import brotli as compat_brotli
+    except ImportError:
+        compat_brotli = None

 WINDOWS_VT_MODE = False if compat_os_name == 'nt' else None

@@ -248,6 +265,7 @@ __all__ = [
    'compat_asyncio_run',
    'compat_b64decode',
    'compat_basestring',
+    'compat_brotli',
    'compat_chr',
    'compat_collections_abc',
    'compat_cookiejar',
@@ -303,6 +321,7 @@ __all__ = [
    'compat_urllib_response',
    'compat_urlparse',
    'compat_urlretrieve',
+    'compat_websockets',
    'compat_xml_parse_error',
    'compat_xpath',
    'compat_zip',
--- a/yt_dlp/cookies.py
+++ b/yt_dlp/cookies.py
@@ -454,7 +454,10 @@ def _extract_safari_cookies(profile, logger):
    cookies_path = os.path.expanduser('~/Library/Cookies/Cookies.binarycookies')

    if not os.path.isfile(cookies_path):
-        raise FileNotFoundError('could not find safari cookies database')
+        logger.debug('Trying secondary cookie location')
+        cookies_path = os.path.expanduser('~/Library/Containers/com.apple.Safari/Data/Library/Cookies/Cookies.binarycookies')
+        if not os.path.isfile(cookies_path):
+            raise FileNotFoundError('could not find safari cookies database')

    with open(cookies_path, 'rb') as f:
        cookies_data = f.read()
--- a/yt_dlp/downloader/init.py
+++ b/yt_dlp/downloader/init.py
@@ -30,6 +30,7 @@ def get_suitable_downloader(info_dict, params={}, default=NO_DEFAULT, protocol=N
 from .common import FileDownloader
 from .dash import DashSegmentsFD
 from .f4m import F4mFD
+from .fc2 import FC2LiveFD
 from .hls import HlsFD
 from .http import HttpFD
 from .rtmp import RtmpFD
@@ -58,6 +59,7 @@ PROTOCOL_MAP = {
    'ism': IsmFD,
    'mhtml': MhtmlFD,
    'niconico_dmc': NiconicoDmcFD,
+    'fc2_live': FC2LiveFD,
    'websocket_frag': WebSocketFragmentFD,
    'youtube_live_chat': YoutubeLiveChatFD,
    'youtube_live_chat_replay': YoutubeLiveChatFD,
@@ -117,7 +119,7 @@ def _get_suitable_downloader(info_dict, protocol, params, default):
            return FFmpegFD
        elif (external_downloader or '').lower() == 'native':
            return HlsFD
-        elif get_suitable_downloader(
+        elif protocol == 'm3u8_native' and get_suitable_downloader(
                info_dict, params, None, protocol='m3u8_frag_urls', to_stdout=info_dict['to_stdout']):
            return HlsFD
        elif params.get('hls_prefer_native') is True:
--- a/yt_dlp/downloader/common.py
+++ b/yt_dlp/downloader/common.py
@@ -210,28 +210,41 @@ class FileDownloader(object):
    def ytdl_filename(self, filename):
        return filename + '.ytdl'

-    def sanitize_open(self, filename, open_mode):
-        file_access_retries = self.params.get('file_access_retries', 10)
-        retry = 0
-        while True:
-            try:
-                return sanitize_open(filename, open_mode)
-            except (IOError, OSError) as err:
-                retry = retry + 1
-                if retry > file_access_retries or err.errno not in (errno.EACCES,):
-                    raise
-                self.to_screen(
-                    '[download] Got file access error. Retrying (attempt %d of %s) ...'
-                    % (retry, self.format_retries(file_access_retries)))
-                time.sleep(0.01)
+    def wrap_file_access(action, *, fatal=False):
+        def outer(func):
+            def inner(self, *args, **kwargs):
+                file_access_retries = self.params.get('file_access_retries', 0)
+                retry = 0
+                while True:
+                    try:
+                        return func(self, *args, **kwargs)
+                    except (IOError, OSError) as err:
+                        retry = retry + 1
+                        if retry > file_access_retries or err.errno not in (errno.EACCES, errno.EINVAL):
+                            if not fatal:
+                                self.report_error(f'unable to {action} file: {err}')
+                                return
+                            raise
+                        self.to_screen(
+                            f'[download] Unable to {action} file due to file access error. '
+                            f'Retrying (attempt {retry} of {self.format_retries(file_access_retries)}) ...')
+                        time.sleep(0.01)
+            return inner
+        return outer

+    @wrap_file_access('open', fatal=True)
+    def sanitize_open(self, filename, open_mode):
+        return sanitize_open(filename, open_mode)
+
+    @wrap_file_access('remove')
+    def try_remove(self, filename):
+        os.remove(filename)
+
+    @wrap_file_access('rename')
    def try_rename(self, old_filename, new_filename):
        if old_filename == new_filename:
            return
-        try:
-            os.replace(old_filename, new_filename)
-        except (IOError, OSError) as err:
-            self.report_error(f'unable to rename file: {err}')
+        os.replace(old_filename, new_filename)

    def try_utime(self, filename, last_modified_hdr):
        """Try to set the last-modified time of the given file."""
--- a/yt_dlp/downloader/external.py
+++ b/yt_dlp/downloader/external.py
@@ -159,9 +159,9 @@ class ExternalFD(FragmentFD):
            dest.write(decrypt_fragment(fragment, src.read()))
            src.close()
            if not self.params.get('keep_fragments', False):
-                os.remove(encodeFilename(fragment_filename))
+                self.try_remove(encodeFilename(fragment_filename))
        dest.close()
-        os.remove(encodeFilename('%s.frag.urls' % tmpfilename))
+        self.try_remove(encodeFilename('%s.frag.urls' % tmpfilename))
        return 0


@@ -253,7 +253,7 @@ class Aria2cFD(ExternalFD):
    def _make_cmd(self, tmpfilename, info_dict):
        cmd = [self.exe, '-c',
               '--console-log-level=warn', '--summary-interval=0', '--download-result=hide',
-               '--file-allocation=none', '-x16', '-j16', '-s16']
+               '--http-accept-gzip=true', '--file-allocation=none', '-x16', '-j16', '-s16']
        if 'fragments' in info_dict:
            cmd += ['--allow-overwrite=true', '--allow-piece-length-change=true']
        else:
--- a/yt_dlp/downloader/fc2.py
+++ b/yt_dlp/downloader/fc2.py
@@ -0,0 +1,41 @@
+from __future__ import division, unicode_literals
+
+import threading
+
+from .common import FileDownloader
+from .external import FFmpegFD
+
+
+class FC2LiveFD(FileDownloader):
+    """
+    Downloads FC2 live without being stopped. <br>
+    Note, this is not a part of public API, and will be removed without notice.
+    DO NOT USE
+    """
+
+    def real_download(self, filename, info_dict):
+        ws = info_dict['ws']
+
+        heartbeat_lock = threading.Lock()
+        heartbeat_state = [None, 1]
+
+        def heartbeat():
+            try:
+                heartbeat_state[1] += 1
+                ws.send('{"name":"heartbeat","arguments":{},"id":%d}' % heartbeat_state[1])
+            except Exception:
+                self.to_screen('[fc2:live] Heartbeat failed')
+
+            with heartbeat_lock:
+                heartbeat_state[0] = threading.Timer(30, heartbeat)
+                heartbeat_state[0]._daemonic = True
+                heartbeat_state[0].start()
+
+        heartbeat()
+
+        new_info_dict = info_dict.copy()
+        new_info_dict.update({
+            'ws': None,
+            'protocol': 'live_ffmpeg',
+        })
+        return FFmpegFD(self.ydl, self.params or {}).download(filename, new_info_dict)
--- a/yt_dlp/downloader/fragment.py
+++ b/yt_dlp/downloader/fragment.py
@@ -25,6 +25,7 @@ from ..utils import (
    error_to_compat_str,
    encodeFilename,
    sanitized_Request,
+    traverse_obj,
 )


@@ -136,7 +137,12 @@ class FragmentFD(FileDownloader):
        if fragment_info_dict.get('filetime'):
            ctx['fragment_filetime'] = fragment_info_dict.get('filetime')
        ctx['fragment_filename_sanitized'] = fragment_filename
-        return True, self._read_fragment(ctx)
+        try:
+            return True, self._read_fragment(ctx)
+        except FileNotFoundError:
+            if not info_dict.get('is_live'):
+                raise
+            return False, None

    def _read_fragment(self, ctx):
        down, frag_sanitized = self.sanitize_open(ctx['fragment_filename_sanitized'], 'rb')
@@ -153,7 +159,7 @@ class FragmentFD(FileDownloader):
            if self.__do_ytdl_file(ctx):
                self._write_ytdl_file(ctx)
            if not self.params.get('keep_fragments', False):
-                os.remove(encodeFilename(ctx['fragment_filename_sanitized']))
+                self.try_remove(encodeFilename(ctx['fragment_filename_sanitized']))
            del ctx['fragment_filename_sanitized']

    def _prepare_frag_download(self, ctx):
@@ -172,7 +178,7 @@ class FragmentFD(FileDownloader):
        dl = HttpQuietDownloader(
            self.ydl,
            {
-                'continuedl': True,
+                'continuedl': self.params.get('continuedl', True),
                'quiet': self.params.get('quiet'),
                'noprogress': True,
                'ratelimit': self.params.get('ratelimit'),
@@ -299,7 +305,7 @@ class FragmentFD(FileDownloader):
        if self.__do_ytdl_file(ctx):
            ytdl_filename = encodeFilename(self.ytdl_filename(ctx['filename']))
            if os.path.isfile(ytdl_filename):
-                os.remove(ytdl_filename)
+                self.try_remove(ytdl_filename)
        elapsed = time.time() - ctx['started']

        if ctx['tmpfilename'] == '-':
@@ -382,6 +388,7 @@ class FragmentFD(FileDownloader):
        max_workers = self.params.get('concurrent_fragment_downloads', 1)
        if max_progress > 1:
            self._prepare_multiline_status(max_progress)
+        is_live = any(traverse_obj(args, (..., 2, 'is_live'), default=[]))

        def thread_func(idx, ctx, fragments, info_dict, tpe):
            ctx['max_progress'] = max_progress
@@ -395,25 +402,43 @@ class FragmentFD(FileDownloader):
            def __exit__(self, exc_type, exc_val, exc_tb):
                pass

-        spins = []
        if compat_os_name == 'nt':
-            self.report_warning('Ctrl+C does not work on Windows when used with parallel threads. '
-                                'This is a known issue and patches are welcome')
+            def bindoj_result(future):
+                while True:
+                    try:
+                        return future.result(0.1)
+                    except KeyboardInterrupt:
+                        raise
+                    except concurrent.futures.TimeoutError:
+                        continue
+        else:
+            def bindoj_result(future):
+                return future.result()
+
+        def interrupt_trigger_iter(fg):
+            for f in fg:
+                if not interrupt_trigger[0]:
+                    break
+                yield f
+
+        spins = []
        for idx, (ctx, fragments, info_dict) in enumerate(args):
            tpe = FTPE(math.ceil(max_workers / max_progress))
-            job = tpe.submit(thread_func, idx, ctx, fragments, info_dict, tpe)
+            job = tpe.submit(thread_func, idx, ctx, interrupt_trigger_iter(fragments), info_dict, tpe)
            spins.append((tpe, job))

        result = True
        for tpe, job in spins:
            try:
-                result = result and job.result()
+                result = result and bindoj_result(job)
            except KeyboardInterrupt:
                interrupt_trigger[0] = False
            finally:
                tpe.shutdown(wait=True)
-        if not interrupt_trigger[0]:
+        if not interrupt_trigger[0] and not is_live:
            raise KeyboardInterrupt()
+        # we expect the user wants to stop and DO WANT the preceding postprocessors to run;
+        # so returning a intermediate result here instead of KeyboardInterrupt on live
        return result

    def download_and_append_fragments(
@@ -431,10 +456,11 @@ class FragmentFD(FileDownloader):
            pack_func = lambda frag_content, _: frag_content

        def download_fragment(fragment, ctx):
+            if not interrupt_trigger[0]:
+                return False, fragment['frag_index']
+
            frag_index = ctx['fragment_index'] = fragment['frag_index']
            ctx['last_error'] = None
-            if not interrupt_trigger[0]:
-                return False, frag_index
            headers = info_dict.get('http_headers', {}).copy()
            byte_range = fragment.get('byte_range')
            if byte_range:
@@ -500,8 +526,6 @@ class FragmentFD(FileDownloader):
            self.report_warning('The download speed shown is only of one thread. This is a known issue and patches are welcome')
            with tpe or concurrent.futures.ThreadPoolExecutor(max_workers) as pool:
                for fragment, frag_content, frag_index, frag_filename in pool.map(_download_fragment, fragments):
-                    if not interrupt_trigger[0]:
-                        break
                    ctx['fragment_filename_sanitized'] = frag_filename
                    ctx['fragment_index'] = frag_index
                    result = append_fragment(decrypt_fragment(fragment, frag_content), frag_index, ctx)
--- a/yt_dlp/downloader/http.py
+++ b/yt_dlp/downloader/http.py
@@ -5,7 +5,6 @@ import os
 import socket
 import time
 import random
-import re

 from .common import FileDownloader
 from ..compat import (
@@ -16,6 +15,7 @@ from ..utils import (
    ContentTooShortError,
    encodeFilename,
    int_or_none,
+    parse_http_range,
    sanitized_Request,
    ThrottledDownload,
    write_xattr,
@@ -59,6 +59,9 @@ class HttpFD(FileDownloader):
        ctx.chunk_size = None
        throttle_start = None

+        # parse given Range
+        req_start, req_end, _ = parse_http_range(headers.get('Range'))
+
        if self.params.get('continuedl', True):
            # Establish possible resume length
            if os.path.isfile(encodeFilename(ctx.tmpfilename)):
@@ -91,6 +94,9 @@ class HttpFD(FileDownloader):
                              if not is_test and chunk_size else chunk_size)
            if ctx.resume_len > 0:
                range_start = ctx.resume_len
+                if req_start is not None:
+                    # offset the beginning of Range to be within request
+                    range_start += req_start
                if ctx.is_resume:
                    self.report_resuming_byte(ctx.resume_len)
                ctx.open_mode = 'ab'
@@ -99,7 +105,17 @@ class HttpFD(FileDownloader):
            else:
                range_start = None
            ctx.is_resume = False
-            range_end = range_start + ctx.chunk_size - 1 if ctx.chunk_size else None
+
+            if ctx.chunk_size:
+                chunk_aware_end = range_start + ctx.chunk_size - 1
+                # we're not allowed to download outside Range
+                range_end = chunk_aware_end if req_end is None else min(chunk_aware_end, req_end)
+            elif req_end is not None:
+                # there's no need for chunked downloads, so download until the end of Range
+                range_end = req_end
+            else:
+                range_end = None
+
            if range_end and ctx.data_len is not None and range_end >= ctx.data_len:
                range_end = ctx.data_len - 1
            has_range = range_start is not None
@@ -124,23 +140,19 @@ class HttpFD(FileDownloader):
                # https://github.com/ytdl-org/youtube-dl/issues/6057#issuecomment-126129799)
                if has_range:
                    content_range = ctx.data.headers.get('Content-Range')
-                    if content_range:
-                        content_range_m = re.search(r'bytes (\d+)-(\d+)?(?:/(\d+))?', content_range)
+                    content_range_start, content_range_end, content_len = parse_http_range(content_range)
+                    if content_range_start is not None and range_start == content_range_start:
                        # Content-Range is present and matches requested Range, resume is possible
-                        if content_range_m:
-                            if range_start == int(content_range_m.group(1)):
-                                content_range_end = int_or_none(content_range_m.group(2))
-                                content_len = int_or_none(content_range_m.group(3))
-                                accept_content_len = (
-                                    # Non-chunked download
-                                    not ctx.chunk_size
-                                    # Chunked download and requested piece or
-                                    # its part is promised to be served
-                                    or content_range_end == range_end
-                                    or content_len < range_end)
-                                if accept_content_len:
-                                    ctx.data_len = content_len
-                                    return
+                        accept_content_len = (
+                            # Non-chunked download
+                            not ctx.chunk_size
+                            # Chunked download and requested piece or
+                            # its part is promised to be served
+                            or content_range_end == range_end
+                            or content_len < range_end)
+                        if accept_content_len:
+                            ctx.data_len = content_len
+                            return
                    # Content-Range is either not present or invalid. Assuming remote webserver is
                    # trying to send the whole file, resume is not possible, so wiping the local file
                    # and performing entire redownload
--- a/yt_dlp/downloader/websocket.py
+++ b/yt_dlp/downloader/websocket.py
@@ -5,9 +5,12 @@ import threading

 try:
    import websockets
-    has_websockets = True
-except ImportError:
+except (ImportError, SyntaxError):
+    # websockets 3.10 on python 3.6 causes SyntaxError
+    # See https://github.com/yt-dlp/yt-dlp/issues/2633
    has_websockets = False
+else:
+    has_websockets = True

 from .common import FileDownloader
 from .external import FFmpegFD
--- a/yt_dlp/downloader/youtube_live_chat.py
+++ b/yt_dlp/downloader/youtube_live_chat.py
@@ -22,6 +22,9 @@ class YoutubeLiveChatFD(FragmentFD):
    def real_download(self, filename, info_dict):
        video_id = info_dict['video_id']
        self.to_screen('[%s] Downloading live chat' % self.FD_NAME)
+        if not self.params.get('skip_download'):
+            self.report_warning('Live chat download runs until the livestream ends. '
+                                'If you wish to download the video simultaneously, run a separate yt-dlp instance')

        fragment_retries = self.params.get('fragment_retries', 0)
        test = self.params.get('test', False)
--- a/yt_dlp/extractor/abc.py
+++ b/yt_dlp/extractor/abc.py
@@ -213,7 +213,7 @@ class ABCIViewIE(InfoExtractor):
                'hdnea': token,
            })

-        for sd in ('720', 'sd', 'sd-low'):
+        for sd in ('1080', '720', 'sd', 'sd-low'):
            sd_url = try_get(
                stream, lambda x: x['streams']['hls'][sd], compat_str)
            if not sd_url:
--- a/yt_dlp/extractor/abematv.py
+++ b/yt_dlp/extractor/abematv.py
@@ -0,0 +1,484 @@
+import io
+import json
+import time
+import hashlib
+import hmac
+import re
+import struct
+from base64 import urlsafe_b64encode
+from binascii import unhexlify
+
+from .common import InfoExtractor
+from ..aes import aes_ecb_decrypt
+from ..compat import (
+    compat_urllib_response,
+    compat_urllib_parse_urlparse,
+    compat_urllib_request,
+)
+from ..utils import (
+    ExtractorError,
+    decode_base,
+    int_or_none,
+    random_uuidv4,
+    request_to_url,
+    time_seconds,
+    update_url_query,
+    traverse_obj,
+    intlist_to_bytes,
+    bytes_to_intlist,
+    urljoin,
+)
+
+
+# NOTE: network handler related code is temporary thing until network stack overhaul PRs are merged (#2861/#2862)
+
+def add_opener(ydl, handler):
+    ''' Add a handler for opening URLs, like _download_webpage '''
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L426
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L605
+    assert isinstance(ydl._opener, compat_urllib_request.OpenerDirector)
+    ydl._opener.add_handler(handler)
+
+
+def remove_opener(ydl, handler):
+    '''
+    Remove handler(s) for opening URLs
+    @param handler Either handler object itself or handler type.
+    Specifying handler type will remove all handler which isinstance returns True.
+    '''
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L426
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L605
+    opener = ydl._opener
+    assert isinstance(ydl._opener, compat_urllib_request.OpenerDirector)
+    if isinstance(handler, (type, tuple)):
+        find_cp = lambda x: isinstance(x, handler)
+    else:
+        find_cp = lambda x: x is handler
+
+    removed = []
+    for meth in dir(handler):
+        if meth in ["redirect_request", "do_open", "proxy_open"]:
+            # oops, coincidental match
+            continue
+
+        i = meth.find("_")
+        protocol = meth[:i]
+        condition = meth[i + 1:]
+
+        if condition.startswith("error"):
+            j = condition.find("_") + i + 1
+            kind = meth[j + 1:]
+            try:
+                kind = int(kind)
+            except ValueError:
+                pass
+            lookup = opener.handle_error.get(protocol, {})
+            opener.handle_error[protocol] = lookup
+        elif condition == "open":
+            kind = protocol
+            lookup = opener.handle_open
+        elif condition == "response":
+            kind = protocol
+            lookup = opener.process_response
+        elif condition == "request":
+            kind = protocol
+            lookup = opener.process_request
+        else:
+            continue
+
+        handlers = lookup.setdefault(kind, [])
+        if handlers:
+            handlers[:] = [x for x in handlers if not find_cp(x)]
+
+        removed.append(x for x in handlers if find_cp(x))
+
+    if removed:
+        for x in opener.handlers:
+            if find_cp(x):
+                x.add_parent(None)
+        opener.handlers[:] = [x for x in opener.handlers if not find_cp(x)]
+
+
+class AbemaLicenseHandler(compat_urllib_request.BaseHandler):
+    handler_order = 499
+    STRTABLE = '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'
+    HKEY = b'3AF0298C219469522A313570E8583005A642E73EDD58E3EA2FB7339D3DF1597E'
+
+    def __init__(self, ie: 'AbemaTVIE'):
+        # the protcol that this should really handle is 'abematv-license://'
+        # abematv_license_open is just a placeholder for development purposes
+        # ref. https://github.com/python/cpython/blob/f4c03484da59049eb62a9bf7777b963e2267d187/Lib/urllib/request.py#L510
+        setattr(self, 'abematv-license_open', getattr(self, 'abematv_license_open'))
+        self.ie = ie
+
+    def _get_videokey_from_ticket(self, ticket):
+        to_show = self.ie._downloader.params.get('verbose', False)
+        media_token = self.ie._get_media_token(to_show=to_show)
+
+        license_response = self.ie._download_json(
+            'https://license.abema.io/abematv-hls', None, note='Requesting playback license' if to_show else False,
+            query={'t': media_token},
+            data=json.dumps({
+                'kv': 'a',
+                'lt': ticket
+            }).encode('utf-8'),
+            headers={
+                'Content-Type': 'application/json',
+            })
+
+        res = decode_base(license_response['k'], self.STRTABLE)
+        encvideokey = bytes_to_intlist(struct.pack('>QQ', res >> 64, res & 0xffffffffffffffff))
+
+        h = hmac.new(
+            unhexlify(self.HKEY),
+            (license_response['cid'] + self.ie._DEVICE_ID).encode('utf-8'),
+            digestmod=hashlib.sha256)
+        enckey = bytes_to_intlist(h.digest())
+
+        return intlist_to_bytes(aes_ecb_decrypt(encvideokey, enckey))
+
+    def abematv_license_open(self, url):
+        url = request_to_url(url)
+        ticket = compat_urllib_parse_urlparse(url).netloc
+        response_data = self._get_videokey_from_ticket(ticket)
+        return compat_urllib_response.addinfourl(io.BytesIO(response_data), headers={
+            'Content-Length': len(response_data),
+        }, url=url, code=200)
+
+
+class AbemaTVBaseIE(InfoExtractor):
+    def _extract_breadcrumb_list(self, webpage, video_id):
+        for jld in re.finditer(
+                r'(?is)</span></li></ul><script[^>]+type=(["\']?)application/ld\+json\1[^>]*>(?P<json_ld>.+?)</script>',
+                webpage):
+            jsonld = self._parse_json(jld.group('json_ld'), video_id, fatal=False)
+            if jsonld:
+                if jsonld.get('@type') != 'BreadcrumbList':
+                    continue
+                trav = traverse_obj(jsonld, ('itemListElement', ..., 'name'))
+                if trav:
+                    return trav
+        return []
+
+
+class AbemaTVIE(AbemaTVBaseIE):
+    _VALID_URL = r'https?://abema\.tv/(?P<type>now-on-air|video/episode|channels/.+?/slots)/(?P<id>[^?/]+)'
+    _NETRC_MACHINE = 'abematv'
+    _TESTS = [{
+        'url': 'https://abema.tv/video/episode/194-25_s2_p1',
+        'info_dict': {
+            'id': '194-25_s2_p1',
+            'title': '第1話 「チーズケーキ」　「モーニング再び」',
+            'series': '異世界食堂２',
+            'series_number': 2,
+            'episode': '第1話 「チーズケーキ」　「モーニング再び」',
+            'episode_number': 1,
+        },
+        'skip': 'expired',
+    }, {
+        'url': 'https://abema.tv/channels/anime-live2/slots/E8tvAnMJ7a9a5d',
+        'info_dict': {
+            'id': 'E8tvAnMJ7a9a5d',
+            'title': 'ゆるキャン△ SEASON２ 全話一挙【無料ビデオ72時間】',
+            'series': 'ゆるキャン△ SEASON２',
+            'episode': 'ゆるキャン△ SEASON２ 全話一挙【無料ビデオ72時間】',
+            'series_number': 2,
+            'episode_number': 1,
+            'description': 'md5:9c5a3172ae763278f9303922f0ea5b17',
+        },
+        'skip': 'expired',
+    }, {
+        'url': 'https://abema.tv/video/episode/87-877_s1282_p31047',
+        'info_dict': {
+            'id': 'E8tvAnMJ7a9a5d',
+            'title': '第5話『光射す』',
+            'description': 'md5:56d4fc1b4f7769ded5f923c55bb4695d',
+            'thumbnail': r're:https://hayabusa\.io/.+',
+            'series': '相棒',
+            'episode': '第5話『光射す』',
+        },
+        'skip': 'expired',
+    }, {
+        'url': 'https://abema.tv/now-on-air/abema-anime',
+        'info_dict': {
+            'id': 'abema-anime',
+            # this varies
+            # 'title': '女子高生の無駄づかい 全話一挙【無料ビデオ72時間】',
+            'description': 'md5:55f2e61f46a17e9230802d7bcc913d5f',
+            'is_live': True,
+        },
+        'skip': 'Not supported until yt-dlp implements native live downloader OR AbemaTV can start a local HTTP server',
+    }]
+    _USERTOKEN = None
+    _DEVICE_ID = None
+    _TIMETABLE = None
+    _MEDIATOKEN = None
+
+    _SECRETKEY = b'v+Gjs=25Aw5erR!J8ZuvRrCx*rGswhB&qdHd_SYerEWdU&a?3DzN9BRbp5KwY4hEmcj5#fykMjJ=AuWz5GSMY-d@H7DMEh3M@9n2G552Us$$k9cD=3TxwWe86!x#Zyhe'
+
+    def _generate_aks(self, deviceid):
+        deviceid = deviceid.encode('utf-8')
+        # add 1 hour and then drop minute and secs
+        ts_1hour = int((time_seconds(hours=9) // 3600 + 1) * 3600)
+        time_struct = time.gmtime(ts_1hour)
+        ts_1hour_str = str(ts_1hour).encode('utf-8')
+
+        tmp = None
+
+        def mix_once(nonce):
+            nonlocal tmp
+            h = hmac.new(self._SECRETKEY, digestmod=hashlib.sha256)
+            h.update(nonce)
+            tmp = h.digest()
+
+        def mix_tmp(count):
+            nonlocal tmp
+            for i in range(count):
+                mix_once(tmp)
+
+        def mix_twist(nonce):
+            nonlocal tmp
+            mix_once(urlsafe_b64encode(tmp).rstrip(b'=') + nonce)
+
+        mix_once(self._SECRETKEY)
+        mix_tmp(time_struct.tm_mon)
+        mix_twist(deviceid)
+        mix_tmp(time_struct.tm_mday % 5)
+        mix_twist(ts_1hour_str)
+        mix_tmp(time_struct.tm_hour % 5)
+
+        return urlsafe_b64encode(tmp).rstrip(b'=').decode('utf-8')
+
+    def _get_device_token(self):
+        if self._USERTOKEN:
+            return self._USERTOKEN
+
+        self._DEVICE_ID = random_uuidv4()
+        aks = self._generate_aks(self._DEVICE_ID)
+        user_data = self._download_json(
+            'https://api.abema.io/v1/users', None, note='Authorizing',
+            data=json.dumps({
+                'deviceId': self._DEVICE_ID,
+                'applicationKeySecret': aks,
+            }).encode('utf-8'),
+            headers={
+                'Content-Type': 'application/json',
+            })
+        self._USERTOKEN = user_data['token']
+
+        # don't allow adding it 2 times or more, though it's guarded
+        remove_opener(self._downloader, AbemaLicenseHandler)
+        add_opener(self._downloader, AbemaLicenseHandler(self))
+
+        return self._USERTOKEN
+
+    def _get_media_token(self, invalidate=False, to_show=True):
+        if not invalidate and self._MEDIATOKEN:
+            return self._MEDIATOKEN
+
+        self._MEDIATOKEN = self._download_json(
+            'https://api.abema.io/v1/media/token', None, note='Fetching media token' if to_show else False,
+            query={
+                'osName': 'android',
+                'osVersion': '6.0.1',
+                'osLang': 'ja_JP',
+                'osTimezone': 'Asia/Tokyo',
+                'appId': 'tv.abema',
+                'appVersion': '3.27.1'
+            }, headers={
+                'Authorization': 'bearer ' + self._get_device_token()
+            })['token']
+
+        return self._MEDIATOKEN
+
+    def _real_initialize(self):
+        self._login()
+
+    def _login(self):
+        username, password = self._get_login_info()
+        # No authentication to be performed
+        if not username:
+            return True
+
+        if '@' in username:  # don't strictly check if it's email address or not
+            ep, method = 'user/email', 'email'
+        else:
+            ep, method = 'oneTimePassword', 'userId'
+
+        login_response = self._download_json(
+            f'https://api.abema.io/v1/auth/{ep}', None, note='Logging in',
+            data=json.dumps({
+                method: username,
+                'password': password
+            }).encode('utf-8'), headers={
+                'Authorization': 'bearer ' + self._get_device_token(),
+                'Origin': 'https://abema.tv',
+                'Referer': 'https://abema.tv/',
+                'Content-Type': 'application/json',
+            })
+
+        self._USERTOKEN = login_response['token']
+        self._get_media_token(True)
+
+    def _real_extract(self, url):
+        # starting download using infojson from this extractor is undefined behavior,
+        # and never be fixed in the future; you must trigger downloads by directly specifing URL.
+        # (unless there's a way to hook before downloading by extractor)
+        video_id, video_type = self._match_valid_url(url).group('id', 'type')
+        headers = {
+            'Authorization': 'Bearer ' + self._get_device_token(),
+        }
+        video_type = video_type.split('/')[-1]
+
+        webpage = self._download_webpage(url, video_id)
+        canonical_url = self._search_regex(
+            r'<link\s+rel="canonical"\s*href="(.+?)"', webpage, 'canonical URL',
+            default=url)
+        info = self._search_json_ld(webpage, video_id, default={})
+
+        title = self._search_regex(
+            r'<span\s*class=".+?EpisodeTitleBlock__title">(.+?)</span>', webpage, 'title', default=None)
+        if not title:
+            jsonld = None
+            for jld in re.finditer(
+                    r'(?is)<span\s*class="com-m-Thumbnail__image">(?:</span>)?<script[^>]+type=(["\']?)application/ld\+json\1[^>]*>(?P<json_ld>.+?)</script>',
+                    webpage):
+                jsonld = self._parse_json(jld.group('json_ld'), video_id, fatal=False)
+                if jsonld:
+                    break
+            if jsonld:
+                title = jsonld.get('caption')
+        if not title and video_type == 'now-on-air':
+            if not self._TIMETABLE:
+                # cache the timetable because it goes to 5MiB in size (!!)
+                self._TIMETABLE = self._download_json(
+                    'https://api.abema.io/v1/timetable/dataSet?debug=false', video_id,
+                    headers=headers)
+            now = time_seconds(hours=9)
+            for slot in self._TIMETABLE.get('slots', []):
+                if slot.get('channelId') != video_id:
+                    continue
+                if slot['startAt'] <= now and now < slot['endAt']:
+                    title = slot['title']
+                    break
+
+        # read breadcrumb on top of page
+        breadcrumb = self._extract_breadcrumb_list(webpage, video_id)
+        if breadcrumb:
+            # breadcrumb list translates to: (example is 1st test for this IE)
+            # Home > Anime (genre) > Isekai Shokudo 2 (series name) > Episode 1 "Cheese cakes" "Morning again" (episode title)
+            # hence this works
+            info['series'] = breadcrumb[-2]
+            info['episode'] = breadcrumb[-1]
+            if not title:
+                title = info['episode']
+
+        description = self._html_search_regex(
+            (r'<p\s+class="com-video-EpisodeDetailsBlock__content"><span\s+class=".+?">(.+?)</span></p><div',
+             r'<span\s+class=".+?SlotSummary.+?">(.+?)</span></div><div',),
+            webpage, 'description', default=None, group=1)
+        if not description:
+            og_desc = self._html_search_meta(
+                ('description', 'og:description', 'twitter:description'), webpage)
+            if og_desc:
+                description = re.sub(r'''(?sx)
+                    ^(.+?)(?:
+                        アニメの動画を無料で見るならABEMA！| # anime
+                        等、.+ # applies for most of categories
+                    )?
+                ''', r'\1', og_desc)
+
+        # canonical URL may contain series and episode number
+        mobj = re.search(r's(\d+)_p(\d+)$', canonical_url)
+        if mobj:
+            seri = int_or_none(mobj.group(1), default=float('inf'))
+            epis = int_or_none(mobj.group(2), default=float('inf'))
+            info['series_number'] = seri if seri < 100 else None
+            # some anime like Detective Conan (though not available in AbemaTV)
+            # has more than 1000 episodes (1026 as of 2021/11/15)
+            info['episode_number'] = epis if epis < 2000 else None
+
+        is_live, m3u8_url = False, None
+        if video_type == 'now-on-air':
+            is_live = True
+            channel_url = 'https://api.abema.io/v1/channels'
+            if video_id == 'news-global':
+                channel_url = update_url_query(channel_url, {'division': '1'})
+            onair_channels = self._download_json(channel_url, video_id)
+            for ch in onair_channels['channels']:
+                if video_id == ch['id']:
+                    m3u8_url = ch['playback']['hls']
+                    break
+            else:
+                raise ExtractorError(f'Cannot find on-air {video_id} channel.', expected=True)
+        elif video_type == 'episode':
+            api_response = self._download_json(
+                f'https://api.abema.io/v1/video/programs/{video_id}', video_id,
+                note='Checking playability',
+                headers=headers)
+            ondemand_types = traverse_obj(api_response, ('terms', ..., 'onDemandType'), default=[])
+            if 3 not in ondemand_types:
+                # cannot acquire decryption key for these streams
+                self.report_warning('This is a premium-only stream')
+
+            m3u8_url = f'https://vod-abematv.akamaized.net/program/{video_id}/playlist.m3u8'
+        elif video_type == 'slots':
+            api_response = self._download_json(
+                f'https://api.abema.io/v1/media/slots/{video_id}', video_id,
+                note='Checking playability',
+                headers=headers)
+            if not traverse_obj(api_response, ('slot', 'flags', 'timeshiftFree'), default=False):
+                self.report_warning('This is a premium-only stream')
+
+            m3u8_url = f'https://vod-abematv.akamaized.net/slot/{video_id}/playlist.m3u8'
+        else:
+            raise ExtractorError('Unreachable')
+
+        if is_live:
+            self.report_warning("This is a livestream; yt-dlp doesn't support downloading natively, but FFmpeg cannot handle m3u8 manifests from AbemaTV")
+            self.report_warning('Please consider using Streamlink to download these streams (https://github.com/streamlink/streamlink)')
+        formats = self._extract_m3u8_formats(
+            m3u8_url, video_id, ext='mp4', live=is_live)
+
+        info.update({
+            'id': video_id,
+            'title': title,
+            'description': description,
+            'formats': formats,
+            'is_live': is_live,
+        })
+        return info
+
+
+class AbemaTVTitleIE(AbemaTVBaseIE):
+    _VALID_URL = r'https?://abema\.tv/video/title/(?P<id>[^?/]+)'
+
+    _TESTS = [{
+        'url': 'https://abema.tv/video/title/90-1597',
+        'info_dict': {
+            'id': '90-1597',
+            'title': 'シャッフルアイランド',
+        },
+        'playlist_mincount': 2,
+    }, {
+        'url': 'https://abema.tv/video/title/193-132',
+        'info_dict': {
+            'id': '193-132',
+            'title': '真心が届く~僕とスターのオフィス・ラブ!?~',
+        },
+        'playlist_mincount': 16,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        playlist_title, breadcrumb = None, self._extract_breadcrumb_list(webpage, video_id)
+        if breadcrumb:
+            playlist_title = breadcrumb[-1]
+
+        playlist = [
+            self.url_result(urljoin('https://abema.tv/', mobj.group(1)))
+            for mobj in re.finditer(r'<li\s*class=".+?EpisodeList.+?"><a\s*href="(/[^"]+?)"', webpage)]
+
+        return self.playlist_result(playlist, playlist_title=playlist_title, playlist_id=video_id)
--- a/yt_dlp/extractor/adobepass.py
+++ b/yt_dlp/extractor/adobepass.py
@@ -1345,6 +1345,11 @@ MSO_INFO = {
        'username_field': 'username',
        'password_field': 'password',
    },
+    'Suddenlink': {
+        'name': 'Suddenlink',
+        'username_field': 'username',
+        'password_field': 'password',
+    },
 }


@@ -1635,6 +1640,52 @@ class AdobePassIE(InfoExtractor):
                        urlh.geturl(), video_id, 'Sending final bookend',
                        query=hidden_data)

+                    post_form(mvpd_confirm_page_res, 'Confirming Login')
+                elif mso_id == 'Suddenlink':
+                    # Suddenlink is similar to SlingTV in using a tab history count and a meta refresh,
+                    # but they also do a dynmaic redirect using javascript that has to be followed as well
+                    first_bookend_page, urlh = post_form(
+                        provider_redirect_page_res, 'Pressing Continue...')
+
+                    hidden_data = self._hidden_inputs(first_bookend_page)
+                    hidden_data['history_val'] = 1
+
+                    provider_login_redirect_page = self._download_webpage(
+                        urlh.geturl(), video_id, 'Sending First Bookend',
+                        query=hidden_data)
+
+                    provider_tryauth_url = self._html_search_regex(
+                        r'url:\s*[\'"]([^\'"]+)', provider_login_redirect_page, 'ajaxurl')
+
+                    provider_tryauth_page = self._download_webpage(
+                        provider_tryauth_url, video_id, 'Submitting TryAuth',
+                        query=hidden_data)
+
+                    provider_login_page_res = self._download_webpage_handle(
+                        f'https://authorize.suddenlink.net/saml/module.php/authSynacor/login.php?AuthState={provider_tryauth_page}',
+                        video_id, 'Getting Login Page',
+                        query=hidden_data)
+
+                    provider_association_redirect, urlh = post_form(
+                        provider_login_page_res, 'Logging in', {
+                            mso_info['username_field']: username,
+                            mso_info['password_field']: password
+                        })
+
+                    provider_refresh_redirect_url = extract_redirect_url(
+                        provider_association_redirect, url=urlh.geturl())
+
+                    last_bookend_page, urlh = self._download_webpage_handle(
+                        provider_refresh_redirect_url, video_id,
+                        'Downloading Auth Association Redirect Page')
+
+                    hidden_data = self._hidden_inputs(last_bookend_page)
+                    hidden_data['history_val'] = 3
+
+                    mvpd_confirm_page_res = self._download_webpage_handle(
+                        urlh.geturl(), video_id, 'Sending Final Bookend',
+                        query=hidden_data)
+
                    post_form(mvpd_confirm_page_res, 'Confirming Login')
                else:
                    # Some providers (e.g. DIRECTV NOW) have another meta refresh
--- a/yt_dlp/extractor/afreecatv.py
+++ b/yt_dlp/extractor/afreecatv.py
@@ -416,26 +416,35 @@ class AfreecaTVLiveIE(AfreecaTVIE):

    def _real_extract(self, url):
        broadcaster_id, broadcast_no = self._match_valid_url(url).group('id', 'bno')
+        password = self.get_param('videopassword')

        info = self._download_json(self._LIVE_API_URL, broadcaster_id, fatal=False,
                                   data=urlencode_postdata({'bid': broadcaster_id})) or {}
        channel_info = info.get('CHANNEL') or {}
        broadcaster_id = channel_info.get('BJID') or broadcaster_id
        broadcast_no = channel_info.get('BNO') or broadcast_no
+        password_protected = channel_info.get('BPWD')
        if not broadcast_no:
            raise ExtractorError(f'Unable to extract broadcast number ({broadcaster_id} may not be live)', expected=True)
+        if password_protected == 'Y' and password is None:
+            raise ExtractorError(
+                'This livestream is protected by a password, use the --video-password option',
+                expected=True)

        formats = []
        quality_key = qualities(self._QUALITIES)
        for quality_str in self._QUALITIES:
+            params = {
+                'bno': broadcast_no,
+                'stream_type': 'common',
+                'type': 'aid',
+                'quality': quality_str,
+            }
+            if password is not None:
+                params['pwd'] = password
            aid_response = self._download_json(
                self._LIVE_API_URL, broadcast_no, fatal=False,
-                data=urlencode_postdata({
-                    'bno': broadcast_no,
-                    'stream_type': 'common',
-                    'type': 'aid',
-                    'quality': quality_str,
-                }),
+                data=urlencode_postdata(params),
                note=f'Downloading access token for {quality_str} stream',
                errnote=f'Unable to download access token for {quality_str} stream')
            aid = traverse_obj(aid_response, ('CHANNEL', 'AID'))
--- a/yt_dlp/extractor/aliexpress.py
+++ b/yt_dlp/extractor/aliexpress.py
@@ -18,7 +18,7 @@ class AliExpressLiveIE(InfoExtractor):
            'id': '2800002704436634',
            'ext': 'mp4',
            'title': 'CASIMA7.22',
-            'thumbnail': r're:http://.*\.jpg',
+            'thumbnail': r're:https?://.*\.jpg',
            'uploader': 'CASIMA Official Store',
            'timestamp': 1500717600,
            'upload_date': '20170722',
--- a/yt_dlp/extractor/alsace20tv.py
+++ b/yt_dlp/extractor/alsace20tv.py
@@ -0,0 +1,87 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import (
+    clean_html,
+    dict_get,
+    get_element_by_class,
+    int_or_none,
+    unified_strdate,
+    url_or_none,
+)
+
+
+class Alsace20TVBaseIE(InfoExtractor):
+    def _extract_video(self, video_id, url=None):
+        info = self._download_json(
+            'https://www.alsace20.tv/visionneuse/visio_v9_js.php?key=%s&habillage=0&mode=html' % (video_id, ),
+            video_id) or {}
+        title = info.get('titre')
+
+        formats = []
+        for res, fmt_url in (info.get('files') or {}).items():
+            formats.extend(
+                self._extract_smil_formats(fmt_url, video_id, fatal=False)
+                if '/smil:_' in fmt_url
+                else self._extract_mpd_formats(fmt_url, video_id, mpd_id=res, fatal=False))
+        self._sort_formats(formats)
+
+        webpage = (url and self._download_webpage(url, video_id, fatal=False)) or ''
+        thumbnail = url_or_none(dict_get(info, ('image', 'preview', )) or self._og_search_thumbnail(webpage))
+        upload_date = self._search_regex(r'/(\d{6})_', thumbnail, 'upload_date', default=None)
+        upload_date = unified_strdate('20%s-%s-%s' % (upload_date[:2], upload_date[2:4], upload_date[4:])) if upload_date else None
+        return {
+            'id': video_id,
+            'title': title,
+            'formats': formats,
+            'description': clean_html(get_element_by_class('wysiwyg', webpage)),
+            'upload_date': upload_date,
+            'thumbnail': thumbnail,
+            'duration': int_or_none(self._og_search_property('video:duration', webpage) if webpage else None),
+            'view_count': int_or_none(info.get('nb_vues')),
+        }
+
+
+class Alsace20TVIE(Alsace20TVBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?alsace20\.tv/(?:[\w-]+/)+[\w-]+-(?P<id>[\w]+)'
+    _TESTS = [{
+        'url': 'https://www.alsace20.tv/VOD/Actu/JT/Votre-JT-jeudi-3-fevrier-lyNHCXpYJh.html',
+        'info_dict': {
+            'id': 'lyNHCXpYJh',
+            'ext': 'mp4',
+            'description': 'md5:fc0bc4a0692d3d2dba4524053de4c7b7',
+            'title': 'Votre JT du jeudi 3 février',
+            'upload_date': '20220203',
+            'thumbnail': r're:https?://.+\.jpg',
+            'duration': 1073,
+            'view_count': int,
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        return self._extract_video(video_id, url)
+
+
+class Alsace20TVEmbedIE(Alsace20TVBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?alsace20\.tv/emb/(?P<id>[\w]+)'
+    _TESTS = [{
+        'url': 'https://www.alsace20.tv/emb/lyNHCXpYJh',
+        # 'md5': 'd91851bf9af73c0ad9b2cdf76c127fbb',
+        'info_dict': {
+            'id': 'lyNHCXpYJh',
+            'ext': 'mp4',
+            'title': 'Votre JT du jeudi 3 février',
+            'upload_date': '20220203',
+            'thumbnail': r're:https?://.+\.jpg',
+            'view_count': int,
+        },
+        'params': {
+            'format': 'bestvideo',
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        return self._extract_video(video_id)
--- a/yt_dlp/extractor/ant1newsgr.py
+++ b/yt_dlp/extractor/ant1newsgr.py
@@ -0,0 +1,143 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+import urllib.parse
+
+from .common import InfoExtractor
+from ..utils import (
+    HEADRequest,
+    ExtractorError,
+    determine_ext,
+    scale_thumbnails_to_max_format_width,
+    unescapeHTML,
+)
+
+
+class Ant1NewsGrBaseIE(InfoExtractor):
+    def _download_and_extract_api_data(self, video_id, netloc, cid=None):
+        url = f'{self.http_scheme()}//{netloc}{self._API_PATH}'
+        info = self._download_json(url, video_id, query={'cid': cid or video_id})
+        try:
+            source = info['url']
+        except KeyError:
+            raise ExtractorError('no source found for %s' % video_id)
+        formats, subs = (self._extract_m3u8_formats_and_subtitles(source, video_id, 'mp4')
+                         if determine_ext(source) == 'm3u8' else ([{'url': source}], {}))
+        self._sort_formats(formats)
+        thumbnails = scale_thumbnails_to_max_format_width(
+            formats, [{'url': info['thumb']}], r'(?<=/imgHandler/)\d+')
+        return {
+            'id': video_id,
+            'title': info.get('title'),
+            'thumbnails': thumbnails,
+            'formats': formats,
+            'subtitles': subs,
+        }
+
+
+class Ant1NewsGrWatchIE(Ant1NewsGrBaseIE):
+    IE_NAME = 'ant1newsgr:watch'
+    IE_DESC = 'ant1news.gr videos'
+    _VALID_URL = r'https?://(?P<netloc>(?:www\.)?ant1news\.gr)/watch/(?P<id>\d+)/'
+    _API_PATH = '/templates/data/player'
+
+    _TESTS = [{
+        'url': 'https://www.ant1news.gr/watch/1506168/ant1-news-09112021-stis-18-45',
+        'md5': '95925e6b32106754235f2417e0d2dfab',
+        'info_dict': {
+            'id': '1506168',
+            'ext': 'mp4',
+            'title': 'md5:0ad00fa66ecf8aa233d26ab0dba7514a',
+            'description': 'md5:18665af715a6dcfeac1d6153a44f16b0',
+            'thumbnail': 'https://ant1media.azureedge.net/imgHandler/640/26d46bf6-8158-4f02-b197-7096c714b2de.jpg',
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id, netloc = self._match_valid_url(url).group('id', 'netloc')
+        webpage = self._download_webpage(url, video_id)
+        info = self._download_and_extract_api_data(video_id, netloc)
+        info['description'] = self._og_search_description(webpage)
+        return info
+
+
+class Ant1NewsGrArticleIE(Ant1NewsGrBaseIE):
+    IE_NAME = 'ant1newsgr:article'
+    IE_DESC = 'ant1news.gr articles'
+    _VALID_URL = r'https?://(?:www\.)?ant1news\.gr/[^/]+/article/(?P<id>\d+)/'
+
+    _TESTS = [{
+        'url': 'https://www.ant1news.gr/afieromata/article/549468/o-tzeims-mpont-sta-meteora-oi-apeiles-kai-o-xesikomos-ton-kalogeron',
+        'md5': '294f18331bb516539d72d85a82887dcc',
+        'info_dict': {
+            'id': '_xvg/m_cmbatw=',
+            'ext': 'mp4',
+            'title': 'md5:a93e8ecf2e4073bfdffcb38f59945411',
+            'timestamp': 1603092840,
+            'upload_date': '20201019',
+            'thumbnail': 'https://ant1media.azureedge.net/imgHandler/640/756206d2-d640-40e2-b201-3555abdfc0db.jpg',
+        },
+    }, {
+        'url': 'https://ant1news.gr/Society/article/620286/symmoria-anilikon-dikigoros-thymaton-ithelan-na-toys-apoteleiosoyn',
+        'info_dict': {
+            'id': '620286',
+            'title': 'md5:91fe569e952e4d146485740ae927662b',
+        },
+        'playlist_mincount': 2,
+        'params': {
+            'skip_download': True,
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        info = self._search_json_ld(webpage, video_id, expected_type='NewsArticle')
+        embed_urls = list(Ant1NewsGrEmbedIE._extract_urls(webpage))
+        if not embed_urls:
+            raise ExtractorError('no videos found for %s' % video_id, expected=True)
+        return self.playlist_from_matches(
+            embed_urls, video_id, info.get('title'), ie=Ant1NewsGrEmbedIE.ie_key(),
+            video_kwargs={'url_transparent': True, 'timestamp': info.get('timestamp')})
+
+
+class Ant1NewsGrEmbedIE(Ant1NewsGrBaseIE):
+    IE_NAME = 'ant1newsgr:embed'
+    IE_DESC = 'ant1news.gr embedded videos'
+    _BASE_PLAYER_URL_RE = r'(?:https?:)?//(?:[a-zA-Z0-9\-]+\.)?(?:antenna|ant1news)\.gr/templates/pages/player'
+    _VALID_URL = rf'{_BASE_PLAYER_URL_RE}\?([^#]+&)?cid=(?P<id>[^#&]+)'
+    _API_PATH = '/news/templates/data/jsonPlayer'
+
+    _TESTS = [{
+        'url': 'https://www.antenna.gr/templates/pages/player?cid=3f_li_c_az_jw_y_u=&w=670&h=377',
+        'md5': 'dfc58c3a11a5a9aad2ba316ed447def3',
+        'info_dict': {
+            'id': '3f_li_c_az_jw_y_u=',
+            'ext': 'mp4',
+            'title': 'md5:a30c93332455f53e1e84ae0724f0adf7',
+            'thumbnail': 'https://ant1media.azureedge.net/imgHandler/640/bbe31201-3f09-4a4e-87f5-8ad2159fffe2.jpg',
+        },
+    }]
+
+    @classmethod
+    def _extract_urls(cls, webpage):
+        _EMBED_URL_RE = rf'{cls._BASE_PLAYER_URL_RE}\?(?:(?!(?P=_q1)).)+'
+        _EMBED_RE = rf'<iframe[^>]+?src=(?P<_q1>["\'])(?P<url>{_EMBED_URL_RE})(?P=_q1)'
+        for mobj in re.finditer(_EMBED_RE, webpage):
+            url = unescapeHTML(mobj.group('url'))
+            if not cls.suitable(url):
+                continue
+            yield url
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        canonical_url = self._request_webpage(
+            HEADRequest(url), video_id,
+            note='Resolve canonical player URL',
+            errnote='Could not resolve canonical player URL').geturl()
+        _, netloc, _, _, query, _ = urllib.parse.urlparse(canonical_url)
+        cid = urllib.parse.parse_qs(query)['cid'][0]
+
+        return self._download_and_extract_api_data(video_id, netloc, cid=cid)
--- a/yt_dlp/extractor/applepodcasts.py
+++ b/yt_dlp/extractor/applepodcasts.py
@@ -3,7 +3,9 @@ from __future__ import unicode_literals

 from .common import InfoExtractor
 from ..utils import (
+    clean_html,
    clean_podcast_url,
+    get_element_by_class,
    int_or_none,
    parse_iso8601,
    try_get,
@@ -14,16 +16,17 @@ class ApplePodcastsIE(InfoExtractor):
    _VALID_URL = r'https?://podcasts\.apple\.com/(?:[^/]+/)?podcast(?:/[^/]+){1,2}.*?\bi=(?P<id>\d+)'
    _TESTS = [{
        'url': 'https://podcasts.apple.com/us/podcast/207-whitney-webb-returns/id1135137367?i=1000482637777',
-        'md5': 'df02e6acb11c10e844946a39e7222b08',
+        'md5': '41dc31cd650143e530d9423b6b5a344f',
        'info_dict': {
            'id': '1000482637777',
            'ext': 'mp3',
            'title': '207 - Whitney Webb Returns',
-            'description': 'md5:13a73bade02d2e43737751e3987e1399',
+            'description': 'md5:75ef4316031df7b41ced4e7b987f79c6',
            'upload_date': '20200705',
-            'timestamp': 1593921600,
-            'duration': 6425,
+            'timestamp': 1593932400,
+            'duration': 6454,
            'series': 'The Tim Dillon Show',
+            'thumbnail': 're:.+[.](png|jpe?g|webp)',
        }
    }, {
        'url': 'https://podcasts.apple.com/podcast/207-whitney-webb-returns/id1135137367?i=1000482637777',
@@ -39,24 +42,47 @@ class ApplePodcastsIE(InfoExtractor):
    def _real_extract(self, url):
        episode_id = self._match_id(url)
        webpage = self._download_webpage(url, episode_id)
-        ember_data = self._parse_json(self._search_regex(
-            r'id="shoebox-ember-data-store"[^>]*>\s*({.+?})\s*<',
-            webpage, 'ember data'), episode_id)
-        ember_data = ember_data.get(episode_id) or ember_data
-        episode = ember_data['data']['attributes']
+        episode_data = {}
+        ember_data = {}
+        # new page type 2021-11
+        amp_data = self._parse_json(self._search_regex(
+            r'(?s)id="shoebox-media-api-cache-amp-podcasts"[^>]*>\s*({.+?})\s*<',
+            webpage, 'AMP data', default='{}'), episode_id, fatal=False) or {}
+        amp_data = try_get(amp_data,
+                           lambda a: self._parse_json(
+                               next(a[x] for x in iter(a) if episode_id in x),
+                               episode_id),
+                           dict) or {}
+        amp_data = amp_data.get('d') or []
+        episode_data = try_get(
+            amp_data,
+            lambda a: next(x for x in a
+                           if x['type'] == 'podcast-episodes' and x['id'] == episode_id),
+            dict)
+        if not episode_data:
+            # try pre 2021-11 page type: TODO: consider deleting if no longer used
+            ember_data = self._parse_json(self._search_regex(
+                r'(?s)id="shoebox-ember-data-store"[^>]*>\s*({.+?})\s*<',
+                webpage, 'ember data'), episode_id) or {}
+            ember_data = ember_data.get(episode_id) or ember_data
+            episode_data = try_get(ember_data, lambda x: x['data'], dict)
+        episode = episode_data['attributes']
        description = episode.get('description') or {}

        series = None
-        for inc in (ember_data.get('included') or []):
+        for inc in (amp_data or ember_data.get('included') or []):
            if inc.get('type') == 'media/podcast':
                series = try_get(inc, lambda x: x['attributes']['name'])
+        series = series or clean_html(get_element_by_class('podcast-header__identity', webpage))

        return {
            'id': episode_id,
-            'title': episode['name'],
+            'title': episode.get('name'),
            'url': clean_podcast_url(episode['assetUrl']),
            'description': description.get('standard') or description.get('short'),
            'timestamp': parse_iso8601(episode.get('releaseDateTime')),
            'duration': int_or_none(episode.get('durationInMilliseconds'), 1000),
            'series': series,
+            'thumbnail': self._og_search_thumbnail(webpage),
+            'vcodec': 'none',
        }
--- a/yt_dlp/extractor/arcpublishing.py
+++ b/yt_dlp/extractor/arcpublishing.py
@@ -124,8 +124,7 @@ class ArcPublishingIE(InfoExtractor):
                formats.extend(smil_formats)
            elif stream_type in ('ts', 'hls'):
                m3u8_formats = self._extract_m3u8_formats(
-                    s_url, uuid, 'mp4', 'm3u8' if is_live else 'm3u8_native',
-                    m3u8_id='hls', fatal=False)
+                    s_url, uuid, 'mp4', live=is_live, m3u8_id='hls', fatal=False)
                if all([f.get('acodec') == 'none' for f in m3u8_formats]):
                    continue
                for f in m3u8_formats:
--- a/yt_dlp/extractor/ard.py
+++ b/yt_dlp/extractor/ard.py
@@ -407,8 +407,9 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
        (?:(?:beta|www)\.)?ardmediathek\.de/
        (?:(?P<client>[^/]+)/)?
        (?:player|live|video|(?P<playlist>sendung|sammlung))/
-        (?:(?P<display_id>[^?#]+)/)?
-        (?P<id>(?(playlist)|Y3JpZDovL)[a-zA-Z0-9]+)'''
+        (?:(?P<display_id>(?(playlist)[^?#]+?|[^?#]+))/)?
+        (?P<id>(?(playlist)|Y3JpZDovL)[a-zA-Z0-9]+)
+        (?(playlist)/(?P<season>\d+)?/?(?:[?#]|$))'''

    _TESTS = [{
        'url': 'https://www.ardmediathek.de/mdr/video/die-robuste-roswita/Y3JpZDovL21kci5kZS9iZWl0cmFnL2Ntcy84MWMxN2MzZC0wMjkxLTRmMzUtODk4ZS0wYzhlOWQxODE2NGI/',
@@ -436,6 +437,13 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
            'description': 'md5:39578c7b96c9fe50afdf5674ad985e6b',
            'upload_date': '20211108',
        },
+    }, {
+        'url': 'https://www.ardmediathek.de/sendung/beforeigners/beforeigners/staffel-1/Y3JpZDovL2Rhc2Vyc3RlLmRlL2JlZm9yZWlnbmVycw/1',
+        'playlist_count': 6,
+        'info_dict': {
+            'id': 'Y3JpZDovL2Rhc2Vyc3RlLmRlL2JlZm9yZWlnbmVycw',
+            'title': 'beforeigners/beforeigners/staffel-1',
+        },
    }, {
        'url': 'https://beta.ardmediathek.de/ard/video/Y3JpZDovL2Rhc2Vyc3RlLmRlL3RhdG9ydC9mYmM4NGM1NC0xNzU4LTRmZGYtYWFhZS0wYzcyZTIxNGEyMDE',
        'only_matching': True,
@@ -561,14 +569,15 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
                break
            pageNumber = pageNumber + 1

-        return self.playlist_result(entries, playlist_title=display_id)
+        return self.playlist_result(entries, playlist_id, playlist_title=display_id)

    def _real_extract(self, url):
-        video_id, display_id, playlist_type, client = self._match_valid_url(url).group(
-            'id', 'display_id', 'playlist', 'client')
+        video_id, display_id, playlist_type, client, season_number = self._match_valid_url(url).group(
+            'id', 'display_id', 'playlist', 'client', 'season')
        display_id, client = display_id or video_id, client or 'ard'

        if playlist_type:
+            # TODO: Extract only specified season
            return self._ARD_extract_playlist(url, video_id, display_id, client, playlist_type)

        player_page = self._download_json(
--- a/yt_dlp/extractor/arte.py
+++ b/yt_dlp/extractor/arte.py
@@ -12,6 +12,7 @@ from ..utils import (
    int_or_none,
    parse_qs,
    qualities,
+    strip_or_none,
    try_get,
    unified_strdate,
    url_or_none,
@@ -253,3 +254,44 @@ class ArteTVPlaylistIE(ArteTVBaseIE):
        title = collection.get('title')
        description = collection.get('shortDescription') or collection.get('teaserText')
        return self.playlist_result(entries, playlist_id, title, description)
+
+
+class ArteTVCategoryIE(ArteTVBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?arte\.tv/(?P<lang>%s)/videos/(?P<id>[\w-]+(?:/[\w-]+)*)/?\s*$' % ArteTVBaseIE._ARTE_LANGUAGES
+    _TESTS = [{
+        'url': 'https://www.arte.tv/en/videos/politics-and-society/',
+        'info_dict': {
+            'id': 'politics-and-society',
+            'title': 'Politics and society',
+            'description': 'Investigative documentary series, geopolitical analysis, and international commentary',
+        },
+        'playlist_mincount': 13,
+    },
+    ]
+
+    @classmethod
+    def suitable(cls, url):
+        return (
+            not any(ie.suitable(url) for ie in (ArteTVIE, ArteTVPlaylistIE, ))
+            and super(ArteTVCategoryIE, cls).suitable(url))
+
+    def _real_extract(self, url):
+        lang, playlist_id = self._match_valid_url(url).groups()
+        webpage = self._download_webpage(url, playlist_id)
+
+        items = []
+        for video in re.finditer(
+                r'<a\b[^>]*?href\s*=\s*(?P<q>"|\'|\b)(?P<url>https?://www\.arte\.tv/%s/videos/[\w/-]+)(?P=q)' % lang,
+                webpage):
+            video = video.group('url')
+            if video == url:
+                continue
+            if any(ie.suitable(video) for ie in (ArteTVIE, ArteTVPlaylistIE, )):
+                items.append(video)
+
+        title = (self._og_search_title(webpage, default=None)
+                 or self._html_search_regex(r'<title\b[^>]*>([^<]+)</title>', default=None))
+        title = strip_or_none(title.rsplit('|', 1)[0]) or self._generic_title(url)
+
+        return self.playlist_from_matches(items, playlist_id=playlist_id, playlist_title=title,
+                                          description=self._og_search_description(webpage, default=None))
--- a/yt_dlp/extractor/atvat.py
+++ b/yt_dlp/extractor/atvat.py
@@ -8,6 +8,7 @@ from ..utils import (
    float_or_none,
    jwt_encode_hs256,
    try_get,
+    ExtractorError,
 )


@@ -94,6 +95,11 @@ class ATVAtIE(InfoExtractor):
            })

        video_id, videos_data = list(videos['data'].items())[0]
+        error_msg = try_get(videos_data, lambda x: x['error']['title'])
+        if error_msg == 'Geo check failed':
+            self.raise_geo_restricted(error_msg)
+        elif error_msg:
+            raise ExtractorError(error_msg)
        entries = [
            self._extract_video_info(url, contentResource[video['id']], video)
            for video in videos_data]
--- a/yt_dlp/extractor/audiomack.py
+++ b/yt_dlp/extractor/audiomack.py
@@ -29,6 +29,7 @@ class AudiomackIE(InfoExtractor):
            }
        },
        # audiomack wrapper around soundcloud song
+        # Needs new test URL.
        {
            'add_ie': ['Soundcloud'],
            'url': 'http://www.audiomack.com/song/hip-hop-daily/black-mamba-freestyle',
--- a/yt_dlp/extractor/bandcamp.py
+++ b/yt_dlp/extractor/bandcamp.py
@@ -183,6 +183,7 @@ class BandcampIE(InfoExtractor):
                            'format_note': f.get('description'),
                            'filesize': parse_filesize(f.get('size_mb')),
                            'vcodec': 'none',
+                            'acodec': format_id.split('-')[0],
                        })

        self._sort_formats(formats)
@@ -212,7 +213,7 @@ class BandcampIE(InfoExtractor):

 class BandcampAlbumIE(BandcampIE):
    IE_NAME = 'Bandcamp:album'
-    _VALID_URL = r'https?://(?:(?P<subdomain>[^.]+)\.)?bandcamp\.com(?!/music)(?:/album/(?P<id>[^/?#&]+))?'
+    _VALID_URL = r'https?://(?:(?P<subdomain>[^.]+)\.)?bandcamp\.com/album/(?P<id>[^/?#&]+)'

    _TESTS = [{
        'url': 'http://blazo.bandcamp.com/album/jazz-format-mixtape-vol-1',
@@ -257,14 +258,6 @@ class BandcampAlbumIE(BandcampIE):
            'id': 'hierophany-of-the-open-grave',
        },
        'playlist_mincount': 9,
-    }, {
-        'url': 'http://dotscale.bandcamp.com',
-        'info_dict': {
-            'title': 'Loom',
-            'id': 'dotscale',
-            'uploader_id': 'dotscale',
-        },
-        'playlist_mincount': 7,
    }, {
        # with escaped quote in title
        'url': 'https://jstrecords.bandcamp.com/album/entropy-ep',
@@ -391,41 +384,63 @@ class BandcampWeeklyIE(BandcampIE):
        }


-class BandcampMusicIE(InfoExtractor):
-    _VALID_URL = r'https?://(?P<id>[^/]+)\.bandcamp\.com/music'
+class BandcampUserIE(InfoExtractor):
+    IE_NAME = 'Bandcamp:user'
+    _VALID_URL = r'https?://(?!www\.)(?P<id>[^.]+)\.bandcamp\.com(?:/music)?/?(?:[#?]|$)'
+
    _TESTS = [{
+        # Type 1 Bandcamp user page.
+        'url': 'https://adrianvonziegler.bandcamp.com',
+        'info_dict': {
+            'id': 'adrianvonziegler',
+            'title': 'Discography of adrianvonziegler',
+        },
+        'playlist_mincount': 23,
+    }, {
+        # Bandcamp user page with only one album
+        'url': 'http://dotscale.bandcamp.com',
+        'info_dict': {
+            'id': 'dotscale',
+            'title': 'Discography of dotscale'
+        },
+        'playlist_count': 1,
+    }, {
+        # Type 2 Bandcamp user page.
+        'url': 'https://nightcallofficial.bandcamp.com',
+        'info_dict': {
+            'id': 'nightcallofficial',
+            'title': 'Discography of nightcallofficial',
+        },
+        'playlist_count': 4,
+    }, {
        'url': 'https://steviasphere.bandcamp.com/music',
        'playlist_mincount': 47,
        'info_dict': {
            'id': 'steviasphere',
+            'title': 'Discography of steviasphere',
        },
    }, {
        'url': 'https://coldworldofficial.bandcamp.com/music',
        'playlist_mincount': 10,
        'info_dict': {
            'id': 'coldworldofficial',
+            'title': 'Discography of coldworldofficial',
        },
    }, {
        'url': 'https://nuclearwarnowproductions.bandcamp.com/music',
        'playlist_mincount': 399,
        'info_dict': {
            'id': 'nuclearwarnowproductions',
+            'title': 'Discography of nuclearwarnowproductions',
        },
-    }
-    ]
-
-    _TYPE_IE_DICT = {
-        'album': BandcampAlbumIE.ie_key(),
-        'track': BandcampIE.ie_key()
-    }
+    }]

    def _real_extract(self, url):
-        id = self._match_id(url)
-        webpage = self._download_webpage(url, id)
-        items = re.findall(r'href\=\"\/(?P<path>(?P<type>album|track)+/[^\"]+)', webpage)
-        entries = [
-            self.url_result(
-                f'https://{id}.bandcamp.com/{item[0]}',
-                ie=self._TYPE_IE_DICT[item[1]])
-            for item in items]
-        return self.playlist_result(entries, id)
+        uploader = self._match_id(url)
+        webpage = self._download_webpage(url, uploader)
+
+        discography_data = (re.findall(r'<li data-item-id=["\'][^>]+>\s*<a href=["\']([^"\']+)', webpage)
+                            or re.findall(r'<div[^>]+trackTitle["\'][^"\']+["\']([^"\']+)', webpage))
+
+        return self.playlist_from_matches(
+            discography_data, uploader, f'Discography of {uploader}', getter=lambda x: urljoin(url, x))
--- a/yt_dlp/extractor/bbc.py
+++ b/yt_dlp/extractor/bbc.py
@@ -11,6 +11,7 @@ from ..compat import (
    compat_etree_Element,
    compat_HTTPError,
    compat_str,
+    compat_urllib_error,
    compat_urlparse,
 )
 from ..utils import (
@@ -38,7 +39,7 @@ from ..utils import (
 class BBCCoUkIE(InfoExtractor):
    IE_NAME = 'bbc.co.uk'
    IE_DESC = 'BBC iPlayer'
-    _ID_REGEX = r'(?:[pbm][\da-z]{7}|w[\da-z]{7,14})'
+    _ID_REGEX = r'(?:[pbml][\da-z]{7}|w[\da-z]{7,14})'
    _VALID_URL = r'''(?x)
                    https?://
                        (?:www\.)?bbc\.co\.uk/
@@ -394,9 +395,17 @@ class BBCCoUkIE(InfoExtractor):
                        formats.extend(self._extract_mpd_formats(
                            href, programme_id, mpd_id=format_id, fatal=False))
                    elif transfer_format == 'hls':
-                        formats.extend(self._extract_m3u8_formats(
-                            href, programme_id, ext='mp4', entry_protocol='m3u8_native',
-                            m3u8_id=format_id, fatal=False))
+                        # TODO: let expected_status be passed into _extract_xxx_formats() instead
+                        try:
+                            fmts = self._extract_m3u8_formats(
+                                href, programme_id, ext='mp4', entry_protocol='m3u8_native',
+                                m3u8_id=format_id, fatal=False)
+                        except ExtractorError as e:
+                            if not (isinstance(e.exc_info[1], compat_urllib_error.HTTPError)
+                                    and e.exc_info[1].code in (403, 404)):
+                                raise
+                            fmts = []
+                        formats.extend(fmts)
                    elif transfer_format == 'hds':
                        formats.extend(self._extract_f4m_formats(
                            href, programme_id, f4m_id=format_id, fatal=False))
@@ -784,21 +793,33 @@ class BBCIE(BBCCoUkIE):
            'timestamp': 1437785037,
            'upload_date': '20150725',
        },
+    }, {
+        # video with window.__INITIAL_DATA__ and value as JSON string
+        'url': 'https://www.bbc.com/news/av/world-europe-59468682',
+        'info_dict': {
+            'id': 'p0b71qth',
+            'ext': 'mp4',
+            'title': 'Why France is making this woman a national hero',
+            'description': 'md5:7affdfab80e9c3a1f976230a1ff4d5e4',
+            'thumbnail': r're:https?://.+/.+\.jpg',
+            'timestamp': 1638230731,
+            'upload_date': '20211130',
+        },
    }, {
        # single video article embedded with data-media-vpid
        'url': 'http://www.bbc.co.uk/sport/rowing/35908187',
        'only_matching': True,
    }, {
+        # bbcthreeConfig
        'url': 'https://www.bbc.co.uk/bbcthree/clip/73d0bbd0-abc3-4cea-b3c0-cdae21905eb1',
        'info_dict': {
            'id': 'p06556y7',
            'ext': 'mp4',
-            'title': 'Transfers: Cristiano Ronaldo to Man Utd, Arsenal to spend?',
-            'description': 'md5:4b7dfd063d5a789a1512e99662be3ddd',
+            'title': 'Things Not To Say to people that live on council estates',
+            'description': "From being labelled a 'chav', to the presumption that they're 'scroungers', people who live on council estates encounter all kinds of prejudices and false assumptions about themselves, their families, and their lifestyles. Here, eight people discuss the common statements, misconceptions, and clichés that they're tired of hearing.",
+            'duration': 360,
+            'thumbnail': r're:https?://.+/.+\.jpg',
        },
-        'params': {
-            'skip_download': True,
-        }
    }, {
        # window.__PRELOADED_STATE__
        'url': 'https://www.bbc.co.uk/radio/play/b0b9z4yl',
@@ -1171,9 +1192,16 @@ class BBCIE(BBCCoUkIE):
                return self.playlist_result(
                    entries, playlist_id, playlist_title, playlist_description)

-        initial_data = self._parse_json(self._search_regex(
-            r'window\.__INITIAL_DATA__\s*=\s*({.+?});', webpage,
-            'preload state', default='{}'), playlist_id, fatal=False)
+        initial_data = self._search_regex(
+            r'window\.__INITIAL_DATA__\s*=\s*("{.+?}")\s*;', webpage,
+            'quoted preload state', default=None)
+        if initial_data is None:
+            initial_data = self._search_regex(
+                r'window\.__INITIAL_DATA__\s*=\s*({.+?})\s*;', webpage,
+                'preload state', default={})
+        else:
+            initial_data = self._parse_json(initial_data or '"{}"', playlist_id, fatal=False)
+        initial_data = self._parse_json(initial_data, playlist_id, fatal=False)
        if initial_data:
            def parse_media(media):
                if not media:
@@ -1214,7 +1242,10 @@ class BBCIE(BBCCoUkIE):
                if name == 'media-experience':
                    parse_media(try_get(resp, lambda x: x['data']['initialItem']['mediaItem'], dict))
                elif name == 'article':
-                    for block in (try_get(resp, lambda x: x['data']['blocks'], list) or []):
+                    for block in (try_get(resp,
+                                          (lambda x: x['data']['blocks'],
+                                           lambda x: x['data']['content']['model']['blocks'],),
+                                          list) or []):
                        if block.get('type') != 'media':
                            continue
                        parse_media(block.get('model'))
--- a/yt_dlp/extractor/beeg.py
+++ b/yt_dlp/extractor/beeg.py
@@ -1,32 +1,45 @@
 from __future__ import unicode_literals

 from .common import InfoExtractor
-from ..compat import (
-    compat_str,
-)
+
 from ..utils import (
    int_or_none,
-    parse_qs,
+    traverse_obj,
+    try_get,
    unified_timestamp,
 )


 class BeegIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?beeg\.(?:com|porn(?:/video)?)/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?beeg\.(?:com(?:/video)?)/-?(?P<id>\d+)'
    _TESTS = [{
-        # api/v6 v1
-        'url': 'http://beeg.com/5416503',
-        'md5': 'a1a1b1a8bc70a89e49ccfd113aed0820',
+        'url': 'https://beeg.com/-0983946056129650',
+        'md5': '51d235147c4627cfce884f844293ff88',
        'info_dict': {
-            'id': '5416503',
+            'id': '0983946056129650',
            'ext': 'mp4',
-            'title': 'Sultry Striptease',
-            'description': 'md5:d22219c09da287c14bed3d6c37ce4bc2',
-            'timestamp': 1391813355,
-            'upload_date': '20140207',
-            'duration': 383,
+            'title': 'sucked cock and fucked in a private plane',
+            'duration': 927,
            'tags': list,
            'age_limit': 18,
+            'upload_date': '20220131',
+            'timestamp': 1643656455,
+            'display_id': 2540839,
+        }
+    }, {
+        'url': 'https://beeg.com/-0599050563103750?t=4-861',
+        'md5': 'bd8b5ea75134f7f07fad63008db2060e',
+        'info_dict': {
+            'id': '0599050563103750',
+            'ext': 'mp4',
+            'title': 'Bad Relatives',
+            'duration': 2060,
+            'tags': list,
+            'age_limit': 18,
+            'description': 'md5:b4fc879a58ae6c604f8f259155b7e3b9',
+            'timestamp': 1643623200,
+            'display_id': 2569965,
+            'upload_date': '20220131',
        }
    }, {
        # api/v6 v2
@@ -36,12 +49,6 @@ class BeegIE(InfoExtractor):
        # api/v6 v2 w/o t
        'url': 'https://beeg.com/1277207756',
        'only_matching': True,
-    }, {
-        'url': 'https://beeg.porn/video/5416503',
-        'only_matching': True,
-    }, {
-        'url': 'https://beeg.porn/5416503',
-        'only_matching': True,
    }]

    def _real_extract(self, url):
@@ -49,68 +56,38 @@ class BeegIE(InfoExtractor):

        webpage = self._download_webpage(url, video_id)

-        beeg_version = self._search_regex(
-            r'beeg_version\s*=\s*([\da-zA-Z_-]+)', webpage, 'beeg version',
-            default='1546225636701')
+        video = self._download_json(
+            'https://store.externulls.com/facts/file/%s' % video_id,
+            video_id, 'Downloading JSON for %s' % video_id)

-        if len(video_id) >= 10:
-            query = {
-                'v': 2,
-            }
-            qs = parse_qs(url)
-            t = qs.get('t', [''])[0].split('-')
-            if len(t) > 1:
-                query.update({
-                    's': t[0],
-                    'e': t[1],
-                })
-        else:
-            query = {'v': 1}
+        fc_facts = video.get('fc_facts')
+        first_fact = {}
+        for fact in fc_facts:
+            if not first_fact or try_get(fact, lambda x: x['id'] < first_fact['id']):
+                first_fact = fact

-        for api_path in ('', 'api.'):
-            video = self._download_json(
-                'https://%sbeeg.com/api/v6/%s/video/%s'
-                % (api_path, beeg_version, video_id), video_id,
-                fatal=api_path == 'api.', query=query)
-            if video:
-                break
+        resources = traverse_obj(video, ('file', 'hls_resources')) or first_fact.get('hls_resources')

        formats = []
-        for format_id, video_url in video.items():
-            if not video_url:
+        for format_id, video_uri in resources.items():
+            if not video_uri:
                continue
-            height = self._search_regex(
-                r'^(\d+)[pP]$', format_id, 'height', default=None)
-            if not height:
-                continue
-            formats.append({
-                'url': self._proto_relative_url(
-                    video_url.replace('{DATA_MARKERS}', 'data=pc_XX__%s_0' % beeg_version), 'https:'),
-                'format_id': format_id,
-                'height': int(height),
-            })
+            height = int_or_none(self._search_regex(r'fl_cdn_(\d+)', format_id, 'height', default=None))
+            current_formats = self._extract_m3u8_formats(f'https://video.beeg.com/{video_uri}', video_id, ext='mp4', m3u8_id=str(height))
+            for f in current_formats:
+                f['height'] = height
+            formats.extend(current_formats)
+
        self._sort_formats(formats)

-        title = video['title']
-        video_id = compat_str(video.get('id') or video_id)
-        display_id = video.get('code')
-        description = video.get('desc')
-        series = video.get('ps_name')
-
-        timestamp = unified_timestamp(video.get('date'))
-        duration = int_or_none(video.get('duration'))
-
-        tags = [tag.strip() for tag in video['tags'].split(',')] if video.get('tags') else None
-
        return {
            'id': video_id,
-            'display_id': display_id,
-            'title': title,
-            'description': description,
-            'series': series,
-            'timestamp': timestamp,
-            'duration': duration,
-            'tags': tags,
+            'display_id': first_fact.get('id'),
+            'title': traverse_obj(video, ('file', 'stuff', 'sf_name')),
+            'description': traverse_obj(video, ('file', 'stuff', 'sf_story')),
+            'timestamp': unified_timestamp(first_fact.get('fc_created')),
+            'duration': int_or_none(traverse_obj(video, ('file', 'fl_duration'))),
+            'tags': traverse_obj(video, ('tags', ..., 'tg_name')),
            'formats': formats,
            'age_limit': self._rta_search(webpage),
        }
--- a/yt_dlp/extractor/bigo.py
+++ b/yt_dlp/extractor/bigo.py
@@ -0,0 +1,59 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import ExtractorError, urlencode_postdata
+
+
+class BigoIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?bigo\.tv/(?:[a-z]{2,}/)?(?P<id>[^/]+)'
+
+    _TESTS = [{
+        'url': 'https://www.bigo.tv/ja/221338632',
+        'info_dict': {
+            'id': '6576287577575737440',
+            'title': '土よ〜💁‍♂️ 休憩室/REST room',
+            'thumbnail': r're:https?://.+',
+            'uploader': '✨Shin💫',
+            'uploader_id': '221338632',
+            'is_live': True,
+        },
+        'skip': 'livestream',
+    }, {
+        'url': 'https://www.bigo.tv/th/Tarlerm1304',
+        'only_matching': True,
+    }, {
+        'url': 'https://bigo.tv/115976881',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        user_id = self._match_id(url)
+
+        info_raw = self._download_json(
+            'https://bigo.tv/studio/getInternalStudioInfo',
+            user_id, data=urlencode_postdata({'siteId': user_id}))
+
+        if not isinstance(info_raw, dict):
+            raise ExtractorError('Received invalid JSON data')
+        if info_raw.get('code'):
+            raise ExtractorError(
+                'Bigo says: %s (code %s)' % (info_raw.get('msg'), info_raw.get('code')), expected=True)
+        info = info_raw.get('data') or {}
+
+        if not info.get('alive'):
+            raise ExtractorError('This user is offline.', expected=True)
+
+        return {
+            'id': info.get('roomId') or user_id,
+            'title': info.get('roomTopic') or info.get('nick_name') or user_id,
+            'formats': [{
+                'url': info.get('hls_src'),
+                'ext': 'mp4',
+                'protocol': 'm3u8',
+            }],
+            'thumbnail': info.get('snapshot'),
+            'uploader': info.get('nick_name'),
+            'uploader_id': user_id,
+            'is_live': True,
+        }
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -225,10 +225,6 @@ class BiliBiliIE(InfoExtractor):
                        'quality': -2 if 'hd.mp4' in backup_url else -3,
                    })

-                for a_format in formats:
-                    a_format.setdefault('http_headers', {}).update({
-                        'Referer': url,
-                    })
            for audio in audios:
                formats.append({
                    'url': audio.get('baseUrl') or audio.get('base_url') or audio.get('url'),
@@ -252,6 +248,9 @@ class BiliBiliIE(InfoExtractor):
                'id': video_id,
                'duration': float_or_none(durl.get('length'), 1000),
                'formats': formats,
+                'http_headers': {
+                    'Referer': url,
+                },
            })
            break

--- a/yt_dlp/extractor/biqle.py
+++ b/yt_dlp/extractor/biqle.py
@@ -3,27 +3,28 @@ from __future__ import unicode_literals

 from .common import InfoExtractor
 from .vk import VKIE
-from ..compat import (
-    compat_b64decode,
-    compat_urllib_parse_unquote,
+from ..compat import compat_b64decode
+from ..utils import (
+    int_or_none,
+    js_to_json,
+    traverse_obj,
+    unified_timestamp,
 )
-from ..utils import int_or_none


 class BIQLEIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?biqle\.(?:com|org|ru)/watch/(?P<id>-?\d+_\d+)'
    _TESTS = [{
-        # Youtube embed
-        'url': 'https://biqle.ru/watch/-115995369_456239081',
-        'md5': '97af5a06ee4c29bbf9c001bdb1cf5c06',
+        'url': 'https://biqle.ru/watch/-2000421746_85421746',
+        'md5': 'ae6ef4f04d19ac84e4658046d02c151c',
        'info_dict': {
-            'id': '8v4f-avW-VI',
+            'id': '-2000421746_85421746',
            'ext': 'mp4',
-            'title': "PASSE-PARTOUT - L'ete c'est fait pour jouer",
-            'description': 'Passe-Partout',
-            'uploader_id': 'mrsimpsonstef3',
-            'uploader': 'Phanolito',
-            'upload_date': '20120822',
+            'title': 'Forsaken By Hope Studio Clip',
+            'description': 'Forsaken By Hope Studio Clip — Смотреть онлайн',
+            'upload_date': '19700101',
+            'thumbnail': r're:https://[^/]+/impf/7vN3ACwSTgChP96OdOfzFjUCzFR6ZglDQgWsIw/KPaACiVJJxM\.jpg\?size=800x450&quality=96&keep_aspect_ratio=1&background=000000&sign=b48ea459c4d33dbcba5e26d63574b1cb&type=video_thumb',
+            'timestamp': 0,
        },
    }, {
        'url': 'http://biqle.org/watch/-44781847_168547604',
@@ -32,53 +33,62 @@ class BIQLEIE(InfoExtractor):
            'id': '-44781847_168547604',
            'ext': 'mp4',
            'title': 'Ребенок в шоке от автоматической мойки',
+            'description': 'Ребенок в шоке от автоматической мойки — Смотреть онлайн',
            'timestamp': 1396633454,
-            'uploader': 'Dmitry Kotov',
            'upload_date': '20140404',
-            'uploader_id': '47850140',
+            'thumbnail': r're:https://[^/]+/c535507/u190034692/video/l_b84df002\.jpg',
        },
    }]

    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
-        embed_url = self._proto_relative_url(self._search_regex(
-            r'<iframe.+?src="((?:https?:)?//(?:daxab\.com|dxb\.to|[^/]+/player)/[^"]+)".*?></iframe>',
-            webpage, 'embed url'))
+
+        title = self._html_search_meta('name', webpage, 'Title', fatal=False)
+        timestamp = unified_timestamp(self._html_search_meta('uploadDate', webpage, 'Upload Date', default=None))
+        description = self._html_search_meta('description', webpage, 'Description', default=None)
+
+        global_embed_url = self._search_regex(
+            r'<script[^<]+?window.globEmbedUrl\s*=\s*\'((?:https?:)?//(?:daxab\.com|dxb\.to|[^/]+/player)/[^\']+)\'',
+            webpage, 'global Embed url')
+        hash = self._search_regex(
+            r'<script id="data-embed-video[^<]+?hash: "([^"]+)"[^<]*</script>', webpage, 'Hash')
+
+        embed_url = global_embed_url + hash
+
        if VKIE.suitable(embed_url):
            return self.url_result(embed_url, VKIE.ie_key(), video_id)

        embed_page = self._download_webpage(
-            embed_url, video_id, headers={'Referer': url})
-        video_ext = self._get_cookies(embed_url).get('video_ext')
-        if video_ext:
-            video_ext = compat_urllib_parse_unquote(video_ext.value)
-        if not video_ext:
-            video_ext = compat_b64decode(self._search_regex(
-                r'video_ext\s*:\s*[\'"]([A-Za-z0-9+/=]+)',
-                embed_page, 'video_ext')).decode()
-        video_id, sig, _, access_token = video_ext.split(':')
+            embed_url, video_id, 'Downloading embed webpage', headers={'Referer': url})
+
+        glob_params = self._parse_json(self._search_regex(
+            r'<script id="globParams">[^<]*window.globParams = ([^;]+);[^<]+</script>',
+            embed_page, 'Global Parameters'), video_id, transform_source=js_to_json)
+        host_name = compat_b64decode(glob_params['server'][::-1]).decode()
+
        item = self._download_json(
-            'https://api.vk.com/method/video.get', video_id,
-            headers={'User-Agent': 'okhttp/3.4.1'}, query={
-                'access_token': access_token,
-                'sig': sig,
-                'v': 5.44,
+            f'https://{host_name}/method/video.get/{video_id}', video_id,
+            headers={'Referer': url}, query={
+                'token': glob_params['video']['access_token'],
                'videos': video_id,
+                'ckey': glob_params['c_key'],
+                'credentials': glob_params['video']['credentials'],
            })['response']['items'][0]
-        title = item['title']

        formats = []
        for f_id, f_url in item.get('files', {}).items():
            if f_id == 'external':
                return self.url_result(f_url)
            ext, height = f_id.split('_')
-            formats.append({
-                'format_id': height + 'p',
-                'url': f_url,
-                'height': int_or_none(height),
-                'ext': ext,
-            })
+            height_extra_key = traverse_obj(glob_params, ('video', 'partial', 'quality', height))
+            if height_extra_key:
+                formats.append({
+                    'format_id': f'{height}p',
+                    'url': f'https://{host_name}/{f_url[8:]}&videos={video_id}&extra_key={height_extra_key}',
+                    'height': int_or_none(height),
+                    'ext': ext,
+                })
        self._sort_formats(formats)

        thumbnails = []
@@ -96,10 +106,9 @@ class BIQLEIE(InfoExtractor):
            'title': title,
            'formats': formats,
            'comment_count': int_or_none(item.get('comments')),
-            'description': item.get('description'),
+            'description': description,
            'duration': int_or_none(item.get('duration')),
            'thumbnails': thumbnails,
-            'timestamp': int_or_none(item.get('date')),
-            'uploader': item.get('owner_id'),
+            'timestamp': timestamp,
            'view_count': int_or_none(item.get('views')),
        }
--- a/yt_dlp/extractor/caltrans.py
+++ b/yt_dlp/extractor/caltrans.py
@@ -0,0 +1,41 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class CaltransIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:[^/]+\.)?ca\.gov/vm/loc/[^/]+/(?P<id>[a-z0-9_]+)\.htm'
+    _TEST = {
+        'url': 'https://cwwp2.dot.ca.gov/vm/loc/d3/hwy50at24th.htm',
+        'info_dict': {
+            'id': 'hwy50at24th',
+            'ext': 'ts',
+            'title': 'US-50 : Sacramento : Hwy 50 at 24th',
+            'live_status': 'is_live',
+            'thumbnail': 'https://cwwp2.dot.ca.gov/data/d3/cctv/image/hwy50at24th/hwy50at24th.jpg',
+        }
+    }
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        global_vars = self._search_regex(
+            r'<script[^<]+?([^<]+\.m3u8[^<]+)</script>',
+            webpage, 'Global Vars')
+        route_place = self._search_regex(r'routePlace\s*=\s*"([^"]+)"', global_vars, 'Route Place', fatal=False)
+        location_name = self._search_regex(r'locationName\s*=\s*"([^"]+)"', global_vars, 'Location Name', fatal=False)
+        poster_url = self._search_regex(r'posterURL\s*=\s*"([^"]+)"', global_vars, 'Poster Url', fatal=False)
+        video_stream = self._search_regex(r'videoStreamURL\s*=\s*"([^"]+)"', global_vars, 'Video Stream URL', fatal=False)
+
+        formats = self._extract_m3u8_formats(video_stream, video_id, 'ts', live=True)
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': f'{route_place} : {location_name}',
+            'is_live': True,
+            'formats': formats,
+            'thumbnail': poster_url,
+        }
--- a/yt_dlp/extractor/ccma.py
+++ b/yt_dlp/extractor/ccma.py
@@ -1,17 +1,14 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import calendar
-import datetime
-
 from .common import InfoExtractor
 from ..utils import (
    clean_html,
-    extract_timezone,
    int_or_none,
    parse_duration,
    parse_resolution,
    try_get,
+    unified_timestamp,
    url_or_none,
 )

@@ -95,14 +92,8 @@ class CCMAIE(InfoExtractor):
        duration = int_or_none(durada.get('milisegons'), 1000) or parse_duration(durada.get('text'))
        tematica = try_get(informacio, lambda x: x['tematica']['text'])

-        timestamp = None
        data_utc = try_get(informacio, lambda x: x['data_emissio']['utc'])
-        try:
-            timezone, data_utc = extract_timezone(data_utc)
-            timestamp = calendar.timegm((datetime.datetime.strptime(
-                data_utc, '%Y-%d-%mT%H:%M:%S') - timezone).timetuple())
-        except TypeError:
-            pass
+        timestamp = unified_timestamp(data_utc)

        subtitles = {}
        subtitols = media.get('subtitols') or []
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -75,6 +75,7 @@ from ..utils import (
    str_to_int,
    strip_or_none,
    traverse_obj,
+    try_get,
    unescapeHTML,
    UnsupportedError,
    unified_strdate,
@@ -225,6 +226,7 @@ class InfoExtractor(object):

    The following fields are optional:

+    direct:         True if a direct video file was given (must only be set by GenericIE)
    alt_title:      A secondary title of the video.
    display_id      An alternative identifier for the video, not necessarily
                    unique, but available before title. Typically, id is
@@ -239,6 +241,7 @@ class InfoExtractor(object):
                        * "resolution" (optional, string "{width}x{height}",
                                        deprecated)
                        * "filesize" (optional, int)
+                        * "http_headers" (dict) - HTTP headers for the request
    thumbnail:      Full URL to a video thumbnail image.
    description:    Full video description.
    uploader:       Full name of the video uploader.
@@ -272,6 +275,8 @@ class InfoExtractor(object):
                        * "url": A URL pointing to the subtitles file
                    It can optionally also have:
                        * "name": Name or description of the subtitles
+                        * "http_headers": A dictionary of additional HTTP headers
+                                  to add to the request.
                    "ext" will be calculated from URL if missing
    automatic_captions: Like 'subtitles'; contains automatically generated
                    captions instead of normal subtitles
@@ -421,8 +426,8 @@ class InfoExtractor(object):
    title, description etc.


-    Subclasses of this one should re-define the _real_initialize() and
-    _real_extract() methods and define a _VALID_URL regexp.
+    Subclasses of this should define a _VALID_URL regexp and, re-define the
+    _real_extract() and (optionally) _real_initialize() methods.
    Probably, they should also be added to the list of extractors.

    Subclasses may also override suitable() if necessary, but ensure the function
@@ -635,7 +640,7 @@ class InfoExtractor(object):
            }
            if hasattr(e, 'countries'):
                kwargs['countries'] = e.countries
-            raise type(e)(e.msg, **kwargs)
+            raise type(e)(e.orig_msg, **kwargs)
        except compat_http_client.IncompleteRead as e:
            raise ExtractorError('A network error has occurred.', cause=e, expected=True, video_id=self.get_temp_id(url))
        except (KeyError, StopIteration) as e:
@@ -657,7 +662,7 @@ class InfoExtractor(object):
        return False

    def set_downloader(self, downloader):
-        """Sets the downloader for this IE."""
+        """Sets a YoutubeDL instance as the downloader for this IE."""
        self._downloader = downloader

    def _real_initialize(self):
@@ -666,7 +671,7 @@ class InfoExtractor(object):

    def _real_extract(self, url):
        """Real extraction process. Redefine in subclasses."""
-        pass
+        raise NotImplementedError('This method must be implemented by subclasses')

    @classmethod
    def ie_key(cls):
@@ -745,7 +750,7 @@ class InfoExtractor(object):

            errmsg = '%s: %s' % (errnote, error_to_compat_str(err))
            if fatal:
-                raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
+                raise ExtractorError(errmsg, cause=err)
            else:
                self.report_warning(errmsg)
                return False
@@ -1097,6 +1102,7 @@ class InfoExtractor(object):
        if metadata_available and (
                self.get_param('ignore_no_formats_error') or self.get_param('wait_for_video')):
            self.report_warning(msg)
+            return
        if method is not None:
            msg = '%s. %s' % (msg, self._LOGIN_HINTS[method])
        raise ExtractorError(msg, expected=True)
@@ -1135,8 +1141,8 @@ class InfoExtractor(object):
            'url': url,
        }

-    def playlist_from_matches(self, matches, playlist_id=None, playlist_title=None, getter=None, ie=None, **kwargs):
-        urls = (self.url_result(self._proto_relative_url(m), ie)
+    def playlist_from_matches(self, matches, playlist_id=None, playlist_title=None, getter=None, ie=None, video_kwargs=None, **kwargs):
+        urls = (self.url_result(self._proto_relative_url(m), ie, **(video_kwargs or {}))
                for m in orderedSet(map(getter, matches) if getter else matches))
        return self.playlist_result(urls, playlist_id, playlist_title, **kwargs)

@@ -1303,6 +1309,10 @@ class InfoExtractor(object):
    def _og_search_url(self, html, **kargs):
        return self._og_search_property('url', html, **kargs)

+    def _html_extract_title(self, html, name, **kwargs):
+        return self._html_search_regex(
+            r'(?s)<title>(.*?)</title>', html, name, **kwargs)
+
    def _html_search_meta(self, name, html, display_name=None, fatal=False, **kwargs):
        name = variadic(name)
        if display_name is None:
@@ -1609,7 +1619,7 @@ class InfoExtractor(object):
            'vcodec': {'type': 'ordered', 'regex': True,
                       'order': ['av0?1', 'vp0?9.2', 'vp0?9', '[hx]265|he?vc?', '[hx]264|avc', 'vp0?8', 'mp4v|h263', 'theora', '', None, 'none']},
            'acodec': {'type': 'ordered', 'regex': True,
-                       'order': ['[af]lac', 'wav|aiff', 'opus', 'vorbis', 'aac', 'mp?4a?', 'mp3', 'e-?a?c-?3', 'ac-?3', 'dts', '', None, 'none']},
+                       'order': ['[af]lac', 'wav|aiff', 'opus', 'vorbis|ogg', 'aac', 'mp?4a?', 'mp3', 'e-?a?c-?3', 'ac-?3', 'dts', '', None, 'none']},
            'hdr': {'type': 'ordered', 'regex': True, 'field': 'dynamic_range',
                    'order': ['dv', '(hdr)?12', r'(hdr)?10\+', '(hdr)?10', 'hlg', '', 'sdr', None]},
            'proto': {'type': 'ordered', 'regex': True, 'field': 'protocol',
@@ -1652,31 +1662,31 @@ class InfoExtractor(object):
            'format_id': {'type': 'alias', 'field': 'id'},
            'preference': {'type': 'alias', 'field': 'ie_pref'},
            'language_preference': {'type': 'alias', 'field': 'lang'},
+            'source_preference': {'type': 'alias', 'field': 'source'},
+            'protocol': {'type': 'alias', 'field': 'proto'},
+            'filesize_approx': {'type': 'alias', 'field': 'fs_approx'},

            # Deprecated
-            'dimension': {'type': 'alias', 'field': 'res'},
-            'resolution': {'type': 'alias', 'field': 'res'},
-            'extension': {'type': 'alias', 'field': 'ext'},
-            'bitrate': {'type': 'alias', 'field': 'br'},
-            'total_bitrate': {'type': 'alias', 'field': 'tbr'},
-            'video_bitrate': {'type': 'alias', 'field': 'vbr'},
-            'audio_bitrate': {'type': 'alias', 'field': 'abr'},
-            'framerate': {'type': 'alias', 'field': 'fps'},
-            'protocol': {'type': 'alias', 'field': 'proto'},
-            'source_preference': {'type': 'alias', 'field': 'source'},
-            'filesize_approx': {'type': 'alias', 'field': 'fs_approx'},
-            'filesize_estimate': {'type': 'alias', 'field': 'size'},
-            'samplerate': {'type': 'alias', 'field': 'asr'},
-            'video_ext': {'type': 'alias', 'field': 'vext'},
-            'audio_ext': {'type': 'alias', 'field': 'aext'},
-            'video_codec': {'type': 'alias', 'field': 'vcodec'},
-            'audio_codec': {'type': 'alias', 'field': 'acodec'},
-            'video': {'type': 'alias', 'field': 'hasvid'},
-            'has_video': {'type': 'alias', 'field': 'hasvid'},
-            'audio': {'type': 'alias', 'field': 'hasaud'},
-            'has_audio': {'type': 'alias', 'field': 'hasaud'},
-            'extractor': {'type': 'alias', 'field': 'ie_pref'},
-            'extractor_preference': {'type': 'alias', 'field': 'ie_pref'},
+            'dimension': {'type': 'alias', 'field': 'res', 'deprecated': True},
+            'resolution': {'type': 'alias', 'field': 'res', 'deprecated': True},
+            'extension': {'type': 'alias', 'field': 'ext', 'deprecated': True},
+            'bitrate': {'type': 'alias', 'field': 'br', 'deprecated': True},
+            'total_bitrate': {'type': 'alias', 'field': 'tbr', 'deprecated': True},
+            'video_bitrate': {'type': 'alias', 'field': 'vbr', 'deprecated': True},
+            'audio_bitrate': {'type': 'alias', 'field': 'abr', 'deprecated': True},
+            'framerate': {'type': 'alias', 'field': 'fps', 'deprecated': True},
+            'filesize_estimate': {'type': 'alias', 'field': 'size', 'deprecated': True},
+            'samplerate': {'type': 'alias', 'field': 'asr', 'deprecated': True},
+            'video_ext': {'type': 'alias', 'field': 'vext', 'deprecated': True},
+            'audio_ext': {'type': 'alias', 'field': 'aext', 'deprecated': True},
+            'video_codec': {'type': 'alias', 'field': 'vcodec', 'deprecated': True},
+            'audio_codec': {'type': 'alias', 'field': 'acodec', 'deprecated': True},
+            'video': {'type': 'alias', 'field': 'hasvid', 'deprecated': True},
+            'has_video': {'type': 'alias', 'field': 'hasvid', 'deprecated': True},
+            'audio': {'type': 'alias', 'field': 'hasaud', 'deprecated': True},
+            'has_audio': {'type': 'alias', 'field': 'hasaud', 'deprecated': True},
+            'extractor': {'type': 'alias', 'field': 'ie_pref', 'deprecated': True},
+            'extractor_preference': {'type': 'alias', 'field': 'ie_pref', 'deprecated': True},
        }

        def __init__(self, ie, field_preference):
@@ -1776,7 +1786,7 @@ class InfoExtractor(object):
                    continue
                if self._get_field_setting(field, 'type') == 'alias':
                    alias, field = field, self._get_field_setting(field, 'field')
-                    if alias not in ('format_id', 'preference', 'language_preference'):
+                    if self._get_field_setting(alias, 'deprecated'):
                        self.ydl.deprecation_warning(
                            f'Format sorting alias {alias} is deprecated '
                            f'and may be removed in a future version. Please use {field} instead')
@@ -2875,7 +2885,8 @@ class InfoExtractor(object):
                            segment_duration = None
                            if 'total_number' not in representation_ms_info and 'segment_duration' in representation_ms_info:
                                segment_duration = float_or_none(representation_ms_info['segment_duration'], representation_ms_info['timescale'])
-                                representation_ms_info['total_number'] = int(math.ceil(float(period_duration) / segment_duration))
+                                representation_ms_info['total_number'] = int(math.ceil(
+                                    float_or_none(period_duration, segment_duration, default=0)))
                            representation_ms_info['fragments'] = [{
                                media_location_key: media_template % {
                                    'Number': segment_number,
@@ -2966,6 +2977,10 @@ class InfoExtractor(object):
                                f['url'] = initialization_url
                            f['fragments'].append({location_key(initialization_url): initialization_url})
                        f['fragments'].extend(representation_ms_info['fragments'])
+                        if not period_duration:
+                            period_duration = try_get(
+                                representation_ms_info,
+                                lambda r: sum(frag['duration'] for frag in r['fragments']), float)
                    else:
                        # Assuming direct URL to unfragmented media.
                        f['url'] = base_url
@@ -3108,7 +3123,7 @@ class InfoExtractor(object):
                    })
        return formats, subtitles

-    def _parse_html5_media_entries(self, base_url, webpage, video_id, m3u8_id=None, m3u8_entry_protocol='m3u8', mpd_id=None, preference=None, quality=None):
+    def _parse_html5_media_entries(self, base_url, webpage, video_id, m3u8_id=None, m3u8_entry_protocol='m3u8_native', mpd_id=None, preference=None, quality=None):
        def absolute_url(item_url):
            return urljoin(base_url, item_url)

@@ -3665,7 +3680,7 @@ class InfoExtractor(object):
    def mark_watched(self, *args, **kwargs):
        if not self.get_param('mark_watched', False):
            return
-        if (self._get_login_info()[0] is not None
+        if (hasattr(self, '_NETRC_MACHINE') and self._get_login_info()[0] is not None
                or self.get_param('cookiefile')
                or self.get_param('cookiesfrombrowser')):
            self._mark_watched(*args, **kwargs)
--- a/yt_dlp/extractor/cpac.py
+++ b/yt_dlp/extractor/cpac.py
@@ -0,0 +1,148 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    int_or_none,
+    str_or_none,
+    try_get,
+    unified_timestamp,
+    update_url_query,
+    urljoin,
+)
+
+# compat_range
+try:
+    if callable(xrange):
+        range = xrange
+except (NameError, TypeError):
+    pass
+
+
+class CPACIE(InfoExtractor):
+    IE_NAME = 'cpac'
+    _VALID_URL = r'https?://(?:www\.)?cpac\.ca/(?P<fr>l-)?episode\?id=(?P<id>[\da-f]{8}(?:-[\da-f]{4}){3}-[\da-f]{12})'
+    _TEST = {
+        # 'url': 'http://www.cpac.ca/en/programs/primetime-politics/episodes/65490909',
+        'url': 'https://www.cpac.ca/episode?id=fc7edcae-4660-47e1-ba61-5b7f29a9db0f',
+        'md5': 'e46ad699caafd7aa6024279f2614e8fa',
+        'info_dict': {
+            'id': 'fc7edcae-4660-47e1-ba61-5b7f29a9db0f',
+            'ext': 'mp4',
+            'upload_date': '20220215',
+            'title': 'News Conference to Celebrate National Kindness Week – February 15, 2022',
+            'description': 'md5:466a206abd21f3a6f776cdef290c23fb',
+            'timestamp': 1644901200,
+        },
+        'params': {
+            'format': 'bestvideo',
+            'hls_prefer_native': True,
+        },
+    }
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        url_lang = 'fr' if '/l-episode?' in url else 'en'
+
+        content = self._download_json(
+            'https://www.cpac.ca/api/1/services/contentModel.json?url=/site/website/episode/index.xml&crafterSite=cpacca&id=' + video_id,
+            video_id)
+        video_url = try_get(content, lambda x: x['page']['details']['videoUrl'], compat_str)
+        formats = []
+        if video_url:
+            content = content['page']
+            title = str_or_none(content['details']['title_%s_t' % (url_lang, )])
+            formats = self._extract_m3u8_formats(video_url, video_id, m3u8_id='hls', ext='mp4')
+            for fmt in formats:
+                # prefer language to match URL
+                fmt_lang = fmt.get('language')
+                if fmt_lang == url_lang:
+                    fmt['language_preference'] = 10
+                elif not fmt_lang:
+                    fmt['language_preference'] = -1
+                else:
+                    fmt['language_preference'] = -10
+
+        self._sort_formats(formats)
+
+        category = str_or_none(content['details']['category_%s_t' % (url_lang, )])
+
+        def is_live(v_type):
+            return (v_type == 'live') if v_type is not None else None
+
+        return {
+            'id': video_id,
+            'formats': formats,
+            'title': title,
+            'description': str_or_none(content['details'].get('description_%s_t' % (url_lang, ))),
+            'timestamp': unified_timestamp(content['details'].get('liveDateTime')),
+            'category': [category] if category else None,
+            'thumbnail': urljoin(url, str_or_none(content['details'].get('image_%s_s' % (url_lang, )))),
+            'is_live': is_live(content['details'].get('type')),
+        }
+
+
+class CPACPlaylistIE(InfoExtractor):
+    IE_NAME = 'cpac:playlist'
+    _VALID_URL = r'(?i)https?://(?:www\.)?cpac\.ca/(?:program|search|(?P<fr>emission|rechercher))\?(?:[^&]+&)*?(?P<id>(?:id=\d+|programId=\d+|key=[^&]+))'
+
+    _TESTS = [{
+        'url': 'https://www.cpac.ca/program?id=6',
+        'info_dict': {
+            'id': 'id=6',
+            'title': 'Headline Politics',
+            'description': 'Watch CPAC’s signature long-form coverage of the day’s pressing political events as they unfold.',
+        },
+        'playlist_count': 10,
+    }, {
+        'url': 'https://www.cpac.ca/search?key=hudson&type=all&order=desc',
+        'info_dict': {
+            'id': 'key=hudson',
+            'title': 'hudson',
+        },
+        'playlist_count': 22,
+    }, {
+        'url': 'https://www.cpac.ca/search?programId=50',
+        'info_dict': {
+            'id': 'programId=50',
+            'title': '50',
+        },
+        'playlist_count': 9,
+    }, {
+        'url': 'https://www.cpac.ca/emission?id=6',
+        'only_matching': True,
+    }, {
+        'url': 'https://www.cpac.ca/rechercher?key=hudson&type=all&order=desc',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        url_lang = 'fr' if any(x in url for x in ('/emission?', '/rechercher?')) else 'en'
+        pl_type, list_type = ('program', 'itemList') if any(x in url for x in ('/program?', '/emission?')) else ('search', 'searchResult')
+        api_url = (
+            'https://www.cpac.ca/api/1/services/contentModel.json?url=/site/website/%s/index.xml&crafterSite=cpacca&%s'
+            % (pl_type, video_id, ))
+        content = self._download_json(api_url, video_id)
+        entries = []
+        total_pages = int_or_none(try_get(content, lambda x: x['page'][list_type]['totalPages']), default=1)
+        for page in range(1, total_pages + 1):
+            if page > 1:
+                api_url = update_url_query(api_url, {'page': '%d' % (page, ), })
+                content = self._download_json(
+                    api_url, video_id,
+                    note='Downloading continuation - %d' % (page, ),
+                    fatal=False)
+
+            for item in try_get(content, lambda x: x['page'][list_type]['item'], list) or []:
+                episode_url = urljoin(url, try_get(item, lambda x: x['url_%s_s' % (url_lang, )]))
+                if episode_url:
+                    entries.append(episode_url)
+
+        return self.playlist_result(
+            (self.url_result(entry) for entry in entries),
+            playlist_id=video_id,
+            playlist_title=try_get(content, lambda x: x['page']['program']['title_%s_t' % (url_lang, )]) or video_id.split('=')[-1],
+            playlist_description=try_get(content, lambda x: x['page']['program']['description_%s_t' % (url_lang, )]),
+        )
--- a/yt_dlp/extractor/crunchyroll.py
+++ b/yt_dlp/extractor/crunchyroll.py
@@ -85,7 +85,7 @@ class CrunchyrollBaseIE(InfoExtractor):
                'session_id': session_id
            }).encode('ascii'))
        if login_response['code'] != 'ok':
-            raise ExtractorError('Login failed. Bad username or password?', expected=True)
+            raise ExtractorError('Login failed. Server message: %s' % login_response['message'], expected=True)
        if not self._get_cookies(self._LOGIN_URL).get('etp_rt'):
            raise ExtractorError('Login succeeded but did not set etp_rt cookie')

--- a/yt_dlp/extractor/cspan.py
+++ b/yt_dlp/extractor/cspan.py
@@ -3,6 +3,7 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
+from ..compat import compat_HTMLParseError
 from ..utils import (
    determine_ext,
    ExtractorError,
@@ -11,9 +12,11 @@ from ..utils import (
    get_element_by_attribute,
    get_element_by_class,
    int_or_none,
+    join_nonempty,
    js_to_json,
    merge_dicts,
    parse_iso8601,
+    parse_qs,
    smuggle_url,
    str_to_int,
    unescapeHTML,
@@ -126,8 +129,12 @@ class CSpanIE(InfoExtractor):
                        ext = 'vtt'
                    subtitle['ext'] = ext
            ld_info = self._search_json_ld(webpage, video_id, default={})
-            title = get_element_by_class('video-page-title', webpage) or \
-                self._og_search_title(webpage)
+            try:
+                title = get_element_by_class('video-page-title', webpage)
+            except compat_HTMLParseError:
+                title = None
+            if title is None:
+                title = self._og_search_title(webpage)
            description = get_element_by_attribute('itemprop', 'description', webpage) or \
                self._html_search_meta(['og:description', 'description'], webpage)
            return merge_dicts(info, ld_info, {
@@ -242,3 +249,42 @@ class CSpanIE(InfoExtractor):
                'title': title,
                'id': 'c' + video_id if video_type == 'clip' else video_id,
            }
+
+
+class CSpanCongressIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?c-span\.org/congress/'
+    _TESTS = [{
+        'url': 'https://www.c-span.org/congress/?chamber=house&date=2017-12-13&t=1513208380',
+        'info_dict': {
+            'id': 'house_2017-12-13',
+            'title': 'Congressional Chronicle - Members of Congress, Hearings and More',
+            'description': 'md5:54c264b7a8f219937987610243305a84',
+            'thumbnail': r're:https://ximage.c-spanvideo.org/.+',
+            'ext': 'mp4'
+        }
+    }]
+
+    def _real_extract(self, url):
+        query = parse_qs(url)
+        video_date = query.get('date', [None])[0]
+        video_id = join_nonempty(query.get('chamber', ['senate'])[0], video_date, delim='_')
+        webpage = self._download_webpage(url, video_id)
+        if not video_date:
+            jwp_date = re.search(r'jwsetup.clipprogdate = \'(?P<date>\d{4}-\d{2}-\d{2})\';', webpage)
+            if jwp_date:
+                video_id = f'{video_id}_{jwp_date.group("date")}'
+        jwplayer_data = self._parse_json(
+            self._search_regex(r'jwsetup\s*=\s*({(?:.|\n)[^;]+});', webpage, 'player config'),
+            video_id, transform_source=js_to_json)
+
+        title = (self._og_search_title(webpage, default=None)
+                 or self._html_search_regex(r'(?s)<title>(.*?)</title>', webpage, 'video title'))
+        description = (self._og_search_description(webpage, default=None)
+                       or self._html_search_meta('description', webpage, 'description', default=None))
+
+        return {
+            **self._parse_jwplayer_data(jwplayer_data, video_id, False),
+            'title': re.sub(r'\s+', ' ', title.split('|')[0]).strip(),
+            'description': description,
+            'http_headers': {'Referer': 'https://www.c-span.org/'},
+        }
--- a/yt_dlp/extractor/dailymotion.py
+++ b/yt_dlp/extractor/dailymotion.py
@@ -259,9 +259,7 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
                    continue
                if media_type == 'application/x-mpegURL':
                    formats.extend(self._extract_m3u8_formats(
-                        media_url, video_id, 'mp4',
-                        'm3u8' if is_live else 'm3u8_native',
-                        m3u8_id='hls', fatal=False))
+                        media_url, video_id, 'mp4', live=is_live, m3u8_id='hls', fatal=False))
                else:
                    f = {
                        'url': media_url,
--- a/yt_dlp/extractor/daystar.py
+++ b/yt_dlp/extractor/daystar.py
@@ -0,0 +1,48 @@
+from .common import InfoExtractor
+from ..utils import js_to_json, urljoin
+
+
+class DaystarClipIE(InfoExtractor):
+    IE_NAME = 'daystar:clip'
+    _VALID_URL = r'https?://player\.daystar\.tv/(?P<id>\w+)'
+    _TESTS = [{
+        'url': 'https://player.daystar.tv/0MTO2ITM',
+        'info_dict': {
+            'id': '0MTO2ITM',
+            'ext': 'mp4',
+            'title': 'The Dark World of COVID Pt. 1 | Aaron Siri',
+            'description': 'a420d320dda734e5f29458df3606c5f4',
+            'thumbnail': r're:^https?://.+\.jpg',
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        src_iframe = self._search_regex(r'\<iframe[^>]+src="([^"]+)"', webpage, 'src iframe')
+        webpage_iframe = self._download_webpage(
+            src_iframe.replace('player.php', 'config2.php'), video_id, headers={'Referer': src_iframe})
+
+        sources = self._parse_json(self._search_regex(
+            r'sources\:\s*(\[.*?\])', webpage_iframe, 'm3u8 source'), video_id, transform_source=js_to_json)
+
+        formats, subtitles = [], {}
+        for source in sources:
+            file = source.get('file')
+            if file and source.get('type') == 'm3u8':
+                fmts, subs = self._extract_m3u8_formats_and_subtitles(
+                    urljoin('https://www.lightcast.com/embed/', file),
+                    video_id, 'mp4', fatal=False, headers={'Referer': src_iframe})
+                formats.extend(fmts)
+                subtitles = self._merge_subtitles(subtitles, subs)
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': self._html_search_meta(['og:title', 'twitter:title'], webpage),
+            'description': self._html_search_meta(['og:description', 'twitter:description'], webpage),
+            'thumbnail': self._search_regex(r'image:\s*"([^"]+)', webpage_iframe, 'thumbnail'),
+            'formats': formats,
+            'subtitles': subtitles,
+        }
--- a/yt_dlp/extractor/dropbox.py
+++ b/yt_dlp/extractor/dropbox.py
@@ -56,8 +56,8 @@ class DropboxIE(InfoExtractor):
            else:
                raise ExtractorError('Password protected video, use --video-password <password>', expected=True)

-        json_string = self._html_search_regex(r'InitReact\.mountComponent.+ "props":(.+), "elem_id"', webpage, 'Info JSON')
-        info_json = self._parse_json(json_string, video_id)
+        json_string = self._html_search_regex(r'InitReact\.mountComponent\(.*?,\s*(\{.+\})\s*?\)', webpage, 'Info JSON')
+        info_json = self._parse_json(json_string, video_id).get('props')
        transcode_url = traverse_obj(info_json, ((None, 'preview'), 'file', 'preview', 'content', 'transcode_url'), get_all=False)
        formats, subtitles = self._extract_m3u8_formats_and_subtitles(transcode_url, video_id)

--- a/yt_dlp/extractor/engadget.py
+++ b/yt_dlp/extractor/engadget.py
@@ -7,16 +7,6 @@ class EngadgetIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?engadget\.com/video/(?P<id>[^/?#]+)'

    _TESTS = [{
-        # video with 5min ID
-        'url': 'http://www.engadget.com/video/518153925/',
-        'md5': 'c6820d4828a5064447a4d9fc73f312c9',
-        'info_dict': {
-            'id': '518153925',
-            'ext': 'mp4',
-            'title': 'Samsung Galaxy Tab Pro 8.4 Review',
-        },
-        'add_ie': ['FiveMin'],
-    }, {
        # video with vidible ID
        'url': 'https://www.engadget.com/video/57a28462134aa15a39f0421a/',
        'only_matching': True,
--- a/yt_dlp/extractor/extractors.py
+++ b/yt_dlp/extractor/extractors.py
@@ -14,6 +14,10 @@ from .abcotvs import (
    ABCOTVSIE,
    ABCOTVSClipsIE,
 )
+from .abematv import (
+    AbemaTVIE,
+    AbemaTVTitleIE,
+)
 from .academicearth import AcademicEarthCourseIE
 from .acast import (
    ACastIE,
@@ -64,6 +68,10 @@ from .anvato import AnvatoIE
 from .aol import AolIE
 from .allocine import AllocineIE
 from .aliexpress import AliExpressLiveIE
+from .alsace20tv import (
+    Alsace20TVIE,
+    Alsace20TVEmbedIE,
+)
 from .apa import APAIE
 from .aparat import AparatIE
 from .appleconnect import AppleConnectIE
@@ -87,6 +95,7 @@ from .arte import (
    ArteTVIE,
    ArteTVEmbedIE,
    ArteTVPlaylistIE,
+    ArteTVCategoryIE,
 )
 from .arnes import ArnesIE
 from .asiancrush import (
@@ -118,7 +127,7 @@ from .bandcamp import (
    BandcampIE,
    BandcampAlbumIE,
    BandcampWeeklyIE,
-    BandcampMusicIE,
+    BandcampUserIE,
 )
 from .bannedvideo import BannedVideoIE
 from .bbc import (
@@ -142,6 +151,7 @@ from .bfmtv import (
 )
 from .bibeltv import BibelTVIE
 from .bigflix import BigflixIE
+from .bigo import BigoIE
 from .bild import BildIE
 from .bilibili import (
    BiliBiliIE,
@@ -194,6 +204,7 @@ from .byutv import BYUtvIE
 from .c56 import C56IE
 from .cableav import CableAVIE
 from .callin import CallinIE
+from .caltrans import CaltransIE
 from .cam4 import CAM4IE
 from .camdemy import (
    CamdemyIE,
@@ -300,6 +311,10 @@ from .commonprotocols import (
 from .condenast import CondeNastIE
 from .contv import CONtvIE
 from .corus import CorusIE
+from .cpac import (
+    CPACIE,
+    CPACPlaylistIE,
+)
 from .cozytv import CozyTVIE
 from .cracked import CrackedIE
 from .crackle import CrackleIE
@@ -314,7 +329,7 @@ from .crunchyroll import (
    CrunchyrollBetaIE,
    CrunchyrollBetaShowIE,
 )
-from .cspan import CSpanIE
+from .cspan import CSpanIE, CSpanCongressIE
 from .ctsnews import CtsNewsIE
 from .ctv import CTVIE
 from .ctvnews import CTVNewsIE
@@ -342,6 +357,7 @@ from .daum import (
    DaumPlaylistIE,
    DaumUserIE,
 )
+from .daystar import DaystarClipIE
 from .dbtv import DBTVIE
 from .dctp import DctpTvIE
 from .deezer import (
@@ -472,6 +488,7 @@ from .faz import FazIE
 from .fc2 import (
    FC2IE,
    FC2EmbedIE,
+    FC2LiveIE,
 )
 from .fczenit import FczenitIE
 from .filmmodu import FilmmoduIE
@@ -481,7 +498,6 @@ from .filmon import (
 )
 from .filmweb import FilmwebIE
 from .firsttv import FirstTVIE
-from .fivemin import FiveMinIE
 from .fivetv import FiveTVIE
 from .flickr import FlickrIE
 from .folketinget import FolketingetIE
@@ -504,6 +520,7 @@ from .foxnews import (
    FoxNewsArticleIE,
 )
 from .foxsports import FoxSportsIE
+from .fptplay import FptplayIE
 from .franceculture import FranceCultureIE
 from .franceinter import FranceInterIE
 from .francetv import (
@@ -513,7 +530,6 @@ from .francetv import (
 )
 from .freesound import FreesoundIE
 from .freespeech import FreespeechIE
-from .freshlive import FreshLiveIE
 from .frontendmasters import (
    FrontendMastersIE,
    FrontendMastersLessonIE,
@@ -548,7 +564,10 @@ from .gazeta import GazetaIE
 from .gdcvault import GDCVaultIE
 from .gedidigital import GediDigitalIE
 from .generic import GenericIE
-from .gettr import GettrIE
+from .gettr import (
+    GettrIE,
+    GettrStreamingIE,
+)
 from .gfycat import GfycatIE
 from .giantbomb import GiantBombIE
 from .giga import GigaIE
@@ -585,7 +604,6 @@ from .hidive import HiDiveIE
 from .historicfilms import HistoricFilmsIE
 from .hitbox import HitboxIE, HitboxLiveIE
 from .hitrecord import HitRecordIE
-from .hornbunny import HornBunnyIE
 from .hotnewhiphop import HotNewHipHopIE
 from .hotstar import (
    HotStarIE,
@@ -655,7 +673,6 @@ from .iqiyi import (
    IqIE,
    IqAlbumIE
 )
-from .ir90tv import Ir90TvIE
 from .itv import (
    ITVIE,
    ITVBTCCIE,
@@ -677,7 +694,6 @@ from .joj import JojIE
 from .jwplatform import JWPlatformIE
 from .kakao import KakaoIE
 from .kaltura import KalturaIE
-from .kankan import KankanIE
 from .karaoketv import KaraoketvIE
 from .karrierevideos import KarriereVideosIE
 from .keezmovies import KeezMoviesIE
@@ -833,6 +849,7 @@ from .microsoftvirtualacademy import (
 from .mildom import (
    MildomIE,
    MildomVodIE,
+    MildomClipIE,
    MildomUserVodIE,
 )
 from .minds import (
@@ -890,6 +907,7 @@ from .mtv import (
    MTVItaliaProgrammaIE,
 )
 from .muenchentv import MuenchenTVIE
+from .murrtube import MurrtubeIE, MurrtubeUserIE
 from .musescore import MuseScoreIE
 from .musicdex import (
    MusicdexSongIE,
@@ -984,6 +1002,7 @@ from .nexx import (
    NexxIE,
    NexxEmbedIE,
 )
+from .nfb import NFBIE
 from .nfhsnetwork import NFHSNetworkIE
 from .nfl import (
    NFLIE,
@@ -992,6 +1011,9 @@ from .nfl import (
 from .nhk import (
    NhkVodIE,
    NhkVodProgramIE,
+    NhkForSchoolBangumiIE,
+    NhkForSchoolSubjectIE,
+    NhkForSchoolProgramListIE,
 )
 from .nhl import NHLIE
 from .nick import (
@@ -1001,14 +1023,16 @@ from .nick import (
    NickNightIE,
    NickRuIE,
 )
-
 from .niconico import (
    NiconicoIE,
    NiconicoPlaylistIE,
    NiconicoUserIE,
+    NiconicoSeriesIE,
+    NiconicoHistoryIE,
    NicovideoSearchDateIE,
    NicovideoSearchIE,
    NicovideoSearchURLIE,
+    NicovideoTagURLIE,
 )
 from .ninecninemedia import (
    NineCNineMediaIE,
@@ -1140,6 +1164,7 @@ from .patreon import (
 )
 from .pbs import PBSIE
 from .pearvideo import PearVideoIE
+from .peekvids import PeekVidsIE, PlayVidsIE
 from .peertube import (
    PeerTubeIE,
    PeerTubePlaylistIE,
@@ -1158,6 +1183,7 @@ from .periscope import (
 from .philharmoniedeparis import PhilharmonieDeParisIE
 from .phoenix import PhoenixIE
 from .photobucket import PhotobucketIE
+from .piapro import PiaproIE
 from .picarto import (
    PicartoIE,
    PicartoVodIE,
@@ -1319,11 +1345,14 @@ from .reuters import ReutersIE
 from .reverbnation import ReverbNationIE
 from .rice import RICEIE
 from .rmcdecouverte import RMCDecouverteIE
-from .ro220 import Ro220IE
 from .rockstargames import RockstarGamesIE
+from .rokfin import (
+    RokfinIE,
+    RokfinStackIE,
+    RokfinChannelIE,
+)
 from .roosterteeth import RoosterTeethIE, RoosterTeethSeriesIE
 from .rottentomatoes import RottenTomatoesIE
-from .roxwel import RoxwelIE
 from .rozhlas import RozhlasIE
 from .rtbf import RTBFIE
 from .rte import RteIE, RteRadioIE
@@ -1374,9 +1403,17 @@ from .megatvcom import (
    MegaTVComIE,
    MegaTVComEmbedIE,
 )
+from .ant1newsgr import (
+    Ant1NewsGrWatchIE,
+    Ant1NewsGrArticleIE,
+    Ant1NewsGrEmbedIE,
+)
 from .rutv import RUTVIE
 from .ruutu import RuutuIE
-from .ruv import RuvIE
+from .ruv import (
+    RuvIE,
+    RuvSpilaIE
+)
 from .safari import (
    SafariIE,
    SafariApiIE,
@@ -1573,6 +1610,7 @@ from .tele13 import Tele13IE
 from .telebruxelles import TeleBruxellesIE
 from .telecinco import TelecincoIE
 from .telegraaf import TelegraafIE
+from .telegram import TelegramEmbedIE
 from .telemb import TeleMBIE
 from .telemundo import TelemundoIE
 from .telequebec import (
@@ -1594,7 +1632,6 @@ from .theplatform import (
    ThePlatformIE,
    ThePlatformFeedIE,
 )
-from .thescene import TheSceneIE
 from .thestar import TheStarIE
 from .thesun import TheSunIE
 from .theta import (
@@ -1616,6 +1653,7 @@ from .tiktok import (
    TikTokSoundIE,
    TikTokEffectIE,
    TikTokTagIE,
+    TikTokVMIE,
    DouyinIE,
 )
 from .tinypic import TinyPicIE
@@ -1813,6 +1851,10 @@ from .vice import (
 from .vidbit import VidbitIE
 from .viddler import ViddlerIE
 from .videa import VideaIE
+from .videocampus_sachsen import (
+    VideocampusSachsenIE,
+    VideocampusSachsenEmbedIE,
+)
 from .videodetective import VideoDetectiveIE
 from .videofyme import VideofyMeIE
 from .videomore import (
@@ -1899,7 +1941,6 @@ from .vrv import (
 from .vshare import VShareIE
 from .vtm import VTMIE
 from .medialaan import MedialaanIE
-from .vube import VubeIE
 from .vuclip import VuClipIE
 from .vupload import VuploadIE
 from .vvvvid import (
@@ -1971,6 +2012,7 @@ from .ximalaya import (
    XimalayaIE,
    XimalayaAlbumIE
 )
+from .xinpianchang import XinpianchangIE
 from .xminus import XMinusIE
 from .xnxx import XNXXIE
 from .xstream import XstreamIE
--- a/yt_dlp/extractor/fc2.py
+++ b/yt_dlp/extractor/fc2.py
@@ -1,14 +1,21 @@
 # coding: utf-8
 from __future__ import unicode_literals

+import re
+
 from .common import InfoExtractor
 from ..compat import (
    compat_parse_qs,
 )
 from ..utils import (
    ExtractorError,
+    WebSocketsWrapper,
+    has_websockets,
+    js_to_json,
    sanitized_Request,
+    std_headers,
    traverse_obj,
+    update_url_query,
    urlencode_postdata,
    urljoin,
 )
@@ -90,7 +97,7 @@ class FC2IE(InfoExtractor):
                webpage,
                'title', fatal=False)
            thumbnail = self._og_search_thumbnail(webpage)
-            description = self._og_search_description(webpage)
+            description = self._og_search_description(webpage, default=None)

        vidplaylist = self._download_json(
            'https://video.fc2.com/api/v3/videoplaylist/%s?sh=1&fs=0' % video_id, video_id,
@@ -105,6 +112,7 @@ class FC2IE(InfoExtractor):
            'title': title,
            'url': vid_url,
            'ext': 'mp4',
+            'protocol': 'm3u8_native',
            'description': description,
            'thumbnail': thumbnail,
        }
@@ -146,3 +154,146 @@ class FC2EmbedIE(InfoExtractor):
            'title': title,
            'thumbnail': thumbnail,
        }
+
+
+class FC2LiveIE(InfoExtractor):
+    _VALID_URL = r'https?://live\.fc2\.com/(?P<id>\d+)'
+    IE_NAME = 'fc2:live'
+
+    _TESTS = [{
+        'url': 'https://live.fc2.com/57892267/',
+        'info_dict': {
+            'id': '57892267',
+            'title': 'どこまで・・・',
+            'uploader': 'あつあげ',
+            'uploader_id': '57892267',
+            'thumbnail': r're:https?://.+fc2.+',
+        },
+        'skip': 'livestream',
+    }]
+
+    def _real_extract(self, url):
+        if not has_websockets:
+            raise ExtractorError('websockets library is not available. Please install it.', expected=True)
+        video_id = self._match_id(url)
+        webpage = self._download_webpage('https://live.fc2.com/%s/' % video_id, video_id)
+
+        self._set_cookie('live.fc2.com', 'js-player_size', '1')
+
+        member_api = self._download_json(
+            'https://live.fc2.com/api/memberApi.php', video_id, data=urlencode_postdata({
+                'channel': '1',
+                'profile': '1',
+                'user': '1',
+                'streamid': video_id
+            }), note='Requesting member info')
+
+        control_server = self._download_json(
+            'https://live.fc2.com/api/getControlServer.php', video_id, note='Downloading ControlServer data',
+            data=urlencode_postdata({
+                'channel_id': video_id,
+                'mode': 'play',
+                'orz': '',
+                'channel_version': member_api['data']['channel_data']['version'],
+                'client_version': '2.1.0\n [1]',
+                'client_type': 'pc',
+                'client_app': 'browser_hls',
+                'ipv6': '',
+            }), headers={'X-Requested-With': 'XMLHttpRequest'})
+        self._set_cookie('live.fc2.com', 'l_ortkn', control_server['orz_raw'])
+
+        ws_url = update_url_query(control_server['url'], {'control_token': control_server['control_token']})
+        playlist_data = None
+
+        self.to_screen('%s: Fetching HLS playlist info via WebSocket' % video_id)
+        ws = WebSocketsWrapper(ws_url, {
+            'Cookie': str(self._get_cookies('https://live.fc2.com/'))[12:],
+            'Origin': 'https://live.fc2.com',
+            'Accept': '*/*',
+            'User-Agent': std_headers['User-Agent'],
+        })
+        ws.__enter__()
+
+        self.write_debug('[debug] Sending HLS server request')
+
+        while True:
+            recv = ws.recv()
+            if not recv:
+                continue
+            data = self._parse_json(recv, video_id, fatal=False)
+            if not data or not isinstance(data, dict):
+                continue
+
+            if data.get('name') == 'connect_complete':
+                break
+        ws.send(r'{"name":"get_hls_information","arguments":{},"id":1}')
+
+        while True:
+            recv = ws.recv()
+            if not recv:
+                continue
+            data = self._parse_json(recv, video_id, fatal=False)
+            if not data or not isinstance(data, dict):
+                continue
+            if data.get('name') == '_response_' and data.get('id') == 1:
+                self.write_debug('[debug] Goodbye.')
+                playlist_data = data
+                break
+            elif self._downloader.params.get('verbose', False):
+                if len(recv) > 100:
+                    recv = recv[:100] + '...'
+                self.to_screen('[debug] Server said: %s' % recv)
+
+        if not playlist_data:
+            raise ExtractorError('Unable to fetch HLS playlist info via WebSocket')
+
+        formats = []
+        for name, playlists in playlist_data['arguments'].items():
+            if not isinstance(playlists, list):
+                continue
+            for pl in playlists:
+                if pl.get('status') == 0 and 'master_playlist' in pl.get('url'):
+                    formats.extend(self._extract_m3u8_formats(
+                        pl['url'], video_id, ext='mp4', m3u8_id=name, live=True,
+                        headers={
+                            'Origin': 'https://live.fc2.com',
+                            'Referer': url,
+                        }))
+
+        self._sort_formats(formats)
+        for fmt in formats:
+            fmt.update({
+                'protocol': 'fc2_live',
+                'ws': ws,
+            })
+
+        title = self._html_search_meta(('og:title', 'twitter:title'), webpage, 'live title', fatal=False)
+        if not title:
+            title = self._html_extract_title(webpage, 'html title', fatal=False)
+            if title:
+                # remove service name in <title>
+                title = re.sub(r'\s+-\s+.+$', '', title)
+        uploader = None
+        if title:
+            match = self._search_regex(r'^(.+?)\s*\[(.+?)\]$', title, 'title and uploader', default=None, group=(1, 2))
+            if match and all(match):
+                title, uploader = match
+
+        live_info_view = self._search_regex(r'(?s)liveInfoView\s*:\s*({.+?}),\s*premiumStateView', webpage, 'user info', fatal=False) or None
+        if live_info_view:
+            # remove jQuery code from object literal
+            live_info_view = re.sub(r'\$\(.+?\)[^,]+,', '"",', live_info_view)
+            live_info_view = self._parse_json(js_to_json(live_info_view), video_id)
+
+        return {
+            'id': video_id,
+            'title': title or traverse_obj(live_info_view, 'title'),
+            'description': self._html_search_meta(
+                ('og:description', 'twitter:description'),
+                webpage, 'live description', fatal=False) or traverse_obj(live_info_view, 'info'),
+            'formats': formats,
+            'uploader': uploader or traverse_obj(live_info_view, 'name'),
+            'uploader_id': video_id,
+            'thumbnail': traverse_obj(live_info_view, 'thumb'),
+            'is_live': True,
+        }
--- a/yt_dlp/extractor/fivemin.py
+++ b/yt_dlp/extractor/fivemin.py
@@ -1,54 +0,0 @@
-from __future__ import unicode_literals
-
-from .common import InfoExtractor
-
-
-class FiveMinIE(InfoExtractor):
-    IE_NAME = '5min'
-    _VALID_URL = r'(?:5min:|https?://(?:[^/]*?5min\.com/|delivery\.vidible\.tv/aol)(?:(?:Scripts/PlayerSeed\.js|playerseed/?)?\?.*?playList=)?)(?P<id>\d+)'
-
-    _TESTS = [
-        {
-            # From http://www.engadget.com/2013/11/15/ipad-mini-retina-display-review/
-            'url': 'http://pshared.5min.com/Scripts/PlayerSeed.js?sid=281&width=560&height=345&playList=518013791',
-            'md5': '4f7b0b79bf1a470e5004f7112385941d',
-            'info_dict': {
-                'id': '518013791',
-                'ext': 'mp4',
-                'title': 'iPad Mini with Retina Display Review',
-                'description': 'iPad mini with Retina Display review',
-                'duration': 177,
-                'uploader': 'engadget',
-                'upload_date': '20131115',
-                'timestamp': 1384515288,
-            },
-            'params': {
-                # m3u8 download
-                'skip_download': True,
-            }
-        },
-        {
-            # From http://on.aol.com/video/how-to-make-a-next-level-fruit-salad-518086247
-            'url': '5min:518086247',
-            'md5': 'e539a9dd682c288ef5a498898009f69e',
-            'info_dict': {
-                'id': '518086247',
-                'ext': 'mp4',
-                'title': 'How to Make a Next-Level Fruit Salad',
-                'duration': 184,
-            },
-            'skip': 'no longer available',
-        },
-        {
-            'url': 'http://embed.5min.com/518726732/',
-            'only_matching': True,
-        },
-        {
-            'url': 'http://delivery.vidible.tv/aol?playList=518013791',
-            'only_matching': True,
-        }
-    ]
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-        return self.url_result('aol-video:%s' % video_id)
--- a/yt_dlp/extractor/fptplay.py
+++ b/yt_dlp/extractor/fptplay.py
@@ -0,0 +1,102 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import hashlib
+import time
+import urllib.parse
+
+from .common import InfoExtractor
+from ..utils import (
+    join_nonempty,
+)
+
+
+class FptplayIE(InfoExtractor):
+    _VALID_URL = r'https?://fptplay\.vn/(?P<type>xem-video)/[^/]+\-(?P<id>\w+)(?:/tap-(?P<episode>[^/]+)?/?(?:[?#]|$)|)'
+    _GEO_COUNTRIES = ['VN']
+    IE_NAME = 'fptplay'
+    IE_DESC = 'fptplay.vn'
+    _TESTS = [{
+        'url': 'https://fptplay.vn/xem-video/nhan-duyen-dai-nhan-xin-dung-buoc-621a123016f369ebbde55945',
+        'md5': 'ca0ee9bc63446c0c3e9a90186f7d6b33',
+        'info_dict': {
+            'id': '621a123016f369ebbde55945',
+            'ext': 'mp4',
+            'title': 'Nhân Duyên Đại Nhân Xin Dừng Bước - Ms. Cupid In Love',
+            'description': 'md5:23cf7d1ce0ade8e21e76ae482e6a8c6c',
+        },
+    }, {
+        'url': 'https://fptplay.vn/xem-video/ma-toi-la-dai-gia-61f3aa8a6b3b1d2e73c60eb5/tap-3',
+        'md5': 'b35be968c909b3e4e1e20ca45dd261b1',
+        'info_dict': {
+            'id': '61f3aa8a6b3b1d2e73c60eb5',
+            'ext': 'mp4',
+            'title': 'Má Tôi Là Đại Gia - 3',
+            'description': 'md5:ff8ba62fb6e98ef8875c42edff641d1c',
+        },
+    }, {
+        'url': 'https://fptplay.vn/xem-video/nha-co-chuyen-hi-alls-well-ends-well-1997-6218995f6af792ee370459f0',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        type_url, video_id, episode = self._match_valid_url(url).group('type', 'id', 'episode')
+        webpage = self._download_webpage(url, video_id=video_id, fatal=False)
+        info = self._download_json(self.get_api_with_st_token(video_id, episode or 0), video_id)
+        formats, subtitles = self._extract_m3u8_formats_and_subtitles(info['data']['url'], video_id, 'mp4')
+        self._sort_formats(formats)
+        return {
+            'id': video_id,
+            'title': join_nonempty(
+                self._html_search_meta(('og:title', 'twitter:title'), webpage), episode, delim=' - '),
+            'description': self._html_search_meta(['og:description', 'twitter:description'], webpage),
+            'formats': formats,
+            'subtitles': subtitles,
+        }
+
+    def get_api_with_st_token(self, video_id, episode):
+        path = f'/api/v6.2_w/stream/vod/{video_id}/{episode}/auto_vip'
+        timestamp = int(time.time()) + 10800
+
+        t = hashlib.md5(f'WEBv6Dkdsad90dasdjlALDDDS{timestamp}{path}'.encode()).hexdigest().upper()
+        r = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
+        n = [int(f'0x{t[2 * o: 2 * o + 2]}', 16) for o in range(len(t) // 2)]
+
+        def convert(e):
+            t = ''
+            n = 0
+            i = [0, 0, 0]
+            a = [0, 0, 0, 0]
+            s = len(e)
+            c = 0
+            for z in range(s, 0, -1):
+                if n <= 3:
+                    i[n] = e[c]
+                n += 1
+                c += 1
+                if 3 == n:
+                    a[0] = (252 & i[0]) >> 2
+                    a[1] = ((3 & i[0]) << 4) + ((240 & i[1]) >> 4)
+                    a[2] = ((15 & i[1]) << 2) + ((192 & i[2]) >> 6)
+                    a[3] = (63 & i[2])
+                    for v in range(4):
+                        t += r[a[v]]
+                    n = 0
+            if n:
+                for o in range(n, 3):
+                    i[o] = 0
+
+                for o in range(n + 1):
+                    a[0] = (252 & i[0]) >> 2
+                    a[1] = ((3 & i[0]) << 4) + ((240 & i[1]) >> 4)
+                    a[2] = ((15 & i[1]) << 2) + ((192 & i[2]) >> 6)
+                    a[3] = (63 & i[2])
+                    t += r[a[o]]
+                n += 1
+                while n < 3:
+                    t += ''
+                    n += 1
+            return t
+
+        st_token = convert(n).replace('+', '-').replace('/', '_').replace('=', '')
+        return f'https://api.fptplay.net{path}?{urllib.parse.urlencode({"st": st_token, "e": timestamp})}'
--- a/yt_dlp/extractor/freshlive.py
+++ b/yt_dlp/extractor/freshlive.py
@@ -1,80 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-from .common import InfoExtractor
-from ..compat import compat_str
-from ..utils import (
-    ExtractorError,
-    int_or_none,
-    try_get,
-    unified_timestamp,
-)
-
-
-class FreshLiveIE(InfoExtractor):
-    _VALID_URL = r'https?://freshlive\.tv/[^/]+/(?P<id>\d+)'
-    _TEST = {
-        'url': 'https://freshlive.tv/satotv/74712',
-        'md5': '9f0cf5516979c4454ce982df3d97f352',
-        'info_dict': {
-            'id': '74712',
-            'ext': 'mp4',
-            'title': 'テスト',
-            'description': 'テスト',
-            'thumbnail': r're:^https?://.*\.jpg$',
-            'duration': 1511,
-            'timestamp': 1483619655,
-            'upload_date': '20170105',
-            'uploader': 'サトTV',
-            'uploader_id': 'satotv',
-            'view_count': int,
-            'comment_count': int,
-            'is_live': False,
-        }
-    }
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-
-        webpage = self._download_webpage(url, video_id)
-
-        options = self._parse_json(
-            self._search_regex(
-                r'window\.__CONTEXT__\s*=\s*({.+?});\s*</script>',
-                webpage, 'initial context'),
-            video_id)
-
-        info = options['context']['dispatcher']['stores']['ProgramStore']['programs'][video_id]
-
-        title = info['title']
-
-        if info.get('status') == 'upcoming':
-            raise ExtractorError('Stream %s is upcoming' % video_id, expected=True)
-
-        stream_url = info.get('liveStreamUrl') or info['archiveStreamUrl']
-
-        is_live = info.get('liveStreamUrl') is not None
-
-        formats = self._extract_m3u8_formats(
-            stream_url, video_id, 'mp4',
-            'm3u8_native', m3u8_id='hls')
-
-        return {
-            'id': video_id,
-            'formats': formats,
-            'title': title,
-            'description': info.get('description'),
-            'thumbnail': info.get('thumbnailUrl'),
-            'duration': int_or_none(info.get('airTime')),
-            'timestamp': unified_timestamp(info.get('createdAt')),
-            'uploader': try_get(
-                info, lambda x: x['channel']['title'], compat_str),
-            'uploader_id': try_get(
-                info, lambda x: x['channel']['code'], compat_str),
-            'uploader_url': try_get(
-                info, lambda x: x['channel']['permalink'], compat_str),
-            'view_count': int_or_none(info.get('viewCount')),
-            'comment_count': int_or_none(info.get('commentCount')),
-            'tags': info.get('tags', []),
-            'is_live': is_live,
-        }
--- a/yt_dlp/extractor/frontendmasters.py
+++ b/yt_dlp/extractor/frontendmasters.py
@@ -252,9 +252,9 @@ class FrontendMastersCourseIE(FrontendMastersPageBaseIE):
        entries = []
        for lesson in lessons:
            lesson_name = lesson.get('slug')
-            if not lesson_name:
-                continue
            lesson_id = lesson.get('hash') or lesson.get('statsId')
+            if not lesson_id or not lesson_name:
+                continue
            entries.append(self._extract_lesson(chapters, lesson_id, lesson))

        title = course.get('title')
--- a/yt_dlp/extractor/fujitv.py
+++ b/yt_dlp/extractor/fujitv.py
@@ -7,6 +7,13 @@ from .common import InfoExtractor
 class FujiTVFODPlus7IE(InfoExtractor):
    _VALID_URL = r'https?://fod\.fujitv\.co\.jp/title/(?P<sid>[0-9a-z]{4})/(?P<id>[0-9a-z]+)'
    _BASE_URL = 'https://i.fod.fujitv.co.jp/'
+    _BITRATE_MAP = {
+        300: (320, 180),
+        800: (640, 360),
+        1200: (1280, 720),
+        2000: (1280, 720),
+        4000: (1920, 1080),
+    }

    _TESTS = [{
        'url': 'https://fod.fujitv.co.jp/title/5d40/5d40110076',
@@ -19,6 +26,17 @@ class FujiTVFODPlus7IE(InfoExtractor):
            'description': 'md5:b3f51dbfdda162ac4f789e0ff4d65750',
            'thumbnail': 'https://i.fod.fujitv.co.jp/img/program/5d40/episode/5d40110076_a.jpg',
        },
+    }, {
+        'url': 'https://fod.fujitv.co.jp/title/5d40/5d40810083',
+        'info_dict': {
+            'id': '5d40810083',
+            'ext': 'mp4',
+            'title': '#1324 『まる子とオニの子』の巻／『結成！2月をムダにしない会』の巻',
+            'description': 'md5:3972d900b896adc8ab1849e310507efa',
+            'series': 'ちびまる子ちゃん',
+            'series_id': '5d40',
+            'thumbnail': 'https://i.fod.fujitv.co.jp/img/program/5d40/episode/5d40810083_a.jpg'},
+        'skip': 'Video available only in one week'
    }]

    def _real_extract(self, url):
@@ -36,6 +54,9 @@ class FujiTVFODPlus7IE(InfoExtractor):
            if not src.get('url'):
                continue
            fmt, subs = self._extract_m3u8_formats_and_subtitles(src['url'], video_id, 'mp4')
+            for f in fmt:
+                f.update(dict(zip(('height', 'width'),
+                                  self._BITRATE_MAP.get(f.get('tbr'), ()))))
            formats.extend(fmt)
            subtitles = self._merge_subtitles(subtitles, subs)
        self._sort_formats(formats, ['tbr'])
--- a/yt_dlp/extractor/generic.py
+++ b/yt_dlp/extractor/generic.py
@@ -103,6 +103,7 @@ from .videopress import VideoPressIE
 from .rutube import RutubeIE
 from .glomex import GlomexEmbedIE
 from .megatvcom import MegaTVComEmbedIE
+from .ant1newsgr import Ant1NewsGrEmbedIE
 from .limelight import LimelightBaseIE
 from .anvato import AnvatoIE
 from .washingtonpost import WashingtonPostIE
@@ -213,7 +214,7 @@ class GenericIE(InfoExtractor):
        {
            'url': 'http://phihag.de/2014/youtube-dl/rss2.xml',
            'info_dict': {
-                'id': 'http://phihag.de/2014/youtube-dl/rss2.xml',
+                'id': 'https://phihag.de/2014/youtube-dl/rss2.xml',
                'title': 'Zero Punctuation',
                'description': 're:.*groundbreaking video review series.*'
            },
@@ -258,6 +259,9 @@ class GenericIE(InfoExtractor):
                    'episode_number': 1,
                    'season_number': 1,
                    'age_limit': 0,
+                    'season': 'Season 1',
+                    'direct': True,
+                    'episode': 'Episode 1',
                },
            }],
            'params': {
@@ -274,6 +278,16 @@ class GenericIE(InfoExtractor):
            },
            'playlist_mincount': 100,
        },
+        # RSS feed with guid
+        {
+            'url': 'https://www.omnycontent.com/d/playlist/a7b4f8fe-59d9-4afc-a79a-a90101378abf/bf2c1d80-3656-4449-9d00-a903004e8f84/efbff746-e7c1-463a-9d80-a903004e8f8f/podcast.rss',
+            'info_dict': {
+                'id': 'https://www.omnycontent.com/d/playlist/a7b4f8fe-59d9-4afc-a79a-a90101378abf/bf2c1d80-3656-4449-9d00-a903004e8f84/efbff746-e7c1-463a-9d80-a903004e8f8f/podcast.rss',
+                'description': 'md5:be809a44b63b0c56fb485caf68685520',
+                'title': 'The Little Red Podcast',
+            },
+            'playlist_mincount': 76,
+        },
        # SMIL from http://videolectures.net/promogram_igor_mekjavic_eng
        {
            'url': 'http://videolectures.net/promogram_igor_mekjavic_eng/video/1/smil.xml',
@@ -1456,24 +1470,6 @@ class GenericIE(InfoExtractor):
                'duration': 45.115,
            },
        },
-        # 5min embed
-        {
-            'url': 'http://techcrunch.com/video/facebook-creates-on-this-day-crunch-report/518726732/',
-            'md5': '4c6f127a30736b59b3e2c19234ee2bf7',
-            'info_dict': {
-                'id': '518726732',
-                'ext': 'mp4',
-                'title': 'Facebook Creates "On This Day" | Crunch Report',
-                'description': 'Amazon updates Fire TV line, Tesla\'s Model X spotted in the wild',
-                'timestamp': 1427237531,
-                'uploader': 'Crunch Report',
-                'upload_date': '20150324',
-            },
-            'params': {
-                # m3u8 download
-                'skip_download': True,
-            },
-        },
        # Crooks and Liars embed
        {
            'url': 'http://crooksandliars.com/2015/04/fox-friends-says-protecting-atheists',
@@ -2536,6 +2532,9 @@ class GenericIE(InfoExtractor):
            if not next_url:
                continue

+            if it.find('guid').text is not None:
+                next_url = smuggle_url(next_url, {'force_videoid': it.find('guid').text})
+
            def itunes(key):
                return xpath_text(
                    it, xpath_with_ns('./itunes:%s' % key, NS_MAP),
@@ -3337,12 +3336,6 @@ class GenericIE(InfoExtractor):
        if mobj is not None:
            return self.url_result(mobj.group('url'))

-        # Look for 5min embeds
-        mobj = re.search(
-            r'<meta[^>]+property="og:video"[^>]+content="https?://embed\.5min\.com/(?P<id>[0-9]+)/?', webpage)
-        if mobj is not None:
-            return self.url_result('5min:%s' % mobj.group('id'), 'FiveMin')
-
        # Look for Crooks and Liars embeds
        mobj = re.search(
            r'<(?:iframe[^>]+src|param[^>]+value)=(["\'])(?P<url>(?:https?:)?//embed\.crooksandliars\.com/(?:embed|v)/.+?)\1', webpage)
@@ -3552,6 +3545,12 @@ class GenericIE(InfoExtractor):
            return self.playlist_from_matches(
                megatvcom_urls, video_id, video_title, ie=MegaTVComEmbedIE.ie_key())

+        # Look for ant1news.gr embeds
+        ant1newsgr_urls = list(Ant1NewsGrEmbedIE._extract_urls(webpage))
+        if ant1newsgr_urls:
+            return self.playlist_from_matches(
+                ant1newsgr_urls, video_id, video_title, ie=Ant1NewsGrEmbedIE.ie_key())
+
        # Look for WashingtonPost embeds
        wapo_urls = WashingtonPostIE._extract_urls(webpage)
        if wapo_urls:
@@ -3999,12 +3998,16 @@ class GenericIE(InfoExtractor):

            # here's a fun little line of code for you:
            video_id = os.path.splitext(video_id)[0]
+            headers = {
+                'referer': full_response.geturl()
+            }

            entry_info_dict = {
                'id': video_id,
                'uploader': video_uploader,
                'title': video_title,
                'age_limit': age_limit,
+                'http_headers': headers,
            }

            if RtmpIE.suitable(video_url):
@@ -4022,11 +4025,11 @@ class GenericIE(InfoExtractor):
            elif ext == 'xspf':
                return self.playlist_result(self._extract_xspf_playlist(video_url, video_id), video_id)
            elif ext == 'm3u8':
-                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(video_url, video_id, ext='mp4')
+                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(video_url, video_id, ext='mp4', headers=headers)
            elif ext == 'mpd':
-                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_mpd_formats_and_subtitles(video_url, video_id)
+                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_mpd_formats_and_subtitles(video_url, video_id, headers=headers)
            elif ext == 'f4m':
-                entry_info_dict['formats'] = self._extract_f4m_formats(video_url, video_id)
+                entry_info_dict['formats'] = self._extract_f4m_formats(video_url, video_id, headers=headers)
            elif re.search(r'(?i)\.(?:ism|smil)/manifest', video_url) and video_url != url:
                # Just matching .ism/manifest is not enough to be reliably sure
                # whether it's actually an ISM manifest or some other streaming
--- a/yt_dlp/extractor/gettr.py
+++ b/yt_dlp/extractor/gettr.py
@@ -3,22 +3,30 @@ from __future__ import unicode_literals

 from .common import InfoExtractor
 from ..utils import (
+    bool_or_none,
    ExtractorError,
    dict_get,
    float_or_none,
    int_or_none,
-    remove_end,
    str_or_none,
+    traverse_obj,
    try_get,
    url_or_none,
    urljoin,
 )


-class GettrIE(InfoExtractor):
-    _VALID_URL = r'https?://(www\.)?gettr\.com/post/(?P<id>[a-z0-9]+)'
+class GettrBaseIE(InfoExtractor):
+    _BASE_REGEX = r'https?://(www\.)?gettr\.com/'
    _MEDIA_BASE_URL = 'https://media.gettr.com/'

+    def _call_api(self, path, video_id, *args, **kwargs):
+        return self._download_json(urljoin('https://api.gettr.com/u/', path), video_id, *args, **kwargs)['result']
+
+
+class GettrIE(GettrBaseIE):
+    _VALID_URL = GettrBaseIE._BASE_REGEX + r'post/(?P<id>[a-z0-9]+)'
+
    _TESTS = [{
        'url': 'https://www.gettr.com/post/pcf6uv838f',
        'info_dict': {
@@ -28,9 +36,11 @@ class GettrIE(InfoExtractor):
            'ext': 'mp4',
            'uploader': 'EpochTV',
            'uploader_id': 'epochtv',
+            'upload_date': '20210927',
            'thumbnail': r're:^https?://.+/out\.jpg',
-            'timestamp': 1632782451058,
+            'timestamp': 1632782451.058,
            'duration': 58.5585,
+            'tags': ['hornofafrica', 'explorations'],
        }
    }, {
        'url': 'https://gettr.com/post/p4iahp',
@@ -41,43 +51,69 @@ class GettrIE(InfoExtractor):
            'ext': 'mp4',
            'uploader': 'Neues Forum Freiheit',
            'uploader_id': 'nf_freiheit',
+            'upload_date': '20210718',
            'thumbnail': r're:^https?://.+/out\.jpg',
-            'timestamp': 1626594455017,
+            'timestamp': 1626594455.017,
            'duration': 23,
+            'tags': 'count:12',
        }
+    }, {
+        # quote post
+        'url': 'https://gettr.com/post/pxn5b743a9',
+        'only_matching': True,
+    }, {
+        # quote with video
+        'url': 'https://gettr.com/post/pxtiiz5ca2',
+        'only_matching': True,
+    }, {
+        # streaming embed
+        'url': 'https://gettr.com/post/pxlu8p3b13',
+        'only_matching': True,
+    }, {
+        # youtube embed
+        'url': 'https://gettr.com/post/pv6wp9e24c',
+        'only_matching': True,
+        'add_ie': ['Youtube'],
    }]

    def _real_extract(self, url):
        post_id = self._match_id(url)
        webpage = self._download_webpage(url, post_id)
+        api_data = self._call_api('post/%s?incl="poststats|userinfo"' % post_id, post_id)

-        api_data = self._download_json(
-            'https://api.gettr.com/u/post/%s?incl="poststats|userinfo"' % post_id, post_id)
+        post_data = api_data.get('data')
+        user_data = try_get(api_data, lambda x: x['aux']['uinf'][post_data['uid']], dict) or {}

-        post_data = try_get(api_data, lambda x: x['result']['data'])
-        user_data = try_get(api_data, lambda x: x['result']['aux']['uinf'][post_data['uid']]) or {}
+        vid = post_data.get('vid')
+        ovid = post_data.get('ovid')

-        if post_data.get('nfound'):
-            raise ExtractorError(post_data.get('txt'), expected=True)
+        if post_data.get('p_type') == 'stream':
+            return self.url_result(f'https://gettr.com/streaming/{post_id}', ie='GettrStreaming', video_id=post_id)
+
+        if not (ovid or vid):
+            embed_url = url_or_none(post_data.get('prevsrc'))
+            shared_post_id = traverse_obj(api_data, ('aux', 'shrdpst', '_id'), ('data', 'rpstIds', 0), expected_type=str)
+
+            if embed_url:
+                return self.url_result(embed_url)
+            elif shared_post_id:
+                return self.url_result(f'https://gettr.com/post/{shared_post_id}', ie='Gettr', video_id=shared_post_id)
+            else:
+                raise ExtractorError('There\'s no video in this post.')

        title = description = str_or_none(
            post_data.get('txt') or self._og_search_description(webpage))

        uploader = str_or_none(
            user_data.get('nickname')
-            or remove_end(self._og_search_title(webpage), ' on GETTR'))
+            or self._search_regex(r'^(.+?) on GETTR', self._og_search_title(webpage, default=''), 'uploader', fatal=False))
+
        if uploader:
            title = '%s - %s' % (uploader, title)

-        if not dict_get(post_data, ['vid', 'ovid']):
-            raise ExtractorError('There\'s no video in this post.')
-
-        vid = post_data.get('vid')
-        ovid = post_data.get('ovid')
-
-        formats = self._extract_m3u8_formats(
+        formats, subtitles = self._extract_m3u8_formats_and_subtitles(
            urljoin(self._MEDIA_BASE_URL, vid), post_id, 'mp4',
-            entry_protocol='m3u8_native', m3u8_id='hls') if vid else []
+            entry_protocol='m3u8_native', m3u8_id='hls', fatal=False) if vid else ([], {})

        if ovid:
            formats.append({
@@ -86,8 +122,6 @@ class GettrIE(InfoExtractor):
                'ext': 'mp4',
                'width': int_or_none(post_data.get('vid_wid')),
                'height': int_or_none(post_data.get('vid_hgt')),
-                'source_preference': 1,
-                'quality': 1,
            })

        self._sort_formats(formats)
@@ -96,15 +130,84 @@ class GettrIE(InfoExtractor):
            'id': post_id,
            'title': title,
            'description': description,
-            'thumbnail': url_or_none(
-                urljoin(self._MEDIA_BASE_URL, post_data.get('main'))
-                or self._og_search_thumbnail(webpage)),
-            'timestamp': int_or_none(post_data.get('cdate')),
+            'formats': formats,
+            'subtitles': subtitles,
+            'uploader': uploader,
            'uploader_id': str_or_none(
                dict_get(user_data, ['_id', 'username'])
                or post_data.get('uid')),
-            'uploader': uploader,
-            'formats': formats,
+            'thumbnail': url_or_none(
+                urljoin(self._MEDIA_BASE_URL, post_data.get('main'))
+                or self._html_search_meta(['og:image', 'image'], webpage, 'thumbnail', fatal=False)),
+            'timestamp': float_or_none(dict_get(post_data, ['cdate', 'udate']), scale=1000),
            'duration': float_or_none(post_data.get('vid_dur')),
            'tags': post_data.get('htgs'),
        }
+
+
+class GettrStreamingIE(GettrBaseIE):
+    _VALID_URL = GettrBaseIE._BASE_REGEX + r'streaming/(?P<id>[a-z0-9]+)'
+
+    _TESTS = [{
+        'url': 'https://gettr.com/streaming/psoiulc122',
+        'info_dict': {
+            'id': 'psoiulc122',
+            'ext': 'mp4',
+            'description': 'md5:56bca4b8f48f1743d9fd03d49c723017',
+            'view_count': int,
+            'uploader': 'Corona Investigative Committee',
+            'uploader_id': 'coronacommittee',
+            'duration': 5180.184,
+            'thumbnail': r're:^https?://.+',
+            'title': 'Day 1: Opening Session of the Grand Jury Proceeding',
+            'timestamp': 1644080997.164,
+            'upload_date': '20220205',
+        }
+    }, {
+        'url': 'https://gettr.com/streaming/psfmeefcc1',
+        'info_dict': {
+            'id': 'psfmeefcc1',
+            'ext': 'mp4',
+            'title': 'Session 90: "The Virus Of Power"',
+            'view_count': int,
+            'uploader_id': 'coronacommittee',
+            'description': 'md5:98986acdf656aa836bf36f9c9704c65b',
+            'uploader': 'Corona Investigative Committee',
+            'thumbnail': r're:^https?://.+',
+            'duration': 21872.507,
+            'timestamp': 1643976662.858,
+            'upload_date': '20220204',
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        video_info = self._call_api('live/join/%s' % video_id, video_id, data={})
+
+        live_info = video_info['broadcast']
+        live_url = url_or_none(live_info.get('url'))
+
+        formats, subtitles = self._extract_m3u8_formats_and_subtitles(
+            live_url, video_id, ext='mp4',
+            entry_protocol='m3u8_native', m3u8_id='hls', fatal=False) if live_url else ([], {})
+
+        thumbnails = [{
+            'url': urljoin(self._MEDIA_BASE_URL, thumbnail),
+        } for thumbnail in try_get(video_info, lambda x: x['postData']['imgs'], list) or []]
+
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': try_get(video_info, lambda x: x['postData']['ttl'], str),
+            'description': try_get(video_info, lambda x: x['postData']['dsc'], str),
+            'formats': formats,
+            'subtitles': subtitles,
+            'thumbnails': thumbnails,
+            'uploader': try_get(video_info, lambda x: x['liveHostInfo']['nickname'], str),
+            'uploader_id': try_get(video_info, lambda x: x['liveHostInfo']['_id'], str),
+            'view_count': int_or_none(live_info.get('viewsCount')),
+            'timestamp': float_or_none(live_info.get('startAt'), scale=1000),
+            'duration': float_or_none(live_info.get('duration'), scale=1000),
+            'is_live': bool_or_none(live_info.get('isLive')),
+        }
--- a/yt_dlp/extractor/globo.py
+++ b/yt_dlp/extractor/globo.py
@@ -139,11 +139,11 @@ class GloboIE(InfoExtractor):
        resource_url = source['scheme'] + '://' + source['domain'] + source['path']
        signed_url = '%s?h=%s&k=html5&a=%s' % (resource_url, signed_hash, 'F' if video.get('subscriber_only') else 'A')

-        formats.extend(self._extract_m3u8_formats(
-            signed_url, video_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls', fatal=False))
+        fmts, subtitles = self._extract_m3u8_formats_and_subtitles(
+            signed_url, video_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls', fatal=False)
+        formats.extend(fmts)
        self._sort_formats(formats)

-        subtitles = {}
        for resource in video['resources']:
            if resource.get('type') == 'subtitle':
                subtitles.setdefault(resource.get('language') or 'por', []).append({
@@ -186,6 +186,7 @@ class GloboArticleIE(InfoExtractor):
        r'\bvideosIDs\s*:\s*["\']?(\d{7,})',
        r'\bdata-id=["\'](\d{7,})',
        r'<div[^>]+\bid=["\'](\d{7,})',
+        r'<bs-player[^>]+\bvideoid=["\'](\d{8,})',
    ]

    _TESTS = [{
@@ -213,6 +214,14 @@ class GloboArticleIE(InfoExtractor):
    }, {
        'url': 'http://oglobo.globo.com/rio/a-amizade-entre-um-entregador-de-farmacia-um-piano-19946271',
        'only_matching': True,
+    }, {
+        'url': 'https://ge.globo.com/video/ta-na-area-como-foi-assistir-ao-jogo-do-palmeiras-que-a-globo-nao-passou-10287094.ghtml',
+        'info_dict': {
+            'id': 'ta-na-area-como-foi-assistir-ao-jogo-do-palmeiras-que-a-globo-nao-passou-10287094',
+            'title': 'Tá na Área: como foi assistir ao jogo do Palmeiras que a Globo não passou',
+            'description': 'md5:2d089d036c4c9675117d3a56f8c61739',
+        },
+        'playlist_count': 1,
    }]

    @classmethod
@@ -228,6 +237,6 @@ class GloboArticleIE(InfoExtractor):
        entries = [
            self.url_result('globo:%s' % video_id, GloboIE.ie_key())
            for video_id in orderedSet(video_ids)]
-        title = self._og_search_title(webpage, fatal=False)
+        title = self._og_search_title(webpage)
        description = self._html_search_meta('description', webpage)
        return self.playlist_result(entries, display_id, title, description)
--- a/yt_dlp/extractor/hornbunny.py
+++ b/yt_dlp/extractor/hornbunny.py
@@ -1,49 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-from .common import InfoExtractor
-from ..utils import (
-    int_or_none,
-    parse_duration,
-)
-
-
-class HornBunnyIE(InfoExtractor):
-    _VALID_URL = r'http?://(?:www\.)?hornbunny\.com/videos/(?P<title_dash>[a-z-]+)-(?P<id>\d+)\.html'
-    _TEST = {
-        'url': 'http://hornbunny.com/videos/panty-slut-jerk-off-instruction-5227.html',
-        'md5': 'e20fd862d1894b67564c96f180f43924',
-        'info_dict': {
-            'id': '5227',
-            'ext': 'mp4',
-            'title': 'panty slut jerk off instruction',
-            'duration': 550,
-            'age_limit': 18,
-            'view_count': int,
-            'thumbnail': r're:^https?://.*\.jpg$',
-        }
-    }
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-
-        webpage = self._download_webpage(url, video_id)
-        title = self._og_search_title(webpage)
-        info_dict = self._parse_html5_media_entries(url, webpage, video_id)[0]
-
-        duration = parse_duration(self._search_regex(
-            r'<strong>Runtime:</strong>\s*([0-9:]+)</div>',
-            webpage, 'duration', fatal=False))
-        view_count = int_or_none(self._search_regex(
-            r'<strong>Views:</strong>\s*(\d+)</div>',
-            webpage, 'view count', fatal=False))
-
-        info_dict.update({
-            'id': video_id,
-            'title': title,
-            'duration': duration,
-            'view_count': view_count,
-            'age_limit': 18,
-        })
-
-        return info_dict
--- a/yt_dlp/extractor/huffpost.py
+++ b/yt_dlp/extractor/huffpost.py
@@ -80,9 +80,6 @@ class HuffPostIE(InfoExtractor):
                    'vcodec': 'none' if key.startswith('audio/') else None,
                })

-        if not formats and data.get('fivemin_id'):
-            return self.url_result('5min:%s' % data['fivemin_id'])
-
        self._sort_formats(formats)

        return {
--- a/yt_dlp/extractor/imggaming.py
+++ b/yt_dlp/extractor/imggaming.py
@@ -96,7 +96,7 @@ class ImgGamingBaseIE(InfoExtractor):
                continue
            if proto == 'hls':
                m3u8_formats = self._extract_m3u8_formats(
-                    media_url, media_id, 'mp4', 'm3u8' if is_live else 'm3u8_native',
+                    media_url, media_id, 'mp4', live=is_live,
                    m3u8_id='hls', fatal=False, headers=self._MANIFEST_HEADERS)
                for f in m3u8_formats:
                    f.setdefault('http_headers', {}).update(self._MANIFEST_HEADERS)
--- a/yt_dlp/extractor/instagram.py
+++ b/yt_dlp/extractor/instagram.py
@@ -17,7 +17,6 @@ from ..utils import (
    get_element_by_attribute,
    int_or_none,
    lowercase_escape,
-    std_headers,
    str_or_none,
    str_to_int,
    traverse_obj,
@@ -503,7 +502,7 @@ class InstagramPlaylistBaseIE(InstagramBaseIE):
                    '%s' % rhx_gis,
                    '',
                    '%s:%s' % (rhx_gis, csrf_token),
-                    '%s:%s:%s' % (rhx_gis, csrf_token, std_headers['User-Agent']),
+                    '%s:%s:%s' % (rhx_gis, csrf_token, self.get_param('http_headers')['User-Agent']),
                ]

            # try all of the ways to generate a GIS query, and not only use the
--- a/yt_dlp/extractor/iqiyi.py
+++ b/yt_dlp/extractor/iqiyi.py
@@ -621,7 +621,7 @@ class IqIE(InfoExtractor):
        preview_time = traverse_obj(
            initial_format_data, ('boss_ts', (None, 'data'), ('previewTime', 'rtime')), expected_type=float_or_none, get_all=False)
        if traverse_obj(initial_format_data, ('boss_ts', 'data', 'prv'), expected_type=int_or_none):
-            self.report_warning('This preview video is limited%s' % format_field(preview_time, template='to %s seconds'))
+            self.report_warning('This preview video is limited%s' % format_field(preview_time, template=' to %s seconds'))

        # TODO: Extract audio-only formats
        for bid in set(traverse_obj(initial_format_data, ('program', 'video', ..., 'bid'), expected_type=str_or_none, default=[])):
--- a/yt_dlp/extractor/ir90tv.py
+++ b/yt_dlp/extractor/ir90tv.py
@@ -1,42 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-from .common import InfoExtractor
-from ..utils import remove_start
-
-
-class Ir90TvIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?90tv\.ir/video/(?P<id>[0-9]+)/.*'
-    _TESTS = [{
-        'url': 'http://90tv.ir/video/95719/%D8%B4%D8%A7%DB%8C%D8%B9%D8%A7%D8%AA-%D9%86%D9%82%D9%84-%D9%88-%D8%A7%D9%86%D8%AA%D9%82%D8%A7%D9%84%D8%A7%D8%AA-%D9%85%D9%87%D9%85-%D9%81%D9%88%D8%AA%D8%A8%D8%A7%D9%84-%D8%A7%D8%B1%D9%88%D9%BE%D8%A7-940218',
-        'md5': '411dbd94891381960cb9e13daa47a869',
-        'info_dict': {
-            'id': '95719',
-            'ext': 'mp4',
-            'title': 'شایعات نقل و انتقالات مهم فوتبال اروپا 94/02/18',
-            'thumbnail': r're:^https?://.*\.jpg$',
-        }
-    }, {
-        'url': 'http://www.90tv.ir/video/95719/%D8%B4%D8%A7%DB%8C%D8%B9%D8%A7%D8%AA-%D9%86%D9%82%D9%84-%D9%88-%D8%A7%D9%86%D8%AA%D9%82%D8%A7%D9%84%D8%A7%D8%AA-%D9%85%D9%87%D9%85-%D9%81%D9%88%D8%AA%D8%A8%D8%A7%D9%84-%D8%A7%D8%B1%D9%88%D9%BE%D8%A7-940218',
-        'only_matching': True,
-    }]
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-        webpage = self._download_webpage(url, video_id)
-
-        title = remove_start(self._html_search_regex(
-            r'<title>([^<]+)</title>', webpage, 'title'), '90tv.ir :: ')
-
-        video_url = self._search_regex(
-            r'<source[^>]+src="([^"]+)"', webpage, 'video url')
-
-        thumbnail = self._search_regex(r'poster="([^"]+)"', webpage, 'thumbnail url', fatal=False)
-
-        return {
-            'url': video_url,
-            'id': video_id,
-            'title': title,
-            'video_url': video_url,
-            'thumbnail': thumbnail,
-        }
--- a/yt_dlp/extractor/kaltura.py
+++ b/yt_dlp/extractor/kaltura.py
@@ -301,6 +301,7 @@ class KalturaIE(InfoExtractor):
            data_url = re.sub(r'/flvclipper/.*', '/serveFlavor', data_url)

        formats = []
+        subtitles = {}
        for f in flavor_assets:
            # Continue if asset is not ready
            if f.get('status') != 2:
@@ -344,13 +345,14 @@ class KalturaIE(InfoExtractor):
        if '/playManifest/' in data_url:
            m3u8_url = sign_url(data_url.replace(
                'format/url', 'format/applehttp'))
-            formats.extend(self._extract_m3u8_formats(
+            fmts, subs = self._extract_m3u8_formats_and_subtitles(
                m3u8_url, entry_id, 'mp4', 'm3u8_native',
-                m3u8_id='hls', fatal=False))
+                m3u8_id='hls', fatal=False)
+            formats.extend(fmts)
+            self._merge_subtitles(subs, target=subtitles)

        self._sort_formats(formats)

-        subtitles = {}
        if captions:
            for caption in captions.get('objects', []):
                # Continue if caption is not ready
--- a/yt_dlp/extractor/kankan.py
+++ b/yt_dlp/extractor/kankan.py
@@ -1,48 +0,0 @@
-from __future__ import unicode_literals
-
-import re
-import hashlib
-
-from .common import InfoExtractor
-
-_md5 = lambda s: hashlib.md5(s.encode('utf-8')).hexdigest()
-
-
-class KankanIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:.*?\.)?kankan\.com/.+?/(?P<id>\d+)\.shtml'
-
-    _TEST = {
-        'url': 'http://yinyue.kankan.com/vod/48/48863.shtml',
-        'md5': '29aca1e47ae68fc28804aca89f29507e',
-        'info_dict': {
-            'id': '48863',
-            'ext': 'flv',
-            'title': 'Ready To Go',
-        },
-        'skip': 'Only available from China',
-    }
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-        webpage = self._download_webpage(url, video_id)
-
-        title = self._search_regex(r'(?:G_TITLE=|G_MOVIE_TITLE = )[\'"](.+?)[\'"]', webpage, 'video title')
-        surls = re.search(r'surls:\[\'.+?\'\]|lurl:\'.+?\.flv\'', webpage).group(0)
-        gcids = re.findall(r'http://.+?/.+?/(.+?)/', surls)
-        gcid = gcids[-1]
-
-        info_url = 'http://p2s.cl.kankan.com/getCdnresource_flv?gcid=%s' % gcid
-        video_info_page = self._download_webpage(
-            info_url, video_id, 'Downloading video url info')
-        ip = self._search_regex(r'ip:"(.+?)"', video_info_page, 'video url ip')
-        path = self._search_regex(r'path:"(.+?)"', video_info_page, 'video url path')
-        param1 = self._search_regex(r'param1:(\d+)', video_info_page, 'param1')
-        param2 = self._search_regex(r'param2:(\d+)', video_info_page, 'param2')
-        key = _md5('xl_mp43651' + param1 + param2)
-        video_url = 'http://%s%s?key=%s&key1=%s' % (ip, path, key, param2)
-
-        return {
-            'id': video_id,
-            'title': title,
-            'url': video_url,
-        }
--- a/yt_dlp/extractor/lbry.py
+++ b/yt_dlp/extractor/lbry.py
@@ -17,6 +17,7 @@ from ..utils import (
    parse_qs,
    OnDemandPagedList,
    try_get,
+    UnsupportedError,
    urljoin,
 )

@@ -196,11 +197,11 @@ class LBRYIE(LBRYBaseIE):
            live_data = self._download_json(
                f'https://api.live.odysee.com/v1/odysee/live/{claim_id}', claim_id,
                note='Downloading livestream JSON metadata')['data']
-            if not live_data['live']:
-                raise ExtractorError('This stream is not live', expected=True)
-            streaming_url = final_url = live_data['url']
+            streaming_url = final_url = live_data.get('url')
+            if not final_url and not live_data.get('live'):
+                self.raise_no_formats('This stream is not live', True, claim_id)
        else:
-            raise ExtractorError('Unsupported URL', expected=True)
+            raise UnsupportedError(url)

        info = self._parse_stream(result, url)
        if determine_ext(final_url) == 'm3u8':
--- a/yt_dlp/extractor/manyvids.py
+++ b/yt_dlp/extractor/manyvids.py
@@ -89,4 +89,5 @@ class ManyVidsIE(InfoExtractor):
            'view_count': view_count,
            'like_count': like_count,
            'formats': formats,
+            'uploader': self._html_search_regex(r'<meta[^>]+name="author"[^>]*>([^<]+)', webpage, 'uploader'),
        }
--- a/yt_dlp/extractor/mediaset.py
+++ b/yt_dlp/extractor/mediaset.py
@@ -7,6 +7,7 @@ import re
 from .theplatform import ThePlatformBaseIE
 from ..utils import (
    ExtractorError,
+    GeoRestrictedError,
    int_or_none,
    OnDemandPagedList,
    parse_qs,
@@ -37,7 +38,7 @@ class MediasetIE(ThePlatformBaseIE):
            'id': 'F310575103000102',
            'ext': 'mp4',
            'title': 'Episodio 1',
-            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'description': 'md5:e8017b7d7194e9bfb75299c2b8d81e02',
            'thumbnail': r're:^https?://.*\.jpg$',
            'duration': 2682.0,
            'upload_date': '20210530',
@@ -45,6 +46,11 @@ class MediasetIE(ThePlatformBaseIE):
            'timestamp': 1622413946,
            'uploader': 'Canale 5',
            'uploader_id': 'C5',
+            'season': 'Season 1',
+            'episode': 'Episode 1',
+            'season_number': 1,
+            'episode_number': 1,
+            'chapters': [{'start_time': 0.0, 'end_time': 439.88}, {'start_time': 439.88, 'end_time': 1685.84}, {'start_time': 1685.84, 'end_time': 2682.0}],
        },
    }, {
        'url': 'https://www.mediasetplay.mediaset.it/video/matrix/puntata-del-25-maggio_F309013801000501',
@@ -53,7 +59,7 @@ class MediasetIE(ThePlatformBaseIE):
            'id': 'F309013801000501',
            'ext': 'mp4',
            'title': 'Puntata del 25 maggio',
-            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'description': 'md5:ee2e456e3eb1dba5e814596655bb5296',
            'thumbnail': r're:^https?://.*\.jpg$',
            'duration': 6565.008,
            'upload_date': '20200903',
@@ -61,6 +67,11 @@ class MediasetIE(ThePlatformBaseIE):
            'timestamp': 1599172492,
            'uploader': 'Canale 5',
            'uploader_id': 'C5',
+            'season': 'Season 5',
+            'episode': 'Episode 5',
+            'season_number': 5,
+            'episode_number': 5,
+            'chapters': [{'start_time': 0.0, 'end_time': 3409.08}, {'start_time': 3409.08, 'end_time': 6565.008}],
        },
    }, {
        'url': 'https://www.mediasetplay.mediaset.it/video/cameracafe5/episodio-69-pezzo-di-luna_F303843101017801',
@@ -69,7 +80,7 @@ class MediasetIE(ThePlatformBaseIE):
            'id': 'F303843101017801',
            'ext': 'mp4',
            'title': 'Episodio 69 - Pezzo di luna',
-            'description': '',
+            'description': 'md5:7c32c8ec4118b72588b9412f11353f73',
            'thumbnail': r're:^https?://.*\.jpg$',
            'duration': 263.008,
            'upload_date': '20200902',
@@ -77,6 +88,11 @@ class MediasetIE(ThePlatformBaseIE):
            'timestamp': 1599064700,
            'uploader': 'Italia 1',
            'uploader_id': 'I1',
+            'season': 'Season 5',
+            'episode': 'Episode 178',
+            'season_number': 5,
+            'episode_number': 178,
+            'chapters': [{'start_time': 0.0, 'end_time': 261.88}, {'start_time': 261.88, 'end_time': 263.008}],
        },
    }, {
        'url': 'https://www.mediasetplay.mediaset.it/video/cameracafe5/episodio-51-tu-chi-sei_F303843107000601',
@@ -85,7 +101,7 @@ class MediasetIE(ThePlatformBaseIE):
            'id': 'F303843107000601',
            'ext': 'mp4',
            'title': 'Episodio 51 - Tu chi sei?',
-            'description': '',
+            'description': 'md5:42ef006e56824cc31787a547590923f4',
            'thumbnail': r're:^https?://.*\.jpg$',
            'duration': 367.021,
            'upload_date': '20200902',
@@ -93,6 +109,28 @@ class MediasetIE(ThePlatformBaseIE):
            'timestamp': 1599069817,
            'uploader': 'Italia 1',
            'uploader_id': 'I1',
+            'season': 'Season 5',
+            'episode': 'Episode 6',
+            'season_number': 5,
+            'episode_number': 6,
+            'chapters': [{'start_time': 0.0, 'end_time': 358.68}, {'start_time': 358.68, 'end_time': 367.021}],
+        },
+    }, {
+        # movie
+        'url': 'https://www.mediasetplay.mediaset.it/movie/selvaggi/selvaggi_F006474501000101',
+        'md5': '720440187a2ae26af8148eb9e6b901ed',
+        'info_dict': {
+            'id': 'F006474501000101',
+            'ext': 'mp4',
+            'title': 'Selvaggi',
+            'description': 'md5:cfdedbbfdd12d4d0e5dcf1fa1b75284f',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            'duration': 5233.01,
+            'upload_date': '20210729',
+            'timestamp': 1627594716,
+            'uploader': 'Cine34',
+            'uploader_id': 'B6',
+            'chapters': [{'start_time': 0.0, 'end_time': 1938.56}, {'start_time': 1938.56, 'end_time': 5233.01}],
        },
    }, {
        # clip
@@ -160,6 +198,22 @@ class MediasetIE(ThePlatformBaseIE):
            video.attrib['src'] = re.sub(r'(https?://vod05)t(-mediaset-it\.akamaized\.net/.+?.mpd)\?.+', r'\1\2', video.attrib['src'])
        return super(MediasetIE, self)._parse_smil_formats(smil, smil_url, video_id, namespace, f4m_params, transform_rtmp_url)

+    def _check_drm_formats(self, tp_formats, video_id):
+        has_nondrm, drm_manifest = False, ''
+        for f in tp_formats:
+            if '_sampleaes/' in (f.get('manifest_url') or ''):
+                drm_manifest = drm_manifest or f['manifest_url']
+                f['has_drm'] = True
+            if not f.get('has_drm') and f.get('manifest_url'):
+                has_nondrm = True
+
+        nodrm_manifest = re.sub(r'_sampleaes/(\w+)_fp_', r'/\1_no_', drm_manifest)
+        if has_nondrm or nodrm_manifest == drm_manifest:
+            return
+
+        tp_formats.extend(self._extract_m3u8_formats(
+            nodrm_manifest, video_id, m3u8_id='hls', fatal=False) or [])
+
    def _real_extract(self, url):
        guid = self._match_id(url)
        tp_path = 'PR1GhC/media/guid/2702976343/' + guid
@@ -167,10 +221,10 @@ class MediasetIE(ThePlatformBaseIE):

        formats = []
        subtitles = {}
-        first_e = None
+        first_e = geo_e = None
        asset_type = 'geoNo:HD,browser,geoIT|geoNo:HD,geoIT|geoNo:SD,browser,geoIT|geoNo:SD,geoIT|geoNo|HD|SD'
        # TODO: fixup ISM+none manifest URLs
-        for f in ('MPEG4', 'MPEG-DASH+none', 'M3U+none'):
+        for f in ('MPEG4', 'M3U'):
            try:
                tp_formats, tp_subtitles = self._extract_theplatform_smil(
                    update_url_query('http://link.theplatform.%s/s/%s' % (self._TP_TLD, tp_path), {
@@ -179,13 +233,19 @@ class MediasetIE(ThePlatformBaseIE):
                        'assetTypes': asset_type,
                    }), guid, 'Downloading %s SMIL data' % (f.split('+')[0]))
            except ExtractorError as e:
+                if not geo_e and isinstance(e, GeoRestrictedError):
+                    geo_e = e
                if not first_e:
                    first_e = e
-                break
+                continue
+            self._check_drm_formats(tp_formats, guid)
            formats.extend(tp_formats)
            subtitles = self._merge_subtitles(subtitles, tp_subtitles)
-        if first_e and not formats:
-            raise first_e
+
+        # check for errors and report them
+        if (first_e or geo_e) and not formats:
+            raise geo_e or first_e
+
        self._sort_formats(formats)

        feed_data = self._download_json(
@@ -201,15 +261,22 @@ class MediasetIE(ThePlatformBaseIE):
                    break

            info.update({
-                'episode_number': int_or_none(feed_data.get('tvSeasonEpisodeNumber')),
-                'season_number': int_or_none(feed_data.get('tvSeasonNumber')),
-                'series': feed_data.get('mediasetprogram$brandTitle'),
+                'description': info.get('description') or feed_data.get('description') or feed_data.get('longDescription'),
                'uploader': publish_info.get('description'),
                'uploader_id': publish_info.get('channel'),
                'view_count': int_or_none(feed_data.get('mediasetprogram$numberOfViews')),
                'thumbnail': thumbnail,
            })

+            if feed_data.get('programType') == 'episode':
+                info.update({
+                    'episode_number': int_or_none(
+                        feed_data.get('tvSeasonEpisodeNumber')),
+                    'season_number': int_or_none(
+                        feed_data.get('tvSeasonNumber')),
+                    'series': feed_data.get('mediasetprogram$brandTitle'),
+                })
+
        info.update({
            'id': guid,
            'formats': formats,
@@ -224,37 +291,29 @@ class MediasetShowIE(MediasetIE):
                        https?://
                            (?:(?:www|static3)\.)?mediasetplay\.mediaset\.it/
                            (?:
-                                (?:fiction|programmi-tv|serie-tv)/(?:.+?/)?
-                                    (?:[a-z]+)_SE(?P<id>\d{12})
+                                (?:fiction|programmi-tv|serie-tv|kids)/(?:.+?/)?
+                                    (?:[a-z-]+)_SE(?P<id>\d{12})
                                    (?:,ST(?P<st>\d{12}))?
                                    (?:,sb(?P<sb>\d{9}))?$
                            )
                    )
                    '''
    _TESTS = [{
-        # TV Show webpage (with a single playlist)
-        'url': 'https://www.mediasetplay.mediaset.it/serie-tv/fireforce/episodi_SE000000001556',
+        # TV Show webpage (general webpage)
+        'url': 'https://www.mediasetplay.mediaset.it/programmi-tv/leiene/leiene_SE000000000061',
        'info_dict': {
-            'id': '000000001556',
-            'title': 'Fire Force',
+            'id': '000000000061',
+            'title': 'Le Iene',
        },
-        'playlist_count': 1,
+        'playlist_mincount': 7,
    }, {
-        # TV Show webpage (with multiple playlists)
+        # TV Show webpage (specific season)
        'url': 'https://www.mediasetplay.mediaset.it/programmi-tv/leiene/leiene_SE000000000061,ST000000002763',
        'info_dict': {
            'id': '000000002763',
            'title': 'Le Iene',
        },
-        'playlist_count': 7,
-    }, {
-        # TV Show specific playlist (single page)
-        'url': 'https://www.mediasetplay.mediaset.it/serie-tv/fireforce/episodi_SE000000001556,ST000000002738,sb100013107',
-        'info_dict': {
-            'id': '100013107',
-            'title': 'Episodi',
-        },
-        'playlist_count': 4,
+        'playlist_mincount': 7,
    }, {
        # TV Show specific playlist (with multiple pages)
        'url': 'https://www.mediasetplay.mediaset.it/programmi-tv/leiene/iservizi_SE000000000061,ST000000002763,sb100013375',
@@ -262,7 +321,7 @@ class MediasetShowIE(MediasetIE):
            'id': '100013375',
            'title': 'I servizi',
        },
-        'playlist_count': 53,
+        'playlist_mincount': 50,
    }]

    _BY_SUBBRAND = 'https://feed.entertainment.tv.theplatform.eu/f/PR1GhC/mediaset-prod-all-programs-v2?byCustomValue={subBrandId}{%s}&sort=:publishInfo_lastPublished|desc,tvSeasonEpisodeNumber|desc&range=%d-%d'
@@ -281,7 +340,7 @@ class MediasetShowIE(MediasetIE):
    def _real_extract(self, url):
        playlist_id, st, sb = self._match_valid_url(url).group('id', 'st', 'sb')
        if not sb:
-            page = self._download_webpage(url, playlist_id)
+            page = self._download_webpage(url, st or playlist_id)
            entries = [self.url_result(urljoin('https://www.mediasetplay.mediaset.it', url))
                       for url in re.findall(r'href="([^<>=]+SE\d{12},ST\d{12},sb\d{9})">[^<]+<', page)]
            title = (self._html_search_regex(r'(?s)<h1[^>]*>(.+?)</h1>', page, 'title', default=None)
--- a/yt_dlp/extractor/mildom.py
+++ b/yt_dlp/extractor/mildom.py
@@ -1,103 +1,42 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import base64
-from datetime import datetime
-import itertools
+import functools
 import json

 from .common import InfoExtractor
 from ..utils import (
-    std_headers,
-    update_url_query,
-    random_uuidv4,
-    try_get,
+    determine_ext,
+    dict_get,
+    ExtractorError,
    float_or_none,
-    dict_get
-)
-from ..compat import (
-    compat_str,
+    OnDemandPagedList,
+    random_uuidv4,
+    traverse_obj,
 )


 class MildomBaseIE(InfoExtractor):
    _GUEST_ID = None
-    _DISPATCHER_CONFIG = None

-    def _call_api(self, url, video_id, query=None, note='Downloading JSON metadata', init=False):
-        query = query or {}
-        if query:
-            query['__platform'] = 'web'
-        url = update_url_query(url, self._common_queries(query, init=init))
-        content = self._download_json(url, video_id, note=note)
-        if content['code'] == 0:
-            return content['body']
-        else:
-            self.raise_no_formats(
-                f'Video not found or premium content. {content["code"]} - {content["message"]}',
+    def _call_api(self, url, video_id, query=None, note='Downloading JSON metadata', body=None):
+        if not self._GUEST_ID:
+            self._GUEST_ID = f'pc-gp-{random_uuidv4()}'
+
+        content = self._download_json(
+            url, video_id, note=note, data=json.dumps(body).encode() if body else None,
+            headers={'Content-Type': 'application/json'} if body else {},
+            query={
+                '__guest_id': self._GUEST_ID,
+                '__platform': 'web',
+                **(query or {}),
+            })
+
+        if content['code'] != 0:
+            raise ExtractorError(
+                f'Mildom says: {content["message"]} (code {content["code"]})',
                expected=True)
-
-    def _common_queries(self, query={}, init=False):
-        dc = self._fetch_dispatcher_config()
-        r = {
-            'timestamp': self.iso_timestamp(),
-            '__guest_id': '' if init else self.guest_id(),
-            '__location': dc['location'],
-            '__country': dc['country'],
-            '__cluster': dc['cluster'],
-            '__platform': 'web',
-            '__la': self.lang_code(),
-            '__pcv': 'v2.9.44',
-            'sfr': 'pc',
-            'accessToken': '',
-        }
-        r.update(query)
-        return r
-
-    def _fetch_dispatcher_config(self):
-        if not self._DISPATCHER_CONFIG:
-            tmp = self._download_json(
-                'https://disp.mildom.com/serverListV2', 'initialization',
-                note='Downloading dispatcher_config', data=json.dumps({
-                    'protover': 0,
-                    'data': base64.b64encode(json.dumps({
-                        'fr': 'web',
-                        'sfr': 'pc',
-                        'devi': 'Windows',
-                        'la': 'ja',
-                        'gid': None,
-                        'loc': '',
-                        'clu': '',
-                        'wh': '1919*810',
-                        'rtm': self.iso_timestamp(),
-                        'ua': std_headers['User-Agent'],
-                    }).encode('utf8')).decode('utf8').replace('\n', ''),
-                }).encode('utf8'))
-            self._DISPATCHER_CONFIG = self._parse_json(base64.b64decode(tmp['data']), 'initialization')
-        return self._DISPATCHER_CONFIG
-
-    @staticmethod
-    def iso_timestamp():
-        'new Date().toISOString()'
-        return datetime.utcnow().isoformat()[0:-3] + 'Z'
-
-    def guest_id(self):
-        'getGuestId'
-        if self._GUEST_ID:
-            return self._GUEST_ID
-        self._GUEST_ID = try_get(
-            self, (
-                lambda x: x._call_api(
-                    'https://cloudac.mildom.com/nonolive/gappserv/guest/h5init', 'initialization',
-                    note='Downloading guest token', init=True)['guest_id'] or None,
-                lambda x: x._get_cookies('https://www.mildom.com').get('gid').value,
-                lambda x: x._get_cookies('https://m.mildom.com').get('gid').value,
-            ), compat_str) or ''
-        return self._GUEST_ID
-
-    def lang_code(self):
-        'getCurrentLangCode'
-        return 'ja'
+        return content['body']


 class MildomIE(MildomBaseIE):
@@ -107,31 +46,13 @@ class MildomIE(MildomBaseIE):

    def _real_extract(self, url):
        video_id = self._match_id(url)
-        url = 'https://www.mildom.com/%s' % video_id
-
-        webpage = self._download_webpage(url, video_id)
+        webpage = self._download_webpage(f'https://www.mildom.com/{video_id}', video_id)

        enterstudio = self._call_api(
            'https://cloudac.mildom.com/nonolive/gappserv/live/enterstudio', video_id,
            note='Downloading live metadata', query={'user_id': video_id})
        result_video_id = enterstudio.get('log_id', video_id)

-        title = try_get(
-            enterstudio, (
-                lambda x: self._html_search_meta('twitter:description', webpage),
-                lambda x: x['anchor_intro'],
-            ), compat_str)
-        description = try_get(
-            enterstudio, (
-                lambda x: x['intro'],
-                lambda x: x['live_intro'],
-            ), compat_str)
-        uploader = try_get(
-            enterstudio, (
-                lambda x: self._html_search_meta('twitter:title', webpage),
-                lambda x: x['loginname'],
-            ), compat_str)
-
        servers = self._call_api(
            'https://cloudac.mildom.com/nonolive/gappserv/live/liveserver', result_video_id,
            note='Downloading live server list', query={
@@ -139,17 +60,20 @@ class MildomIE(MildomBaseIE):
                'live_server_type': 'hls',
            })

-        stream_query = self._common_queries({
-            'streamReqId': random_uuidv4(),
-            'is_lhls': '0',
-        })
-        m3u8_url = update_url_query(servers['stream_server'] + '/%s_master.m3u8' % video_id, stream_query)
-        formats = self._extract_m3u8_formats(m3u8_url, result_video_id, 'mp4', headers={
-            'Referer': 'https://www.mildom.com/',
-            'Origin': 'https://www.mildom.com',
-        }, note='Downloading m3u8 information')
+        playback_token = self._call_api(
+            'https://cloudac.mildom.com/nonolive/gappserv/live/token', result_video_id,
+            note='Obtaining live playback token', body={'host_id': video_id, 'type': 'hls'})
+        playback_token = traverse_obj(playback_token, ('data', ..., 'token'), get_all=False)
+        if not playback_token:
+            raise ExtractorError('Failed to obtain live playback token')
+
+        formats = self._extract_m3u8_formats(
+            f'{servers["stream_server"]}/{video_id}_master.m3u8?{playback_token}',
+            result_video_id, 'mp4', headers={
+                'Referer': 'https://www.mildom.com/',
+                'Origin': 'https://www.mildom.com',
+            })

-        del stream_query['streamReqId'], stream_query['timestamp']
        for fmt in formats:
            fmt.setdefault('http_headers', {})['Referer'] = 'https://www.mildom.com/'

@@ -157,10 +81,10 @@ class MildomIE(MildomBaseIE):

        return {
            'id': result_video_id,
-            'title': title,
-            'description': description,
+            'title': self._html_search_meta('twitter:description', webpage, default=None) or traverse_obj(enterstudio, 'anchor_intro'),
+            'description': traverse_obj(enterstudio, 'intro', 'live_intro', expected_type=str),
            'timestamp': float_or_none(enterstudio.get('live_start_ms'), scale=1000),
-            'uploader': uploader,
+            'uploader': self._html_search_meta('twitter:title', webpage, default=None) or traverse_obj(enterstudio, 'loginname'),
            'uploader_id': video_id,
            'formats': formats,
            'is_live': True,
@@ -169,7 +93,7 @@ class MildomIE(MildomBaseIE):

 class MildomVodIE(MildomBaseIE):
    IE_NAME = 'mildom:vod'
-    IE_DESC = 'Download a VOD in Mildom'
+    IE_DESC = 'VOD in Mildom'
    _VALID_URL = r'https?://(?:(?:www|m)\.)mildom\.com/playback/(?P<user_id>\d+)/(?P<id>(?P=user_id)-[a-zA-Z0-9]+-?[0-9]*)'
    _TESTS = [{
        'url': 'https://www.mildom.com/playback/10882672/10882672-1597662269',
@@ -216,11 +140,8 @@ class MildomVodIE(MildomBaseIE):
    }]

    def _real_extract(self, url):
-        m = self._match_valid_url(url)
-        user_id, video_id = m.group('user_id'), m.group('id')
-        url = 'https://www.mildom.com/playback/%s/%s' % (user_id, video_id)
-
-        webpage = self._download_webpage(url, video_id)
+        user_id, video_id = self._match_valid_url(url).group('user_id', 'id')
+        webpage = self._download_webpage(f'https://www.mildom.com/playback/{user_id}/{video_id}', video_id)

        autoplay = self._call_api(
            'https://cloudac.mildom.com/nonolive/videocontent/playback/getPlaybackDetail', video_id,
@@ -228,20 +149,6 @@ class MildomVodIE(MildomBaseIE):
                'v_id': video_id,
            })['playback']

-        title = try_get(
-            autoplay, (
-                lambda x: self._html_search_meta('og:description', webpage),
-                lambda x: x['title'],
-            ), compat_str)
-        description = try_get(
-            autoplay, (
-                lambda x: x['video_intro'],
-            ), compat_str)
-        uploader = try_get(
-            autoplay, (
-                lambda x: x['author_info']['login_name'],
-            ), compat_str)
-
        formats = [{
            'url': autoplay['audio_url'],
            'format_id': 'audio',
@@ -266,17 +173,81 @@ class MildomVodIE(MildomBaseIE):

        return {
            'id': video_id,
-            'title': title,
-            'description': description,
-            'timestamp': float_or_none(autoplay['publish_time'], scale=1000),
-            'duration': float_or_none(autoplay['video_length'], scale=1000),
+            'title': self._html_search_meta(('og:description', 'description'), webpage, default=None) or autoplay.get('title'),
+            'description': traverse_obj(autoplay, 'video_intro'),
+            'timestamp': float_or_none(autoplay.get('publish_time'), scale=1000),
+            'duration': float_or_none(autoplay.get('video_length'), scale=1000),
            'thumbnail': dict_get(autoplay, ('upload_pic', 'video_pic')),
-            'uploader': uploader,
+            'uploader': traverse_obj(autoplay, ('author_info', 'login_name')),
            'uploader_id': user_id,
            'formats': formats,
        }


+class MildomClipIE(MildomBaseIE):
+    IE_NAME = 'mildom:clip'
+    IE_DESC = 'Clip in Mildom'
+    _VALID_URL = r'https?://(?:(?:www|m)\.)mildom\.com/clip/(?P<id>(?P<user_id>\d+)-[a-zA-Z0-9]+)'
+    _TESTS = [{
+        'url': 'https://www.mildom.com/clip/10042245-63921673e7b147ebb0806d42b5ba5ce9',
+        'info_dict': {
+            'id': '10042245-63921673e7b147ebb0806d42b5ba5ce9',
+            'title': '全然違ったよ',
+            'timestamp': 1619181890,
+            'duration': 59,
+            'thumbnail': r're:https?://.+',
+            'uploader': 'ざきんぽ',
+            'uploader_id': '10042245',
+        },
+    }, {
+        'url': 'https://www.mildom.com/clip/10111524-ebf4036e5aa8411c99fb3a1ae0902864',
+        'info_dict': {
+            'id': '10111524-ebf4036e5aa8411c99fb3a1ae0902864',
+            'title': 'かっこいい',
+            'timestamp': 1621094003,
+            'duration': 59,
+            'thumbnail': r're:https?://.+',
+            'uploader': '(ルーキー',
+            'uploader_id': '10111524',
+        },
+    }, {
+        'url': 'https://www.mildom.com/clip/10660174-2c539e6e277c4aaeb4b1fbe8d22cb902',
+        'info_dict': {
+            'id': '10660174-2c539e6e277c4aaeb4b1fbe8d22cb902',
+            'title': 'あ',
+            'timestamp': 1614769431,
+            'duration': 31,
+            'thumbnail': r're:https?://.+',
+            'uploader': 'ドルゴルスレンギーン＝ダグワドルジ',
+            'uploader_id': '10660174',
+        },
+    }]
+
+    def _real_extract(self, url):
+        user_id, video_id = self._match_valid_url(url).group('user_id', 'id')
+        webpage = self._download_webpage(f'https://www.mildom.com/clip/{video_id}', video_id)
+
+        clip_detail = self._call_api(
+            'https://cloudac-cf-jp.mildom.com/nonolive/videocontent/clip/detail', video_id,
+            note='Downloading playback metadata', query={
+                'clip_id': video_id,
+            })
+
+        return {
+            'id': video_id,
+            'title': self._html_search_meta(
+                ('og:description', 'description'), webpage, default=None) or clip_detail.get('title'),
+            'timestamp': float_or_none(clip_detail.get('create_time')),
+            'duration': float_or_none(clip_detail.get('length')),
+            'thumbnail': clip_detail.get('cover'),
+            'uploader': traverse_obj(clip_detail, ('user_info', 'loginname')),
+            'uploader_id': user_id,
+
+            'url': clip_detail['url'],
+            'ext': determine_ext(clip_detail.get('url'), 'mp4'),
+        }
+
+
 class MildomUserVodIE(MildomBaseIE):
    IE_NAME = 'mildom:user:vod'
    IE_DESC = 'Download all VODs from specific user in Mildom'
@@ -287,29 +258,32 @@ class MildomUserVodIE(MildomBaseIE):
            'id': '10093333',
            'title': 'Uploads from ねこばたけ',
        },
-        'playlist_mincount': 351,
+        'playlist_mincount': 732,
    }, {
        'url': 'https://www.mildom.com/profile/10882672',
        'info_dict': {
            'id': '10882672',
            'title': 'Uploads from kson組長(けいそん)',
        },
-        'playlist_mincount': 191,
+        'playlist_mincount': 201,
    }]

-    def _entries(self, user_id):
-        for page in itertools.count(1):
-            reply = self._call_api(
-                'https://cloudac.mildom.com/nonolive/videocontent/profile/playbackList',
-                user_id, note='Downloading page %d' % page, query={
-                    'user_id': user_id,
-                    'page': page,
-                    'limit': '30',
-                })
-            if not reply:
-                break
-            for x in reply:
-                yield self.url_result('https://www.mildom.com/playback/%s/%s' % (user_id, x['v_id']))
+    def _fetch_page(self, user_id, page):
+        page += 1
+        reply = self._call_api(
+            'https://cloudac.mildom.com/nonolive/videocontent/profile/playbackList',
+            user_id, note=f'Downloading page {page}', query={
+                'user_id': user_id,
+                'page': page,
+                'limit': '30',
+            })
+        if not reply:
+            return
+        for x in reply:
+            v_id = x.get('v_id')
+            if not v_id:
+                continue
+            yield self.url_result(f'https://www.mildom.com/playback/{user_id}/{v_id}')

    def _real_extract(self, url):
        user_id = self._match_id(url)
@@ -320,4 +294,5 @@ class MildomUserVodIE(MildomBaseIE):
            query={'user_id': user_id}, note='Downloading user profile')['user_info']

        return self.playlist_result(
-            self._entries(user_id), user_id, 'Uploads from %s' % profile['loginname'])
+            OnDemandPagedList(functools.partial(self._fetch_page, user_id), 30),
+            user_id, f'Uploads from {profile["loginname"]}')
--- a/yt_dlp/extractor/mirrativ.py
+++ b/yt_dlp/extractor/mirrativ.py
@@ -19,9 +19,25 @@ class MirrativBaseIE(InfoExtractor):
 class MirrativIE(MirrativBaseIE):
    IE_NAME = 'mirrativ'
    _VALID_URL = r'https?://(?:www\.)?mirrativ\.com/live/(?P<id>[^/?#&]+)'
-    LIVE_API_URL = 'https://www.mirrativ.com/api/live/live?live_id=%s'

    TESTS = [{
+        'url': 'https://mirrativ.com/live/UQomuS7EMgHoxRHjEhNiHw',
+        'info_dict': {
+            'id': 'UQomuS7EMgHoxRHjEhNiHw',
+            'title': 'ねむいぃ、。『参加型』🔰jcが初めてやるCOD✨初見さん大歓迎💗',
+            'is_live': True,
+            'description': 'md5:bfcd8f77f2fab24c3c672e5620f3f16e',
+            'thumbnail': r're:https?://.+',
+            'uploader': '# あ ち ゅ 。💡',
+            'uploader_id': '118572165',
+            'duration': None,
+            'view_count': 1241,
+            'release_timestamp': 1646229192,
+            'timestamp': 1646229167,
+            'was_live': False,
+        },
+        'skip': 'livestream',
+    }, {
        'url': 'https://mirrativ.com/live/POxyuG1KmW2982lqlDTuPw',
        'only_matching': True,
    }]
@@ -29,12 +45,11 @@ class MirrativIE(MirrativBaseIE):
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage('https://www.mirrativ.com/live/%s' % video_id, video_id)
-        live_response = self._download_json(self.LIVE_API_URL % video_id, video_id)
+        live_response = self._download_json(f'https://www.mirrativ.com/api/live/live?live_id={video_id}', video_id)
        self.assert_error(live_response)

        hls_url = dict_get(live_response, ('archive_url_hls', 'streaming_url_hls'))
        is_live = bool(live_response.get('is_live'))
-        was_live = bool(live_response.get('is_archive'))
        if not hls_url:
            raise ExtractorError('Neither archive nor live is available.', expected=True)

@@ -42,55 +57,29 @@ class MirrativIE(MirrativBaseIE):
            hls_url, video_id,
            ext='mp4', entry_protocol='m3u8_native',
            m3u8_id='hls', live=is_live)
-        rtmp_url = live_response.get('streaming_url_edge')
-        if rtmp_url:
-            keys_to_copy = ('width', 'height', 'vcodec', 'acodec', 'tbr')
-            fmt = {
-                'format_id': 'rtmp',
-                'url': rtmp_url,
-                'protocol': 'rtmp',
-                'ext': 'mp4',
-            }
-            fmt.update({k: traverse_obj(formats, (0, k)) for k in keys_to_copy})
-            formats.append(fmt)
        self._sort_formats(formats)

-        title = self._og_search_title(webpage, default=None) or self._search_regex(
-            r'<title>\s*(.+?) - Mirrativ\s*</title>', webpage) or live_response.get('title')
-        description = live_response.get('description')
-        thumbnail = live_response.get('image_url')
-
-        duration = try_get(live_response, lambda x: x['ended_at'] - x['started_at'])
-        view_count = live_response.get('total_viewer_num')
-        release_timestamp = live_response.get('started_at')
-        timestamp = live_response.get('created_at')
-
-        owner = live_response.get('owner', {})
-        uploader = owner.get('name')
-        uploader_id = owner.get('user_id')
-
        return {
            'id': video_id,
-            'title': title,
+            'title': self._og_search_title(webpage, default=None) or self._search_regex(
+                r'<title>\s*(.+?) - Mirrativ\s*</title>', webpage) or live_response.get('title'),
            'is_live': is_live,
-            'description': description,
+            'description': live_response.get('description'),
            'formats': formats,
-            'thumbnail': thumbnail,
-            'uploader': uploader,
-            'uploader_id': uploader_id,
-            'duration': duration,
-            'view_count': view_count,
-            'release_timestamp': release_timestamp,
-            'timestamp': timestamp,
-            'was_live': was_live,
+            'thumbnail': live_response.get('image_url'),
+            'uploader': traverse_obj(live_response, ('owner', 'name')),
+            'uploader_id': traverse_obj(live_response, ('owner', 'user_id')),
+            'duration': try_get(live_response, lambda x: x['ended_at'] - x['started_at']) if not is_live else None,
+            'view_count': live_response.get('total_viewer_num'),
+            'release_timestamp': live_response.get('started_at'),
+            'timestamp': live_response.get('created_at'),
+            'was_live': bool(live_response.get('is_archive')),
        }


 class MirrativUserIE(MirrativBaseIE):
    IE_NAME = 'mirrativ:user'
    _VALID_URL = r'https?://(?:www\.)?mirrativ\.com/user/(?P<id>\d+)'
-    LIVE_HISTORY_API_URL = 'https://www.mirrativ.com/api/live/live_history?user_id=%s&page=%d'
-    USER_INFO_API_URL = 'https://www.mirrativ.com/api/user/profile?user_id=%s'

    _TESTS = [{
        # Live archive is available up to 3 days
@@ -104,8 +93,8 @@ class MirrativUserIE(MirrativBaseIE):
        page = 1
        while page is not None:
            api_response = self._download_json(
-                self.LIVE_HISTORY_API_URL % (user_id, page), user_id,
-                note='Downloading page %d' % page)
+                f'https://www.mirrativ.com/api/live/live_history?user_id={user_id}&page={page}', user_id,
+                note=f'Downloading page {page}')
            self.assert_error(api_response)
            lives = api_response.get('lives')
            if not lives:
@@ -123,12 +112,10 @@ class MirrativUserIE(MirrativBaseIE):
    def _real_extract(self, url):
        user_id = self._match_id(url)
        user_info = self._download_json(
-            self.USER_INFO_API_URL % user_id, user_id,
+            f'https://www.mirrativ.com/api/user/profile?user_id={user_id}', user_id,
            note='Downloading user info', fatal=False)
        self.assert_error(user_info)

-        uploader = user_info.get('name')
-        description = user_info.get('description')
-
-        entries = self._entries(user_id)
-        return self.playlist_result(entries, user_id, uploader, description)
+        return self.playlist_result(
+            self._entries(user_id), user_id,
+            user_info.get('name'), user_info.get('description'))
--- a/yt_dlp/extractor/murrtube.py
+++ b/yt_dlp/extractor/murrtube.py
@@ -0,0 +1,165 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import functools
+import json
+
+from .common import InfoExtractor
+from ..utils import (
+    ExtractorError,
+    OnDemandPagedList,
+    determine_ext,
+    int_or_none,
+    try_get,
+)
+
+
+class MurrtubeIE(InfoExtractor):
+    _VALID_URL = r'''(?x)
+                        (?:
+                            murrtube:|
+                            https?://murrtube\.net/videos/(?P<slug>[a-z0-9\-]+)\-
+                        )
+                        (?P<id>[a-f0-9]{8}\-[a-f0-9]{4}\-[a-f0-9]{4}\-[a-f0-9]{4}\-[a-f0-9]{12})
+                    '''
+    _TEST = {
+        'url': 'https://murrtube.net/videos/inferno-x-skyler-148b6f2a-fdcc-4902-affe-9c0f41aaaca0',
+        'md5': '169f494812d9a90914b42978e73aa690',
+        'info_dict': {
+            'id': '148b6f2a-fdcc-4902-affe-9c0f41aaaca0',
+            'ext': 'mp4',
+            'title': 'Inferno X Skyler',
+            'description': 'Humping a very good slutty sheppy (roomate)',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            'duration': 284,
+            'uploader': 'Inferno Wolf',
+            'age_limit': 18,
+            'comment_count': int,
+            'view_count': int,
+            'like_count': int,
+            'tags': ['hump', 'breed', 'Fursuit', 'murrsuit', 'bareback'],
+        }
+    }
+
+    def _download_gql(self, video_id, op, note=None, fatal=True):
+        result = self._download_json(
+            'https://murrtube.net/graphql',
+            video_id, note, data=json.dumps(op).encode(), fatal=fatal,
+            headers={'Content-Type': 'application/json'})
+        return result['data']
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        data = self._download_gql(video_id, {
+            'operationName': 'Medium',
+            'variables': {
+                'id': video_id,
+            },
+            'query': '''\
+query Medium($id: ID!) {
+  medium(id: $id) {
+    title
+    description
+    key
+    duration
+    commentsCount
+    likesCount
+    viewsCount
+    thumbnailKey
+    tagList
+    user {
+      name
+      __typename
+    }
+    __typename
+  }
+}'''})
+        meta = data['medium']
+
+        storage_url = 'https://storage.murrtube.net/murrtube/'
+        format_url = storage_url + meta.get('key', '')
+        thumbnail = storage_url + meta.get('thumbnailKey', '')
+
+        if determine_ext(format_url) == 'm3u8':
+            formats = self._extract_m3u8_formats(
+                format_url, video_id, 'mp4', entry_protocol='m3u8_native', fatal=False)
+        else:
+            formats = [{'url': format_url}]
+
+        return {
+            'id': video_id,
+            'title': meta.get('title'),
+            'description': meta.get('description'),
+            'formats': formats,
+            'thumbnail': thumbnail,
+            'duration': int_or_none(meta.get('duration')),
+            'uploader': try_get(meta, lambda x: x['user']['name']),
+            'view_count': meta.get('viewsCount'),
+            'like_count': meta.get('likesCount'),
+            'comment_count': meta.get('commentsCount'),
+            'tags': meta.get('tagList'),
+            'age_limit': 18,
+        }
+
+
+class MurrtubeUserIE(MurrtubeIE):
+    IE_DESC = 'Murrtube user profile'
+    _VALID_URL = r'https?://murrtube\.net/(?P<id>[^/]+)$'
+    _TEST = {
+        'url': 'https://murrtube.net/stormy',
+        'info_dict': {
+            'id': 'stormy',
+        },
+        'playlist_mincount': 27,
+    }
+    _PAGE_SIZE = 10
+
+    def _fetch_page(self, username, user_id, page):
+        data = self._download_gql(username, {
+            'operationName': 'Media',
+            'variables': {
+                'limit': self._PAGE_SIZE,
+                'offset': page * self._PAGE_SIZE,
+                'sort': 'latest',
+                'userId': user_id,
+            },
+            'query': '''\
+query Media($q: String, $sort: String, $userId: ID, $offset: Int!, $limit: Int!) {
+  media(q: $q, sort: $sort, userId: $userId, offset: $offset, limit: $limit) {
+    id
+    __typename
+  }
+}'''},
+            'Downloading page {0}'.format(page + 1))
+        if data is None:
+            raise ExtractorError(f'Failed to retrieve video list for page {page + 1}')
+
+        media = data['media']
+
+        for entry in media:
+            yield self.url_result('murrtube:{0}'.format(entry['id']), MurrtubeIE.ie_key())
+
+    def _real_extract(self, url):
+        username = self._match_id(url)
+        data = self._download_gql(username, {
+            'operationName': 'User',
+            'variables': {
+                'id': username,
+            },
+            'query': '''\
+query User($id: ID!) {
+  user(id: $id) {
+    id
+    __typename
+  }
+}'''},
+            'Downloading user info')
+        if data is None:
+            raise ExtractorError('Failed to fetch user info')
+
+        user = data['user']
+
+        entries = OnDemandPagedList(functools.partial(
+            self._fetch_page, username, user.get('id')), self._PAGE_SIZE)
+
+        return self.playlist_result(entries, username)
--- a/yt_dlp/extractor/nfb.py
+++ b/yt_dlp/extractor/nfb.py
@@ -0,0 +1,62 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import int_or_none
+
+
+class NFBIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?nfb\.ca/film/(?P<id>[^/?#&]+)'
+    _TESTS = [{
+        'url': 'https://www.nfb.ca/film/trafficopter/',
+        'info_dict': {
+            'id': 'trafficopter',
+            'ext': 'mp4',
+            'title': 'Trafficopter',
+            'description': 'md5:060228455eb85cf88785c41656776bc0',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            'uploader': 'Barrie Howells',
+            'release_year': 1972,
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        webpage = self._download_webpage('https://www.nfb.ca/film/%s/' % video_id, video_id)
+
+        iframe = self._html_search_regex(
+            r'<[^>]+\bid=["\']player-iframe["\'][^>]*src=["\']([^"\']+)',
+            webpage, 'iframe', default=None, fatal=True)
+        if iframe.startswith('/'):
+            iframe = f'https://www.nfb.ca{iframe}'
+
+        player = self._download_webpage(iframe, video_id)
+
+        source = self._html_search_regex(
+            r'source:\s*\'([^\']+)',
+            player, 'source', default=None, fatal=True)
+
+        formats, subtitles = self._extract_m3u8_formats_and_subtitles(source, video_id, ext='mp4')
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': self._html_search_regex(
+                r'<[^>]+\bid=["\']titleHeader["\'][^>]*>\s*<h1[^>]*>\s*([^<]+?)\s*</h1>',
+                webpage, 'title', default=None),
+            'description': self._html_search_regex(
+                r'<[^>]+\bid=["\']tabSynopsis["\'][^>]*>\s*<p[^>]*>\s*([^<]+)',
+                webpage, 'description', default=None),
+            'thumbnail': self._html_search_regex(
+                r'poster:\s*\'([^\']+)',
+                player, 'thumbnail', default=None),
+            'uploader': self._html_search_regex(
+                r'<[^>]+\bitemprop=["\']name["\'][^>]*>([^<]+)',
+                webpage, 'uploader', default=None),
+            'release_year': int_or_none(self._html_search_regex(
+                r'<[^>]+\bitemprop=["\']datePublished["\'][^>]*>([^<]+)',
+                webpage, 'release_year', default=None)),
+            'formats': formats,
+            'subtitles': subtitles,
+        }
--- a/yt_dlp/extractor/nhk.py
+++ b/yt_dlp/extractor/nhk.py
@@ -1,8 +1,15 @@
 from __future__ import unicode_literals

+import re

 from .common import InfoExtractor
-from ..utils import urljoin
+from ..utils import (
+    parse_duration,
+    traverse_obj,
+    unescapeHTML,
+    unified_timestamp,
+    urljoin
+)


 class NhkBaseIE(InfoExtractor):
@@ -176,3 +183,143 @@ class NhkVodProgramIE(NhkBaseIE):
            program_title = entries[0].get('series')

        return self.playlist_result(entries, program_id, program_title)
+
+
+class NhkForSchoolBangumiIE(InfoExtractor):
+    _VALID_URL = r'https?://www2\.nhk\.or\.jp/school/movie/(?P<type>bangumi|clip)\.cgi\?das_id=(?P<id>[a-zA-Z0-9_-]+)'
+    _TESTS = [{
+        'url': 'https://www2.nhk.or.jp/school/movie/bangumi.cgi?das_id=D0005150191_00000',
+        'info_dict': {
+            'id': 'D0005150191_00003',
+            'title': 'にている かな',
+            'duration': 599.999,
+            'timestamp': 1396414800,
+
+            'upload_date': '20140402',
+            'ext': 'mp4',
+
+            'chapters': 'count:12'
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+    }]
+
+    def _real_extract(self, url):
+        program_type, video_id = self._match_valid_url(url).groups()
+
+        webpage = self._download_webpage(
+            f'https://www2.nhk.or.jp/school/movie/{program_type}.cgi?das_id={video_id}', video_id)
+
+        # searches all variables
+        base_values = {g.group(1): g.group(2) for g in re.finditer(r'var\s+([a-zA-Z_]+)\s*=\s*"([^"]+?)";', webpage)}
+        # and programObj values too
+        program_values = {g.group(1): g.group(3) for g in re.finditer(r'(?:program|clip)Obj\.([a-zA-Z_]+)\s*=\s*(["\'])([^"]+?)\2;', webpage)}
+        # extract all chapters
+        chapter_durations = [parse_duration(g.group(1)) for g in re.finditer(r'chapterTime\.push\(\'([0-9:]+?)\'\);', webpage)]
+        chapter_titles = [' '.join([g.group(1) or '', unescapeHTML(g.group(2))]).strip() for g in re.finditer(r'<div class="cpTitle"><span>(scene\s*\d+)?</span>([^<]+?)</div>', webpage)]
+
+        # this is how player_core.js is actually doing (!)
+        version = base_values.get('r_version') or program_values.get('version')
+        if version:
+            video_id = f'{video_id.split("_")[0]}_{version}'
+
+        formats = self._extract_m3u8_formats(
+            f'https://nhks-vh.akamaihd.net/i/das/{video_id[0:8]}/{video_id}_V_000.f4v/master.m3u8',
+            video_id, ext='mp4', m3u8_id='hls')
+        self._sort_formats(formats)
+
+        duration = parse_duration(base_values.get('r_duration'))
+
+        chapters = None
+        if chapter_durations and chapter_titles and len(chapter_durations) == len(chapter_titles):
+            start_time = chapter_durations
+            end_time = chapter_durations[1:] + [duration]
+            chapters = [{
+                'start_time': s,
+                'end_time': e,
+                'title': t,
+            } for s, e, t in zip(start_time, end_time, chapter_titles)]
+
+        return {
+            'id': video_id,
+            'title': program_values.get('name'),
+            'duration': parse_duration(base_values.get('r_duration')),
+            'timestamp': unified_timestamp(base_values['r_upload']),
+            'formats': formats,
+            'chapters': chapters,
+        }
+
+
+class NhkForSchoolSubjectIE(InfoExtractor):
+    IE_DESC = 'Portal page for each school subjects, like Japanese (kokugo, 国語) or math (sansuu/suugaku or 算数・数学)'
+    KNOWN_SUBJECTS = (
+        'rika', 'syakai', 'kokugo',
+        'sansuu', 'seikatsu', 'doutoku',
+        'ongaku', 'taiiku', 'zukou',
+        'gijutsu', 'katei', 'sougou',
+        'eigo', 'tokkatsu',
+        'tokushi', 'sonota',
+    )
+    _VALID_URL = r'https?://www\.nhk\.or\.jp/school/(?P<id>%s)/?(?:[\?#].*)?$' % '|'.join(re.escape(s) for s in KNOWN_SUBJECTS)
+
+    _TESTS = [{
+        'url': 'https://www.nhk.or.jp/school/sougou/',
+        'info_dict': {
+            'id': 'sougou',
+            'title': '総合的な学習の時間',
+        },
+        'playlist_mincount': 16,
+    }, {
+        'url': 'https://www.nhk.or.jp/school/rika/',
+        'info_dict': {
+            'id': 'rika',
+            'title': '理科',
+        },
+        'playlist_mincount': 15,
+    }]
+
+    def _real_extract(self, url):
+        subject_id = self._match_id(url)
+        webpage = self._download_webpage(url, subject_id)
+
+        return self.playlist_from_matches(
+            re.finditer(rf'href="((?:https?://www\.nhk\.or\.jp)?/school/{re.escape(subject_id)}/[^/]+/)"', webpage),
+            subject_id,
+            self._html_search_regex(r'(?s)<span\s+class="subjectName">\s*<img\s*[^<]+>\s*([^<]+?)</span>', webpage, 'title', fatal=False),
+            lambda g: urljoin(url, g.group(1)))
+
+
+class NhkForSchoolProgramListIE(InfoExtractor):
+    _VALID_URL = r'https?://www\.nhk\.or\.jp/school/(?P<id>(?:%s)/[a-zA-Z0-9_-]+)' % (
+        '|'.join(re.escape(s) for s in NhkForSchoolSubjectIE.KNOWN_SUBJECTS)
+    )
+    _TESTS = [{
+        'url': 'https://www.nhk.or.jp/school/sougou/q/',
+        'info_dict': {
+            'id': 'sougou/q',
+            'title': 'Ｑ～こどものための哲学',
+        },
+        'playlist_mincount': 20,
+    }]
+
+    def _real_extract(self, url):
+        program_id = self._match_id(url)
+
+        webpage = self._download_webpage(f'https://www.nhk.or.jp/school/{program_id}/', program_id)
+
+        title = self._og_search_title(webpage, fatal=False) or self._html_extract_title(webpage, fatal=False) or self._html_search_regex(r'<h3>([^<]+?)とは？\s*</h3>', webpage, 'title', fatal=False)
+        title = re.sub(r'\s*\|\s*NHK\s+for\s+School\s*$', '', title) if title else None
+        description = self._html_search_regex(
+            r'(?s)<div\s+class="programDetail\s*">\s*<p>[^<]+</p>',
+            webpage, 'description', fatal=False, group=0)
+
+        bangumi_list = self._download_json(
+            f'https://www.nhk.or.jp/school/{program_id}/meta/program.json', program_id)
+        # they're always bangumi
+        bangumis = [
+            self.url_result(f'https://www2.nhk.or.jp/school/movie/bangumi.cgi?das_id={x}')
+            for x in traverse_obj(bangumi_list, ('part', ..., 'part-video-dasid')) or []]
+
+        return self.playlist_result(bangumis, program_id, title, description)
--- a/yt_dlp/extractor/niconico.py
+++ b/yt_dlp/extractor/niconico.py
@@ -3,6 +3,7 @@ from __future__ import unicode_literals

 import datetime
 import itertools
+import functools
 import json
 import re

@@ -12,6 +13,7 @@ from ..compat import (
    compat_str,
    compat_parse_qs,
    compat_urllib_parse_urlparse,
+    compat_HTTPError,
 )
 from ..utils import (
    ExtractorError,
@@ -24,7 +26,9 @@ from ..utils import (
    PostProcessingError,
    remove_start,
    str_or_none,
+    traverse_obj,
    try_get,
+    unescapeHTML,
    unified_timestamp,
    urlencode_postdata,
    xpath_text,
@@ -606,8 +610,61 @@ class NiconicoIE(InfoExtractor):
        }


-class NiconicoPlaylistIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?nicovideo\.jp/(?:user/\d+/|my/)?mylist/(?P<id>\d+)'
+class NiconicoPlaylistBaseIE(InfoExtractor):
+    _PAGE_SIZE = 100
+
+    _API_HEADERS = {
+        'X-Frontend-ID': '6',
+        'X-Frontend-Version': '0',
+        'X-Niconico-Language': 'en-us'
+    }
+
+    def _call_api(self, list_id, resource, query):
+        "Implement this in child class"
+        pass
+
+    @staticmethod
+    def _parse_owner(item):
+        return {
+            'uploader': traverse_obj(item, ('owner', 'name')),
+            'uploader_id': traverse_obj(item, ('owner', 'id')),
+        }
+
+    def _fetch_page(self, list_id, page):
+        page += 1
+        resp = self._call_api(list_id, 'page %d' % page, {
+            'page': page,
+            'pageSize': self._PAGE_SIZE,
+        })
+        # this is needed to support both mylist and user
+        for video in traverse_obj(resp, ('items', ..., ('video', None))) or []:
+            video_id = video.get('id')
+            if not video_id:
+                # skip {"video": {"id": "blablabla", ...}}
+                continue
+            count = video.get('count') or {}
+            get_count = lambda x: int_or_none(count.get(x))
+            yield {
+                '_type': 'url',
+                'id': video_id,
+                'title': video.get('title'),
+                'url': f'https://www.nicovideo.jp/watch/{video_id}',
+                'description': video.get('shortDescription'),
+                'duration': int_or_none(video.get('duration')),
+                'view_count': get_count('view'),
+                'comment_count': get_count('comment'),
+                'thumbnail': traverse_obj(video, ('thumbnail', ('nHdUrl', 'largeUrl', 'listingUrl', 'url'))),
+                'ie_key': NiconicoIE.ie_key(),
+                **self._parse_owner(video),
+            }
+
+    def _entries(self, list_id):
+        return OnDemandPagedList(functools.partial(self._fetch_page, list_id), self._PAGE_SIZE)
+
+
+class NiconicoPlaylistIE(NiconicoPlaylistBaseIE):
+    IE_NAME = 'niconico:playlist'
+    _VALID_URL = r'https?://(?:(?:www\.|sp\.)?nicovideo\.jp|nico\.ms)/(?:user/\d+/)?(?:my/)?mylist/(?:#/)?(?P<id>\d+)'

    _TESTS = [{
        'url': 'http://www.nicovideo.jp/mylist/27411728',
@@ -618,51 +675,115 @@ class NiconicoPlaylistIE(InfoExtractor):
            'uploader': 'のっく',
            'uploader_id': '805442',
        },
-        'playlist_mincount': 225,
+        'playlist_mincount': 291,
    }, {
        'url': 'https://www.nicovideo.jp/user/805442/mylist/27411728',
        'only_matching': True,
+    }, {
+        'url': 'https://www.nicovideo.jp/my/mylist/#/68048635',
+        'only_matching': True,
    }]

-    _API_HEADERS = {
-        'X-Frontend-ID': '6',
-        'X-Frontend-Version': '0'
-    }
+    def _call_api(self, list_id, resource, query):
+        return self._download_json(
+            f'https://nvapi.nicovideo.jp/v2/mylists/{list_id}', list_id,
+            f'Downloading {resource}', query=query,
+            headers=self._API_HEADERS)['data']['mylist']

    def _real_extract(self, url):
        list_id = self._match_id(url)
+        mylist = self._call_api(list_id, 'list', {
+            'pageSize': 1,
+        })
+        return self.playlist_result(
+            self._entries(list_id), list_id,
+            mylist.get('name'), mylist.get('description'), **self._parse_owner(mylist))

-        def get_page_data(pagenum, pagesize):
-            return self._download_json(
-                'http://nvapi.nicovideo.jp/v2/mylists/' + list_id, list_id,
-                query={'page': 1 + pagenum, 'pageSize': pagesize},
-                headers=self._API_HEADERS).get('data').get('mylist')

-        data = get_page_data(0, 1)
-        title = data.get('name')
-        description = data.get('description')
-        uploader = data.get('owner').get('name')
-        uploader_id = data.get('owner').get('id')
+class NiconicoSeriesIE(InfoExtractor):
+    IE_NAME = 'niconico:series'
+    _VALID_URL = r'https?://(?:(?:www\.|sp\.)?nicovideo\.jp|nico\.ms)/series/(?P<id>\d+)'

-        def pagefunc(pagenum):
-            data = get_page_data(pagenum, 25)
-            return ({
-                '_type': 'url',
-                'url': 'http://www.nicovideo.jp/watch/' + item.get('watchId'),
-            } for item in data.get('items'))
+    _TESTS = [{
+        'url': 'https://www.nicovideo.jp/series/110226',
+        'info_dict': {
+            'id': '110226',
+            'title': 'ご立派ァ！のシリーズ',
+        },
+        'playlist_mincount': 10,  # as of 2021/03/17
+    }, {
+        'url': 'https://www.nicovideo.jp/series/12312/',
+        'info_dict': {
+            'id': '12312',
+            'title': 'バトルスピリッツ　お勧めカード紹介(調整中)',
+        },
+        'playlist_mincount': 97,  # as of 2021/03/17
+    }, {
+        'url': 'https://nico.ms/series/203559',
+        'only_matching': True,
+    }]

-        return {
-            '_type': 'playlist',
-            'id': list_id,
-            'title': title,
-            'description': description,
-            'uploader': uploader,
-            'uploader_id': uploader_id,
-            'entries': OnDemandPagedList(pagefunc, 25),
-        }
+    def _real_extract(self, url):
+        list_id = self._match_id(url)
+        webpage = self._download_webpage(f'https://www.nicovideo.jp/series/{list_id}', list_id)
+
+        title = self._search_regex(
+            (r'<title>「(.+)（全',
+             r'<div class="TwitterShareButton"\s+data-text="(.+)\s+https:'),
+            webpage, 'title', fatal=False)
+        if title:
+            title = unescapeHTML(title)
+        playlist = [
+            self.url_result(f'https://www.nicovideo.jp/watch/{v_id}', video_id=v_id)
+            for v_id in re.findall(r'href="/watch/([a-z0-9]+)" data-href="/watch/\1', webpage)]
+        return self.playlist_result(playlist, list_id, title)
+
+
+class NiconicoHistoryIE(NiconicoPlaylistBaseIE):
+    IE_NAME = 'niconico:history'
+    IE_DESC = 'NicoNico user history. Requires cookies.'
+    _VALID_URL = r'https?://(?:www\.|sp\.)?nicovideo\.jp/my/history'
+
+    _TESTS = [{
+        'note': 'PC page, with /video',
+        'url': 'https://www.nicovideo.jp/my/history/video',
+        'only_matching': True,
+    }, {
+        'note': 'PC page, without /video',
+        'url': 'https://www.nicovideo.jp/my/history',
+        'only_matching': True,
+    }, {
+        'note': 'mobile page, with /video',
+        'url': 'https://sp.nicovideo.jp/my/history/video',
+        'only_matching': True,
+    }, {
+        'note': 'mobile page, without /video',
+        'url': 'https://sp.nicovideo.jp/my/history',
+        'only_matching': True,
+    }]
+
+    def _call_api(self, list_id, resource, query):
+        return self._download_json(
+            'https://nvapi.nicovideo.jp/v1/users/me/watch/history', 'history',
+            f'Downloading {resource}', query=query,
+            headers=self._API_HEADERS)['data']
+
+    def _real_extract(self, url):
+        list_id = 'history'
+        try:
+            mylist = self._call_api(list_id, 'list', {
+                'pageSize': 1,
+            })
+        except ExtractorError as e:
+            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
+                self.raise_login_required('You have to be logged in to get your watch history')
+            raise
+        return self.playlist_result(self._entries(list_id), list_id, **self._parse_owner(mylist))


 class NicovideoSearchBaseIE(InfoExtractor):
+    _SEARCH_TYPE = 'search'
+
    def _entries(self, url, item_id, query=None, note='Downloading page %(page)s'):
        query = query or {}
        pages = [query['page']] if 'page' in query else itertools.count(1)
@@ -677,7 +798,7 @@ class NicovideoSearchBaseIE(InfoExtractor):

    def _search_results(self, query):
        return self._entries(
-            self._proto_relative_url(f'//www.nicovideo.jp/search/{query}'), query)
+            self._proto_relative_url(f'//www.nicovideo.jp/{self._SEARCH_TYPE}/{query}'), query)


 class NicovideoSearchIE(NicovideoSearchBaseIE, SearchInfoExtractor):
@@ -757,6 +878,25 @@ class NicovideoSearchDateIE(NicovideoSearchBaseIE, SearchInfoExtractor):
        yield from super()._entries(url, item_id, query=query, note=note)


+class NicovideoTagURLIE(NicovideoSearchBaseIE):
+    IE_NAME = 'niconico:tag'
+    IE_DESC = 'NicoNico video tag URLs'
+    _SEARCH_TYPE = 'tag'
+    _VALID_URL = r'https?://(?:www\.)?nicovideo\.jp/tag/(?P<id>[^?#&]+)?'
+    _TESTS = [{
+        'url': 'https://www.nicovideo.jp/tag/ドキュメンタリー淫夢',
+        'info_dict': {
+            'id': 'ドキュメンタリー淫夢',
+            'title': 'ドキュメンタリー淫夢'
+        },
+        'playlist_mincount': 400,
+    }]
+
+    def _real_extract(self, url):
+        query = self._match_id(url)
+        return self.playlist_result(self._entries(url, query), query, query)
+
+
 class NiconicoUserIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?nicovideo\.jp/user/(?P<id>\d+)/?(?:$|[#?])'
    _TEST = {
--- a/yt_dlp/extractor/nrk.py
+++ b/yt_dlp/extractor/nrk.py
@@ -8,6 +8,7 @@ import re
 from .common import InfoExtractor
 from ..compat import compat_str
 from ..utils import (
+    compat_HTTPError,
    determine_ext,
    ExtractorError,
    int_or_none,
@@ -147,10 +148,14 @@ class NRKIE(NRKBaseIE):
    def _real_extract(self, url):
        video_id = self._match_id(url).split('/')[-1]

-        path_templ = 'playback/%s/program/' + video_id
-
        def call_playback_api(item, query=None):
-            return self._call_api(path_templ % item, video_id, item, query=query)
+            try:
+                return self._call_api(f'playback/{item}/program/{video_id}', video_id, item, query=query)
+            except ExtractorError as e:
+                if isinstance(e.cause, compat_HTTPError) and e.cause.code == 400:
+                    return self._call_api(f'playback/{item}/{video_id}', video_id, item, query=query)
+                raise
+
        # known values for preferredCdn: akamai, iponly, minicdn and telenor
        manifest = call_playback_api('manifest', {'preferredCdn': 'akamai'})

--- a/yt_dlp/extractor/nuvid.py
+++ b/yt_dlp/extractor/nuvid.py
@@ -1,11 +1,14 @@
 # coding: utf-8
 from __future__ import unicode_literals
+import re

 from .common import InfoExtractor
 from ..utils import (
    parse_duration,
    int_or_none,
-    try_get,
+    strip_or_none,
+    traverse_obj,
+    url_or_none,
 )


@@ -20,14 +23,30 @@ class NuvidIE(InfoExtractor):
            'title': 'italian babe',
            'duration': 321.0,
            'age_limit': 18,
+            'thumbnail': r're:https?://.+\.jpg',
        }
    }, {
        'url': 'https://m.nuvid.com/video/6523263',
+        'md5': 'ebd22ce8e47e1d9a4d0756a15c67da52',
        'info_dict': {
            'id': '6523263',
            'ext': 'mp4',
-            'age_limit': 18,
            'title': 'Slut brunette college student anal dorm',
+            'duration': 421.0,
+            'age_limit': 18,
+            'thumbnail': r're:https?://.+\.jpg',
+            'thumbnails': list,
+        }
+    }, {
+        'url': 'http://m.nuvid.com/video/6415801/',
+        'md5': '638d5ececb138d5753593f751ae3f697',
+        'info_dict': {
+            'id': '6415801',
+            'ext': 'mp4',
+            'title': 'My best friend wanted to fuck my wife for a long time',
+            'duration': 1882,
+            'age_limit': 18,
+            'thumbnail': r're:https?://.+\.jpg',
        }
    }]

@@ -46,6 +65,16 @@ class NuvidIE(InfoExtractor):
                'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8',
            })

+        webpage = self._download_webpage(
+            'http://m.nuvid.com/video/%s' % (video_id, ),
+            video_id, 'Downloading video page', fatal=False) or ''
+
+        title = strip_or_none(video_data.get('title') or self._html_search_regex(
+            (r'''<span\s[^>]*?\btitle\s*=\s*(?P<q>"|'|\b)(?P<title>[^"]+)(?P=q)\s*>''',
+                r'''<div\s[^>]*?\bclass\s*=\s*(?P<q>"|'|\b)thumb-holder video(?P=q)>\s*<h5\b[^>]*>(?P<title>[^<]+)</h5''',
+                r'''<span\s[^>]*?\bclass\s*=\s*(?P<q>"|'|\b)title_thumb(?P=q)>(?P<title>[^<]+)</span'''),
+            webpage, 'title', group='title'))
+
        formats = [{
            'url': source,
            'format_id': qualities.get(quality),
@@ -55,19 +84,19 @@ class NuvidIE(InfoExtractor):
        self._check_formats(formats, video_id)
        self._sort_formats(formats)

-        title = video_data.get('title')
-        thumbnail_base_url = try_get(video_data, lambda x: x['thumbs']['url'])
-        thumbnail_extension = try_get(video_data, lambda x: x['thumbs']['extension'])
-        thumbnail_id = self._search_regex(
-            r'/media/videos/tmb/6523263/preview/(/d+)' + thumbnail_extension, video_data.get('poster', ''), 'thumbnail id', default=19)
-        thumbnail = f'{thumbnail_base_url}player/{thumbnail_id}{thumbnail_extension}'
-        duration = parse_duration(video_data.get('duration') or video_data.get('duration_format'))
+        duration = parse_duration(traverse_obj(video_data, 'duration', 'duration_format'))
+        thumbnails = [
+            {'url': thumb_url} for thumb_url in re.findall(
+                r'<div\s+class\s*=\s*"video-tmb-wrap"\s*>\s*<img\s+src\s*=\s*"([^"]+)"\s*/>', webpage)
+            if url_or_none(thumb_url)]
+        if url_or_none(video_data.get('poster')):
+            thumbnails.append({'url': video_data['poster'], 'preference': 1})

        return {
            'id': video_id,
            'formats': formats,
            'title': title,
-            'thumbnail': thumbnail,
+            'thumbnails': thumbnails,
            'duration': duration,
            'age_limit': 18,
        }
--- a/yt_dlp/extractor/openload.py
+++ b/yt_dlp/extractor/openload.py
@@ -16,7 +16,6 @@ from ..utils import (
    ExtractorError,
    get_exe_version,
    is_outdated_version,
-    std_headers,
    Popen,
 )

@@ -208,7 +207,7 @@ class PhantomJSwrapper(object):

        replaces = self.options
        replaces['url'] = url
-        user_agent = headers.get('User-Agent') or std_headers['User-Agent']
+        user_agent = headers.get('User-Agent') or self.get_param('http_headers')['User-Agent']
        replaces['ua'] = user_agent.replace('"', '\\"')
        replaces['jscode'] = jscode

--- a/yt_dlp/extractor/openrec.py
+++ b/yt_dlp/extractor/openrec.py
@@ -42,8 +42,7 @@ class OpenRecBaseIE(InfoExtractor):
            if not m3u8_url:
                continue
            formats.extend(self._extract_m3u8_formats(
-                m3u8_url, video_id, ext='mp4', entry_protocol='m3u8',
-                m3u8_id='hls-%s' % name, live=True))
+                m3u8_url, video_id, ext='mp4', live=is_live, m3u8_id='hls-%s' % name))

        self._sort_formats(formats)

--- a/yt_dlp/extractor/peekvids.py
+++ b/yt_dlp/extractor/peekvids.py
@@ -0,0 +1,81 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class PeekVidsIE(InfoExtractor):
+    _VALID_URL = r'''(?x)
+        https?://(?:www\.)?peekvids\.com/
+        (?:(?:[^/?#]+/){2}|embed/?\?(?:[^#]*&)?v=)
+        (?P<id>[^/?&#]*)
+    '''
+    _TESTS = [{
+        'url': 'https://peekvids.com/pc/dane-jones-cute-redhead-with-perfect-tits-with-mini-vamp/BSyLMbN0YCd',
+        'md5': 'a00940646c428e232407e3e62f0e8ef5',
+        'info_dict': {
+            'id': 'BSyLMbN0YCd',
+            'title': ' Dane Jones - Cute redhead with perfect tits with Mini Vamp, SEXYhub',
+            'ext': 'mp4',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            'description': 'Watch  Dane Jones - Cute redhead with perfect tits with Mini Vamp (7 min), uploaded by SEXYhub.com',
+            'timestamp': 1642579329,
+            'upload_date': '20220119',
+            'duration': 416,
+            'view_count': int,
+            'age_limit': 18,
+        },
+    }]
+    _DOMAIN = 'www.peekvids.com'
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        short_video_id = self._html_search_regex(r'<video [^>]*data-id="(.+?)"', webpage, 'short video ID')
+        srcs = self._download_json(
+            f'https://{self._DOMAIN}/v-alt/{short_video_id}', video_id,
+            note='Downloading list of source files')
+        formats = [{
+            'url': url,
+            'ext': 'mp4',
+            'format_id': name[8:],
+        } for name, url in srcs.items() if len(name) > 8 and name.startswith('data-src')]
+        if not formats:
+            formats = [{'url': url} for url in srcs.values()]
+        self._sort_formats(formats)
+
+        info = self._search_json_ld(webpage, video_id, expected_type='VideoObject')
+        info.update({
+            'id': video_id,
+            'age_limit': 18,
+            'formats': formats,
+        })
+        return info
+
+
+class PlayVidsIE(PeekVidsIE):
+    _VALID_URL = r'https?://(?:www\.)?playvids\.com/(?:embed/|[^/]{2}/)?(?P<id>[^/?#]*)'
+    _TESTS = [{
+        'url': 'https://www.playvids.com/U3pBrYhsjXM/pc/dane-jones-cute-redhead-with-perfect-tits-with-mini-vamp',
+        'md5': 'cd7dfd8a2e815a45402369c76e3c1825',
+        'info_dict': {
+            'id': 'U3pBrYhsjXM',
+            'title': ' Dane Jones - Cute redhead with perfect tits with Mini Vamp, SEXYhub',
+            'ext': 'mp4',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            'description': 'Watch  Dane Jones - Cute redhead with perfect tits with Mini Vamp video in HD, uploaded by SEXYhub.com',
+            'timestamp': 1640435839,
+            'upload_date': '20211225',
+            'duration': 416,
+            'view_count': int,
+            'age_limit': 18,
+        },
+    }, {
+        'url': 'https://www.playvids.com/es/U3pBrYhsjXM/pc/dane-jones-cute-redhead-with-perfect-tits-with-mini-vamp',
+        'only_matching': True,
+    }, {
+        'url': 'https://www.playvids.com/embed/U3pBrYhsjXM',
+        'only_matching': True,
+    }]
+    _DOMAIN = 'www.playvids.com'
--- a/yt_dlp/extractor/peertube.py
+++ b/yt_dlp/extractor/peertube.py
@@ -87,6 +87,7 @@ class PeerTubeIE(InfoExtractor):
                            maindreieck-tv\.de|
                            mani\.tube|
                            manicphase\.me|
+                            media\.fsfe\.org|
                            media\.gzevd\.de|
                            media\.inno3\.cricket|
                            media\.kaitaia\.life|
--- a/yt_dlp/extractor/periscope.py
+++ b/yt_dlp/extractor/periscope.py
@@ -33,7 +33,7 @@ class PeriscopeBaseIE(InfoExtractor):

        return {
            'id': broadcast.get('id') or video_id,
-            'title': self._live_title(title) if is_live else title,
+            'title': title,
            'timestamp': parse_iso8601(broadcast.get('created_at')),
            'uploader': uploader,
            'uploader_id': broadcast.get('user_id') or broadcast.get('username'),
--- a/yt_dlp/extractor/piapro.py
+++ b/yt_dlp/extractor/piapro.py
@@ -0,0 +1,100 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_urlparse
+from ..utils import (
+    ExtractorError,
+    parse_duration,
+    parse_filesize,
+    str_to_int,
+    unified_timestamp,
+    urlencode_postdata,
+)
+
+
+class PiaproIE(InfoExtractor):
+    _NETRC_MACHINE = 'piapro'
+    _VALID_URL = r'https?://piapro\.jp/t/(?P<id>\w+)/?'
+    _TESTS = [{
+        'url': 'https://piapro.jp/t/NXYR',
+        'md5': 'a9d52f27d13bafab7ee34116a7dcfa77',
+        'info_dict': {
+            'id': 'NXYR',
+            'ext': 'mp3',
+            'uploader': 'wowaka',
+            'uploader_id': 'wowaka',
+            'title': '裏表ラバーズ',
+            'thumbnail': r're:^https?://.*\.jpg$',
+        }
+    }]
+
+    def _real_initialize(self):
+        self._login_status = self._login()
+
+    def _login(self):
+        username, password = self._get_login_info()
+        if not username:
+            return False
+        login_ok = True
+        login_form_strs = {
+            '_username': username,
+            '_password': password,
+            '_remember_me': 'on',
+            'login': 'ログイン'
+        }
+        self._request_webpage('https://piapro.jp/login/', None)
+        urlh = self._request_webpage(
+            'https://piapro.jp/login/exe', None,
+            note='Logging in', errnote='Unable to log in',
+            data=urlencode_postdata(login_form_strs))
+        if urlh is False:
+            login_ok = False
+        else:
+            parts = compat_urlparse.urlparse(urlh.geturl())
+            if parts.path != '/':
+                login_ok = False
+        if not login_ok:
+            self.report_warning(
+                'unable to log in: bad username or password')
+        return login_ok
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        category_id = self._search_regex(r'categoryId=(.+)">', webpage, 'category ID')
+        if category_id not in ('1', '2', '21', '22', '23', '24', '25'):
+            raise ExtractorError('The URL does not contain audio.', expected=True)
+
+        str_duration, str_filesize = self._search_regex(
+            r'サイズ：</span>(.+?)/\(([0-9,]+?[KMG]?B)）', webpage, 'duration and size',
+            group=(1, 2), default=(None, None))
+        str_viewcount = self._search_regex(r'閲覧数：</span>([0-9,]+)\s+', webpage, 'view count', fatal=False)
+
+        uploader_id, uploader = self._search_regex(
+            r'<a\s+class="cd_user-name"\s+href="/(.*)">([^<]+)さん<', webpage, 'uploader',
+            group=(1, 2), default=(None, None))
+        content_id = self._search_regex(r'contentId\:\'(.+)\'', webpage, 'content ID')
+        create_date = self._search_regex(r'createDate\:\'(.+)\'', webpage, 'timestamp')
+
+        player_webpage = self._download_webpage(
+            f'https://piapro.jp/html5_player_popup/?id={content_id}&cdate={create_date}',
+            video_id, note='Downloading player webpage')
+
+        return {
+            'id': video_id,
+            'title': self._html_search_regex(r'<h1\s+class="cd_works-title">(.+?)</h1>', webpage, 'title', fatal=False),
+            'description': self._html_search_regex(r'<p\s+class="cd_dtl_cap">(.+?)</p>\s*<div', webpage, 'description', fatal=False),
+            'uploader': uploader,
+            'uploader_id': uploader_id,
+            'timestamp': unified_timestamp(create_date, False),
+            'duration': parse_duration(str_duration),
+            'view_count': str_to_int(str_viewcount),
+            'thumbnail': self._html_search_meta('twitter:image', webpage),
+
+            'filesize_approx': parse_filesize(str_filesize.replace(',', '')),
+            'url': self._search_regex(r'mp3:\s*\'(.*?)\'\}', player_webpage, 'url'),
+            'ext': 'mp3',
+            'vcodec': 'none',
+        }
--- a/yt_dlp/extractor/ro220.py
+++ b/yt_dlp/extractor/ro220.py
@@ -1,43 +0,0 @@
-from __future__ import unicode_literals
-
-from .common import InfoExtractor
-from ..compat import compat_urllib_parse_unquote
-
-
-class Ro220IE(InfoExtractor):
-    IE_NAME = '220.ro'
-    _VALID_URL = r'(?x)(?:https?://)?(?:www\.)?220\.ro/(?P<category>[^/]+)/(?P<shorttitle>[^/]+)/(?P<id>[^/]+)'
-    _TEST = {
-        'url': 'http://www.220.ro/sport/Luati-Le-Banii-Sez-4-Ep-1/LYV6doKo7f/',
-        'md5': '03af18b73a07b4088753930db7a34add',
-        'info_dict': {
-            'id': 'LYV6doKo7f',
-            'ext': 'mp4',
-            'title': 'Luati-le Banii sez 4 ep 1',
-            'description': r're:^Iata-ne reveniti dupa o binemeritata vacanta\. +Va astept si pe Facebook cu pareri si comentarii.$',
-        }
-    }
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-
-        webpage = self._download_webpage(url, video_id)
-        url = compat_urllib_parse_unquote(self._search_regex(
-            r'(?s)clip\s*:\s*{.*?url\s*:\s*\'([^\']+)\'', webpage, 'url'))
-        title = self._og_search_title(webpage)
-        description = self._og_search_description(webpage)
-        thumbnail = self._og_search_thumbnail(webpage)
-
-        formats = [{
-            'format_id': 'sd',
-            'url': url,
-            'ext': 'mp4',
-        }]
-
-        return {
-            'id': video_id,
-            'formats': formats,
-            'title': title,
-            'description': description,
-            'thumbnail': thumbnail,
-        }
--- a/yt_dlp/extractor/rokfin.py
+++ b/yt_dlp/extractor/rokfin.py
@@ -0,0 +1,256 @@
+# coding: utf-8
+import itertools
+from datetime import datetime
+
+from .common import InfoExtractor
+from ..utils import (
+    determine_ext,
+    ExtractorError,
+    float_or_none,
+    format_field,
+    int_or_none,
+    str_or_none,
+    traverse_obj,
+    unified_timestamp,
+    url_or_none,
+)
+
+
+_API_BASE_URL = 'https://prod-api-v2.production.rokfin.com/api/v2/public/'
+
+
+class RokfinIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?rokfin\.com/(?P<id>(?P<type>post|stream)/\d+)'
+    _TESTS = [{
+        'url': 'https://www.rokfin.com/post/57548/Mitt-Romneys-Crazy-Solution-To-Climate-Change',
+        'info_dict': {
+            'id': 'post/57548',
+            'ext': 'mp4',
+            'title': 'Mitt Romney\'s Crazy Solution To Climate Change',
+            'thumbnail': r're:https://img\.production\.rokfin\.com/.+',
+            'upload_date': '20211023',
+            'timestamp': 1634998029,
+            'channel': 'Jimmy Dore',
+            'channel_id': 65429,
+            'channel_url': 'https://rokfin.com/TheJimmyDoreShow',
+            'duration': 213.0,
+            'availability': 'public',
+            'live_status': 'not_live',
+            'dislike_count': int,
+            'like_count': int,
+        }
+    }, {
+        'url': 'https://rokfin.com/post/223/Julian-Assange-Arrested-Streaming-In-Real-Time',
+        'info_dict': {
+            'id': 'post/223',
+            'ext': 'mp4',
+            'title': 'Julian Assange Arrested: Streaming In Real Time',
+            'thumbnail': r're:https://img\.production\.rokfin\.com/.+',
+            'upload_date': '20190412',
+            'timestamp': 1555052644,
+            'channel': 'Ron Placone',
+            'channel_id': 10,
+            'channel_url': 'https://rokfin.com/RonPlacone',
+            'availability': 'public',
+            'live_status': 'not_live',
+            'dislike_count': int,
+            'like_count': int,
+            'tags': ['FreeThinkingMedia^', 'RealProgressives^'],
+        }
+    }, {
+        'url': 'https://www.rokfin.com/stream/10543/Its-A-Crazy-Mess-Regional-Director-Blows-Whistle-On-Pfizers-Vaccine-Trial-Data',
+        'info_dict': {
+            'id': 'stream/10543',
+            'ext': 'mp4',
+            'title': '"It\'s A Crazy Mess" Regional Director Blows Whistle On Pfizer\'s Vaccine Trial Data',
+            'thumbnail': r're:https://img\.production\.rokfin\.com/.+',
+            'description': 'md5:324ce2d3e3b62e659506409e458b9d8e',
+            'channel': 'Ryan Cristián',
+            'channel_id': 53856,
+            'channel_url': 'https://rokfin.com/TLAVagabond',
+            'availability': 'public',
+            'is_live': False,
+            'was_live': True,
+            'live_status': 'was_live',
+            'timestamp': 1635874720,
+            'release_timestamp': 1635874720,
+            'release_date': '20211102',
+            'upload_date': '20211102',
+            'dislike_count': int,
+            'like_count': int,
+            'tags': ['FreeThinkingMedia^'],
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id, video_type = self._match_valid_url(url).group('id', 'type')
+
+        metadata = self._download_json(f'{_API_BASE_URL}{video_id}', video_id)
+
+        scheduled = unified_timestamp(metadata.get('scheduledAt'))
+        live_status = ('was_live' if metadata.get('stoppedAt')
+                       else 'is_upcoming' if scheduled
+                       else 'is_live' if video_type == 'stream'
+                       else 'not_live')
+
+        video_url = traverse_obj(metadata, 'url', ('content', 'contentUrl'), expected_type=url_or_none)
+        formats, subtitles = [{'url': video_url}] if video_url else [], {}
+        if determine_ext(video_url) == 'm3u8':
+            formats, subtitles = self._extract_m3u8_formats_and_subtitles(
+                video_url, video_id, fatal=False, live=live_status == 'is_live')
+
+        if not formats:
+            if traverse_obj(metadata, 'premiumPlan', 'premium'):
+                self.raise_login_required('This video is only available to premium users', True, method='cookies')
+            elif scheduled:
+                self.raise_no_formats(
+                    f'Stream is offline; sheduled for {datetime.fromtimestamp(scheduled).strftime("%Y-%m-%d %H:%M:%S")}',
+                    video_id=video_id, expected=True)
+        self._sort_formats(formats)
+
+        uploader = traverse_obj(metadata, ('createdBy', 'username'), ('creator', 'username'))
+        timestamp = (scheduled or float_or_none(metadata.get('postedAtMilli'), 1000)
+                     or unified_timestamp(metadata.get('creationDateTime')))
+        return {
+            'id': video_id,
+            'formats': formats,
+            'subtitles': subtitles,
+            'title': str_or_none(traverse_obj(metadata, 'title', ('content', 'contentTitle'))),
+            'duration': float_or_none(traverse_obj(metadata, ('content', 'duration'))),
+            'thumbnail': url_or_none(traverse_obj(metadata, 'thumbnail', ('content', 'thumbnailUrl1'))),
+            'description': str_or_none(traverse_obj(metadata, 'description', ('content', 'contentDescription'))),
+            'like_count': int_or_none(metadata.get('likeCount')),
+            'dislike_count': int_or_none(metadata.get('dislikeCount')),
+            'channel': str_or_none(traverse_obj(metadata, ('createdBy', 'name'), ('creator', 'name'))),
+            'channel_id': traverse_obj(metadata, ('createdBy', 'id'), ('creator', 'id')),
+            'channel_url': url_or_none(f'https://rokfin.com/{uploader}') if uploader else None,
+            'timestamp': timestamp,
+            'release_timestamp': timestamp if live_status != 'not_live' else None,
+            'tags': traverse_obj(metadata, ('tags', ..., 'title'), expected_type=str_or_none),
+            'live_status': live_status,
+            'availability': self._availability(
+                needs_premium=bool(traverse_obj(metadata, 'premiumPlan', 'premium')),
+                is_private=False, needs_subscription=False, needs_auth=False, is_unlisted=False),
+            # 'comment_count': metadata.get('numComments'), # Data provided by website is wrong
+            '__post_extractor': self.extract_comments(video_id) if video_type == 'post' else None,
+        }
+
+    def _get_comments(self, video_id):
+        pages_total = None
+        for page_n in itertools.count():
+            raw_comments = self._download_json(
+                f'{_API_BASE_URL}comment?postId={video_id[5:]}&page={page_n}&size=50',
+                video_id, note=f'Downloading viewer comments page {page_n + 1}{format_field(pages_total, template=" of %s")}',
+                fatal=False) or {}
+
+            for comment in raw_comments.get('content') or []:
+                yield {
+                    'text': str_or_none(comment.get('comment')),
+                    'author': str_or_none(comment.get('name')),
+                    'id': comment.get('commentId'),
+                    'author_id': comment.get('userId'),
+                    'parent': 'root',
+                    'like_count': int_or_none(comment.get('numLikes')),
+                    'dislike_count': int_or_none(comment.get('numDislikes')),
+                    'timestamp': unified_timestamp(comment.get('postedAt'))
+                }
+
+            pages_total = int_or_none(raw_comments.get('totalPages')) or None
+            is_last = raw_comments.get('last')
+            if not raw_comments.get('content') or is_last or (page_n > pages_total if pages_total else is_last is not False):
+                return
+
+
+class RokfinPlaylistBaseIE(InfoExtractor):
+    _TYPES = {
+        'video': 'post',
+        'audio': 'post',
+        'stream': 'stream',
+        'dead_stream': 'stream',
+        'stack': 'stack',
+    }
+
+    def _get_video_data(self, metadata):
+        for content in metadata.get('content') or []:
+            media_type = self._TYPES.get(content.get('mediaType'))
+            video_id = content.get('id') if media_type == 'post' else content.get('mediaId')
+            if not media_type or not video_id:
+                continue
+
+            yield self.url_result(f'https://rokfin.com/{media_type}/{video_id}', video_id=f'{media_type}/{video_id}',
+                                  video_title=str_or_none(traverse_obj(content, ('content', 'contentTitle'))))
+
+
+class RokfinStackIE(RokfinPlaylistBaseIE):
+    IE_NAME = 'rokfin:stack'
+    _VALID_URL = r'https?://(?:www\.)?rokfin\.com/stack/(?P<id>[^/]+)'
+    _TESTS = [{
+        'url': 'https://www.rokfin.com/stack/271/Tulsi-Gabbard-Portsmouth-Townhall-FULL--Feb-9-2020',
+        'playlist_count': 8,
+        'info_dict': {
+            'id': '271',
+        },
+    }]
+
+    def _real_extract(self, url):
+        list_id = self._match_id(url)
+        return self.playlist_result(self._get_video_data(
+            self._download_json(f'{_API_BASE_URL}stack/{list_id}', list_id)), list_id)
+
+
+class RokfinChannelIE(RokfinPlaylistBaseIE):
+    IE_NAME = 'rokfin:channel'
+    _VALID_URL = r'https?://(?:www\.)?rokfin\.com/(?!((feed/?)|(discover/?)|(channels/?))$)(?P<id>[^/]+)/?$'
+    _TESTS = [{
+        'url': 'https://rokfin.com/TheConvoCouch',
+        'playlist_mincount': 100,
+        'info_dict': {
+            'id': '12071-new',
+            'title': 'TheConvoCouch - New',
+            'description': 'md5:bb622b1bca100209b91cd685f7847f06',
+        },
+    }]
+
+    _TABS = {
+        'new': 'posts',
+        'top': 'top',
+        'videos': 'video',
+        'podcasts': 'audio',
+        'streams': 'stream',
+        'stacks': 'stack',
+    }
+
+    def _real_initialize(self):
+        self._validate_extractor_args()
+
+    def _validate_extractor_args(self):
+        requested_tabs = self._configuration_arg('tab', None)
+        if requested_tabs is not None and (len(requested_tabs) > 1 or requested_tabs[0] not in self._TABS):
+            raise ExtractorError(f'Invalid extractor-arg "tab". Must be one of {", ".join(self._TABS)}', expected=True)
+
+    def _entries(self, channel_id, channel_name, tab):
+        pages_total = None
+        for page_n in itertools.count(0):
+            if tab in ('posts', 'top'):
+                data_url = f'{_API_BASE_URL}user/{channel_name}/{tab}?page={page_n}&size=50'
+            else:
+                data_url = f'{_API_BASE_URL}post/search/{tab}?page={page_n}&size=50&creator={channel_id}'
+            metadata = self._download_json(
+                data_url, channel_name,
+                note=f'Downloading video metadata page {page_n + 1}{format_field(pages_total, template=" of %s")}')
+
+            yield from self._get_video_data(metadata)
+            pages_total = int_or_none(metadata.get('totalPages')) or None
+            is_last = metadata.get('last')
+            if is_last or (page_n > pages_total if pages_total else is_last is not False):
+                return
+
+    def _real_extract(self, url):
+        channel_name = self._match_id(url)
+        channel_info = self._download_json(f'{_API_BASE_URL}user/{channel_name}', channel_name)
+        channel_id = channel_info['id']
+        tab = self._configuration_arg('tab', default=['new'])[0]
+
+        return self.playlist_result(
+            self._entries(channel_id, channel_name, self._TABS[tab]),
+            f'{channel_id}-{tab}', f'{channel_name} - {tab.title()}', str_or_none(channel_info.get('description')))
--- a/yt_dlp/extractor/roxwel.py
+++ b/yt_dlp/extractor/roxwel.py
@@ -1,52 +0,0 @@
-from __future__ import unicode_literals
-
-
-from .common import InfoExtractor
-from ..utils import unified_strdate, determine_ext
-
-
-class RoxwelIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?roxwel\.com/player/(?P<filename>.+?)(\.|\?|$)'
-
-    _TEST = {
-        'url': 'http://www.roxwel.com/player/passionpittakeawalklive.html',
-        'info_dict': {
-            'id': 'passionpittakeawalklive',
-            'ext': 'flv',
-            'title': 'Take A Walk (live)',
-            'uploader': 'Passion Pit',
-            'uploader_id': 'passionpit',
-            'upload_date': '20120928',
-            'description': 'Passion Pit performs "Take A Walk\" live at The Backyard in Austin, Texas. ',
-        },
-        'params': {
-            # rtmp download
-            'skip_download': True,
-        }
-    }
-
-    def _real_extract(self, url):
-        mobj = self._match_valid_url(url)
-        filename = mobj.group('filename')
-        info_url = 'http://www.roxwel.com/api/videos/%s' % filename
-        info = self._download_json(info_url, filename)
-
-        rtmp_rates = sorted([int(r.replace('flv_', '')) for r in info['media_rates'] if r.startswith('flv_')])
-        best_rate = rtmp_rates[-1]
-        url_page_url = 'http://roxwel.com/pl_one_time.php?filename=%s&quality=%s' % (filename, best_rate)
-        rtmp_url = self._download_webpage(url_page_url, filename, 'Downloading video url')
-        ext = determine_ext(rtmp_url)
-        if ext == 'f4v':
-            rtmp_url = rtmp_url.replace(filename, 'mp4:%s' % filename)
-
-        return {
-            'id': filename,
-            'title': info['title'],
-            'url': rtmp_url,
-            'ext': 'flv',
-            'description': info['description'],
-            'thumbnail': info.get('player_image_url') or info.get('image_url_large'),
-            'uploader': info['artist'],
-            'uploader_id': info['artistname'],
-            'upload_date': unified_strdate(info['dbdate']),
-        }
--- a/yt_dlp/extractor/rtve.py
+++ b/yt_dlp/extractor/rtve.py
@@ -17,7 +17,6 @@ from ..utils import (
    qualities,
    remove_end,
    remove_start,
-    std_headers,
    try_get,
 )

@@ -71,7 +70,7 @@ class RTVEALaCartaIE(InfoExtractor):
    }]

    def _real_initialize(self):
-        user_agent_b64 = base64.b64encode(std_headers['User-Agent'].encode('utf-8')).decode('utf-8')
+        user_agent_b64 = base64.b64encode(self.get_param('http_headers')['User-Agent'].encode('utf-8')).decode('utf-8')
        self._manager = self._download_json(
            'http://www.rtve.es/odin/loki/' + user_agent_b64,
            None, 'Fetching manager info')['manager']
--- a/yt_dlp/extractor/rtvs.py
+++ b/yt_dlp/extractor/rtvs.py
@@ -1,11 +1,19 @@
 # coding: utf-8
 from __future__ import unicode_literals

+import re
+
 from .common import InfoExtractor

+from ..utils import (
+    parse_duration,
+    traverse_obj,
+    unified_timestamp,
+)
+

 class RTVSIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?rtvs\.sk/(?:radio|televizia)/archiv/\d+/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?rtvs\.sk/(?:radio|televizia)/archiv(?:/\d+)?/(?P<id>\d+)/?(?:[#?]|$)'
    _TESTS = [{
        # radio archive
        'url': 'http://www.rtvs.sk/radio/archiv/11224/414872',
@@ -13,23 +21,37 @@ class RTVSIE(InfoExtractor):
        'info_dict': {
            'id': '414872',
            'ext': 'mp3',
-            'title': 'Ostrov pokladov 1 časť.mp3'
-        },
-        'params': {
-            'skip_download': True,
+            'title': 'Ostrov pokladov 1 časť.mp3',
+            'duration': 2854,
+            'thumbnail': 'https://www.rtvs.sk/media/a501/image/file/2/0000/b1R8.rtvs.jpg',
+            'display_id': '135331',
        }
    }, {
        # tv archive
        'url': 'http://www.rtvs.sk/televizia/archiv/8249/63118',
-        'md5': '85e2c55cf988403b70cac24f5c086dc6',
        'info_dict': {
            'id': '63118',
            'ext': 'mp4',
            'title': 'Amaro Džives - Náš deň',
-            'description': 'Galavečer pri príležitosti Medzinárodného dňa Rómov.'
-        },
-        'params': {
-            'skip_download': True,
+            'description': 'Galavečer pri príležitosti Medzinárodného dňa Rómov.',
+            'thumbnail': 'https://www.rtvs.sk/media/a501/image/file/2/0031/L7Qm.amaro_dzives_png.jpg',
+            'timestamp': 1428555900,
+            'upload_date': '20150409',
+            'duration': 4986,
+        }
+    }, {
+        # tv archive
+        'url': 'https://www.rtvs.sk/televizia/archiv/18083?utm_source=web&utm_medium=rozcestnik&utm_campaign=Robin',
+        'info_dict': {
+            'id': '18083',
+            'ext': 'mp4',
+            'title': 'Robin',
+            'description': 'md5:2f70505a7b8364491003d65ff7a0940a',
+            'timestamp': 1636652760,
+            'display_id': '307655',
+            'duration': 831,
+            'upload_date': '20211111',
+            'thumbnail': 'https://www.rtvs.sk/media/a501/image/file/2/0916/robin.jpg',
        }
    }]

@@ -37,11 +59,31 @@ class RTVSIE(InfoExtractor):
        video_id = self._match_id(url)

        webpage = self._download_webpage(url, video_id)
+        iframe_id = self._search_regex(
+            r'<iframe[^>]+id\s*=\s*"player_[^_]+_([0-9]+)"', webpage, 'Iframe ID')
+        iframe_url = self._search_regex(
+            fr'<iframe[^>]+id\s*=\s*"player_[^_]+_{re.escape(iframe_id)}"[^>]+src\s*=\s*"([^"]+)"', webpage, 'Iframe URL')

-        playlist_url = self._search_regex(
-            r'playlist["\']?\s*:\s*(["\'])(?P<url>(?:(?!\1).)+)\1', webpage,
-            'playlist url', group='url')
+        webpage = self._download_webpage(iframe_url, video_id, 'Downloading iframe')
+        json_url = self._search_regex(r'var\s+url\s*=\s*"([^"]+)"\s*\+\s*ruurl', webpage, 'json URL')
+        data = self._download_json(f'https:{json_url}b=mozilla&p=win&v=97&f=0&d=1', video_id)

-        data = self._download_json(
-            playlist_url, video_id, 'Downloading playlist')[0]
-        return self._parse_jwplayer_data(data, video_id=video_id)
+        if data.get('clip'):
+            data['playlist'] = [data['clip']]
+
+        if traverse_obj(data, ('playlist', 0, 'sources', 0, 'type')) == 'audio/mp3':
+            formats = [{'url': traverse_obj(data, ('playlist', 0, 'sources', 0, 'src'))}]
+        else:
+            formats = self._extract_m3u8_formats(traverse_obj(data, ('playlist', 0, 'sources', 0, 'src')), video_id)
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'display_id': iframe_id,
+            'title': traverse_obj(data, ('playlist', 0, 'title')),
+            'description': traverse_obj(data, ('playlist', 0, 'description')),
+            'duration': parse_duration(traverse_obj(data, ('playlist', 0, 'length'))),
+            'thumbnail': traverse_obj(data, ('playlist', 0, 'image')),
+            'timestamp': unified_timestamp(traverse_obj(data, ('playlist', 0, 'datetime_create'))),
+            'formats': formats
+        }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
github-actions	5a93dc1b85	[version] update Created by: pukkandan :ci skip all	2022-03-08 19:57:15 +00:00
pukkandan	535eb16a44	Release 2022.03.08	2022-03-09 01:24:27 +05:30
P-reducible	9461cb586a	[Rokfin] Fix `availability` (#1534 ) Authored by: P-reducible	2022-03-08 11:42:00 -08:00
pukkandan	a405b38f20	[youtube] Further de-prioritize 3gp format	2022-03-08 23:02:38 +05:30
pukkandan	08d30158ec	[cleanup, docs] Misc cleanup Closes #2828, closes #2734, closes #2802, closes #2937	2022-03-08 22:38:06 +05:30
Ha Tien Loi	c89bec262c	[xinpianchang] Add extractor (#2963 ) Authored by: hatienl0i261299	2022-03-08 08:55:40 -08:00
Ha Tien Loi	151f8f1c02	[fptplay] Add extractor (#2949 ) Closes #2857 Authored by: hatienl0i261299	2022-03-08 08:52:51 -08:00
Max Mehl	a35155be17	[peertube] Add media.fsfe.org (#2986 ) Authored by: mxmehl	2022-03-08 08:48:35 -08:00
nyuszika7h	e66662b1e0	[ccma] Fix timestamp parsing (#2989 ) Authored by: nyuszika7h	2022-03-08 08:45:23 -08:00
coletdev	4390d5ec12	Add brotli content-encoding support (#2433 ) Authored by: coletdjnz	2022-03-08 08:44:05 -08:00
CplPwnies	9e0e6adb2d	[adobepass] Add Suddenlink MSO (#2977 ) Closes #2704 Authored by: CplPwnies	2022-03-08 08:18:52 -08:00
Lesmiscore	b637c4e22e	[mildom] Fix linter	2022-03-08 23:56:30 +09:00
Lesmiscore (Naoya Ozaki)	fb6e3f4389	[mildom] Rework extractors (#2940 ) Authored by: Lesmiscore	2022-03-08 23:49:10 +09:00
pukkandan	409cdd1ec9	[ard] Fix valid URL Partial fix for #2975	2022-03-08 12:58:26 +05:30
coletdev	992f9a730b	[youtube] Prefer UTC upload date for videos (#2223 ) Except for live/scheduled streams/premieres. Closes #1881 Related: #2402 Authored-by: coletdjnz	2022-03-08 12:58:19 +05:30
pukkandan	497d2fab6c	[utils] Better traceback for `ExtractorError`	2022-03-08 12:04:49 +05:30
pukkandan	2807d1709b	[nrk] Add fallback API Closes #1891	2022-03-08 11:13:25 +05:30
shirt	b46ccbc6d4	[build] Update pyinstaller to 4.10	2022-03-07 23:02:27 -05:00
Lesmiscore	1ed7953a74	[utils] render_table: Fix character calculation for removing extra gap without this fix, the column next to delimiter will lack leading spaces on terminal (see https://github.com/yt-dlp/yt-dlp/pull/920#issuecomment-1059914615 for the situation)	2022-03-06 17:11:10 +09:00
pukkandan	d49669acad	[youtube] Fix automatic captions Closes #2956	2022-03-05 09:42:12 +05:30
foghawk	bed30106f5	[tumblr] Fix extractor (#2883 ) Authored by: foghawk	2022-03-04 19:24:49 -08:00
Zenon Mousmoulas	27231526ae	[ant1newsgr] Add extractor (#1982 ) Authored by: zmousm	2022-03-04 13:52:48 -08:00
pukkandan	50e93e03a7	Update to ytdl-commit-6508688 Make default upload_/release_date a compat_str `6508688e88` Except: * "[NDR] Overhaul NDR and NJoy extractors" https://github.com/ytdl-org/youtube-dl/pull/30531 - `01824d275b` - `39a98b09a2` - `f0a05a55c2` - `4186e81777`	2022-03-05 02:24:17 +05:30
FestplattenSchnitzel	72e995f122	[VideocampusSachsen] Add extractors (#2787 ) Authored by: FestplattenSchnitzel	2022-03-04 08:19:07 -08:00
pukkandan	8b7539d27c	Implement `--add-header` without modifying `std_headers` Closes #2526, #1614	2022-03-04 20:59:03 +05:30
pukkandan	e48b3875ec	Revert `2e4cacd038` Closes #2923	2022-03-04 20:18:14 +05:30
pukkandan	2a938746f3	Fix verbose log when stdout/stderr encoding is `None` See: `5c10453827`	2022-03-04 19:49:39 +05:30
pukkandan	933dbf5a55	[bandcamp] Detect acodec	2022-03-04 19:49:38 +05:30
pukkandan	a10aa588b0	[FormatSort] Consider `acodec`=`ogg` as `vorbis`	2022-03-04 19:49:38 +05:30
pukkandan	be8cd3cb1d	[twitch] Fix field name of `view_count`	2022-03-04 19:49:37 +05:30
pukkandan	319b6059d2	Better error message when no --live-from-start format	2022-03-04 19:49:36 +05:30
pukkandan	4c3f8c3fb6	Handle negative duration from extractor Closes #2921	2022-03-04 19:49:36 +05:30
pukkandan	7265a2190c	Fix doubling of `video_id` in `ExtractorError`	2022-03-04 19:37:43 +05:30
pukkandan	3a4bb9f751	[lbry] Fix `--ignore-no-formats-error` Closes #2942	2022-03-04 19:24:52 +05:30
i6t	b90dbe6c19	[Gettr] Improve extractor (#2920 ) Authored by: i6t	2022-03-04 05:53:43 -08:00
Jeff Huffman	97bef011ee	[crunchyroll] Better error reporting on login failure (#2938 ) Authored by: tejing1	2022-03-04 03:27:35 -08:00
Ha Tien Loi	ecca4519b7	[zingmp3] Fix extractor (#2889 ) Authored by: hatienl0i261299	2022-03-04 03:22:45 -08:00
Ha Tien Loi	761fba6d22	[daystar] Add extractor (#2924 ) Closes #2887 Authored by; hatienl0i261299	2022-03-04 03:19:57 -08:00
Ha Tien Loi	5bcccbfec3	[telegram] Add extractor (#2922 ) Closes #2910 Authored by: hatienl0i261299	2022-03-04 03:18:46 -08:00
coletdev	ded9f32667	[extractor] Support `--mark-watched` without `_NETRC_MACHINE` (#2939 ) Authored by: coletdjnz	2022-03-03 23:27:09 -08:00
Emanuel Hoogeveen	45806d44a7	[downloader] Obey `--file-access-retries` when deleting/renaming (#2224 ) Authored by: ehoogeveen-medweb	2022-03-03 06:33:32 -08:00
pukkandan	747c0bd127	[utils] Improve file locking * Implement non-blocking locks for windows * Don't raise error when closing a closed file	2022-03-03 19:28:47 +05:30
Justin Keogh	acea8d7cfb	[utils] Fix file locking for AOSP (#2714 ) Closes #2080, #2670 Authored by: jakeogh	2022-03-03 05:09:00 -08:00
pukkandan	f1d130902b	[utils] OnDemandPagedList: Do not download pages after error	2022-03-03 02:43:16 +05:30
pukkandan	c2ae48dbd5	[manyvids] Extract `uploader` (#2913 ) Authored by: regarten	2022-03-03 01:21:05 +05:30
pukkandan	a5c0c20252	[cleanup] Don't pass protocol to `_extract_m3u8_formats` for live videos `live` parameter already handles changing the protocol	2022-03-02 22:59:48 +05:30
Lesmiscore (Naoya Ozaki)	f494ddada8	[niconico] Add playlist extractors and refactor (#2915 ) Authored by: Lesmiscore	2022-03-03 02:08:13 +09:00
Lesmiscore (Naoya Ozaki)	02fc6feb6e	[mirrativ] Cleanup extractor code (#2925 ) Authored by: Lesmiscore	2022-03-03 02:06:34 +09:00
pukkandan	7eaf7f9aba	[rokfin] Add stack and channel extractors (#1534 ) Authored by: P-reducible, pukkandan	2022-03-02 21:39:08 +05:30
pukkandan	334b1c4800	[rokfin] Add extractor (#1534 ) Authored by: P-reducible, pukkandan	2022-03-02 19:27:34 +05:30
coletdev	7c219ea601	[youtube:tab] Follow redirect to regional channel (#2884 ) Closes #2694 Authored by: coletdjnz	2022-02-28 21:08:19 -08:00
Lesmiscore (Naoya Ozaki)	93c8410d33	[downloader/fragment] Fix bugs around resuming with Range (#2901 ) Authored by: Lesmiscore	2022-02-28 13:10:54 +09:00
Lesmiscore	195c22840c	[downloader/fragment] Ignore `FileNotFoundError` when downloading livestreams when `--live-from-start` is used for YouTube and the live ends, request for the last segment prematurely ends (or 404, 403). this is causing lack of the file and `FileNotFoundError` lacking segment doesn't have any data, so it's safe to ignore	2022-02-26 12:34:36 +09:00
Lesmiscore	f0734e1190	[downloader/fragment] Fix problem where multiple threads can share one iterator which causes "ValueError: generator already executing" error Closes #2881	2022-02-25 13:22:17 +09:00
Lesmiscore (Naoya Ozaki)	15dfb3929c	[fc2:live] Add extractor (#2418 ) Authored by: Lesmiscore	2022-02-25 11:16:23 +09:00
Lesmiscore (Naoya Ozaki)	3e9b66d761	[AbemaTV] Add extractors (#1688 ) Authored by: Lesmiscore	2022-02-25 11:14:04 +09:00
Lesmiscore (Naoya Ozaki)	a539f06570	[downloader/fragment] Improve `--live-from-start` for YouTube livestreams (#2870 )	2022-02-25 02:00:46 +09:00
pukkandan	b440e1bb22	[devscripts] Improve `prepare_manpage` Closes #2873	2022-02-24 17:02:52 +05:30
Lesmiscore (Naoya Ozaki)	03f830040a	[YoutubeDL] Fill more fields for playlists (#2824 )	2022-02-24 18:42:53 +09:00
pukkandan	09b49e1f68	Add pre-processor stage `after_filter` * Move `_match_entry` and `post_extract` to `process_video_result`. It is also left in `process_info` for API compat * `--list-...` options and `--force-write-archive` now obey filtering options * Move `SponsorBlockPP` to `after_filter`. Closes https://github.com/yt-dlp/yt-dlp/issues/2536 * Reverts `4ec82a72bb` since this commit addresses the issue it was solving	2022-02-23 04:26:48 +05:30
pukkandan	1108613f02	[youtube:tab] Reject webpage data if redirected to home page Closes #2660	2022-02-23 04:25:55 +05:30
pukkandan	a30a6ed3e4	[youtube:tab] Add `approximate_date` extractor-arg	2022-02-23 04:25:55 +05:30
pukkandan	65d151d58f	[spiegel] Fix `_VALID_URL` Closes #2842	2022-02-23 04:25:48 +05:30
pukkandan	72073451be	[ThumbnailsConvertor] Support `webp` Closes #2226	2022-02-23 03:51:13 +05:30
Lesmiscore (Naoya Ozaki)	77cc7c6e60	[nhk] Add support for NHK for School (#2850 ) Authored by: Lesmiscore	2022-02-23 01:15:08 +09:00
i6t	971c4847d7	[Gettr] Fix formats order (#2832 ) Closes #2557 Authored by: i6t	2022-02-22 06:24:36 -08:00
Nil Admirari	7a34b5d628	[SponsorBlock] Fixes for highlight and "full video labels" (#2849 ) Authored by: nihil-admirari	2022-02-22 06:18:44 -08:00
Aniruddh Joshi	4d4f9a029f	[zee5] Support web-series (#2827 ) Authored by: Aniruddh-J	2022-02-21 00:07:36 -08:00
Lesmiscore (Naoya Ozaki)	f099df1463	[TwitCasting] Check for password protection (#2838 ) Authored by: Lesmiscore	2022-02-20 20:48:26 +09:00
pukkandan	3f4faff748	[generic] Pass referer to extracted formats Closes #2839	2022-02-20 17:14:31 +05:30
Daniel.Zeng	be8d623455	[Bilibili] Pass referer for all formats (#2834 ) Authored by: blackgear	2022-02-20 03:27:02 -08:00
Lesmiscore	a7d4acc018	[youtube] Escape possible `$` in `_extract_n_function_name` regex	2022-02-20 17:33:58 +09:00
Bepis	febff4c119	[tubitv] Fix/improve TV series extraction (#2829 ) Authored by: bbepis	2022-02-19 04:00:51 -08:00
pukkandan	ed66a17ef0	[FFmpegConcat] Abort on `--simulate`	2022-02-18 23:17:37 +05:30
Bricio	5625e6073f	[Biqle] Fix extractor (#2731 ) Closes #193 Authored by: Bricio	2022-02-18 08:02:14 -08:00
pukkandan	0ad92dfb18	[youtube] De-prioritize potentially damaged formats Closes #2823	2022-02-18 19:41:37 +05:30
pukkandan	60f3e99592	Tolerate failure to `--write-link` due to unknown URL Closes #2724	2022-02-18 18:14:50 +05:30
pukkandan	8d93e69d67	Create necessary directories for `--print-to-file` Closes #2721	2022-02-18 18:03:21 +05:30
pukkandan	3aa915400d	Fix `-all` for `--sub-langs` Closes #2703	2022-02-18 18:03:20 +05:30
pukkandan	dcd55f766d	[aria2c] Add `--http-accept-gzip=true` Closes #1936, #1236	2022-02-18 18:03:20 +05:30
pukkandan	2e4cacd038	[youtube] Fix intermittent failure of embed-based age-gate bypass	2022-02-18 18:03:13 +05:30
Ronnnny	c15c316b21	[abc] Support 1080p (#2819 ) Authored by: Ronnnny	2022-02-18 00:25:47 -08:00
Bricio	549cb2a836	[rtvs] Fix extractor (#2795 ) Closes #2758 Authored by: Bricio	2022-02-18 00:15:17 -08:00
MinePlayersPE	c571b3a6ab	[youtube] Fix n-sig extraction for phone player JS (#2816 ) Authored by: MinePlayersPE	2022-02-18 00:12:20 -08:00
Bricio	5b804e3906	[washingtonpost] Fix extractor (#2796 ) Closes #2778 Authored by: Bricio	2022-02-17 09:38:58 -08:00
Lesmiscore (Naoya Ozaki)	6bb608d055	[piapro] Add extractor (#2801 ) Based on https://github.com/ytdl-org/youtube-dl/pull/25922 Closes #2710, https://github.com/ytdl-org/youtube-dl/issues/5856 Authored by: pycabbage, Lesmiscore	2022-02-17 09:15:29 -08:00
Nil Admirari	ae419aa94f	[Sponsorblock] minor fixes (#2793 ) * preserve mtime - Closes #2769 * keep concat spec on failure Authored by: nihil-admirari	2022-02-17 09:10:34 -08:00
ajj8	ac184ab742	[bbc] Fix extraction of news articles (#2811 ) Closes #1374 Authored by: ajj8	2022-02-17 07:54:53 -08:00
pukkandan	5c10453827	Fix for when stdout/stderr encoding is None Closes #2711	2022-02-17 19:21:59 +05:30
pukkandan	ffa89477ea	[extractor] Fix for manifests without period duration Closes #2705 Authored by: dirkf, pukkandan	2022-02-17 19:07:23 +05:30
zenerdi0de	db74de8c54	[dropbox] fix regex (#2814 ) Closes #2812 Authored by: zenerdi0de	2022-02-17 04:20:47 -08:00
Grabien	edecb5f81f	[extractor/cspan] Support of C-Span congress videos (#2295 ) Authored by: Grabien	2022-02-16 11:21:05 -08:00
lyz-code	85a0ad0117	[bandcamp] Fix user URLs (#2800 ) Authored by: lyz-code	2022-02-16 07:56:17 -08:00
Lesmiscore (Naoya Ozaki)	07ea0014ae	[twitcasting] Add fallback for finding running live (#2803 ) Authored by: Lesmiscore	2022-02-16 20:32:14 +09:00
schn0sch	e1f7f235bd	[peekvids] Use JSON-LD (#2784 ) Authored by: schn0sch	2022-02-16 01:32:24 -08:00
shirt	fc259cc249	[build] Update pyinstaller to 4.9	2022-02-15 17:48:02 -05:00
Lesmiscore (Naoya Ozaki)	9a5b012575	[niconico:tag] Add support for searching tags (#2789 )	2022-02-16 02:12:39 +09:00
Lesmiscore (Naoya Ozaki)	df635a09a4	[twitcasting] Fix extraction (#2788 ) Authored by: Lesmiscore	2022-02-15 23:30:11 +09:00
cyberfox1691	812283199a	[murrtube] Add extractor (#2387 ) Authored by: cyberfox1691	2022-02-15 03:10:16 -08:00
marieell	5c6dfc1f79	[ATVAt] Detect geo-restriction (#2777 ) Authored by: marieell	2022-02-15 01:16:49 -08:00
schn0sch	c2a8547fdc	[peekvids] Add extractor (#2414 ) Authored by: schn0sch	2022-02-14 19:21:27 -08:00
Bricio	0a19532ead	[Caltrans] Add extractor (#2781 ) Closes #2775 Authored by: Bricio	2022-02-14 18:45:36 -08:00
Ronald Ip	2d41e2eceb	[twitter] Fix for private videos (#2772 ) Closes #2762, https://github.com/ytdl-org/youtube-dl/issues/27643 Authored by: iphoting	2022-02-14 08:37:21 -08:00
Lesmiscore (Naoya Ozaki)	81c5f44c0f	[fc2] Fix extraction (#2776 ) Closes #2774 Authored by: Lesmiscore	2022-02-15 01:35:20 +09:00
Michael Pauley	1f7db8533a	[cookies] Update MacOS12 `Cookies.binarycookies` location (#2742 ) Authored by: mdpauley	2022-02-14 06:36:51 -08:00
pukkandan	e8969bda94	Obey `--abort-on-error` for "ffmpeg not installed" Closes #1840	2022-02-14 14:40:19 +05:30
chris	c82f051dbb	[ruv.is] Add extractor (#2665 ) Closes: #2122 Authored by: iw0nderhow	2022-02-13 13:40:50 -08:00
pukkandan	49895f062e	[tiktok] Fix `vt.tiktok` URLs and add test	2022-02-14 03:06:51 +05:30
coletdev	60f393e48b	[youtube] Ensure subtitle urls are absolute (#2765 ) Closes #2755 Authored by: coletdjnz	2022-02-13 13:36:01 -08:00
pukkandan	88afe05695	[tiktok] Fix `vm.tiktok` URLs Closes #2396	2022-02-13 21:15:59 +05:30
pukkandan	57ebfca39b	Set `webpage_url_...` from `webpage_url` and not input URL Closes #2756	2022-02-13 21:15:50 +05:30
YuenSzeHong	b1cb0525ac	[fujitv] Extract resolution for free sources (#2685 ) Authored by: YuenSzeHong	2022-02-13 06:39:01 -08:00
Lesmiscore (Naoya Ozaki)	da42679b87	[utils] WebSockets wrapper for non-async functions (#2417 ) Authored by: Lesmiscore	2022-02-13 14:58:21 +09:00
Lesmiscore	2944835080	[bigo] Fix extractor to not to use `form_params`	2022-02-13 00:01:04 +09:00
Tom	a3eb987e0e	[zoom] Add support for screen cast (#2699 ) Authored by: Mipsters	2022-02-12 06:22:51 -08:00
Lesmiscore (Naoya Ozaki)	7bc33ad0e9	[bigo] Add extractor (#2749 ) Fixes https://github.com/ytdl-org/youtube-dl/issues/18357 Authored by: Lesmiscore	2022-02-12 06:07:10 -08:00
Bricio	2068a60318	[generic] Set rss `guid` as video id (#2741 ) Closes #2424 Authored by: Bricio	2022-02-11 15:32:58 -08:00
Lukas Fink	1ce9a3cb49	Add regex operator and quoting to format filters (#2698 ) Closes #2681 Authored by: lukasfink1	2022-02-11 13:35:34 -08:00
pukkandan	d49f8db39f	[utils] Validate `DateRange` input Closes #2641	2022-02-12 02:46:05 +05:30
pukkandan	ab6df717d1	[youtube] Differentiate descriptive audio by language code Related: #2677	2022-02-12 02:13:17 +05:30
pukkandan	0c8d9e5fec	[youtube] Label original auto-subs Closes #2655	2022-02-12 01:50:49 +05:30
Felix S	3f047fc406	[extractor] Extract subtitles from manifests for more sites (#2686 ) vimeo, globo, kaltura, svt Authored by: fstirlitz	2022-02-11 11:03:33 -08:00
i6t	82b5176783	[Gettr] Add `GettrStreamingIE` (#2661 ) Closes #2654 Authored by: i6t	2022-02-11 10:36:16 -08:00
Bricio	17b183886f	[globo] Expand valid URL (#2732 ) Closes #2730 Authored by: Bricio	2022-02-11 10:08:55 -08:00
Bricio	cd170e8184	[beeg] Fix extractor (#2616 ) Closes #2592 Authored by: Bricio	2022-02-11 10:05:23 -08:00
pukkandan	297e9952b6	[extractor] Allow `http_headers` to be specified for `thumbnails`	2022-02-11 23:31:12 +05:30
marieell	dca4f46274	[cleanup] Remove extractors for some dead websites (#2739 ) 90tv.ir, HornBunny.com, 220.ro, 5min.com, Kankan.com, Roxwel.com, FreshLive.tv, TheScene.com, Vube.com Authored by: marieell	2022-02-11 09:46:29 -08:00
Luc Ritchie	5dee3ad037	[afreecatv] Support password-protected livestreams (#2738 ) Authored by: wlritchi	2022-02-11 06:15:59 -08:00
pukkandan	079a7cfc71	[downloader] Do not use aria2c for non-native `m3u8` Closes #2718	2022-02-11 12:09:03 +05:30
pukkandan	3856407a86	[options] Rename `--clean-infojson` to `--clean-info-json`	2022-02-11 12:07:10 +05:30
pukkandan	db2e129ca0	[options] Better ambiguous option resolution Eg: `--write-auto` no longer results in > ambiguous option: --write-auto (--write-auto-subs, --write-automatic-subs?)	2022-02-11 12:07:03 +05:30
marieell	1209b6ca5b	[zaq1] Remove dead extractor (#2728 ) Was already partially removed in `29f7c58aaf` Authored-by: marieell	2022-02-11 02:15:38 +00:00
Justin Keogh	a3125791c7	[utils] Use `locked_file` for `sanitize_open` (#1066 ) Authored by: jakeogh	2022-02-05 16:15:51 +05:30
ofkz	f1657a98cb	[nfb] Add extractor (#2579 ) Authored by: ofkz	2022-02-05 02:52:30 +05:30
github-actions	b761428226	[version] update Created by: pukkandan :ci skip all	2022-02-04 06:39:10 +00:00
pukkandan	c1653e9efb	Release 2022.02.04	2022-02-04 12:03:56 +05:30
pukkandan	84bbc54599	[youtube:search] Add tests	2022-02-04 12:02:01 +05:30
pukkandan	1e5d87beee	[websocket] Make syntax error in `websockets` module non-fatal Closes #2633	2022-02-04 12:02:01 +05:30
nixxo	22219f2d1f	[mediaset] Fix extractor (#2158 ) Closes #2149 Authored by: nixxo	2022-02-04 11:29:38 +05:30
Lesmiscore (Naoya Ozaki)	5a13fdd225	[twitcasting] Enforce UTF-8 for POST payload (#2521 ) Authored by: Lesmiscore	2022-02-04 11:24:33 +05:30
coletdjnz	af5c1c553e	[youtube] Fix search extractor Regression introduced in `16aa9ea41d`. Closes #2628 Authored-by: coletdjnz	2022-02-04 10:32:56 +13:00