Release 2021.07.21

and fix some typos
Closes #538

.gitattributes

@@ -1 +1,4 @@
+* text=auto
+
 Makefile* text whitespace=-tab-in-indent
+*.sh text eol=lf

.github/ISSUE_TEMPLATE/1_broken_site.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support
-- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -42,9 +42,9 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.04.03
+ [debug] yt-dlp version 2021.07.07
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/2_site_support_request.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a new site support request
-- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones

.github/ISSUE_TEMPLATE/3_site_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a site feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE/4_bug_report.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support issue
-- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -44,9 +44,9 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.04.03
+ [debug] yt-dlp version 2021.07.07
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/5_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE_tmpl/1_broken_site.md

@@ -42,7 +42,7 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
  [debug] yt-dlp version %(version)s
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2

.github/ISSUE_TEMPLATE_tmpl/4_bug_report.md

@@ -44,7 +44,7 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
  [debug] yt-dlp version %(version)s
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2

.github/workflows/build.yml

@@ -7,13 +7,13 @@ on:
 
 jobs:
   build_unix:
-
     runs-on: ubuntu-latest
 
     outputs:
       ytdlp_version: ${{ steps.bump_version.outputs.ytdlp_version }}
       upload_url: ${{ steps.create_release.outputs.upload_url }}
-      sha2_unix: ${{ steps.sha2_file.outputs.sha2_unix }}
+      sha256_unix: ${{ steps.sha256_file.outputs.sha256_unix }}
+      sha512_unix: ${{ steps.sha512_file.outputs.sha512_unix }}
 
     steps:
     - uses: actions/checkout@v2
@@ -29,7 +29,7 @@ jobs:
     - name: Print version
       run: echo "${{ steps.bump_version.outputs.ytdlp_version }}"
     - name: Run Make
-      run: make
+      run: make all tar
     - name: Create Release
       id: create_release
       uses: actions/create-release@v1
@@ -44,7 +44,7 @@ jobs:
         draft: false
         prerelease: false
     - name: Upload yt-dlp Unix binary
-      id: upload-release-asset 
+      id: upload-release-asset
       uses: actions/upload-release-asset@v1
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -53,9 +53,21 @@ jobs:
         asset_path: ./yt-dlp
         asset_name: yt-dlp
         asset_content_type: application/octet-stream
+    - name: Upload Source tar
+      uses: actions/upload-release-asset@v1
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./yt-dlp.tar.gz
+        asset_name: yt-dlp.tar.gz
+        asset_content_type: application/gzip
     - name: Get SHA2-256SUMS for yt-dlp
-      id: sha2_file
-      run: echo "::set-output name=sha2_unix::$(sha256sum yt-dlp | awk '{print $1}')"
+      id: sha256_file
+      run: echo "::set-output name=sha256_unix::$(sha256sum yt-dlp | awk '{print $1}')"
+    - name: Get SHA2-512SUMS for yt-dlp
+      id: sha512_file
+      run: echo "::set-output name=sha512_unix::$(sha512sum yt-dlp | awk '{print $1}')"
     - name: Install dependencies for pypi
       env:
         PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
@@ -74,24 +86,24 @@ jobs:
         twine upload dist/*
 
   build_windows:
-
     runs-on: windows-latest
+    needs: build_unix
 
     outputs:
-      sha2_windows: ${{ steps.sha2_file_win.outputs.sha2_windows }}
-
-    needs: build_unix
+      sha256_windows: ${{ steps.sha256_file_win.outputs.sha256_windows }}
+      sha512_windows: ${{ steps.sha512_file_win.outputs.sha512_windows }}
 
     steps:
     - uses: actions/checkout@v2
-    - name: Set up Python
+    # 3.8 is used for Win7 support
+    - name: Set up Python 3.8
       uses: actions/setup-python@v2
       with:
           python-version: '3.8'
     - name: Upgrade pip and enable wheel support
       run: python -m pip install --upgrade pip setuptools wheel
     - name: Install Requirements
-      run: pip install pyinstaller mutagen pycryptodome
+      run: pip install pyinstaller mutagen pycryptodome websockets
     - name: Bump version
       id: bump_version
       run: python devscripts/update-version.py
@@ -110,29 +122,32 @@ jobs:
         asset_name: yt-dlp.exe
         asset_content_type: application/vnd.microsoft.portable-executable
     - name: Get SHA2-256SUMS for yt-dlp.exe
-      id: sha2_file_win
-      run: echo "::set-output name=sha2_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())"
+      id: sha256_file_win
+      run: echo "::set-output name=sha256_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())"
+    - name: Get SHA2-512SUMS for yt-dlp.exe
+      id: sha512_file_win
+      run: echo "::set-output name=sha512_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA512).Hash.ToLower())"
 
   build_windows32:
-
     runs-on: windows-latest
+    needs: [build_unix, build_windows]
 
     outputs:
-      sha2_windows32: ${{ steps.sha2_file_win32.outputs.sha2_windows32 }}
-
-    needs: [build_unix, build_windows]
+      sha256_windows32: ${{ steps.sha256_file_win32.outputs.sha256_windows32 }}
+      sha512_windows32: ${{ steps.sha512_file_win32.outputs.sha512_windows32 }}
 
     steps:
     - uses: actions/checkout@v2
-    - name: Set up Python 3.4.4 32-Bit
+    # 3.7 is used for Vista support. See https://github.com/yt-dlp/yt-dlp/issues/390
+    - name: Set up Python 3.7 32-Bit
       uses: actions/setup-python@v2
       with:
-          python-version: '3.4.4'
+          python-version: '3.7'
           architecture: 'x86'
     - name: Upgrade pip and enable wheel support
-      run: python -m pip install pip==19.1.1 setuptools==43.0.0 wheel==0.33.6
-    - name: Install Requirements for 32 Bit
-      run: pip install pyinstaller==3.5 mutagen==1.42.0 pycryptodome==3.9.4
+      run: python -m pip install --upgrade pip setuptools wheel
+    - name: Install Requirements
+      run: pip install pyinstaller mutagen pycryptodome websockets
     - name: Bump version
       id: bump_version
       run: python devscripts/update-version.py
@@ -151,20 +166,28 @@ jobs:
         asset_name: yt-dlp_x86.exe
         asset_content_type: application/vnd.microsoft.portable-executable
     - name: Get SHA2-256SUMS for yt-dlp_x86.exe
-      id: sha2_file_win32
-      run: echo "::set-output name=sha2_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())"
+      id: sha256_file_win32
+      run: echo "::set-output name=sha256_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())"
+    - name: Get SHA2-512SUMS for yt-dlp_x86.exe
+      id: sha512_file_win32
+      run: echo "::set-output name=sha512_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA512).Hash.ToLower())"
+
+  finish:
+    runs-on: ubuntu-latest
+    needs: [build_unix, build_windows, build_windows32]
+
+    steps:
     - name: Make SHA2-256SUMS file
       env:
-        SHA2_WINDOWS: ${{ needs.build_windows.outputs.sha2_windows }}
-        SHA2_WINDOWS32: ${{ steps.sha2_file_win32.outputs.sha2_windows32 }}
-        SHA2_UNIX: ${{ needs.build_unix.outputs.sha2_unix }}
+        SHA256_WINDOWS: ${{ needs.build_windows.outputs.sha256_windows }}
+        SHA256_WINDOWS32: ${{ needs.build_windows32.outputs.sha256_windows32 }}
+        SHA256_UNIX: ${{ needs.build_unix.outputs.sha256_unix }}
         YTDLP_VERSION: ${{ needs.build_unix.outputs.ytdlp_version }}
       run: |
-        echo "version:${env:YTDLP_VERSION}" >> SHA2-256SUMS
-        echo "yt-dlp.exe:${env:SHA2_WINDOWS}" >> SHA2-256SUMS
-        echo "yt-dlp_x86.exe:${env:SHA2_WINDOWS32}" >> SHA2-256SUMS
-        echo "yt-dlp:${env:SHA2_UNIX}" >> SHA2-256SUMS
-
+        echo "version:${{ env.YTDLP_VERSION }}" >> SHA2-256SUMS
+        echo "yt-dlp.exe:${{ env.SHA256_WINDOWS }}" >> SHA2-256SUMS
+        echo "yt-dlp_x86.exe:${{ env.SHA256_WINDOWS32 }}" >> SHA2-256SUMS
+        echo "yt-dlp:${{ env.SHA256_UNIX }}" >> SHA2-256SUMS
     - name: Upload 256SUMS file
       id: upload-sums
       uses: actions/upload-release-asset@v1
@@ -175,3 +198,22 @@ jobs:
         asset_path: ./SHA2-256SUMS
         asset_name: SHA2-256SUMS
         asset_content_type: text/plain
+    - name: Make SHA2-512SUMS file
+      env:
+        SHA512_WINDOWS: ${{ needs.build_windows.outputs.sha512_windows }}
+        SHA512_WINDOWS32: ${{ needs.build_windows32.outputs.sha512_windows32 }}
+        SHA512_UNIX: ${{ needs.build_unix.outputs.sha512_unix }}
+      run: |
+        echo "${{ env.SHA512_WINDOWS }}  yt-dlp.exe" >> SHA2-512SUMS
+        echo "${{ env.SHA512_WINDOWS32 }}  yt-dlp_x86.exe" >> SHA2-512SUMS
+        echo "${{ env.SHA512_UNIX }}  yt-dlp" >> SHA2-512SUMS
+    - name: Upload 512SUMS file
+      id: upload-512sums
+      uses: actions/upload-release-asset@v1
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        upload_url: ${{ needs.build_unix.outputs.upload_url }}
+        asset_path: ./SHA2-512SUMS
+        asset_name: SHA2-512SUMS
+        asset_content_type: text/plain

.github/workflows/core.yml

@@ -6,49 +6,28 @@ jobs:
     if: "!contains(github.event.head_commit.message, 'ci skip')"
     runs-on: ${{ matrix.os }}
     strategy:
-      fail-fast: true
+      fail-fast: false
       matrix:
         os: [ubuntu-18.04]
-        # TODO: python 2.6
-        python-version: [2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, pypy-2.7, pypy-3.6, pypy-3.7]
-        python-impl: [cpython]
-        ytdl-test-set: [core]
+        # py3.9 is in quick-test
+        python-version: [3.7, 3.8, pypy-3.6, pypy-3.7]
         run-tests-ext: [sh]
         include:
-        # python 3.2 is only available on windows via setup-python
+        # atleast one of the tests must be in windows
         - os: windows-latest
-          python-version: 3.2
-          python-impl: cpython
-          ytdl-test-set: core
+          python-version: 3.6
           run-tests-ext: bat
-        # jython
-        - os: ubuntu-latest
-          python-impl: jython
-          ytdl-test-set: core
-          run-tests-ext: sh
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
-      if: ${{ matrix.python-impl == 'cpython' }}
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Set up Java 8
-      if: ${{ matrix.python-impl == 'jython' }}
-      uses: actions/setup-java@v1
-      with:
-        java-version: 8
-    - name: Install Jython
-      if: ${{ matrix.python-impl == 'jython' }}
-      run: |
-        wget http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar -O jython-installer.jar
-        java -jar jython-installer.jar -s -d "$HOME/jython"
-        echo "$HOME/jython/bin" >> $GITHUB_PATH
     - name: Install nose
       run: pip install nose
     - name: Run tests
-      continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
+      continue-on-error: False
       env:
-        YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
+        YTDL_TEST_SET: core
       run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}
   # Linter is in quick-test

.github/workflows/download.yml

@@ -9,45 +9,22 @@ jobs:
       fail-fast: true
       matrix:
         os: [ubuntu-18.04]
-        # TODO: python 2.6
-        python-version: [2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, pypy-2.7, pypy-3.6, pypy-3.7]
-        python-impl: [cpython]
-        ytdl-test-set: [download]
+        python-version: [3.7, 3.8, 3.9, pypy-3.6, pypy-3.7]
         run-tests-ext: [sh]
         include:
-        # python 3.2 is only available on windows via setup-python
         - os: windows-latest
-          python-version: 3.2
-          python-impl: cpython
-          ytdl-test-set: download
+          python-version: 3.6
           run-tests-ext: bat
-        # jython - disable for now since it takes too long to complete
-        # - os: ubuntu-latest
-        #   python-impl: jython
-        #   ytdl-test-set: download
-        #   run-tests-ext: sh
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
-      if: ${{ matrix.python-impl == 'cpython' }}
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Set up Java 8
-      if: ${{ matrix.python-impl == 'jython' }}
-      uses: actions/setup-java@v1
-      with:
-        java-version: 8
-    - name: Install Jython
-      if: ${{ matrix.python-impl == 'jython' }}
-      run: |
-        wget http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar -O jython-installer.jar
-        java -jar jython-installer.jar -s -d "$HOME/jython"
-        echo "$HOME/jython/bin" >> $GITHUB_PATH
     - name: Install nose
       run: pip install nose
     - name: Run tests
-      continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
+      continue-on-error: true
       env:
-        YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
+        YTDL_TEST_SET: download
       run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}

.github/workflows/quick-test.yml

@@ -11,8 +11,8 @@ jobs:
       uses: actions/setup-python@v2
       with:
         python-version: 3.9
-    - name: Install nose
-      run: pip install nose
+    - name: Install test requirements
+      run: pip install nose pycryptodome
     - name: Run tests
       env:
         YTDL_TEST_SET: core

.gitignore

@@ -1,3 +1,47 @@
+# Config
+*.conf
+*.spec
+cookies
+cookies.txt
+
+# Downloaded
+*.srt
+*.ttml
+*.sbv
+*.vtt
+*.flv
+*.mp4
+*.m4a
+*.m4v
+*.mp3
+*.3gp
+*.webm
+*.wav
+*.ape
+*.mkv
+*.swf
+*.part
+*.part-*
+*.ytdl
+*.dump
+*.frag
+*.frag.urls
+*.aria2
+*.swp
+*.ogg
+*.opus
+*.info.json
+*.live_chat.json
+*.jpg
+*.jpeg
+*.png
+*.webp
+*.annotations.xml
+*.description
+
+# Allow config/media files in testdata
+!test/testdata/**
+
 # Python
 *.pyc
 *.pyo
@@ -43,48 +87,6 @@ README.txt
 yt-dlp.zip
 *.exe
 
-# Downloaded
-*.srt
-*.ttml
-*.sbv
-*.vtt
-*.flv
-*.mp4
-*.m4a
-*.m4v
-*.mp3
-*.3gp
-*.webm
-*.wav
-*.ape
-*.mkv
-*.swf
-*.part
-*.part-*
-*.ytdl
-*.dump
-*.frag
-*.frag.urls
-*.aria2
-*.swp
-*.ogg
-*.opus
-*.info.json
-*.live_chat.json
-*.jpg
-*.png
-*.webp
-*.annotations.xml
-*.description
-
-# Config
-*.conf
-*.spec
-cookies
-cookies.txt
-
-
-
 # Text Editor / IDE
 .idea
 *.iml

CONTRIBUTING.md

@@ -3,7 +3,7 @@
 $ youtube-dl -v <your command line>
 [debug] System config: []
 [debug] User config: []
-[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj']
+[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] youtube-dl version 2015.12.06
 [debug] Git HEAD: 135392e

CONTRIBUTORS

@@ -1,6 +1,7 @@
 pukkandan (owner)
 shirt-dev (collaborator)
 colethedj (collaborator)
+Ashish0804 (collaborator)
 h-h-h-h
 pauldubois98
 nixxo
@@ -20,11 +21,9 @@ FelixFrog
 Zocker1999NET
 nao20010128nao
 kurumigi
-tsukumi
 bbepis
 animelover1984
 Pccode66
-Ashish0804
 RobinD42
 hseg
 DennyDai
@@ -37,4 +36,30 @@ CXwudi
 xtkoba
 llacb47
 hheimbuerger
-B0pol
+B0pol
+lkho
+fstirlitz
+Lamieur
+tsukumijima
+Hadi0609
+b5eff52
+craftingmod
+tpikonen
+tripulse
+king-millez
+alex-gedeon
+hhirtz
+louie-github
+MinePlayersPE
+olifre
+rhsmachine/zenerdi0de
+nihil-admirari
+krichbanana
+ohmybahgosh
+nyuszika7h
+blackjack4494
+pyx
+TpmKranz
+mzbaulhaque
+zackmark29
+mbway

Changelog.md

@@ -10,7 +10,7 @@
 * Commit to master as `Release <version>`
 * Push to origin/release using `git push origin master:release`
     build task will now run
-* Update version.py using devscripts\update-version.py
+* Update version.py using `devscripts\update-version.py`
 * Run `make issuetemplates`
 * Commit to master as `[version] update :ci skip all`
 * Push to origin/master
@@ -18,6 +18,360 @@
 
 -->
 
+### 2021.07.21
+
+* **Add option `--cookies-from-browser`** to load cookies from a browser by [mbway](https://github.com/mbway)
+    * Usage: `--cookies-from-browser BROWSER[:PROFILE_NAME_OR_PATH]`
+    * Also added `--no-cookies-from-browser`
+    * To decrypt chromium cookies, `keyring` is needed for UNIX and `pycryptodome` for Windows
+* Add option `--exec-before-download`
+* Add field `live_status`
+* [FFmpegMetadata] Add language of each stream and some refactoring
+* [douyin] Add extractor by [pukkandan](https://github.com/pukkandan), [pyx](https://github.com/pyx)
+* [pornflip] Add extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
+* **[youtube] Extract data from multiple clients** by [pukkandan](https://github.com/pukkandan), [colethedj](https://github.com/colethedj)
+    * `player_client` now accepts multiple clients
+    * Default `player_client` = `android,web`
+        * This uses twice as many requests, but avoids throttling for most videos while also not losing any formats
+    * Music clients can be specifically requested and is enabled by default if `music.youtube.com`
+    * Added `player_client=ios` (Known issue: formats from ios are not sorted correctly)
+    * Add age-gate bypass for android and ios clients
+* [youtube] Extract more thumbnails
+    * The thumbnail URLs are hard-coded and their actual existence is tested lazily
+    * Added option `--no-check-formats` to not test them
+* [youtube] Misc fixes
+    * Improve extraction of livestream metadata by [pukkandan](https://github.com/pukkandan), [krichbanana](https://github.com/krichbanana)
+    * Hide live dash formats since they can't be downloaded anyway
+    * Fix authentication when using multiple accounts by [colethedj](https://github.com/colethedj)
+    * Fix controversial videos when requested via API by [colethedj](https://github.com/colethedj)
+    * Fix session index extraction and headers for non-web player clients by [colethedj](https://github.com/colethedj)
+    * Make `--extractor-retries` work for more errors
+    * Fix sorting of 3gp format
+    * Sanity check `chapters` (and refactor related code)
+    * Make `parse_time_text` and `_extract_chapters` non-fatal
+    * Misc cleanup and bug fixes by [colethedj](https://github.com/colethedj)
+* [youtube:tab] Fix channels tab
+* [youtube:tab] Extract playlist availability by [colethedj](https://github.com/colethedj)
+* **[youtube:comments] Move comment extraction to new API** by [colethedj](https://github.com/colethedj)
+* [youtube:comments] Fix `is_favorited`, improve `like_count` parsing by [colethedj](https://github.com/colethedj)
+* [BravoTV] Improve metadata extraction by [kevinoconnor7](https://github.com/kevinoconnor7)
+* [crunchyroll:playlist] Force http
+* [yahoo:gyao:player] Relax `_VALID_URL` by [nao20010128nao](https://github.com/nao20010128nao)
+* [nebula] Authentication via tokens from cookie jar by [hheimbuerger](https://github.com/hheimbuerger), [TpmKranz](https://github.com/TpmKranz)
+* [RTP] Fix extraction and add subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [viki] Rewrite extractors and add extractor-arg `video_types` to `vikichannel` by [zackmark29](https://github.com/zackmark29), [pukkandan](https://github.com/pukkandan)
+* [vlive] Extract thumbnail directly in addition to the one from Naver
+* [generic] Extract previously missed subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [generic] Extract everything in the SMIL manifest and detect discarded subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [embedthumbnail] Fix `_get_thumbnail_resolution`
+* [metadatafromfield] Do not detect numbers as field names
+* Fix selectors `all`, `mergeall` and add tests
+* Errors in playlist extraction should obey `--ignore-errors`
+* Fix bug where `original_url` was not propagated when `_type`=`url`
+* Revert "Merge webm formats into mkv if thumbnails are to be embedded (#173)"
+    * This was wrongly checking for `write_thumbnail`
+* Improve `extractor_args` parsing
+* Rename `NOTE` in `-F` to `MORE INFO` since it's often confused to be the same as `format_note`
+* Add `only_once` param for `write_debug` and `report_warning`
+* [extractor] Allow extracting multiple groups in `_search_regex` by [fstirlitz](https://github.com/fstirlitz)
+* [utils] Improve `traverse_obj`
+* [utils] Add `variadic`
+* [utils] Improve `js_to_json` comment regex by [fstirlitz](https://github.com/fstirlitz)
+* [webtt] Fix timestamps
+* [compat] Remove unnecessary code
+* [doc] fix default of multistreams
+
+
+### 2021.07.07
+
+* Merge youtube-dl: Upto [commit/a803582](https://github.com/ytdl-org/youtube-dl/commit/a8035827177d6b59aca03bd717acb6a9bdd75ada)
+* Add `--extractor-args` to pass some extractor-specific arguments. See [readme](https://github.com/yt-dlp/yt-dlp#extractor-arguments)
+    * Add extractor option `skip` for `youtube`. Eg: `--extractor-args youtube:skip=hls,dash`
+    * Deprecates `--youtube-skip-dash-manifest`, `--youtube-skip-hls-manifest`, `--youtube-include-dash-manifest`, `--youtube-include-hls-manifest`
+* Allow `--list...` options to work with `--print`, `--quiet` and other `--list...` options
+* [youtube] Use `player` API for additional video extraction requests by [colethedj](https://github.com/colethedj)
+    * **Fixes youtube premium music** (format 141) extraction
+    * Adds extractor option `player_client` = `web`/`android`
+        * **`--extractor-args youtube:player_client=android` works around the throttling** for the time-being
+    * Adds extractor option `player_skip=config`
+    * Adds age-gate fallback using embedded client
+* [youtube] Choose correct Live chat API for upcoming streams by [krichbanana](https://github.com/krichbanana)
+* [youtube] Fix subtitle names for age-gated videos
+* [youtube:comments] Fix error handling and add `itct` to params by [colethedj](https://github.com/colethedj)
+* [youtube_live_chat] Fix download with cookies by [siikamiika](https://github.com/siikamiika)
+* [youtube_live_chat] use `clickTrackingParams` by [siikamiika](https://github.com/siikamiika)
+* [Funimation] Rewrite extractor
+    * Add `FunimationShowIE` by [Mevious](https://github.com/Mevious)
+    * **Treat the different versions of an episode as different formats of a single video**
+        * This changes the video `id` and will break break existing archives
+        * Compat option `seperate-video-versions` to fall back to old behavior including using the old video ids
+    * Support direct `/player/` URL
+    * Extractor options `language` and `version` to pre-select them during extraction
+        * These options may be removed in the future if we can extract all formats without additional network requests
+        * Do not rely on these for format selection and use `-f` filters instead
+* [AdobePass] Add Spectrum MSO by [kevinoconnor7](https://github.com/kevinoconnor7), [ohmybahgosh](https://github.com/ohmybahgosh)
+* [facebook] Extract description and fix title
+* [fancode] Fix extraction, support live and allow login with refresh token by [zenerdi0de](https://github.com/zenerdi0de)
+* [plutotv] Improve `_VALID_URL`
+* [RCTIPlus] Add extractor by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [Soundcloud] Allow login using oauth token by [blackjack4494](https://github.com/blackjack4494)
+* [TBS] Support livestreams by [llacb47](https://github.com/llacb47)
+* [videa] Fix extraction by [nyuszika7h](https://github.com/nyuszika7h)
+* [yahoo] Fix extraction by [llacb47](https://github.com/llacb47), [pukkandan](https://github.com/pukkandan)
+* Process videos when using `--ignore-no-formats-error` by [krichbanana](https://github.com/krichbanana)
+* Fix `--throttled-rate` when using `--load-info-json`
+* Fix `--flat-playlist` when entry has no `ie_key`
+* Fix `check_formats` catching `ExtractorError` instead of `DownloadError`
+* Fix deprecated option `--list-formats-old`
+* [downloader/ffmpeg] Fix `--ppa` when using simultaneous download
+* [extractor] Prevent unnecessary download of hls manifests and refactor `hls_split_discontinuity`
+* [fragment] Handle status of download and errors in threads correctly; and minor refactoring
+* [thumbnailsconvertor] Treat `jpeg` as `jpg`
+* [utils] Fix issues with `LazyList` reversal
+* [extractor] Allow extractors to set their own login hint
+* [cleanup] Simplify format selector code with `LazyList` and `yield from`
+* [cleanup] Clean `extractor.common._merge_subtitles` signature
+* [cleanup] Fix some typos
+
+
+### 2021.06.23
+
+* Merge youtube-dl: Upto [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)
+* **Add option `--throttled-rate`** below which video data is re-extracted
+* [fragment] **Merge during download for `-N`**, and refactor `hls`/`dash`
+* [websockets] Add `WebSocketFragmentFD` by [nao20010128nao](https://github.com/nao20010128nao), [pukkandan](https://github.com/pukkandan)
+* Allow `images` formats in addition to video/audio
+* [downloader/mhtml] Add new downloader for slideshows/storyboards by [fstirlitz](https://github.com/fstirlitz)
+* [youtube] Temporary **fix for age-gate**
+* [youtube] Support ongoing live chat by [siikamiika](https://github.com/siikamiika)
+* [youtube] Improve SAPISID cookie handling by [colethedj](https://github.com/colethedj)
+* [youtube] Login is not needed for `:ytrec`
+* [youtube] Non-fatal alert reporting for unavailable videos page by [colethedj](https://github.com/colethedj)
+* [twitcasting] Websocket support by [nao20010128nao](https://github.com/nao20010128nao)
+* [mediasite] Extract slides by [fstirlitz](https://github.com/fstirlitz)
+* [funimation] Extract subtitles 
+* [pornhub] Extract `cast`
+* [hotstar] Use server time for authentication instead of local time
+* [EmbedThumbnail] Fix for already downloaded thumbnail
+* [EmbedThumbnail] Add compat-option `embed-thumbnail-atomicparsley`
+* Expand `--check-formats` to thumbnails
+* Fix id sanitization in filenames
+* Skip fixup of existing files and add `--fixup force` to force it
+* Better error handling of syntax errors in `-f`
+* Use `NamedTemporaryFile` for `--check-formats`
+* [aria2c] Lower `--min-split-size` for HTTP downloads
+* [options] Rename `--add-metadata` to `--embed-metadata`
+* [utils] Improve `LazyList` and add tests
+* [build] Build Windows x86 version with py3.7 and remove redundant tests by [pukkandan](https://github.com/pukkandan), [shirt](https://github.com/shirt-dev)
+* [docs] Clarify that `--embed-metadata` embeds chapter markers
+* [cleanup] Refactor fixup
+
+
+### 2021.06.09
+
+* Fix bug where `%(field)d` in filename template throws error
+* Improve offset parsing in outtmpl
+* [test] More rigorous tests for `prepare_filename`
+
+### 2021.06.08
+
+* Remove support for obsolete Python versions: Only 3.6+ is now supported
+* Merge youtube-dl: Upto [commit/c2350ca](https://github.com/ytdl-org/youtube-dl/commit/c2350cac243ba1ec1586fe85b0d62d1b700047a2)
+* [hls] Fix decryption for multithreaded downloader
+* [extractor] Fix pre-checking archive for some extractors
+* [extractor] Fix FourCC fallback when parsing ISM by [fstirlitz](https://github.com/fstirlitz)
+* [twitcasting] Add TwitCastingUserIE, TwitCastingLiveIE by [pukkandan](https://github.com/pukkandan), [nao20010128nao](https://github.com/nao20010128nao)
+* [vidio] Add VidioPremierIE and VidioLiveIE by [MinePlayersPE](Https://github.com/MinePlayersPE)
+* [viki] Fix extraction from [ytdl-org/youtube-dl@59e583f](https://github.com/ytdl-org/youtube-dl/commit/59e583f7e8530ca92776c866897d895c072e2a82)
+* [youtube] Support shorts URL
+* [zoom] Extract transcripts as subtitles
+* Add field `original_url` with the user-inputted URL
+* Fix and refactor `prepare_outtmpl`
+* Make more fields available for `--print` when used with `--flat-playlist`
+* [utils] Generalize `traverse_dict` to `traverse_obj`
+* [downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode by [fstirlitz](https://github.com/fstirlitz)
+* [build] Release `yt-dlp.tar.gz`
+* [build,update] Add GNU-style SHA512 and prepare updater for simlar SHA256 by [nihil-admirari](https://github.com/nihil-admirari)
+* [pyinst] Show Python version in exe metadata by [nihil-admirari](https://github.com/nihil-admirari)
+* [docs] Improve documentation of dependencies
+* [cleanup] Mark unused files
+* [cleanup] Point all shebang to `python3` by [fstirlitz](https://github.com/fstirlitz)
+* [cleanup] Remove duplicate file `trovolive.py`
+
+
+### 2021.06.01
+
+* Merge youtube-dl: Upto [commit/d495292](https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf)
+* Pre-check archive and filters during playlist extraction
+* Handle Basic Auth `user:pass` in URLs by [hhirtz](https://github.com/hhirtz) and [pukkandan](https://github.com/pukkandan)
+* [archiveorg] Add YoutubeWebArchiveIE by [colethedj](https://github.com/colethedj) and [alex-gedeon](https://github.com/alex-gedeon)
+* [fancode] Add extractor by [rhsmachine](https://github.com/rhsmachine)
+* [patreon] Support vimeo embeds by [rhsmachine](https://github.com/rhsmachine)
+* [Saitosan] Add new extractor by [llacb47](https://github.com/llacb47)
+* [ShemarooMe] Add extractor by [Ashish0804](https://github.com/Ashish0804) and [pukkandan](https://github.com/pukkandan)
+* [telemundo] Add extractor by [king-millez](https://github.com/king-millez)
+* [SonyLIV] Add SonyLIVSeriesIE and subtitle support by [Ashish0804](https://github.com/Ashish0804)
+* [Hotstar] Add HotStarSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [Voot] Add VootSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [vidio] Support login and premium videos by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [fragment] When using `-N`, do not keep the fragment content in memory
+* [ffmpeg] Download and merge in a single step if possible
+* [ThumbnailsConvertor] Support conversion to `png` and make it the default by [louie-github](https://github.com/louie-github)
+* [VideoConvertor] Generalize with remuxer and allow conditional recoding
+* [EmbedThumbnail] Embed in `mp4`/`m4a` using mutagen by [tripulse](https://github.com/tripulse) and [pukkandan](https://github.com/pukkandan)
+* [EmbedThumbnail] Embed if any thumbnail was downloaded, not just the best
+* [EmbedThumbnail] Correctly escape filename
+* [update] replace self without launching a subprocess in windows
+* [update] Block further update for unsupported systems
+* Refactor `__process_playlist` by creating `LazyList`
+* Write messages to `stderr` when both `quiet` and `verbose`
+* Sanitize and sort playlist thumbnails
+* Remove `None` values from `info.json`
+* [extractor] Always prefer native hls downloader by default
+* [extractor] Skip subtitles without URI in m3u8 manifests by [hheimbuerger](https://github.com/hheimbuerger)
+* [extractor] Functions to parse `socket.io` response as `json` by [pukkandan](https://github.com/pukkandan) and [llacb47](https://github.com/llacb47)
+* [extractor] Allow `note=False` when extracting manifests
+* [utils] Escape URLs in `sanitized_Request`, not `sanitize_url`
+* [hls] Disable external downloader for `webtt`
+* [youtube] `/live` URLs should raise error if channel is not live
+* [youtube] Bug fixes
+* [zee5] Fix m3u8 formats' extension
+* [ard] Allow URLs without `-` before id by [olifre](https://github.com/olifre)
+* [cleanup] `YoutubeDL._match_entry`
+* [cleanup] Refactor updater
+* [cleanup] Refactor ffmpeg convertors
+* [cleanup] setup.py
+
+
+### 2021.05.20
+
+* **Youtube improvements**: 
+    * Support youtube music `MP`, `VL` and `browse` pages
+    * Extract more formats for youtube music by [craftingmod](https://github.com/craftingmod), [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
+    * Extract multiple subtitles in same language by [pukkandan](https://github.com/pukkandan) and [tpikonen](https://github.com/tpikonen)
+    * Redirect channels that doesn't have a `videos` tab to their `UU` playlists
+    * Support in-channel search
+    * Sort audio-only formats correctly
+    * Always extract `maxresdefault` thumbnail
+    * Extract audio language
+    * Add subtitle language names by [nixxo](https://github.com/nixxo) and [tpikonen](https://github.com/tpikonen)
+    * Show alerts only from the final webpage
+    * Add `html5=1` param to `get_video_info` page requests by [colethedj](https://github.com/colethedj)
+    * Better message when login required
+* **Add option `--print`**: to print any field/template
+    * Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
+* Field `additional_urls` to download additional videos from metadata using [`--parse-metadata`](https://github.com/yt-dlp/yt-dlp#modifying-metadata)
+* Merge youtube-dl: Upto [commit/dfbbe29](https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b)
+* Write thumbnail of playlist and add `pl_thumbnail` outtmpl key
+* [embedthumbnail] Add `flac` support and refactor `mutagen` code by [pukkandan](https://github.com/pukkandan) and [tripulse](https://github.com/tripulse)
+* [audius:artist] Add extractor by [king-millez](https://github.com/king-millez)
+* [parlview] Add extractor by [king-millez](https://github.com/king-millez)
+* [tenplay] Fix extractor by [king-millez](https://github.com/king-millez)
+* [rmcdecouverte] Generalize `_VALID_URL`
+* Add compat-option `no-attach-infojson`
+* Add field `name` for subtitles
+* Ensure `post_extract` and `pre_process` only run once
+* Fix `--check-formats` when there is network error
+* Standardize `write_debug` and `get_param`
+* [options] Alias `--write-comments`, `--no-write-comments`
+* [options] Refactor callbacks
+* [test:download] Only extract enough videos for `playlist_mincount`
+* [extractor] bugfix for when `compat_opts` is not given
+* [build] Fix x86 build by [shirt](https://github.com/shirt-dev)
+* [cleanup] code formatting, youtube tests and readme
+
+### 2021.05.11
+* **Deprecate support for python versions < 3.6**
+* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
+* **Improve output template:**
+    * Allow slicing lists/strings using `field.start:end:step`
+    * A field can also be used as offset like `field1+num+field2`
+    * A default value can be given using `field|default`
+    * Prevent invalid fields from causing errors
+* **Merge youtube-dl**: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
+* **Remove options** `-l`, `-t`, `-A` completely and disable `--auto-number`, `--title`, `--literal`, `--id`
+* [Plugins] Prioritize plugins over standard extractors and prevent plugins from overwriting the standard extractor classes
+* [downloader] Fix `quiet` and `to_stderr`
+* [fragment] Ensure the file is closed on error
+* [fragment] Make sure first segment is not skipped
+* [aria2c] Fix whitespace being stripped off
+* [embedthumbnail] Fix bug where jpeg thumbnails were converted again
+* [FormatSort] Fix for when some formats have quality and others don't
+* [utils] Add `network_exceptions`
+* [utils] Escape URL while sanitizing
+* [ukcolumn] Add Extractor
+* [whowatch] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
+* [CBS] Improve `_VALID_URL` to support movies
+* [crackle] Improve extraction
+* [curiositystream] Fix collections
+* [francetvinfo] Improve video id extraction
+* [generic] Respect the encoding in manifest
+* [limelight] Obey `allow_unplayable_formats`
+* [mediasite] Generalize URL pattern by [fstirlitz](https://github.com/fstirlitz)
+* [mxplayer] Add MxplayerShowIE by [Ashish0804](https://github.com/Ashish0804)
+* [nebula] Move to nebula.app by [Lamieur](https://github.com/Lamieur)
+* [niconico] Fix HLS formats by [CXwudi](https://github.com/CXwudi), [tsukumijima](https://github.com/tsukumijima), [nao20010128nao](https://github.com/nao20010128nao) and [pukkandan](https://github.com/pukkandan)
+* [niconico] Fix title and thumbnail extraction by [CXwudi](https://github.com/CXwudi)
+* [plutotv] Extract subtitles from manifests
+* [plutotv] Fix format extraction for some urls
+* [rmcdecouverte] Improve `_VALID_URL`
+* [sonyliv] Fix `title` and `series` extraction by [Ashish0804](https://github.com/Ashish0804)
+* [tubi] Raise "no video formats" error when video url is empty
+* [youtube:tab] Detect playlists inside community posts
+* [youtube] Add `oembed` to reserved names
+* [zee5] Fix extraction for some URLs by [Hadi0609](https://github.com/Hadi0609)
+* [zee5] Fix py2 compatibility
+* Fix `playlist_index` and add `playlist_autonumber`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details
+* Add experimental option `--check-formats` to test the URLs before format selection
+* Option `--compat-options` to revert [some of yt-dlp's changes](https://github.com/yt-dlp/yt-dlp#differences-in-default-behavior)
+    * Deprecates `--list-formats-as-table`, `--list-formats-old`
+* Fix number of digits in `%(playlist_index)s`
+* Fix case sensitivity of format selector
+* Revert "[core] be able to hand over id and title using url_result"
+* Do not strip out whitespaces in `-o` and `-P`
+* Fix `preload_download_archive` writing verbose message to `stdout`
+* Move option warnings to `YoutubeDL`so that they obey `--no-warnings` and can output colors
+* Py2 compatibility for `FileNotFoundError`
+
+
+### 2021.04.22
+* **Improve output template:**
+    * Objects can be traversed like `%(field.key1.key2)s`
+    * An offset can be added to numeric fields as `%(field+N)s`
+    * Deprecates `--autonumber-start`
+* **Improve `--sub-langs`:**
+    * Treat `--sub-langs` entries as regex
+    * `all` can be used to refer to all the subtitles
+    * language codes can be prefixed with `-` to exclude it
+    * Deprecates `--all-subs`
+* Add option `--ignore-no-formats-error` to ignore the "no video format" and similar errors
+* Add option `--skip-playlist-after-errors` to skip the rest of a playlist after a given number of errors are encountered
+* Merge youtube-dl: Upto [commit/7e8b3f9](https://github.com/ytdl-org/youtube-dl/commit/7e8b3f9439ebefb3a3a4e5da9c0bd2b595976438)
+* [downloader] Fix bug in downloader selection
+* [BilibiliChannel] Fix pagination by [nao20010128nao](https://github.com/nao20010128nao) and [pukkandan](https://github.com/pukkandan)
+* [rai] Add support for http formats by [nixxo](https://github.com/nixxo)
+* [TubiTv] Add TubiTvShowIE by [Ashish0804](https://github.com/Ashish0804)
+* [twitcasting] Fix extractor
+* [viu:ott] Fix extractor and support series by [lkho](https://github.com/lkho) and [pukkandan](https://github.com/pukkandan)
+* [youtube:tab] Show unavailable videos in playlists by [colethedj](https://github.com/colethedj)
+* [youtube:tab] Reload with unavailable videos for all playlists
+* [youtube] Ignore invalid stretch ratio
+* [youtube] Improve channel syncid extraction to support ytcfg by [colethedj](https://github.com/colethedj)
+* [youtube] Standardize API calls for tabs, mixes and search by [colethedj](https://github.com/colethedj)
+* [youtube] Bugfix in `_extract_ytcfg`
+* [mildom:user:vod] Download only necessary amount of pages
+* [mildom] Remove proxy completely by [fstirlitz](https://github.com/fstirlitz)
+* [go] Fix `_VALID_URL`
+* [MetadataFromField] Improve regex and add tests
+* [Exec] Ensure backward compatibility when the command contains `%`
+* [extractor] Fix inconsistent use of `report_warning`
+* Ensure `mergeall` selects best format when multistreams are disabled
+* Improve the yt-dlp.sh script by [fstirlitz](https://github.com/fstirlitz)
+* [lazy_extractor] Do not load plugins
+* [ci] Disable fail-fast
+* [documentation] Clarify which deprecated options still work
+* [documentation] Fix typos
+
 
 ### 2021.04.11
 * Add option `--convert-thumbnails` (only jpg currently supported)
@@ -35,7 +389,7 @@
 * [youtube] Fix thumbnail URL
 * [youtube] Parse API parameters from initial webpage by [colethedj](https://github.com/colethedj)
 * [youtube] Extract comments' approximate timestamp by [colethedj](https://github.com/colethedj)
-* [youtube] Fix `\_extract_alerts`
+* [youtube] Fix alert extraction
 * [bilibili] Fix uploader
 * [utils] Add `datetime_from_str` and `datetime_add_months` by [colethedj](https://github.com/colethedj)
 * Run some `postprocessors` before actual download
@@ -226,7 +580,7 @@
 
 ### 2021.02.15
 * Merge youtube-dl: Upto [2021.02.10](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.02.10) (except archive.org)
-* [niconico] Improved extraction and support encrypted/SMILE movies by [kurumigi](https://github.com/kurumigi), [tsukumi](https://github.com/tsukumi), [bbepis](https://github.com/bbepis), [pukkandan](https://github.com/pukkandan)
+* [niconico] Improved extraction and support encrypted/SMILE movies by [kurumigi](https://github.com/kurumigi), [tsukumijima](https://github.com/tsukumijima), [bbepis](https://github.com/bbepis), [pukkandan](https://github.com/pukkandan)
 * Fix HLS AES-128 with multiple keys in external downloaders by [shirt](https://github.com/shirt-dev)
 * [youtube_live_chat] Fix by using POST API by [siikamiika](https://github.com/siikamiika)
 * [rumble] Add support for video page
@@ -480,4 +834,4 @@
 * [generic] Extract embedded youtube and twitter videos by [diegorodriguezv](https://github.com/diegorodriguezv)
 * [ffmpeg] Ensure all streams are copied by [pukkandan](https://github.com/pukkandan)
 * [embedthumbnail] Fix for os.rename error by [pukkandan](https://github.com/pukkandan)
-* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)
+* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)

Makefile

@@ -25,6 +25,7 @@ completion-zsh: completions/zsh/_yt-dlp
 lazy-extractors: yt_dlp/extractor/lazy_extractors.py
 
 PREFIX ?= /usr/local
+DESTDIR ?= .
 BINDIR ?= $(PREFIX)/bin
 MANDIR ?= $(PREFIX)/man
 SHAREDIR ?= $(PREFIX)/share

README.md

@@ -3,17 +3,16 @@
 # YT-DLP
 A command-line program to download videos from YouTube and many other [video platforms](supportedsites.md)
 
-<!-- GHA doesnot have for-the-badge style
+<!-- GHA doesn't have for-the-badge style
 [![CI Status](https://github.com/yt-dlp/yt-dlp/workflows/Core%20Tests/badge.svg?branch=master)](https://github.com/yt-dlp/yt-dlp/actions)
 -->
 [![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Release&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest)
 [![License: Unlicense](https://img.shields.io/badge/License-Unlicense-blue.svg?style=for-the-badge)](LICENSE)
 [![Doc Status](https://readthedocs.org/projects/yt-dlp/badge/?version=latest&style=for-the-badge)](https://yt-dlp.readthedocs.io)
-[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=discord&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r)  
-
+[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=discord&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r)
 [![Commits](https://img.shields.io/github/commit-activity/m/yt-dlp/yt-dlp?label=commits&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
 [![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
-[![Downloads](https://img.shields.io/github/downloads/yt-dlp/yt-dlp/total?style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest)
+[![Downloads](https://img.shields.io/github/downloads/yt-dlp/yt-dlp/total?style=for-the-badge&color=blue)](https://github.com/yt-dlp/yt-dlp/releases/latest)
 [![PyPi Downloads](https://img.shields.io/pypi/dm/yt-dlp?label=PyPi&style=for-the-badge)](https://pypi.org/project/yt-dlp)
 
 </div>
@@ -21,9 +20,10 @@ A command-line program to download videos from YouTube and many other [video pla
 yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on the now inactive [youtube-dlc](https://github.com/blackjack4494/yt-dlc). The main focus of this project is adding new features and patches while also keeping up to date with the original project
 
 * [NEW FEATURES](#new-features)
+    * [Differences in default behavior](#differences-in-default-behavior)
 * [INSTALLATION](#installation)
-    * [Dependancies](#dependancies)
     * [Update](#update)
+    * [Dependencies](#dependencies)
     * [Compile](#compile)
 * [USAGE AND OPTIONS](#usage-and-options)
     * [General Options](#general-options)
@@ -53,6 +53,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
     * [Format Selection examples](#format-selection-examples)
 * [MODIFYING METADATA](#modifying-metadata)
     * [Modifying metadata examples](#modifying-metadata-examples)
+* [EXTRACTOR ARGUMENTS](#extractor-arguments)
 * [PLUGINS](#plugins)
 * [DEPRECATED OPTIONS](#deprecated-options)
 * [MORE](#more)
@@ -66,14 +67,20 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/
 
 * **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))
 
-* **Merged with youtube-dl v2021.04.07**: You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
+* **Merged with youtube-dl [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)**: (v2021.06.06) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
 
-* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--get-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, Playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
+* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
 
 * **Youtube improvements**:
-    * All Youtube Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) works correctly and supports downloading multiple pages of content
-    * Youtube search (`ytsearch:`, `ytsearchdate:`) along with Search URLs works correctly
+    * All Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) supports downloading multiple pages of content
+    * Search (`ytsearch:`, `ytsearchdate:`), search URLs and in-channel search works
+    * Mixes supports downloading multiple pages of content
+    * Partial workarounds for age-gate and throttling issues
     * Redirect channel's home URL automatically to `/video` to preserve the old behaviour
+    * `255kbps` audio is extracted from youtube music if premium cookies are given
+    * Youtube music Albums, channels etc can be downloaded
+
+* **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[:PROFILE]`
 
 * **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters`
 
@@ -81,17 +88,21 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/
 
 * **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats
 
-* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula
+* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip
 
-* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter
+* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki
+
+* **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
 
 * **Multiple paths and output templates**: You can give different [output templates](#output-template) and download paths for different types of files. You can also set a temporary path where intermediary files are downloaded to using `--paths` (`-P`)
 
 * **Portable Configuration**: Configuration files are automatically loaded from the home and root directories. See [configuration](#configuration) for details
 
-* **Other new options**: `--parse-metadata`, `--list-formats-as-table`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
+* **Output template improvements**: Output templates can now have date-time formatting, numeric offsets, object traversal etc. See [output template](#output-template) for details. Even more advanced operations can also be done with the help of `--parse-metadata`
 
-* **Improvements**: Multiple `--postprocessor-args` and `--external-downloader-args`, Date/time formatting in `-o`, faster archive checking, more [format selection options](#format-selection) etc
+* **Other new options**: `--sleep-requests`, `--convert-thumbnails`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
+
+* **Improvements**: Multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection) etc
 
 * **Plugin extractors**: Extractors can be loaded from an external file. See [plugins](#plugins) for details
 
@@ -105,15 +116,44 @@ See [changelog](Changelog.md) or [commits](https://github.com/yt-dlp/yt-dlp/comm
 
 If you are coming from [youtube-dl](https://github.com/ytdl-org/youtube-dl), the amount of changes are very large. Compare [options](#options) and [supported sites](supportedsites.md) with youtube-dl's to get an idea of the massive number of features/patches [youtube-dlc](https://github.com/blackjack4494/yt-dlc) has accumulated.
 
+### Differences in default behavior
+
+Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc.
+
+* The options `--id`, `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
+* `avconv` is not supported as as an alternative to `ffmpeg`
+* The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s.%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
+* The default [format sorting](sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
+* The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be prefered. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
+* Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
+* `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
+* When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files
+* `--add-metadata` attaches the `infojson` to `mkv` files in addition to writing the metadata when used with `--write-infojson`. Use `--compat-options no-attach-info-json` to revert this
+* `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
+* The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
+* All *experiences* of a funimation episode are considered as a single video. This behavior breaks existing archives. Use `--compat-options seperate-video-versions` to extract information from only the default player
+* Youtube live chat (if available) is considered as a subtitle. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent live chat from downloading
+* Youtube channel URLs are automatically redirected to `/video`. Append a `/featured` to the URL to download only the videos in the home page. If the channel does not have a videos tab, we try to download the equivalent `UU` playlist instead. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections
+* Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
+* If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this
+* Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead
+
+For ease of use, a few more compat options are available:
+* `--compat-options all`: Use all compat options
+* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams`
+* `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
+
 
 # INSTALLATION
 yt-dlp is not platform specific. So it should work on your Unix box, on Windows or on macOS
 
 You can install yt-dlp using one of the following methods:
 * Download the binary from the [latest release](https://github.com/yt-dlp/yt-dlp/releases/latest) (recommended method)
-* Use [PyPI package](https://pypi.org/project/yt-dlp): `python -m pip install --upgrade yt-dlp`
-* Use pip+git: `python -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp.git@release`
-* Install master branch: `python -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp`
+* Use [PyPI package](https://pypi.org/project/yt-dlp): `python3 -m pip install --upgrade yt-dlp`
+* Use pip+git: `python3 -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp.git@release`
+* Install master branch: `python3 -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp`
+
+Note that on some systems, you may need to use `py` or `python` instead of `python3`
 
 UNIX users (Linux, macOS, BSD) can also install the [latest release](https://github.com/yt-dlp/yt-dlp/releases/latest) one of the following ways:
 
@@ -132,28 +172,47 @@ sudo aria2c https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o
 sudo chmod a+rx /usr/local/bin/yt-dlp
 ```
 
-### DEPENDANCIES
-
-Python versions 2.6, 2.7, or 3.2+ are currently supported. However, 3.2+ is strongly recommended and python2 support will be deprecated in the future.
-
-Although there are no required dependancies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependancies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.
-
 ### UPDATE
 You can use `yt-dlp -U` to update if you are using the provided release.
 If you are using `pip`, simply re-run the same command that was used to install the program.
 
+### DEPENDENCIES
+Python versions 3.6+ (CPython and PyPy) are supported. Other versions and implementations may or may not work correctly.
+
+<!-- https://www.microsoft.com/en-us/download/details.aspx?id=26999 -->
+On windows, [Microsoft Visual C++ 2010 SP1 Redistributable Package (x86)](https://download.microsoft.com/download/1/6/5/165255E7-1014-4D0A-B094-B6A430A6BFFC/vcredist_x86.exe) is also necessary to run yt-dlp. You probably already have this, but if the executable throws an error due to missing `MSVCR100.dll` you need to install it manually.
+
+While all the other dependancies are optional, `ffmpeg` and `ffprobe` are highly recommended
+* [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging seperate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. Licence [depends on the build](https://www.ffmpeg.org/legal.html)
+* [**sponskrub**](https://github.com/faissaloo/SponSkrub) - For using the [sponskrub options](#sponskrub-sponsorblock-options). Licenced under [GPLv3+](https://github.com/faissaloo/SponSkrub/blob/master/LICENCE.md)
+* [**mutagen**](https://github.com/quodlibet/mutagen) - For embedding thumbnail in certain formats. Licenced under [GPLv2+](https://github.com/quodlibet/mutagen/blob/master/COPYING)
+* [**pycryptodome**](https://github.com/Legrandin/pycryptodome) - For decrypting various data. Licenced under [BSD2](https://github.com/Legrandin/pycryptodome/blob/master/LICENSE.rst)
+* [**websockets**](https://github.com/aaugustin/websockets) - For downloading over websocket. Licenced under [BSD3](https://github.com/aaugustin/websockets/blob/main/LICENSE)
+* [**keyring**](https://github.com/jaraco/keyring) - For decrypting cookies of chromium-based browsers on Linux. Licenced under [MIT](https://github.com/jaraco/keyring/blob/main/LICENSE)
+* [**AtomicParsley**](https://github.com/wez/atomicparsley) - For embedding thumbnail in mp4/m4a if mutagen is not present. Licenced under [GPLv2+](https://github.com/wez/atomicparsley/blob/master/COPYING)
+* [**rtmpdump**](http://rtmpdump.mplayerhq.hu) - For downloading `rtmp` streams. ffmpeg will be used as a fallback. Licenced under [GPLv2+](http://rtmpdump.mplayerhq.hu)
+* [**mplayer**](http://mplayerhq.hu/design7/info.html) or [**mpv**](https://mpv.io) - For downloading `rstp` streams. ffmpeg will be used as a fallback. Licenced under [GPLv2+](https://github.com/mpv-player/mpv/blob/master/Copyright)
+* [**phantomjs**](https://github.com/ariya/phantomjs) - Used in extractors where javascript needs to be run. Licenced under [BSD3](https://github.com/ariya/phantomjs/blob/master/LICENSE.BSD)
+* Any external downloader that you want to use with `--downloader`
+
+To use or redistribute the dependencies, you must agree to their respective licensing terms.
+
+Note that the windows releases are already built with the python interpreter, mutagen, pycryptodome and websockets included.
+
 ### COMPILE
 
 **For Windows**:
-To build the Windows executable, you must have pyinstaller (and optionally mutagen and pycryptodome)
+To build the Windows executable, you must have pyinstaller (and optionally mutagen, pycryptodome, websockets)
 
-    python -m pip install --upgrade pyinstaller mutagen pycryptodome
+    python3 -m pip install --upgrade pyinstaller mutagen pycryptodome websockets
 
-Once you have all the necessary dependancies installed, just run `py pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it. It is strongly reccomended to use python3 although python2.6+ is supported.
+Once you have all the necessary dependencies installed, just run `py pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it.
 
 You can also build the executable without any version info or metadata by using:
 
     pyinstaller.exe yt_dlp\__main__.py --onefile --name yt-dlp
+    
+Note that pyinstaller [does not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment
 
 **For Unix**:
 You will need the required build tools: `python`, `make` (GNU), `pandoc`, `zip`, `nosetests`  
@@ -166,7 +225,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     yt-dlp [OPTIONS] [--] URL [URL...]
 
 `Ctrl+F` is your friend :D
-<!-- Autogenerated -->
+<!-- Auto generated -->
 
 ## General Options:
     -h, --help                       Print this help text and exit
@@ -211,6 +270,11 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --mark-watched                   Mark videos watched (YouTube only)
     --no-mark-watched                Do not mark videos watched (default)
     --no-colors                      Do not emit color codes in output
+    --compat-options OPTS            Options that can help keep compatibility
+                                     with youtube-dl and youtube-dlc
+                                     configurations by reverting some of the
+                                     changes made in yt-dlp. See "Differences in
+                                     default behavior" for details
 
 ## Network Options:
     --proxy URL                      Use the specified HTTP/HTTPS/SOCKS proxy.
@@ -306,6 +370,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      a file that is in the archive
     --break-on-reject                Stop the download process when encountering
                                      a file that has been filtered out
+    --skip-playlist-after-errors N   Number of allowed failures until the rest
+                                     of the playlist is skipped
     --no-download-archive            Do not use archive file (default)
 
 ## Download Options:
@@ -314,6 +380,9 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      (default is 1)
     -r, --limit-rate RATE            Maximum download rate in bytes per second
                                      (e.g. 50K or 4.2M)
+    --throttled-rate RATE            Minimum download rate in bytes per second
+                                     below which throttling is assumed and the
+                                     video data is re-extracted (e.g. 100K)
     -R, --retries RETRIES            Number of retries (default is 10), or
                                      "infinite"
     --fragment-retries RETRIES       Number of retries for a fragment (default
@@ -370,7 +439,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --downloader-args NAME:ARGS      Give these arguments to the external
                                      downloader. Specify the downloader name and
                                      the arguments separated by a colon ":". You
-                                     can use this option multiple times
+                                     can use this option multiple times to give
+                                     different arguments to different downloaders
                                      (Alias: --external-downloader-args)
 
 ## Filesystem Options:
@@ -394,8 +464,6 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --output-na-placeholder TEXT     Placeholder value for unavailable meta
                                      fields in output filename template
                                      (default: "NA")
-    --autonumber-start NUMBER        Specify the start value for %(autonumber)s
-                                     (default is 1)
     --restrict-filenames             Restrict filenames to only ASCII
                                      characters, and avoid "&" and spaces in
                                      filenames
@@ -415,7 +483,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     -c, --continue                   Resume partially downloaded files/fragments
                                      (default)
     --no-continue                    Do not resume partially downloaded
-                                     fragments. If the file is unfragmented,
+                                     fragments. If the file is not fragmented,
                                      restart download of the entire file
     --part                           Use .part files instead of writing directly
                                      into output file (default)
@@ -444,24 +512,36 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      could still contain some personal
                                      information (default)
     --no-clean-infojson              Write all fields to the infojson
-    --get-comments                   Retrieve video comments to be placed in the
-                                     .info.json file. The comments are fetched
-                                     even without this option if the extraction
-                                     is known to be quick
+    --write-comments                 Retrieve video comments to be placed in the
+                                     infojson. The comments are fetched even
+                                     without this option if the extraction is
+                                     known to be quick (Alias: --get-comments)
+    --no-write-comments              Do not retrieve video comments unless the
+                                     extraction is known to be quick
+                                     (Alias: --no-get-comments)
     --load-info-json FILE            JSON file containing the video information
                                      (created with the "--write-info-json"
                                      option)
     --cookies FILE                   File to read cookies from and dump cookie
                                      jar in
-    --no-cookies                     Do not read/dump cookies (default)
+    --no-cookies                     Do not read/dump cookies from/to file
+                                     (default)
+    --cookies-from-browser BROWSER[:PROFILE]
+                                     Load cookies from a user profile of the
+                                     given web browser. Currently supported
+                                     browsers are: brave|chrome|chromium|edge|fi
+                                     refox|opera|safari|vivaldi. You can specify
+                                     the user profile name or directory using
+                                     "BROWSER:PROFILE_NAME" or
+                                     "BROWSER:PROFILE_PATH". If no profile is
+                                     given, the most recently accessed one is
+                                     used
+    --no-cookies-from-browser        Do not load cookies from browser (default)
     --cache-dir DIR                  Location in the filesystem where youtube-dl
-                                     can store some downloaded information
-                                     permanently. By default
-                                     $XDG_CACHE_HOME/youtube-dl or
-                                     ~/.cache/youtube-dl . At the moment, only
-                                     YouTube player files (for videos with
-                                     obfuscated signatures) are cached, but that
-                                     may change
+                                     can store some downloaded information (such
+                                     as client ids and signatures) permanently.
+                                     By default $XDG_CACHE_HOME/youtube-dl or
+                                     ~/.cache/youtube-dl
     --no-cache-dir                   Disable filesystem caching
     --rm-cache-dir                   Delete all filesystem cache files
 
@@ -487,16 +567,17 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --no-warnings                    Ignore warnings
     -s, --simulate                   Do not download the video and do not write
                                      anything to disk
+    --ignore-no-formats-error        Ignore "No video formats" error. Usefull
+                                     for extracting metadata even if the video
+                                     is not actually available for download
+                                     (experimental)
+    --no-ignore-no-formats-error     Throw error when no downloadable video
+                                     formats are found (default)
     --skip-download                  Do not download the video but write all
                                      related files (Alias: --no-download)
-    -g, --get-url                    Simulate, quiet but print URL
-    -e, --get-title                  Simulate, quiet but print title
-    --get-id                         Simulate, quiet but print id
-    --get-thumbnail                  Simulate, quiet but print thumbnail URL
-    --get-description                Simulate, quiet but print video description
-    --get-duration                   Simulate, quiet but print video length
-    --get-filename                   Simulate, quiet but print output filename
-    --get-format                     Simulate, quiet but print output format
+    -O, --print TEMPLATE             Simulate, quiet but print the given fields.
+                                     Either a field name or similar formatting
+                                     as the output template can be used
     -j, --dump-json                  Simulate, quiet but print JSON information.
                                      See "OUTPUT TEMPLATE" for a description of
                                      available keys
@@ -507,8 +588,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --print-json                     Be quiet and print the video information as
                                      JSON (video is still being downloaded)
     --force-write-archive            Force download archive entries to be
-                                     written as far as no errors occur,even if
-                                     -s or another simulation switch is used
+                                     written as far as no errors occur, even if
+                                     -s or another simulation option is used
                                      (Alias: --force-download-archive)
     --newline                        Output progress bar as new lines
     --no-progress                    Do not print progress bar
@@ -572,18 +653,18 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      containers irrespective of quality
     --no-prefer-free-formats         Don't give any special preference to free
                                      containers (default)
+    --check-formats                  Check that the formats selected are
+                                     actually downloadable
+    --no-check-formats               Do not check that the formats selected are
+                                     actually downloadable
     -F, --list-formats               List all available formats of requested
                                      videos
-    --list-formats-as-table          Present the output of -F in tabular form
-                                     (default)
-    --list-formats-old               Present the output of -F in the old form
-                                     (Alias: --no-list-formats-as-table)
     --merge-output-format FORMAT     If a merge is required (e.g.
                                      bestvideo+bestaudio), output to given
                                      container format. One of mkv, mp4, ogg,
                                      webm, flv. Ignored if no merge is required
     --allow-unplayable-formats       Allow unplayable formats to be listed and
-                                     downloaded. All video postprocessing will
+                                     downloaded. All video post-processing will
                                      also be turned off
     --no-allow-unplayable-formats    Do not allow unplayable formats to be
                                      listed or downloaded (default)
@@ -595,15 +676,17 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      (Alias: --write-automatic-subs)
     --no-write-auto-subs             Do not write auto-generated subtitles
                                      (default) (Alias: --no-write-automatic-subs)
-    --all-subs                       Download all the available subtitles of the
-                                     video
     --list-subs                      List all available subtitles for the video
     --sub-format FORMAT              Subtitle format, accepts formats
                                      preference, for example: "srt" or
                                      "ass/srt/best"
-    --sub-langs LANGS                Languages of the subtitles to download
-                                     (optional) separated by commas, use --list-
-                                     subs for available language tags
+    --sub-langs LANGS                Languages of the subtitles to download (can
+                                     be regex) or "all" separated by commas.
+                                     (Eg: --sub-langs en.*,ja) You can prefix
+                                     the language code with a "-" to exempt it
+                                     from the requested languages. (Eg: --sub-
+                                     langs all,-live_chat) Use --list-subs for a
+                                     list of available language tags
 
 ## Authentication Options:
     -u, --username USERNAME          Login with this account ID
@@ -625,10 +708,10 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
 ## Post-Processing Options:
     -x, --extract-audio              Convert video files to audio-only files
                                      (requires ffmpeg and ffprobe)
-    --audio-format FORMAT            Specify audio format: "best", "aac",
-                                     "flac", "mp3", "m4a", "opus", "vorbis", or
-                                     "wav"; "best" by default; No effect without
-                                     -x
+    --audio-format FORMAT            Specify audio format to convert the audio
+                                     to when -x is used. Currently supported
+                                     formats are: best (default) or one of
+                                     aac|flac|mp3|m4a|opus|vorbis|wav
     --audio-quality QUALITY          Specify ffmpeg audio quality, insert a
                                      value between 0 (better) and 9 (worse) for
                                      VBR or a specific bitrate like 128K
@@ -638,12 +721,12 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      |webm|mov|avi|mp3|mka|m4a|ogg|opus). If
                                      target container does not support the
                                      video/audio codec, remuxing will fail. You
-                                     can specify multiple rules; eg.
+                                     can specify multiple rules; Eg.
                                      "aac>m4a/mov>mp4/mkv" will remux aac to
                                      m4a, mov to mp4 and anything else to mkv.
     --recode-video FORMAT            Re-encode the video into another format if
-                                     re-encoding is necessary. The supported
-                                     formats are the same as --remux-video
+                                     re-encoding is necessary. The syntax and
+                                     supported formats are the same as --remux-video
     --postprocessor-args NAME:ARGS   Give these arguments to the postprocessors.
                                      Specify the postprocessor/executable name
                                      and the arguments separated by a colon ":"
@@ -653,7 +736,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      Metadata, EmbedSubtitle, EmbedThumbnail,
                                      SubtitlesConvertor, ThumbnailsConvertor,
                                      VideoRemuxer, VideoConvertor, SponSkrub,
-                                     FixupStretched, FixupM4a and FixupM3u8. The
+                                     FixupStretched, FixupM4a, FixupM3u8,
+                                     FixupTimestamp and FixupDuration. The
                                      supported executables are: AtomicParsley,
                                      FFmpeg, FFprobe, and SponSkrub. You can
                                      also specify "PP+EXE:ARGS" to give the
@@ -677,10 +761,13 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --embed-subs                     Embed subtitles in the video (only for mp4,
                                      webm and mkv videos)
     --no-embed-subs                  Do not embed subtitles (default)
-    --embed-thumbnail                Embed thumbnail in the audio as cover art
+    --embed-thumbnail                Embed thumbnail in the video as cover art
     --no-embed-thumbnail             Do not embed thumbnail (default)
-    --add-metadata                   Write metadata to the video file
-    --no-add-metadata                Do not write metadata (default)
+    --embed-metadata                 Embed metadata including chapter markers
+                                     (if supported by the format) to the video
+                                     file (Alias: --add-metadata)
+    --no-embed-metadata              Do not write metadata (default)
+                                     (Alias: --no-add-metadata)
     --parse-metadata FROM:TO         Parse additional metadata like title/artist
                                      from other fields; see "MODIFYING METADATA"
                                      for details
@@ -690,7 +777,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      file. One of never (do nothing), warn (only
                                      emit a warning), detect_or_warn (the
                                      default; fix file if we can, warn
-                                     otherwise)
+                                     otherwise), force (try fixing even if file
+                                     already exists
     --ffmpeg-location PATH           Location of the ffmpeg binary; either the
                                      path to the binary or its containing
                                      directory
@@ -703,11 +791,13 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                      downloaded file is also available. If no
                                      fields are passed, "%(filepath)s" is
                                      appended to the end of the command
+    --exec-before-download CMD       Execute a command before the actual
+                                     download. The syntax is the same as --exec
     --convert-subs FORMAT            Convert the subtitles to another format
-                                     (currently supported: srt|ass|vtt|lrc)
+                                     (currently supported: srt|vtt|ass|lrc)
                                      (Alias: --convert-subtitles)
     --convert-thumbnails FORMAT      Convert the thumbnails to another format
-                                     (currently supported: jpg)
+                                     (currently supported: jpg|png)
     --split-chapters                 Split video into multiple files based on
                                      internal chapters. The "chapter:" prefix
                                      can be used with "--paths" and "--output"
@@ -749,18 +839,10 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
     --no-hls-split-discontinuity     Do not split HLS playlists to different
                                      formats at discontinuities such as ad
                                      breaks (default)
-    --youtube-include-dash-manifest  Download the DASH manifests and related
-                                     data on YouTube videos (default)
-                                     (Alias: --no-youtube-skip-dash-manifest)
-    --youtube-skip-dash-manifest     Do not download the DASH manifests and
-                                     related data on YouTube videos
-                                     (Alias: --no-youtube-include-dash-manifest)
-    --youtube-include-hls-manifest   Download the HLS manifests and related data
-                                     on YouTube videos (default)
-                                     (Alias: --no-youtube-skip-hls-manifest)
-    --youtube-skip-hls-manifest      Do not download the HLS manifests and
-                                     related data on YouTube videos
-                                     (Alias: --no-youtube-include-hls-manifest)
+    --extractor-args KEY:ARGS        Pass these arguments to the extractor. See
+                                     "EXTRACTOR ARGUMENTS" for details. You can
+                                     use this option multiple times to give
+                                     arguments for different extractors
 
 # CONFIGURATION
 
@@ -778,7 +860,7 @@ You can configure yt-dlp by placing any supported command line option to a confi
     * `~/yt-dlp.conf.txt`
 
     Note that `~` points to `C:\Users\<user name>` on windows. Also, `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined
-1. **System Configuration**: `/etc/yt-dlp.conf` or `/etc/yt-dlp.conf`
+1. **System Configuration**: `/etc/yt-dlp.conf`
 
 For example, with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
 ```
@@ -830,9 +912,22 @@ The `-o` option is used to indicate a template for the output file names while `
 
 **tl;dr:** [navigate me to examples](#output-template-examples).
 
-The basic usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is not recommended). However, it may contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. Date/time fields can also be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it inside the parantheses separated from the field name using a `>`. For example, `%(duration>%H-%M-%S)s`.
+The simplest usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is _not_ recommended and could break some post-processing).
 
-Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different filetypes supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.
+It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.
+
+The field names themselves (the part inside the parenthesis) can also have some special formatting:
+1. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. You can also do python slicing using `:`. Eg: `%(tags.0)s`, `%(subtitles.en.-1.ext)`, `%(id.3:7:-1)s`. Note that the fields that become available using this method are not listed below. Use `-j` to see such fields
+1. **Addition**: Addition and subtraction of numeric fields can be done using `+` and `-` respectively. Eg: `%(playlist_index+10)03d`, `%(n_entries+1-playlist_index)d`
+1. **Date/time Formatting**: Date/time fields can be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it separated from the field name using a `>`. Eg: `%(duration>%H-%M-%S)s`, `%(upload_date>%Y-%m-%d)s`, `%(epoch-3600>%H-%M-%S)s`
+1. **Default**: A default value can be specified for when the field is empty using a `|` seperator. This overrides `--output-na-template`. Eg: `%(uploader|Unknown)s`
+
+To summarize, the general syntax for a field is:
+```
+%(name[.keys][addition][>strf][|default])[flags][width][.precision][length]type
+```
+
+Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_thumbnail`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.
 
 The available fields are:
 
@@ -854,7 +949,7 @@ The available fields are:
  - `channel_id` (string): Id of the channel
  - `location` (string): Physical location where the video was filmed
  - `duration` (numeric): Length of the video in seconds
- - `duration_string` (string): Length of the video (HH-mm-ss)
+ - `duration_string` (string): Length of the video (HH:mm:ss)
  - `view_count` (numeric): How many users have watched the video on the platform
  - `like_count` (numeric): Number of positive ratings of the video
  - `dislike_count` (numeric): Number of negative ratings of the video
@@ -862,6 +957,7 @@ The available fields are:
  - `average_rating` (numeric): Average rating give by users, the scale used depends on the webpage
  - `comment_count` (numeric): Number of comments on the video (For some extractors, comments are only downloaded at the end, and so this field cannot be used)
  - `age_limit` (numeric): Age restriction for the video (years)
+ - `live_status` (string): One of 'is_live', 'was_live', 'upcoming', 'not_live'
  - `is_live` (boolean): Whether this video is a live stream or a fixed-length video
  - `was_live` (boolean): Whether this video was originally a live stream
  - `playable_in_embed` (string): Whether this video is allowed to play in embedded players on other sites
@@ -895,6 +991,8 @@ The available fields are:
  - `playlist_title` (string): Playlist title
  - `playlist_uploader` (string): Full name of the playlist uploader
  - `playlist_uploader_id` (string): Nickname or id of the playlist uploader
+ - `webpage_url` (string): A URL to the video webpage which if given to yt-dlp should allow to get the same result again
+ - `original_url` (string): The URL given by the user (or same as `webpage_url` for playlist entries)
 
 Available for the video that belongs to some logical chapter or section:
 
@@ -932,9 +1030,14 @@ Available for `chapter:` prefix when using `--split-chapters` for videos with in
  - `section_start` (numeric): Start time of the chapter in seconds
  - `section_end` (numeric): End time of the chapter in seconds
 
+Available only when used in `--print`:
+
+ - `urls` (string): The URLs of all requested formats, one in each line
+ - `filename` (string): Name of the video file. Note that the actual filename may be different due to post-processing. Use `--exec echo` to get the name after all postprocessing is complete
+
 Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).
 
-For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKcj`, this will result in a `yt-dlp test video-BaW_jenozKcj.mp4` file created in the current directory.
+For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKc`, this will result in a `yt-dlp test video-BaW_jenozKc.mp4` file created in the current directory.
 
 For numeric sequences you can use numeric related formatting, for example, `%(view_count)05d` will result in a string with view count padded with zeros up to 5 characters, like in `00042`.
 
@@ -989,7 +1092,7 @@ The general syntax for format selection is `-f FORMAT` (or `--format FORMAT`) wh
 
 **tl;dr:** [navigate me to examples](#format-selection-examples).
 
-The simplest case is requesting a specific format, for example with `-f 22` you can download the format with format code equal to 22. You can get the list of available format codes for particular video using `--list-formats` or `-F`. Note that these format codes are extractor specific. 
+The simplest case is requesting a specific format, for example with `-f 22` you can download the format with format code equal to 22. You can get the list of available format codes for particular video using `--list-formats` or `-F`. Note that these format codes are extractor specific.
 
 You can also use a file extension (currently `3gp`, `aac`, `flv`, `m4a`, `mp3`, `mp4`, `ogg`, `wav`, `webm` are supported) to download the best quality format of a particular file extension served as a single file, e.g. `-f webm` will download the best quality format with the `webm` extension served as a single file.
 
@@ -1010,7 +1113,7 @@ You can also use special names to select particular edge case formats:
  - `ba*`, `bestaudio*`: Select the best quality format that contains audio. It may also contain video. Equivalent to `best*[acodec!=none]`
  - `wa*`, `worstaudio*`: Select the worst quality format that contains audio. It may also contain video. Equivalent to `worst*[acodec!=none]`
 
-For example, to download the worst quality video-only format you can use `-f worstvideo`. It is however recomended not to use `worst` and related options. When your format selector is `worst`, the format which is worst in all respects is selected. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps` instead of `-f worst`. See [sorting formats](#sorting-formats) for more details.
+For example, to download the worst quality video-only format you can use `-f worstvideo`. It is however recommended not to use `worst` and related options. When your format selector is `worst`, the format which is worst in all respects is selected. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps` instead of `-f worst`. See [sorting formats](#sorting-formats) for more details.
 
 You can select the n'th best format of a type by using `best<type>.<n>`. For example, `best.2` will select the 2nd best combined format. Similarly, `bv*.3` will select the 3rd best format that contains a video stream.
 
@@ -1018,7 +1121,7 @@ If you want to download multiple videos and they don't have the same formats ava
 
 If you want to download several formats of the same video use a comma as a separator, e.g. `-f 22,17,18` will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: `-f 136/137/mp4/bestvideo,140/m4a/bestaudio`.
 
-You can merge the video and audio of multiple formats into a single file using `-f <format1>+<format2>+...` (requires ffmpeg installed), for example `-f bestvideo+bestaudio` will download the best video-only format, the best audio-only format and mux them together with ffmpeg. If `--no-video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, if `--no-audio-multistreams` is used, all formats with an audio stream except the first one are ignored. For example, `-f bestvideo+best+bestaudio` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.
+You can merge the video and audio of multiple formats into a single file using `-f <format1>+<format2>+...` (requires ffmpeg installed), for example `-f bestvideo+bestaudio` will download the best video-only format, the best audio-only format and mux them together with ffmpeg. Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. For example, `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.
 
 ## Filtering Formats
 
@@ -1047,7 +1150,7 @@ Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends
 
 Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain).
 
-Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the video hoster. Any other field made available by the extractor can also be used for filtering.
+Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering.
 
 Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height<=?720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. You can also use the filters with `all` to download all formats that satisfy the filter. For example, `-f "all[vcodec=none]"` selects all audio-only formats.
 
@@ -1063,7 +1166,7 @@ You can change the criteria for being considered the `best` by using `-S` (`--fo
  - `lang`: Language preference as given by the extractor
  - `quality`: The quality of the format as given by the extractor
  - `source`: Preference of the source as given by the extractor
- - `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8-native` > `m3u8` > `http-dash-segments` > other > `mms`/`rtsp` > unknown > `f4f`/`f4m`)
+ - `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8_native`/`m3u8` > `http_dash_segments`> `websocket_frag` > other > `mms`/`rtsp` > unknown > `f4f`/`f4m`)
  - `vcodec`: Video Codec (`av01` > `vp9.2` > `vp9` > `h265` > `h264` > `vp8` > `h263` > `theora` > other > unknown)
  - `acodec`: Audio Codec (`opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `ac3` > `dts` > other > unknown)
  - `codec`: Equivalent to `vcodec,acodec`
@@ -1083,11 +1186,11 @@ You can change the criteria for being considered the `best` by using `-S` (`--fo
  - `br`: Equivalent to using `tbr,vbr,abr`
  - `asr`: Audio sample rate in Hz
 
-Note that any other **numerical** field made available by the extractor can also be used. All fields, unless specified otherwise, are sorted in decending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a prefered value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two prefered values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB.
+Note that any other **numerical** field made available by the extractor can also be used. All fields, unless specified otherwise, are sorted in descending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a preferred value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two preferred values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB.
 
 The fields `hasvid`, `ie_pref`, `lang` are always given highest priority in sorting, irrespective of the user-defined order. This behaviour can be changed by using `--force-format-sort`. Apart from these, the default order used is: `quality,res,fps,codec:vp9.2,size,br,asr,proto,ext,hasaud,source,id`. Note that the extractors may override this default order, but they cannot override the user-provided order.
 
-If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all repects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`.
+If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all respects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`.
 
 **Tip**: You can use the `-v -F` to see how the formats have been sorted (worst to best).
 
@@ -1096,7 +1199,7 @@ If your format selector is `worst`, the last item is selected after sorting. Thi
 Note that on Windows you may need to use double quotes instead of single.
 
 ```bash
-# Download and merge the best best video-only format and the best audio-only format,
+# Download and merge the best video-only format and the best audio-only format,
 # or download the best combined format if video-only format is not available
 $ yt-dlp -f 'bv+ba/b'
 
@@ -1198,20 +1301,20 @@ $ yt-dlp -S '+codec:h264'
 
 # More complex examples
 
-# Download the best video no better than 720p prefering framerate greater than 30,
-# or the worst video (still prefering framerate greater than 30) if there is no such video
+# Download the best video no better than 720p preferring framerate greater than 30,
+# or the worst video (still preferring framerate greater than 30) if there is no such video
 $ yt-dlp -f '((bv*[fps>30]/bv*)[height<=720]/(wv*[fps>30]/wv*)) + ba / (b[fps>30]/b)[height<=720]/(w[fps>30]/w)'
 
 # Download the video with the largest resolution no better than 720p,
 # or the video with the smallest resolution available if there is no such video,
-# prefering larger framerate for formats with the same resolution
+# preferring larger framerate for formats with the same resolution
 $ yt-dlp -S 'res:720,fps'
 
 
 
 # Download the video with smallest resolution no worse than 480p,
 # or the video with the largest resolution available if there is no such video,
-# prefering better codec and then larger total bitrate for the same resolution
+# preferring better codec and then larger total bitrate for the same resolution
 $ yt-dlp -S '+res:480,codec,br'
 ```
 
@@ -1221,7 +1324,9 @@ The metadata obtained the the extractors can be modified by using `--parse-metad
 
 Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.
 
-You can also use this to change only the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. You can use this to set a different "description" and "synopsis", for example.
+This option also has a few special uses:
+* You can use this to change the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. You can use this to set a different "description" and "synopsis", for example
+* You can download an additional URL based on the metadata of the currently downloaded video. To do this, set the field `additional_urls` to the URL that you want to download. Eg: `--parse-metadata "description:(?P<additional_urls>https?://www\.vimeo\.com/\d+)` will download the first vimeo video found in the description
 
 ## Modifying metadata examples
 
@@ -1242,6 +1347,29 @@ $ yt-dlp --parse-metadata 'description:(?s)(?P<meta_comment>.+)' --add-metadata
 
 ```
 
+# EXTRACTOR ARGUMENTS
+
+Some extractors accept additional arguments which can be passed using `--extractor-args KEY:ARGS`. `ARGS` is a `;` (semicolon) seperated string of `ARG=VAL1,VAL2`. Eg: `--extractor-args "youtube:skip=dash,hls;player_client=android" --extractor-args "funimation:version=uncut"`
+
+The following extractors use this feature:
+* **youtube**
+    * `skip`: `hls` or `dash` (or both) to skip download of the respective manifests
+    * `player_client`: Clients to extract video data from - one or more of `web`, `android`, `ios`, `web_music`, `android_music`, `ios_music`. By default, `android,web` is used. If the URL is from `music.youtube.com`, `android,web,android_music,web_music` is used
+    * `player_skip`: `configs` - skip any requests for client configs and use defaults
+    * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side).
+    * `max_comments`: maximum amount of comments to download (default all).
+    * `max_comment_depth`: maximum depth for nested comments. YouTube supports depths 1 or 2 (default). 
+
+* **funimation**
+    * `language`: Languages to extract. Eg: `funimation:language=english,japanese`
+    * `version`: The video version to extract - `uncut` or `simulcast`
+
+* **vikiChannel**
+    * `video_types`: Types of videos to download - one or more of `episodes`, `movies`, `clips`, `trailers`
+
+NOTE: These options may be changed/removed in the future without concern for backward compatibility
+
+
 # PLUGINS
 
 Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`. Currently only `extractor` plugins are supported. Support for `downloader` and `postprocessor` plugins may be added in the future. See [ytdlp_plugins](ytdlp_plugins) for example.
@@ -1252,39 +1380,71 @@ Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`. Currently
 
 These are all the deprecated options and the current alternative to achieve the same effect
 
+#### Not recommended
+While these options still work, their use is not recommended since there are other alternatives to achieve the same
+
+    --get-description                --print description
+    --get-duration                   --print duration_string
+    --get-filename                   --print filename
+    --get-format                     --print format
+    --get-id                         --print id
+    --get-thumbnail                  --print thumbnail
+    -e, --get-title                  --print title
+    -g, --get-url                    --print urls
+    --all-formats                    -f all
+    --all-subs                       --sub-langs all --write-subs
+    --autonumber-size NUMBER         Use string formatting. Eg: %(autonumber)03d
+    --autonumber-start NUMBER        Use internal field formatting like %(autonumber+NUMBER)s
+    --metadata-from-title FORMAT     --parse-metadata "%(title)s:FORMAT"
+    --hls-prefer-native              --downloader "m3u8:native"
+    --hls-prefer-ffmpeg              --downloader "m3u8:ffmpeg"
+    --list-formats-old               --compat-options list-formats (Alias: --no-list-formats-as-table)
+    --list-formats-as-table          --compat-options -list-formats [Default] (Alias: --no-list-formats-old)
+    --sponskrub-args ARGS            --ppa "sponskrub:ARGS"
+    --youtube-skip-dash-manifest     --extractor-args "youtube:skip=dash" (Alias: --no-youtube-include-dash-manifest)
+    --youtube-skip-hls-manifest      --extractor-args "youtube:skip=hls" (Alias: --no-youtube-include-hls-manifest)
+    --youtube-include-dash-manifest  Default (Alias: --no-youtube-skip-dash-manifest)
+    --youtube-include-hls-manifest   Default (Alias: --no-youtube-skip-hls-manifest)
+    --test                           Used by developers for testing extractors. Not intended for the end user
+    --youtube-print-sig-code         Used for testing youtube signatures
+
+
+#### Old aliases
+These are aliases that are no longer documented for various reasons
+
+    --avconv-location                --ffmpeg-location
     --cn-verification-proxy URL      --geo-verification-proxy URL
+    --dump-headers                   --print-traffic
+    --dump-intermediate-pages        --dump-pages
+    --force-write-download-archive   --force-write-archive
+    --load-info                      --load-info-json
+    --no-split-tracks                --no-split-chapters
+    --no-write-srt                   --no-write-subs
+    --prefer-unsecure                --prefer-insecure
+    --rate-limit RATE                --limit-rate RATE
+    --split-tracks                   --split-chapters
+    --srt-lang LANGS                 --sub-langs LANGS
+    --trim-file-names LENGTH         --trim-filenames LENGTH
+    --write-srt                      --write-subs
+    --yes-overwrites                 --force-overwrites
+
+#### No longer supported
+These options may no longer work as intended
+
+    --prefer-avconv                  avconv is not officially supported by yt-dlp (Alias: --no-prefer-ffmpeg)
+    --prefer-ffmpeg                  Default (Alias: --no-prefer-avconv)
+    -C, --call-home                  Not implemented
+    --no-call-home                   Default
+    --include-ads                    No longer supported
+    --no-include-ads                 Default
+
+#### Removed
+These options were deprecated since 2014 and have now been entirely removed
+
     --id                             -o "%(id)s.%(ext)s"
     -A, --auto-number                -o "%(autonumber)s-%(id)s.%(ext)s"
     -t, --title                      -o "%(title)s-%(id)s.%(ext)s"
     -l, --literal                    -o accepts literal names
-    --all-formats                    -f all
-    --autonumber-size NUMBER         Use string formatting. Eg: %(autonumber)03d
-    --metadata-from-title FORMAT     --parse-metadata "%(title)s:FORMAT"
-    --prefer-avconv                  avconv is no longer officially supported (Alias: --no-prefer-ffmpeg)
-    --prefer-ffmpeg                  Default (Alias: --no-prefer-avconv)
-    --hls-prefer-native              --downloader "m3u8:native"
-    --hls-prefer-ffmpeg              --downloader "m3u8:ffmpeg"
-    --avconv-location                avconv is no longer officially supported
-    -C, --call-home                  Not implemented
-    --no-call-home                   Default
-    --include-ads                    Not implemented
-    --no-include-ads                 Default
-    --write-srt                      --write-subs
-    --no-write-srt                   --no-write-subs
-    --srt-lang LANGS                 --sub-langs LANGS
-    --prefer-unsecure                --prefer-insecure
-    --rate-limit RATE                --limit-rate RATE
-    --force-write-download-archive   --force-write-archive
-    --dump-intermediate-pages        --dump-pages
-    --dump-headers                   --print-traffic
-    --youtube-print-sig-code         No longer supported
-    --trim-file-names LENGTH         --trim-filenames LENGTH
-    --yes-overwrites                 --force-overwrites
-    --load-info                      --load-info-json
-    --split-tracks                   --split-chapters
-    --no-split-tracks                --no-split-chapters
-    --sponskrub-args ARGS            --ppa "sponskrub:ARGS"
-    --test                           Only used for testing extractors
 
 
 # MORE

devscripts/bash-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import os

devscripts/buildserver.py

@@ -1,3 +1,5 @@
+# UNUSED
+
 #!/usr/bin/python3
 
 import argparse

devscripts/check-porn.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 """

devscripts/create-github-release.py

@@ -1,6 +1,6 @@
 # Unused
 
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/fish-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import optparse

devscripts/generate_aes_testdata.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import codecs

devscripts/gh-pages.unused/update-copyright.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import with_statement, unicode_literals

devscripts/lazy_load_template.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

devscripts/make_contributing.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # import io

devscripts/make_issue_template.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/make_lazy_extractors.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals, print_function
 
 from inspect import getsource
@@ -14,9 +15,14 @@ lazy_extractors_filename = sys.argv[1]
 if os.path.exists(lazy_extractors_filename):
     os.remove(lazy_extractors_filename)
 
+# Block plugins from loading
+os.rename('ytdlp_plugins', 'ytdlp_plugins_blocked')
+
 from yt_dlp.extractor import _ALL_CLASSES
 from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
 
+os.rename('ytdlp_plugins_blocked', 'ytdlp_plugins')
+
 with open('devscripts/lazy_load_template.py', 'rt') as f:
     module_template = f.read()
 

devscripts/make_readme.py

@@ -1,3 +1,8 @@
+#!/usr/bin/env python3
+
+# yt-dlp --help | make_readme.py
+# This must be run in a console of correct width
+
 from __future__ import unicode_literals
 
 import io

devscripts/make_supportedsites.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/prepare_manpage.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/release.sh

@@ -1,4 +1,5 @@
 # Unused
+
 #!/bin/bash
 
 # IMPORTANT: the following assumptions are made

devscripts/show-downloads-statistics.py

@@ -1,6 +1,6 @@
 # Unused
 
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import itertools

devscripts/update-version.py

@@ -1,4 +1,6 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals
+
 from datetime import datetime
 # import urllib.request
 

devscripts/wine-py2exe.sh

@@ -1,3 +1,5 @@
+# UNUSED
+
 #!/bin/bash
 
 # Run with as parameter a setup.py that works in the current directory

devscripts/zsh-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import os

pyinst.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -6,6 +6,7 @@ import sys
 # import os
 import platform
 
+from PyInstaller.utils.hooks import collect_submodules
 from PyInstaller.utils.win32.versioninfo import (
     VarStruct, VarFileInfo, StringStruct, StringTable,
     StringFileInfo, FixedFileInfo, VSVersionInfo, SetVersion,
@@ -17,7 +18,7 @@ assert arch in ('32', '64')
 print('Building %sbit version' % arch)
 _x86 = '_x86' if arch == '32' else ''
 
-FILE_DESCRIPTION = 'Media Downloader%s' % (' (32 Bit)' if _x86 else '')
+FILE_DESCRIPTION = 'yt-dlp%s' % (' (32 Bit)' if _x86 else '')
 
 # root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
 # print('Changing working directory to %s' % root_dir)
@@ -58,21 +59,23 @@ VERSION_FILE = VSVersionInfo(
                     ),
                     StringStruct('OriginalFilename', 'yt-dlp%s.exe' % _x86),
                     StringStruct('ProductName', 'yt-dlp%s' % _x86),
-                    StringStruct('ProductVersion', '%s%s' % (VERSION, _x86)),
+                    StringStruct(
+                        'ProductVersion',
+                        '%s%s on Python %s' % (VERSION, _x86, platform.python_version())),
                 ])]),
         VarFileInfo([VarStruct('Translation', [0, 1200])])
     ]
 )
 
+dependancies = ['Crypto', 'mutagen'] + collect_submodules('websockets')
+excluded_modules = ['test', 'ytdlp_plugins', 'youtube-dl', 'youtube-dlc']
+
 PyInstaller.__main__.run([
     '--name=yt-dlp%s' % _x86,
     '--onefile',
     '--icon=devscripts/cloud.ico',
-    '--exclude-module=youtube_dl',
-    '--exclude-module=test',
-    '--exclude-module=ytdlp_plugins',
-    '--hidden-import=mutagen',
-    '--hidden-import=Crypto',
+    *[f'--exclude-module={module}' for module in excluded_modules],
+    *[f'--hidden-import={module}' for module in dependancies],
     '--upx-exclude=vcruntime140.dll',
     'yt_dlp/__main__.py',
 ])

requirements.txt

@@ -1,2 +1,3 @@
 mutagen
 pycryptodome
+websockets

setup.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from setuptools import setup, Command, find_packages
@@ -9,45 +9,44 @@ from distutils.spawn import spawn
 
 
 # Get the version from yt_dlp/version.py without importing the package
-exec(compile(open('yt_dlp/version.py').read(),
-             'yt_dlp/version.py', 'exec'))
+exec(compile(open('yt_dlp/version.py').read(), 'yt_dlp/version.py', 'exec'))
 
 
 DESCRIPTION = 'Command-line program to download videos from YouTube.com and many other other video platforms.'
 
 LONG_DESCRIPTION = '\n\n'.join((
     'Official repository: <https://github.com/yt-dlp/yt-dlp>',
-    '**PS**: Many links in this document will not work since this is a copy of the README.md from Github',
-    open("README.md", "r", encoding="utf-8").read()))
+    '**PS**: Some links in this document will not work since this is a copy of the README.md from Github',
+    open('README.md', 'r', encoding='utf-8').read()))
 
-REQUIREMENTS = ['mutagen', 'pycryptodome']
+REQUIREMENTS = ['mutagen', 'pycryptodome', 'websockets']
+
+if sys.argv[1:2] == ['py2exe']:
+    raise NotImplementedError('py2exe is not currently supported; instead, use "pyinst.py" to build with pyinstaller')
 
 
-if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
-    print("inv")
-else:
-    files_spec = [
-        ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
-        ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
-        ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
-        ('share/doc/yt_dlp', ['README.txt']),
-        ('share/man/man1', ['yt-dlp.1'])
-    ]
-    root = os.path.dirname(os.path.abspath(__file__))
-    data_files = []
-    for dirname, files in files_spec:
-        resfiles = []
-        for fn in files:
-            if not os.path.exists(fn):
-                warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first.' % fn)
-            else:
-                resfiles.append(fn)
-        data_files.append((dirname, resfiles))
+files_spec = [
+    ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
+    ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
+    ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
+    ('share/doc/yt_dlp', ['README.txt']),
+    ('share/man/man1', ['yt-dlp.1'])
+]
+root = os.path.dirname(os.path.abspath(__file__))
+data_files = []
+for dirname, files in files_spec:
+    resfiles = []
+    for fn in files:
+        if not os.path.exists(fn):
+            warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first' % fn)
+        else:
+            resfiles.append(fn)
+    data_files.append((dirname, resfiles))
 
-    params = {
-        'data_files': data_files,
-    }
-    params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
+params = {
+    'data_files': data_files,
+}
+params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
 
 
 class build_lazy_extractors(Command):
@@ -61,23 +60,21 @@ class build_lazy_extractors(Command):
         pass
 
     def run(self):
-        spawn(
-            [sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
-            dry_run=self.dry_run,
-        )
+        spawn([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
+              dry_run=self.dry_run)
 
 
-packages = find_packages(exclude=("youtube_dl", "test", "ytdlp_plugins"))
+packages = find_packages(exclude=('youtube_dl', 'test', 'ytdlp_plugins'))
 
 setup(
-    name="yt-dlp",
+    name='yt-dlp',
     version=__version__,
-    maintainer="pukkandan",
-    maintainer_email="pukkandan.ytdlp@gmail.com",
+    maintainer='pukkandan',
+    maintainer_email='pukkandan.ytdlp@gmail.com',
     description=DESCRIPTION,
     long_description=LONG_DESCRIPTION,
-    long_description_content_type="text/markdown",
-    url="https://github.com/yt-dlp/yt-dlp",
+    long_description_content_type='text/markdown',
+    url='https://github.com/yt-dlp/yt-dlp',
     packages=packages,
     install_requires=REQUIREMENTS,
     project_urls={
@@ -87,30 +84,20 @@ setup(
         #'Funding': 'https://donate.pypi.org',
     },
     classifiers=[
-        "Topic :: Multimedia :: Video",
-        "Development Status :: 5 - Production/Stable",
-        "Environment :: Console",
-        "Programming Language :: Python",
-        "Programming Language :: Python :: 2",
-        "Programming Language :: Python :: 2.6",
-        "Programming Language :: Python :: 2.7",
-        "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.2",
-        "Programming Language :: Python :: 3.3",
-        "Programming Language :: Python :: 3.4",
-        "Programming Language :: Python :: 3.5",
-        "Programming Language :: Python :: 3.6",
-        "Programming Language :: Python :: 3.7",
-        "Programming Language :: Python :: 3.8",
-        "Programming Language :: Python :: Implementation",
-        "Programming Language :: Python :: Implementation :: CPython",
-        "Programming Language :: Python :: Implementation :: IronPython",
-        "Programming Language :: Python :: Implementation :: Jython",
-        "Programming Language :: Python :: Implementation :: PyPy",
-        "License :: Public Domain",
-        "Operating System :: OS Independent",
+        'Topic :: Multimedia :: Video',
+        'Development Status :: 5 - Production/Stable',
+        'Environment :: Console',
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: Implementation',
+        'Programming Language :: Python :: Implementation :: CPython',
+        'Programming Language :: Python :: Implementation :: PyPy',
+        'License :: Public Domain',
+        'Operating System :: OS Independent',
     ],
-    python_requires='>=2.6',
+    python_requires='>=3.6',
 
     cmdclass={'build_lazy_extractors': build_lazy_extractors},
     **params

supportedsites.md

@@ -82,6 +82,7 @@
  - **audiomack**
  - **audiomack:album**
  - **Audius**: Audius.co
+ - **audius:artist**: Audius.co profile/artist pages
  - **audius:playlist**: Audius.co playlists
  - **audius:track**: Audius track ID or API link. Prepend with "audius:"
  - **AWAAN**
@@ -130,7 +131,6 @@
  - **bitwave:stream**
  - **BleacherReport**
  - **BleacherReportCMS**
- - **blinkx**
  - **Bloomberg**
  - **BokeCC**
  - **BongaCams**
@@ -259,6 +259,7 @@
  - **dlive:vod**
  - **DoodStream**
  - **Dotsub**
+ - **Douyin**
  - **DouyuShow**
  - **DouyuTV**: 斗鱼
  - **DPlay**
@@ -306,6 +307,8 @@
  - **EyedoTV**
  - **facebook**
  - **FacebookPluginsVideo**
+ - **fancode:live**
+ - **fancode:vod**
  - **faz.net**
  - **fc2**
  - **fc2:embed**
@@ -342,6 +345,8 @@
  - **FrontendMastersLesson**
  - **FujiTVFODPlus7**
  - **Funimation**
+ - **funimation:page**
+ - **funimation:show**
  - **Funk**
  - **Fusion**
  - **Fux**
@@ -391,6 +396,7 @@
  - **HotNewHipHop**
  - **hotstar**
  - **hotstar:playlist**
+ - **hotstar:series**
  - **Howcast**
  - **HowStuffWorks**
  - **hrfernsehen**
@@ -494,8 +500,6 @@
  - **LinuxAcademy**
  - **LiTV**
  - **LiveJournal**
- - **LiveLeak**
- - **LiveLeakEmbed**
  - **livestream**
  - **livestream:original**
  - **LnkGo**
@@ -584,6 +588,7 @@
  - **Mwave**
  - **MwaveMeetGreet**
  - **Mxplayer**
+ - **MxplayerShow**
  - **MyChannels**
  - **MySpace**
  - **MySpace:album**
@@ -724,6 +729,7 @@
  - **pandora.tv**: 판도라TV
  - **ParamountNetwork**
  - **parliamentlive.tv**: UK parliament videos
+ - **Parlview**
  - **Patreon**
  - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
  - **PearVideo**
@@ -747,6 +753,7 @@
  - **play.fm**
  - **player.sky.it**
  - **PlayPlusTV**
+ - **PlayStuff**
  - **PlaysTV**
  - **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
  - **Playvid**
@@ -763,6 +770,7 @@
  - **PopcornTV**
  - **PornCom**
  - **PornerBros**
+ - **PornFlip**
  - **PornHd**
  - **PornHub**: PornHub and Thumbzilla
  - **PornHubPagedVideoList**
@@ -805,6 +813,8 @@
  - **RCS**
  - **RCSEmbeds**
  - **RCSVarious**
+ - **RCTIPlus**
+ - **RCTIPlusSeries**
  - **RDS**: RDS.ca
  - **RedBull**
  - **RedBullEmbed**
@@ -855,6 +865,7 @@
  - **safari**: safaribooksonline.com online video
  - **safari:api**
  - **safari:course**: safaribooksonline.com online courses
+ - **Saitosan**
  - **SAKTV**
  - **SaltTV**
  - **SampleFocus**
@@ -879,6 +890,7 @@
  - **Shahid**
  - **ShahidShow**
  - **Shared**: shared.sx
+ - **ShemarooMe**
  - **ShowRoomLive**
  - **simplecast**
  - **simplecast:episode**
@@ -898,6 +910,7 @@
  - **Snotr**
  - **Sohu**
  - **SonyLIV**
+ - **SonyLIVSeries**
  - **soundcloud**
  - **soundcloud:playlist**
  - **soundcloud:search**: Soundcloud search
@@ -976,6 +989,7 @@
  - **Telecinco**: telecinco.es, cuatro.com and mediaset.es
  - **Telegraaf**
  - **TeleMB**
+ - **Telemundo**
  - **TeleQuebec**
  - **TeleQuebecEmission**
  - **TeleQuebecLive**
@@ -1016,6 +1030,7 @@
  - **TruTV**
  - **Tube8**
  - **TubiTv**
+ - **TubiTvShow**
  - **Tumblr**
  - **tunein:clip**
  - **tunein:program**
@@ -1058,6 +1073,8 @@
  - **TVPlayHome**
  - **Tweakers**
  - **TwitCasting**
+ - **TwitCastingLive**
+ - **TwitCastingUser**
  - **twitch:clips**
  - **twitch:stream**
  - **twitch:vod**
@@ -1075,6 +1092,7 @@
  - **UDNEmbed**: 聯合影音
  - **UFCArabia**
  - **UFCTV**
+ - **ukcolumn**
  - **UKTVPlay**
  - **umg:de**: Universal Music Deutschland
  - **Unistra**
@@ -1118,6 +1136,8 @@
  - **videomore:video**
  - **VideoPress**
  - **Vidio**
+ - **VidioLive**
+ - **VidioPremier**
  - **VidLii**
  - **vidme**
  - **vidme:user**
@@ -1157,6 +1177,7 @@
  - **VODPlatform**
  - **VoiceRepublic**
  - **Voot**
+ - **VootSeries**
  - **VoxMedia**
  - **VoxMediaVolume**
  - **vpro**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
@@ -1186,6 +1207,7 @@
  - **wdr:mobile**
  - **WDRElefant**
  - **WDRPage**
+ - **web.archive:youtube**: web.archive.org saved youtube videos
  - **Webcaster**
  - **WebcasterFeed**
  - **WebOfStories**
@@ -1193,6 +1215,7 @@
  - **Weibo**
  - **WeiboMobile**
  - **WeiqiTV**: WQTV
+ - **whowatch**
  - **WimTV**
  - **Wistia**
  - **WistiaPlaylist**
@@ -1203,7 +1226,7 @@
  - **WWE**
  - **XBef**
  - **XboxClips**
- - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, XVideoSharing
+ - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, WolfStream, XVideoSharing
  - **XHamster**
  - **XHamsterEmbed**
  - **XHamsterUser**

test/parameters.json

@@ -1,41 +1,40 @@
 {
-    "consoletitle": false, 
-    "continuedl": true, 
-    "forcedescription": false, 
-    "forcefilename": false, 
-    "forceformat": false, 
-    "forcethumbnail": false, 
-    "forcetitle": false, 
-    "forceurl": false, 
+    "consoletitle": false,
+    "continuedl": true,
+    "forcedescription": false,
+    "forcefilename": false,
+    "forceformat": false,
+    "forcethumbnail": false,
+    "forcetitle": false,
+    "forceurl": false,
     "force_write_download_archive": false,
     "format": "best",
-    "ignoreerrors": false, 
-    "listformats": null, 
-    "logtostderr": false, 
-    "matchtitle": null, 
-    "max_downloads": null, 
-    "overwrites": null, 
-    "nopart": false, 
-    "noprogress": false, 
-    "outtmpl": "%(id)s.%(ext)s", 
-    "password": null, 
-    "playlistend": -1, 
-    "playliststart": 1, 
-    "prefer_free_formats": false, 
-    "quiet": false, 
-    "ratelimit": null, 
-    "rejecttitle": null, 
-    "retries": 10, 
-    "simulate": false, 
-    "subtitleslang": null, 
+    "ignoreerrors": false,
+    "listformats": null,
+    "logtostderr": false,
+    "matchtitle": null,
+    "max_downloads": null,
+    "overwrites": null,
+    "nopart": false,
+    "noprogress": false,
+    "outtmpl": "%(id)s.%(ext)s",
+    "password": null,
+    "playliststart": 1,
+    "prefer_free_formats": false,
+    "quiet": false,
+    "ratelimit": null,
+    "rejecttitle": null,
+    "retries": 10,
+    "simulate": false,
+    "subtitleslang": null,
     "subtitlesformat": "best",
-    "test": true, 
-    "updatetime": true, 
-    "usenetrc": false, 
-    "username": null, 
-    "verbose": true, 
-    "writedescription": false, 
-    "writeinfojson": true, 
+    "test": true,
+    "updatetime": true,
+    "usenetrc": false,
+    "username": null,
+    "verbose": true,
+    "writedescription": false,
+    "writeinfojson": true,
     "writeannotations": false,
     "writelink": false,
     "writeurllink": false,

test/test_InfoExtractor.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -450,7 +450,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': 'aud2-English',
@@ -458,7 +458,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': 'aud3-English',
@@ -466,14 +466,14 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': '530',
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -482,7 +482,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -491,7 +491,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -500,7 +500,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -509,7 +509,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -518,7 +518,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -527,7 +527,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -536,7 +536,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -545,7 +545,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -554,7 +554,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -563,7 +563,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -572,7 +572,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -581,7 +581,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -590,7 +590,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -599,7 +599,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -608,7 +608,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -617,7 +617,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -626,7 +626,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -635,7 +635,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -644,7 +644,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -653,7 +653,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -662,7 +662,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -671,7 +671,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -680,21 +680,190 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
-                }]
+                }],
+                {}
+            ),
+            (
+                'bipbop_16x9',
+                'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                [{
+                    'format_id': 'bipbop_audio-BipBop Audio 2',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/alternate_audio_aac/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'language': 'eng',
+                    'ext': 'mp4',
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'vcodec': 'none',
+                    'audio_ext': 'mp4',
+                    'video_ext': 'none',
+                }, {
+                    'format_id': '41',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear0/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 41.457,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'vcodec': 'none',
+                    'acodec': 'mp4a.40.2',
+                    'audio_ext': 'mp4',
+                    'video_ext': 'none',
+                    'abr': 41.457,
+                }, {
+                    'format_id': '263',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear1/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 263.851,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 416,
+                    'height': 234,
+                    'vcodec': 'avc1.4d400d',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 263.851,
+                    'abr': 0,
+                }, {
+                    'format_id': '577',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear2/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 577.61,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 640,
+                    'height': 360,
+                    'vcodec': 'avc1.4d401e',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 577.61,
+                    'abr': 0,
+                }, {
+                    'format_id': '915',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear3/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 915.905,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 960,
+                    'height': 540,
+                    'vcodec': 'avc1.4d401f',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 915.905,
+                    'abr': 0,
+                }, {
+                    'format_id': '1030',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear4/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 1030.138,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 1280,
+                    'height': 720,
+                    'vcodec': 'avc1.4d401f',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 1030.138,
+                    'abr': 0,
+                }, {
+                    'format_id': '1924',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear5/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 1924.009,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 1920,
+                    'height': 1080,
+                    'vcodec': 'avc1.4d401f',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 1924.009,
+                    'abr': 0,
+                }],
+                {
+                    'en': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                    'fr': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                    'es': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                    'ja': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                }
             ),
         ]
 
-        for m3u8_file, m3u8_url, expected_formats in _TEST_CASES:
+        for m3u8_file, m3u8_url, expected_formats, expected_subs in _TEST_CASES:
             with io.open('./test/testdata/m3u8/%s.m3u8' % m3u8_file,
                          mode='r', encoding='utf-8') as f:
-                formats = self.ie._parse_m3u8_formats(
+                formats, subs = self.ie._parse_m3u8_formats_and_subtitles(
                     f.read(), m3u8_url, ext='mp4')
                 self.ie._sort_formats(formats)
                 expect_value(self, formats, expected_formats, None)
+                expect_value(self, subs, expected_subs, None)
 
     def test_parse_mpd_formats(self):
         _TEST_CASES = [
@@ -780,7 +949,8 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'tbr': 5997.485,
                     'width': 1920,
                     'height': 1080,
-                }]
+                }],
+                {},
             ), (
                 # https://github.com/ytdl-org/youtube-dl/pull/14844
                 'urls_only',
@@ -863,7 +1033,8 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'tbr': 4400,
                     'width': 1920,
                     'height': 1080,
-                }]
+                }],
+                {},
             ), (
                 # https://github.com/ytdl-org/youtube-dl/issues/20346
                 # Media considered unfragmented even though it contains
@@ -909,18 +1080,328 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'width': 360,
                     'height': 360,
                     'fps': 30,
-                }]
+                }],
+                {},
+            ), (
+                'subtitles',
+                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/',
+                [{
+                    'format_id': 'audio=128001',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'm4a',
+                    'tbr': 128.001,
+                    'asr': 48000,
+                    'format_note': 'DASH audio',
+                    'container': 'm4a_dash',
+                    'vcodec': 'none',
+                    'acodec': 'mp4a.40.2',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'audio_ext': 'm4a',
+                    'video_ext': 'none',
+                    'abr': 128.001,
+                }, {
+                    'format_id': 'video=100000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 336,
+                    'height': 144,
+                    'tbr': 100,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 100,
+                }, {
+                    'format_id': 'video=326000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 562,
+                    'height': 240,
+                    'tbr': 326,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 326,
+                }, {
+                    'format_id': 'video=698000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 844,
+                    'height': 360,
+                    'tbr': 698,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 698,
+                }, {
+                    'format_id': 'video=1493000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 1126,
+                    'height': 480,
+                    'tbr': 1493,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 1493,
+                }, {
+                    'format_id': 'video=4482000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 1688,
+                    'height': 720,
+                    'tbr': 4482,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 4482,
+                }],
+                {
+                    'en': [
+                        {
+                            'ext': 'mp4',
+                            'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                            'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                            'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                            'protocol': 'http_dash_segments',
+                        }
+                    ]
+                },
             )
         ]
 
-        for mpd_file, mpd_url, mpd_base_url, expected_formats in _TEST_CASES:
+        for mpd_file, mpd_url, mpd_base_url, expected_formats, expected_subtitles in _TEST_CASES:
             with io.open('./test/testdata/mpd/%s.mpd' % mpd_file,
                          mode='r', encoding='utf-8') as f:
-                formats = self.ie._parse_mpd_formats(
+                formats, subtitles = self.ie._parse_mpd_formats_and_subtitles(
                     compat_etree_fromstring(f.read().encode('utf-8')),
                     mpd_base_url=mpd_base_url, mpd_url=mpd_url)
                 self.ie._sort_formats(formats)
                 expect_value(self, formats, expected_formats, None)
+                expect_value(self, subtitles, expected_subtitles, None)
+
+    def test_parse_ism_formats(self):
+        _TEST_CASES = [
+            (
+                'sintel',
+                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                [{
+                    'format_id': 'audio-128',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'isma',
+                    'tbr': 128,
+                    'asr': 48000,
+                    'vcodec': 'none',
+                    'acodec': 'AACL',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'audio',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 0,
+                        'height': 0,
+                        'fourcc': 'AACL',
+                        'codec_private_data': '1190',
+                        'sampling_rate': 48000,
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'audio_ext': 'isma',
+                    'video_ext': 'none',
+                    'abr': 128,
+                }, {
+                    'format_id': 'video-100',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 336,
+                    'height': 144,
+                    'tbr': 100,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 336,
+                        'height': 144,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 100,
+                }, {
+                    'format_id': 'video-326',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 562,
+                    'height': 240,
+                    'tbr': 326,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 562,
+                        'height': 240,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 326,
+                }, {
+                    'format_id': 'video-698',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 844,
+                    'height': 360,
+                    'tbr': 698,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 844,
+                        'height': 360,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 698,
+                }, {
+                    'format_id': 'video-1493',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 1126,
+                    'height': 480,
+                    'tbr': 1493,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 1126,
+                        'height': 480,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 1493,
+                }, {
+                    'format_id': 'video-4482',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 1688,
+                    'height': 720,
+                    'tbr': 4482,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 1688,
+                        'height': 720,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 4482,
+                }],
+                {
+                    'eng': [
+                        {
+                            'ext': 'ismt',
+                            'protocol': 'ism',
+                            'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                            'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                            '_download_params': {
+                                'stream_type': 'text',
+                                'duration': 8880746666,
+                                'timescale': 10000000,
+                                'fourcc': 'TTML',
+                                'codec_private_data': ''
+                            }
+                        }
+                    ]
+                },
+            ),
+        ]
+
+        for ism_file, ism_url, expected_formats, expected_subtitles in _TEST_CASES:
+            with io.open('./test/testdata/ism/%s.Manifest' % ism_file,
+                         mode='r', encoding='utf-8') as f:
+                formats, subtitles = self.ie._parse_ism_formats_and_subtitles(
+                    compat_etree_fromstring(f.read().encode('utf-8')), ism_url=ism_url)
+                self.ie._sort_formats(formats)
+                expect_value(self, formats, expected_formats, None)
+                expect_value(self, subtitles, expected_subtitles, None)
 
     def test_parse_f4m_formats(self):
         _TEST_CASES = [

test/test_YoutubeDL.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -17,7 +17,7 @@ from yt_dlp.compat import compat_str, compat_urllib_error
 from yt_dlp.extractor import YoutubeIE
 from yt_dlp.extractor.common import InfoExtractor
 from yt_dlp.postprocessor.common import PostProcessor
-from yt_dlp.utils import ExtractorError, match_filter_func
+from yt_dlp.utils import ExtractorError, int_or_none, match_filter_func
 
 TEST_URL = 'http://localhost/sample.mp4'
 
@@ -29,11 +29,15 @@ class YDL(FakeYDL):
         self.msgs = []
 
     def process_info(self, info_dict):
+        info_dict.pop('__original_infodict', None)
         self.downloaded_info_dicts.append(info_dict)
 
     def to_screen(self, msg):
         self.msgs.append(msg)
 
+    def dl(self, *args, **kwargs):
+        assert False, 'Downloader must not be invoked for test_YoutubeDL'
+
 
 def _make_result(formats, **kwargs):
     res = {
@@ -116,35 +120,24 @@ class TestFormatSelection(unittest.TestCase):
         ]
         info_dict = _make_result(formats)
 
-        ydl = YDL({'format': '20/47'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '47')
+        def test(inp, *expected, multi=False):
+            ydl = YDL({
+                'format': inp,
+                'allow_multiple_video_streams': multi,
+                'allow_multiple_audio_streams': multi,
+            })
+            ydl.process_ie_result(info_dict.copy())
+            downloaded = map(lambda x: x['format_id'], ydl.downloaded_info_dicts)
+            self.assertEqual(list(downloaded), list(expected))
 
-        ydl = YDL({'format': '20/71/worst'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '35')
-
-        ydl = YDL()
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '2')
-
-        ydl = YDL({'format': 'webm/mp4'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '47')
-
-        ydl = YDL({'format': '3gp/40/mp4'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '35')
-
-        ydl = YDL({'format': 'example-with-dashes'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], 'example-with-dashes')
+        test('20/47', '47')
+        test('20/71/worst', '35')
+        test(None, '2')
+        test('webm/mp4', '47')
+        test('3gp/40/mp4', '35')
+        test('example-with-dashes', 'example-with-dashes')
+        test('all', '35', 'example-with-dashes', '45', '47', '2')  # Order doesn't actually matter for this
+        test('mergeall', '2+47+45+example-with-dashes+35', multi=True)
 
     def test_format_selection_audio(self):
         formats = [
@@ -311,8 +304,8 @@ class TestFormatSelection(unittest.TestCase):
         self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
 
     def test_youtube_format_selection(self):
+        # FIXME: Rewrite in accordance with the new format sorting options
         return
-        # disabled for now - this needs some changes
 
         order = [
             '38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '17', '36', '13',
@@ -460,14 +453,13 @@ class TestFormatSelection(unittest.TestCase):
 
     def test_invalid_format_specs(self):
         def assert_syntax_error(format_spec):
-            ydl = YDL({'format': format_spec})
-            info_dict = _make_result([{'format_id': 'foo', 'url': TEST_URL}])
-            self.assertRaises(SyntaxError, ydl.process_ie_result, info_dict)
+            self.assertRaises(SyntaxError, YDL, {'format': format_spec})
 
         assert_syntax_error('bestvideo,,best')
         assert_syntax_error('+bestaudio')
         assert_syntax_error('bestvideo+')
         assert_syntax_error('/')
+        assert_syntax_error('[720<height]')
 
     def test_format_filtering(self):
         formats = [
@@ -601,6 +593,26 @@ class TestYoutubeDL(unittest.TestCase):
         self.assertTrue(subs)
         self.assertEqual(set(subs.keys()), set(['es', 'fr']))
 
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['all', '-en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['es', 'fr']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['en', 'fr', '-en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['fr']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['-en', 'en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['en']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['e.+']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['es', 'en']))
+
         result = get_info({'writesubtitles': True, 'writeautomaticsub': True, 'subtitleslangs': ['es', 'pt']})
         subs = result['requested_subtitles']
         self.assertTrue(subs)
@@ -627,47 +639,131 @@ class TestYoutubeDL(unittest.TestCase):
         self.assertEqual(test_dict['extractor'], 'Foo')
         self.assertEqual(test_dict['playlist'], 'funny videos')
 
-    def test_prepare_filename(self):
-        info = {
-            'id': '1234',
-            'ext': 'mp4',
-            'width': None,
-            'height': 1080,
-            'title1': '$PATH',
-            'title2': '%PATH%',
-        }
+    outtmpl_info = {
+        'id': '1234',
+        'ext': 'mp4',
+        'width': None,
+        'height': 1080,
+        'title1': '$PATH',
+        'title2': '%PATH%',
+        'title3': 'foo/bar\\test',
+        'timestamp': 1618488000,
+        'duration': 100000,
+        'playlist_index': 1,
+        '_last_playlist_index': 100,
+        'n_entries': 10,
+        'formats': [{'id': 'id1'}, {'id': 'id2'}, {'id': 'id3'}]
+    }
 
-        def fname(templ, na_placeholder='NA'):
-            params = {'outtmpl': templ}
-            if na_placeholder != 'NA':
-                params['outtmpl_na_placeholder'] = na_placeholder
+    def test_prepare_outtmpl_and_filename(self):
+        def test(tmpl, expected, *, info=None, **params):
+            params['outtmpl'] = tmpl
             ydl = YoutubeDL(params)
-            return ydl.prepare_filename(info)
-        self.assertEqual(fname('%(id)s.%(ext)s'), '1234.mp4')
-        self.assertEqual(fname('%(id)s-%(width)s.%(ext)s'), '1234-NA.mp4')
-        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(id)s.%(ext)s'
-        # Replace missing fields with 'NA' by default
-        self.assertEqual(fname(NA_TEST_OUTTMPL), 'NA-NA-1234.mp4')
-        # Or by provided placeholder
-        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder='none'), 'none-none-1234.mp4')
-        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder=''), '--1234.mp4')
-        self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
-        self.assertEqual(fname('%(height)6d.%(ext)s'), '  1080.mp4')
-        self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080  .mp4')
-        self.assertEqual(fname('%(height)06d.%(ext)s'), '001080.mp4')
-        self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)   06d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)0   6d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)   0   6d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%%'), '%')
-        self.assertEqual(fname('%%%%'), '%%')
-        self.assertEqual(fname('%%(height)06d.%(ext)s'), '%(height)06d.mp4')
-        self.assertEqual(fname('%(width)06d.%(ext)s'), 'NA.mp4')
-        self.assertEqual(fname('%(width)06d.%%(ext)s'), 'NA.%(ext)s')
-        self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
-        self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
-        self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
+            ydl._num_downloads = 1
+            self.assertEqual(ydl.validate_outtmpl(tmpl), None)
+
+            outtmpl, tmpl_dict = ydl.prepare_outtmpl(tmpl, info or self.outtmpl_info)
+            out = outtmpl % tmpl_dict
+            fname = ydl.prepare_filename(info or self.outtmpl_info)
+
+            if callable(expected):
+                self.assertTrue(expected(out))
+                self.assertTrue(expected(fname))
+            elif isinstance(expected, compat_str):
+                self.assertEqual((out, fname), (expected, expected))
+            else:
+                self.assertEqual((out, fname), expected)
+
+        # Auto-generated fields
+        test('%(id)s.%(ext)s', '1234.mp4')
+        test('%(duration_string)s', ('27:46:40', '27-46-40'))
+        test('%(epoch)d', int_or_none)
+        test('%(resolution)s', '1080p')
+        test('%(playlist_index)s', '001')
+        test('%(autonumber)s', '00001')
+        test('%(autonumber+2)03d', '005', autonumber_start=3)
+        test('%(autonumber)s', '001', autonumber_size=3)
+
+        # Escaping %
+        test('%%', '%')
+        test('%%%%', '%%')
+        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+        test('%(width)06d.%(ext)s', 'NA.mp4')
+        test('%(width)06d.%%(ext)s', 'NA.%(ext)s')
+        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+
+        # ID sanitization
+        test('%(id)s', '_abcd', info={'id': '_abcd'})
+        test('%(some_id)s', '_abcd', info={'some_id': '_abcd'})
+        test('%(formats.0.id)s', '_abcd', info={'formats': [{'id': '_abcd'}]})
+        test('%(id)s', '-abcd', info={'id': '-abcd'})
+        test('%(id)s', '.abcd', info={'id': '.abcd'})
+        test('%(id)s', 'ab__cd', info={'id': 'ab__cd'})
+        test('%(id)s', ('ab:cd', 'ab -cd'), info={'id': 'ab:cd'})
+
+        # Invalid templates
+        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%'), ValueError))
+        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%(title)'), ValueError))
+        test('%(invalid@tmpl|def)s', 'none', outtmpl_na_placeholder='none')
+        test('%()s', 'NA')
+        test('%s', '%s')
+        test('%d', '%d')
+
+        # NA placeholder
+        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(x|def)s-%(id)s.%(ext)s'
+        test(NA_TEST_OUTTMPL, 'NA-NA-def-1234.mp4')
+        test(NA_TEST_OUTTMPL, 'none-none-def-1234.mp4', outtmpl_na_placeholder='none')
+        test(NA_TEST_OUTTMPL, '--def-1234.mp4', outtmpl_na_placeholder='')
+
+        # String formatting
+        FMT_TEST_OUTTMPL = '%%(height)%s.%%(ext)s'
+        test(FMT_TEST_OUTTMPL % 's', '1080.mp4')
+        test(FMT_TEST_OUTTMPL % 'd', '1080.mp4')
+        test(FMT_TEST_OUTTMPL % '6d', '  1080.mp4')
+        test(FMT_TEST_OUTTMPL % '-6d', '1080  .mp4')
+        test(FMT_TEST_OUTTMPL % '06d', '001080.mp4')
+        test(FMT_TEST_OUTTMPL % ' 06d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '   06d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '0 6d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '0   6d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '   0   6d', ' 01080.mp4')
+
+        # Type casting
+        test('%(id)d', '1234')
+        test('%(height)c', '1')
+        test('%(ext)c', 'm')
+        test('%(id)d %(id)r', "1234 '1234'")
+        test('%(id)r %(height)r', "'1234' 1080")
+        test('%(ext)s-%(ext|def)d', 'mp4-def')
+        test('%(width|0)04d', '0000')
+        test('a%(width|)d', 'a', outtmpl_na_placeholder='none')
+
+        # Internal formatting
+        FORMATS = self.outtmpl_info['formats']
+        test('%(timestamp-1000>%H-%M-%S)s', '11-43-20')
+        test('%(id+1-height+3)05d', '00158')
+        test('%(width+100)05d', 'NA')
+        test('%(formats.0) 15s', ('% 15s' % FORMATS[0], '% 15s' % str(FORMATS[0]).replace(':', ' -')))
+        test('%(formats.0)r', (repr(FORMATS[0]), repr(FORMATS[0]).replace(':', ' -')))
+        test('%(height.0)03d', '001')
+        test('%(-height.0)04d', '-001')
+        test('%(formats.-1.id)s', FORMATS[-1]['id'])
+        test('%(formats.0.id.-1)d', FORMATS[0]['id'][-1])
+        test('%(formats.3)s', 'NA')
+        test('%(formats.:2:-1)r', repr(FORMATS[:2:-1]))
+        test('%(formats.0.id.-1+id)f', '1235.000000')
+        test('%(formats.0.id.-1+formats.1.id.-1)d', '3')
+
+        # Empty filename
+        test('%(foo|)s-%(bar|)s.%(ext)s', '-.mp4')
+        # test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4'))  # fixme
+        # test('%(foo|)s', ('', '_'))  # fixme
+
+        # Path expansion and escaping
+        test('Hello %(title1)s', 'Hello $PATH')
+        test('Hello %(title2)s', 'Hello %PATH%')
+        test('%(title3)s', ('foo/bar\\test', 'foo_bar_test'))
+        test('folder/%(title3)s', ('folder/foo/bar\\test', 'folder%sfoo_bar_test' % os.path.sep))
 
     def test_format_note(self):
         ydl = YoutubeDL()
@@ -726,7 +822,7 @@ class TestYoutubeDL(unittest.TestCase):
             def process_info(self, info_dict):
                 super(YDL, self).process_info(info_dict)
 
-            def _match_entry(self, info_dict, incomplete):
+            def _match_entry(self, info_dict, incomplete=False):
                 res = super(FilterYDL, self)._match_entry(info_dict, incomplete)
                 if res is None:
                     self.downloaded_info_dicts.append(info_dict)

test/test_YoutubeDLCookieJar.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_aes.py

@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_age_restriction.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_all_urls.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -72,15 +72,6 @@ class TestAllURLsMatching(unittest.TestCase):
         self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
         self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])
 
-    def test_youtube_extract(self):
-        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
-        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
-        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
-
     def test_facebook_matching(self):
         self.assertTrue(FacebookIE.suitable('https://www.facebook.com/Shiniknoh#!/photo.php?v=10153317450565268'))
         self.assertTrue(FacebookIE.suitable('https://www.facebook.com/cindyweather?fref=ts#!/photo.php?v=10152183998945793'))

test/test_cache.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_compat.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_cookies.py

@@ -0,0 +1,96 @@
+import unittest
+from datetime import datetime, timezone
+
+from yt_dlp import cookies
+from yt_dlp.cookies import (
+    CRYPTO_AVAILABLE,
+    LinuxChromeCookieDecryptor,
+    MacChromeCookieDecryptor,
+    WindowsChromeCookieDecryptor,
+    YDLLogger,
+    parse_safari_cookies,
+    pbkdf2_sha1,
+)
+
+
+class MonkeyPatch:
+    def __init__(self, module, temporary_values):
+        self._module = module
+        self._temporary_values = temporary_values
+        self._backup_values = {}
+
+    def __enter__(self):
+        for name, temp_value in self._temporary_values.items():
+            self._backup_values[name] = getattr(self._module, name)
+            setattr(self._module, name, temp_value)
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        for name, backup_value in self._backup_values.items():
+            setattr(self._module, name, backup_value)
+
+
+class TestCookies(unittest.TestCase):
+    def test_chrome_cookie_decryptor_linux_derive_key(self):
+        key = LinuxChromeCookieDecryptor.derive_key(b'abc')
+        self.assertEqual(key, b'7\xa1\xec\xd4m\xfcA\xc7\xb19Z\xd0\x19\xdcM\x17')
+
+    def test_chrome_cookie_decryptor_mac_derive_key(self):
+        key = MacChromeCookieDecryptor.derive_key(b'abc')
+        self.assertEqual(key, b'Y\xe2\xc0\xd0P\xf6\xf4\xe1l\xc1\x8cQ\xcb|\xcdY')
+
+    def test_chrome_cookie_decryptor_linux_v10(self):
+        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b''}):
+            encrypted_value = b'v10\xccW%\xcd\xe6\xe6\x9fM" \xa7\xb0\xca\xe4\x07\xd6'
+            value = 'USD'
+            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_chrome_cookie_decryptor_linux_v11(self):
+        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b'',
+                                   'KEYRING_AVAILABLE': True}):
+            encrypted_value = b'v11#\x81\x10>`w\x8f)\xc0\xb2\xc1\r\xf4\x1al\xdd\x93\xfd\xf8\xf8N\xf2\xa9\x83\xf1\xe9o\x0elVQd'
+            value = 'tz=Europe.London'
+            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    @unittest.skipIf(not CRYPTO_AVAILABLE, 'cryptography library not available')
+    def test_chrome_cookie_decryptor_windows_v10(self):
+        with MonkeyPatch(cookies, {
+            '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&'
+        }):
+            encrypted_value = b'v10T\xb8\xf3\xb8\x01\xa7TtcV\xfc\x88\xb8\xb8\xef\x05\xb5\xfd\x18\xc90\x009\xab\xb1\x893\x85)\x87\xe1\xa9-\xa3\xad='
+            value = '32101439'
+            decryptor = WindowsChromeCookieDecryptor('', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_chrome_cookie_decryptor_mac_v10(self):
+        with MonkeyPatch(cookies, {'_get_mac_keyring_password': lambda *args, **kwargs: b'6eIDUdtKAacvlHwBVwvg/Q=='}):
+            encrypted_value = b'v10\xb3\xbe\xad\xa1[\x9fC\xa1\x98\xe0\x9a\x01\xd9\xcf\xbfc'
+            value = '2021-06-01-22'
+            decryptor = MacChromeCookieDecryptor('', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_safari_cookie_parsing(self):
+        cookies = \
+            b'cook\x00\x00\x00\x01\x00\x00\x00i\x00\x00\x01\x00\x01\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00Y' \
+            b'\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x008\x00\x00\x00B\x00\x00\x00F\x00\x00\x00H' \
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\xa5>\xc3A\x00\x00\x80\xc3\x07:\xc3A' \
+            b'localhost\x00foo\x00/\x00test%20%3Bcookie\x00\x00\x00\x054\x07\x17 \x05\x00\x00\x00Kbplist00\xd1\x01' \
+            b'\x02_\x10\x18NSHTTPCookieAcceptPolicy\x10\x02\x08\x0b&\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00' \
+            b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00('
+
+        jar = parse_safari_cookies(cookies)
+        self.assertEqual(len(jar), 1)
+        cookie = list(jar)[0]
+        self.assertEqual(cookie.domain, 'localhost')
+        self.assertEqual(cookie.port, None)
+        self.assertEqual(cookie.path, '/')
+        self.assertEqual(cookie.name, 'foo')
+        self.assertEqual(cookie.value, 'test%20%3Bcookie')
+        self.assertFalse(cookie.secure)
+        expected_expiration = datetime(2021, 6, 18, 21, 39, 19, tzinfo=timezone.utc)
+        self.assertEqual(cookie.expires, int(expected_expiration.timestamp()))
+
+    def test_pbkdf2_sha1(self):
+        key = pbkdf2_sha1(b'peanuts', b' ' * 16, 1, 16)
+        self.assertEqual(key, b'g\xe1\x8e\x0fQ\x1c\x9b\xf3\xc9`!\xaa\x90\xd9\xd34')

test/test_download.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -121,6 +121,7 @@ def generator(test_case, tname):
         params['outtmpl'] = tname + '_' + params['outtmpl']
         if is_playlist and 'playlist' not in test_case:
             params.setdefault('extract_flat', 'in_playlist')
+            params.setdefault('playlistend', test_case.get('playlist_mincount'))
             params.setdefault('skip_download', True)
 
         ydl = YoutubeDL(params, auto_init=False)

test/test_downloader_http.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_execution.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -39,6 +39,16 @@ class TestExecution(unittest.TestCase):
         _, stderr = p.communicate()
         self.assertFalse(stderr)
 
+    def test_lazy_extractors(self):
+        try:
+            subprocess.check_call([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'], cwd=rootDir, stdout=_DEV_NULL)
+            subprocess.check_call([sys.executable, 'test/test_all_urls.py'], cwd=rootDir, stdout=_DEV_NULL)
+        finally:
+            try:
+                os.remove('yt_dlp/extractor/lazy_extractors.py')
+            except (IOError, OSError):
+                pass
+
 
 if __name__ == '__main__':
     unittest.main()

test/test_http.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_iqiyi_sdk_interpreter.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_jsinterp.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_overwrites.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import os

test/test_post_hooks.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_postprocessors.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -8,7 +8,14 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from yt_dlp.postprocessor import MetadataFromFieldPP, MetadataFromTitlePP
+from yt_dlp import YoutubeDL
+from yt_dlp.compat import compat_shlex_quote
+from yt_dlp.postprocessor import (
+    ExecAfterDownloadPP,
+    FFmpegThumbnailsConvertorPP,
+    MetadataFromFieldPP,
+    MetadataFromTitlePP,
+)
 
 
 class TestMetadataFromField(unittest.TestCase):
@@ -16,8 +23,49 @@ class TestMetadataFromField(unittest.TestCase):
         pp = MetadataFromFieldPP(None, ['title:%(title)s - %(artist)s'])
         self.assertEqual(pp._data[0]['regex'], r'(?P<title>.+)\ \-\ (?P<artist>.+)')
 
+    def test_field_to_outtmpl(self):
+        pp = MetadataFromFieldPP(None, ['title:%(title)s : %(artist)s'])
+        self.assertEqual(pp._data[0]['tmpl'], '%(title)s')
+
+    def test_in_out_seperation(self):
+        pp = MetadataFromFieldPP(None, ['%(title)s \\: %(artist)s:%(title)s : %(artist)s'])
+        self.assertEqual(pp._data[0]['in'], '%(title)s : %(artist)s')
+        self.assertEqual(pp._data[0]['out'], '%(title)s : %(artist)s')
+
 
 class TestMetadataFromTitle(unittest.TestCase):
     def test_format_to_regex(self):
         pp = MetadataFromTitlePP(None, '%(title)s - %(artist)s')
         self.assertEqual(pp._titleregex, r'(?P<title>.+)\ \-\ (?P<artist>.+)')
+
+
+class TestConvertThumbnail(unittest.TestCase):
+    def test_escaping(self):
+        pp = FFmpegThumbnailsConvertorPP()
+        if not pp.available:
+            print('Skipping: ffmpeg not found')
+            return
+
+        file = 'test/testdata/thumbnails/foo %d bar/foo_%d.{}'
+        tests = (('webp', 'png'), ('png', 'jpg'))
+
+        for inp, out in tests:
+            out_file = file.format(out)
+            if os.path.exists(out_file):
+                os.remove(out_file)
+            pp.convert_thumbnail(file.format(inp), out)
+            assert os.path.exists(out_file)
+
+        for _, out in tests:
+            os.remove(file.format(out))
+
+
+class TestExecAfterDownload(unittest.TestCase):
+    def test_parse_cmd(self):
+        pp = ExecAfterDownloadPP(YoutubeDL(), '')
+        info = {'filepath': 'file name'}
+        quoted_filepath = compat_shlex_quote(info['filepath'])
+
+        self.assertEqual(pp.parse_cmd('echo', info), 'echo %s' % quoted_filepath)
+        self.assertEqual(pp.parse_cmd('echo.{}', info), 'echo.%s' % quoted_filepath)
+        self.assertEqual(pp.parse_cmd('echo "%(filepath)s"', info), 'echo "%s"' % info['filepath'])

test/test_socks.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_subtitles.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_swfinterp.py.disabled

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_update.py.disabled

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_utils.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -12,6 +12,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
 # Various small unit tests
 import io
+import itertools
 import json
 import xml.etree.ElementTree
 
@@ -66,6 +67,7 @@ from yt_dlp.utils import (
     sanitize_filename,
     sanitize_path,
     sanitize_url,
+    sanitized_Request,
     expand_path,
     prepend_extension,
     replace_extension,
@@ -107,6 +109,7 @@ from yt_dlp.utils import (
     cli_bool_option,
     parse_codecs,
     iri_to_uri,
+    LazyList,
 )
 from yt_dlp.compat import (
     compat_chr,
@@ -125,6 +128,7 @@ class TestUtil(unittest.TestCase):
         self.assertTrue(timeconvert('bougrg') is None)
 
     def test_sanitize_filename(self):
+        self.assertEqual(sanitize_filename(''), '')
         self.assertEqual(sanitize_filename('abc'), 'abc')
         self.assertEqual(sanitize_filename('abc_d-e'), 'abc_d-e')
 
@@ -238,6 +242,16 @@ class TestUtil(unittest.TestCase):
         self.assertEqual(sanitize_url('httpss://foo.bar'), 'https://foo.bar')
         self.assertEqual(sanitize_url('rmtps://foo.bar'), 'rtmps://foo.bar')
         self.assertEqual(sanitize_url('https://foo.bar'), 'https://foo.bar')
+        self.assertEqual(sanitize_url('foo bar'), 'foo bar')
+
+    def test_extract_basic_auth(self):
+        auth_header = lambda url: sanitized_Request(url).get_header('Authorization')
+        self.assertFalse(auth_header('http://foo.bar'))
+        self.assertFalse(auth_header('http://:foo.bar'))
+        self.assertEqual(auth_header('http://@foo.bar'), 'Basic Og==')
+        self.assertEqual(auth_header('http://:pass@foo.bar'), 'Basic OnBhc3M=')
+        self.assertEqual(auth_header('http://user:@foo.bar'), 'Basic dXNlcjo=')
+        self.assertEqual(auth_header('http://user:pass@foo.bar'), 'Basic dXNlcjpwYXNz')
 
     def test_expand_path(self):
         def env(var):
@@ -1040,6 +1054,9 @@ class TestUtil(unittest.TestCase):
         on = js_to_json('{ "040": "040" }')
         self.assertEqual(json.loads(on), {'040': '040'})
 
+        on = js_to_json('[1,//{},\n2]')
+        self.assertEqual(json.loads(on), [1, 2])
+
     def test_js_to_json_malformed(self):
         self.assertEqual(js_to_json('42a1'), '42"a1"')
         self.assertEqual(js_to_json('42a-1'), '42"a"-1')
@@ -1513,6 +1530,47 @@ Line 1
         self.assertEqual(clean_podcast_url('https://www.podtrac.com/pts/redirect.mp3/chtbl.com/track/5899E/traffic.megaphone.fm/HSW7835899191.mp3'), 'https://traffic.megaphone.fm/HSW7835899191.mp3')
         self.assertEqual(clean_podcast_url('https://play.podtrac.com/npr-344098539/edge1.pod.npr.org/anon.npr-podcasts/podcast/npr/waitwait/2020/10/20201003_waitwait_wwdtmpodcast201003-015621a5-f035-4eca-a9a1-7c118d90bc3c.mp3'), 'https://edge1.pod.npr.org/anon.npr-podcasts/podcast/npr/waitwait/2020/10/20201003_waitwait_wwdtmpodcast201003-015621a5-f035-4eca-a9a1-7c118d90bc3c.mp3')
 
+    def test_LazyList(self):
+        it = list(range(10))
+
+        self.assertEqual(list(LazyList(it)), it)
+        self.assertEqual(LazyList(it).exhaust(), it)
+        self.assertEqual(LazyList(it)[5], it[5])
+
+        self.assertEqual(LazyList(it)[::2], it[::2])
+        self.assertEqual(LazyList(it)[1::2], it[1::2])
+        self.assertEqual(LazyList(it)[6:2:-2], it[6:2:-2])
+        self.assertEqual(LazyList(it)[::-1], it[::-1])
+
+        self.assertTrue(LazyList(it))
+        self.assertFalse(LazyList(range(0)))
+        self.assertEqual(len(LazyList(it)), len(it))
+        self.assertEqual(repr(LazyList(it)), repr(it))
+        self.assertEqual(str(LazyList(it)), str(it))
+
+        self.assertEqual(list(LazyList(it).reverse()), it[::-1])
+        self.assertEqual(list(LazyList(it).reverse()[1:3:7]), it[::-1][1:3:7])
+
+    def test_LazyList_laziness(self):
+
+        def test(ll, idx, val, cache):
+            self.assertEqual(ll[idx], val)
+            self.assertEqual(getattr(ll, '_LazyList__cache'), list(cache))
+
+        ll = LazyList(range(10))
+        test(ll, 0, 0, range(1))
+        test(ll, 5, 5, range(6))
+        test(ll, -3, 7, range(10))
+
+        ll = LazyList(range(10)).reverse()
+        test(ll, -1, 0, range(1))
+        test(ll, 3, 6, range(10))
+
+        ll = LazyList(itertools.count())
+        test(ll, 10, 10, range(11))
+        ll.reverse()
+        test(ll, -15, 14, range(15))
+
 
 if __name__ == '__main__':
     unittest.main()

test/test_verbose_output.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_write_annotations.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_youtube_lists.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_youtube_misc.py

@@ -0,0 +1,26 @@
+#!/usr/bin/env python3
+from __future__ import unicode_literals
+
+# Allow direct execution
+import os
+import sys
+import unittest
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+
+from yt_dlp.extractor import YoutubeIE
+
+
+class TestYoutubeMisc(unittest.TestCase):
+    def test_youtube_extract(self):
+        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
+        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
+        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
+
+
+if __name__ == '__main__':
+    unittest.main()

test/test_youtube_signature.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/testdata/ism/sintel.Manifest

@@ -0,0 +1,988 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Created with Unified Streaming Platform (version=1.10.18-20255) -->
+<SmoothStreamingMedia
+  MajorVersion="2"
+  MinorVersion="0"
+  TimeScale="10000000"
+  Duration="8880746666">
+  <StreamIndex
+    Type="audio"
+    QualityLevels="1"
+    TimeScale="10000000"
+    Name="audio"
+    Chunks="445"
+    Url="QualityLevels({bitrate})/Fragments(audio={start time})">
+    <QualityLevel
+      Index="0"
+      Bitrate="128001"
+      CodecPrivateData="1190"
+      SamplingRate="48000"
+      Channels="2"
+      BitsPerSample="16"
+      PacketSize="4"
+      AudioTag="255"
+      FourCC="AACL" />
+    <c t="0" d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="746666" />
+  </StreamIndex>
+  <StreamIndex
+    Type="text"
+    QualityLevels="1"
+    TimeScale="10000000"
+    Language="eng"
+    Subtype="CAPT"
+    Name="textstream_eng"
+    Chunks="11"
+    Url="QualityLevels({bitrate})/Fragments(textstream_eng={start time})">
+    <QualityLevel
+      Index="0"
+      Bitrate="1000"
+      CodecPrivateData=""
+      FourCC="TTML" />
+    <c t="0" d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="240000000" />
+  </StreamIndex>
+  <StreamIndex
+    Type="video"
+    QualityLevels="5"
+    TimeScale="10000000"
+    Name="video"
+    Chunks="444"
+    Url="QualityLevels({bitrate})/Fragments(video={start time})"
+    MaxWidth="1688"
+    MaxHeight="720"
+    DisplayWidth="1689"
+    DisplayHeight="720">
+    <QualityLevel
+      Index="0"
+      Bitrate="100000"
+      CodecPrivateData="00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8"
+      MaxWidth="336"
+      MaxHeight="144"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="1"
+      Bitrate="326000"
+      CodecPrivateData="00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8"
+      MaxWidth="562"
+      MaxHeight="240"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="2"
+      Bitrate="698000"
+      CodecPrivateData="00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8"
+      MaxWidth="844"
+      MaxHeight="360"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="3"
+      Bitrate="1493000"
+      CodecPrivateData="00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8"
+      MaxWidth="1126"
+      MaxHeight="480"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="4"
+      Bitrate="4482000"
+      CodecPrivateData="00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8"
+      MaxWidth="1688"
+      MaxHeight="720"
+      FourCC="AVC1" />
+    <c t="0" d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+  </StreamIndex>
+</SmoothStreamingMedia>

test/testdata/m3u8/bipbop_16x9.m3u8

@@ -0,0 +1,38 @@
+#EXTM3U
+
+#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
+#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac/prog_index.m3u8"
+
+
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="en",URI="subtitles/eng_forced/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/fra/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="fr",URI="subtitles/fra_forced/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="es",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/spa/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="es",URI="subtitles/spa_forced/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="ja",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/jpn/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語 (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="ja",URI="subtitles/jpn_forced/prog_index.m3u8"
+
+
+#EXT-X-STREAM-INF:BANDWIDTH=263851,CODECS="mp4a.40.2, avc1.4d400d",RESOLUTION=416x234,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear1/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=28451,CODECS="avc1.4d400d",URI="gear1/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear2/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=181534,CODECS="avc1.4d401e",URI="gear2/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear3/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=297056,CODECS="avc1.4d401f",URI="gear3/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=1030138,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1280x720,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear4/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=339492,CODECS="avc1.4d401f",URI="gear4/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=1924009,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1920x1080,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear5/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=669554,CODECS="avc1.4d401f",URI="gear5/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=41457,CODECS="mp4a.40.2",AUDIO="bipbop_audio",SUBTITLES="subs"
+gear0/prog_index.m3u8

test/testdata/mpd/subtitles.mpd

@@ -0,0 +1,351 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Created with Unified Streaming Platform (version=1.10.18-20255) -->
+<MPD
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xmlns="urn:mpeg:dash:schema:mpd:2011"
+  xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
+  type="static"
+  mediaPresentationDuration="PT14M48S"
+  maxSegmentDuration="PT1M"
+  minBufferTime="PT10S"
+  profiles="urn:mpeg:dash:profile:isoff-live:2011">
+  <Period
+    id="1"
+    duration="PT14M48S">
+    <BaseURL>dash/</BaseURL>
+    <AdaptationSet
+      id="1"
+      group="1"
+      contentType="audio"
+      segmentAlignment="true"
+      audioSamplingRate="48000"
+      mimeType="audio/mp4"
+      codecs="mp4a.40.2"
+      startWithSAP="1">
+      <AudioChannelConfiguration
+        schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011"
+        value="2" />
+      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
+      <SegmentTemplate
+        timescale="48000"
+        initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
+        media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
+        <SegmentTimeline>
+          <S t="0" d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="3584" />
+        </SegmentTimeline>
+      </SegmentTemplate>
+      <Representation
+        id="audio=128001"
+        bandwidth="128001">
+      </Representation>
+    </AdaptationSet>
+    <AdaptationSet
+      id="2"
+      group="3"
+      contentType="text"
+      lang="en"
+      mimeType="application/mp4"
+      codecs="stpp"
+      startWithSAP="1">
+      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="subtitle" />
+      <SegmentTemplate
+        timescale="1000"
+        initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
+        media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
+        <SegmentTimeline>
+          <S t="0" d="60000" r="9" />
+          <S d="24000" />
+        </SegmentTimeline>
+      </SegmentTemplate>
+      <Representation
+        id="textstream_eng=1000"
+        bandwidth="1000">
+      </Representation>
+    </AdaptationSet>
+    <AdaptationSet
+      id="3"
+      group="2"
+      contentType="video"
+      par="960:409"
+      minBandwidth="100000"
+      maxBandwidth="4482000"
+      maxWidth="1689"
+      maxHeight="720"
+      segmentAlignment="true"
+      mimeType="video/mp4"
+      codecs="avc1.4D401F"
+      startWithSAP="1">
+      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
+      <SegmentTemplate
+        timescale="12288"
+        initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
+        media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
+        <SegmentTimeline>
+          <S t="0" d="24576" r="443" />
+        </SegmentTimeline>
+      </SegmentTemplate>
+      <Representation
+        id="video=100000"
+        bandwidth="100000"
+        width="336"
+        height="144"
+        sar="2880:2863"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=326000"
+        bandwidth="326000"
+        width="562"
+        height="240"
+        sar="115200:114929"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=698000"
+        bandwidth="698000"
+        width="844"
+        height="360"
+        sar="86400:86299"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=1493000"
+        bandwidth="1493000"
+        width="1126"
+        height="480"
+        sar="230400:230267"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=4482000"
+        bandwidth="4482000"
+        width="1688"
+        height="720"
+        sar="86400:86299"
+        scanType="progressive">
+      </Representation>
+    </AdaptationSet>
+  </Period>
+</MPD>

yt-dlp.sh

@@ -1,2 +1,2 @@
-#!/bin/bash
-python3 "$(dirname $(realpath $0))/yt_dlp/__main__.py" "$@"
+#!/bin/sh
+exec python3 "$(dirname "$(realpath "$0")")/yt_dlp/__main__.py" "$@"