Release 2021.07.21

and fix some typos
Closes #538

.gitattributes

@@ -1 +1,4 @@
+* text=auto
+
 Makefile* text whitespace=-tab-in-indent
+*.sh text eol=lf

.github/ISSUE_TEMPLATE/1_broken_site.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support
-- [ ] I've verified that I'm running yt-dlp version **2021.03.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -42,9 +42,9 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.03.24
+ [debug] yt-dlp version 2021.07.07
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/2_site_support_request.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a new site support request
-- [ ] I've verified that I'm running yt-dlp version **2021.03.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones

.github/ISSUE_TEMPLATE/3_site_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a site feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.03.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE/4_bug_report.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support issue
-- [ ] I've verified that I'm running yt-dlp version **2021.03.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -44,9 +44,9 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.03.24
+ [debug] yt-dlp version 2021.07.07
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/5_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.07. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.03.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.07**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE_tmpl/1_broken_site.md

@@ -42,7 +42,7 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
  [debug] yt-dlp version %(version)s
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2

.github/ISSUE_TEMPLATE_tmpl/4_bug_report.md

@@ -44,7 +44,7 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
  [debug] yt-dlp version %(version)s
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2

.github/workflows/build.yml

@@ -7,13 +7,13 @@ on:
 
 jobs:
   build_unix:
-
     runs-on: ubuntu-latest
 
     outputs:
       ytdlp_version: ${{ steps.bump_version.outputs.ytdlp_version }}
       upload_url: ${{ steps.create_release.outputs.upload_url }}
-      sha2_unix: ${{ steps.sha2_file.outputs.sha2_unix }}
+      sha256_unix: ${{ steps.sha256_file.outputs.sha256_unix }}
+      sha512_unix: ${{ steps.sha512_file.outputs.sha512_unix }}
 
     steps:
     - uses: actions/checkout@v2
@@ -29,7 +29,7 @@ jobs:
     - name: Print version
       run: echo "${{ steps.bump_version.outputs.ytdlp_version }}"
     - name: Run Make
-      run: make
+      run: make all tar
     - name: Create Release
       id: create_release
       uses: actions/create-release@v1
@@ -44,7 +44,7 @@ jobs:
         draft: false
         prerelease: false
     - name: Upload yt-dlp Unix binary
-      id: upload-release-asset 
+      id: upload-release-asset
       uses: actions/upload-release-asset@v1
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -53,9 +53,21 @@ jobs:
         asset_path: ./yt-dlp
         asset_name: yt-dlp
         asset_content_type: application/octet-stream
+    - name: Upload Source tar
+      uses: actions/upload-release-asset@v1
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./yt-dlp.tar.gz
+        asset_name: yt-dlp.tar.gz
+        asset_content_type: application/gzip
     - name: Get SHA2-256SUMS for yt-dlp
-      id: sha2_file
-      run: echo "::set-output name=sha2_unix::$(sha256sum yt-dlp | awk '{print $1}')"
+      id: sha256_file
+      run: echo "::set-output name=sha256_unix::$(sha256sum yt-dlp | awk '{print $1}')"
+    - name: Get SHA2-512SUMS for yt-dlp
+      id: sha512_file
+      run: echo "::set-output name=sha512_unix::$(sha512sum yt-dlp | awk '{print $1}')"
     - name: Install dependencies for pypi
       env:
         PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
@@ -74,24 +86,24 @@ jobs:
         twine upload dist/*
 
   build_windows:
-
     runs-on: windows-latest
+    needs: build_unix
 
     outputs:
-      sha2_windows: ${{ steps.sha2_file_win.outputs.sha2_windows }}
-
-    needs: build_unix
+      sha256_windows: ${{ steps.sha256_file_win.outputs.sha256_windows }}
+      sha512_windows: ${{ steps.sha512_file_win.outputs.sha512_windows }}
 
     steps:
     - uses: actions/checkout@v2
-    - name: Set up Python
+    # 3.8 is used for Win7 support
+    - name: Set up Python 3.8
       uses: actions/setup-python@v2
       with:
           python-version: '3.8'
     - name: Upgrade pip and enable wheel support
       run: python -m pip install --upgrade pip setuptools wheel
     - name: Install Requirements
-      run: pip install pyinstaller mutagen pycryptodome
+      run: pip install pyinstaller mutagen pycryptodome websockets
     - name: Bump version
       id: bump_version
       run: python devscripts/update-version.py
@@ -110,29 +122,32 @@ jobs:
         asset_name: yt-dlp.exe
         asset_content_type: application/vnd.microsoft.portable-executable
     - name: Get SHA2-256SUMS for yt-dlp.exe
-      id: sha2_file_win
-      run: echo "::set-output name=sha2_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())"
+      id: sha256_file_win
+      run: echo "::set-output name=sha256_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())"
+    - name: Get SHA2-512SUMS for yt-dlp.exe
+      id: sha512_file_win
+      run: echo "::set-output name=sha512_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA512).Hash.ToLower())"
 
   build_windows32:
-
     runs-on: windows-latest
+    needs: [build_unix, build_windows]
 
     outputs:
-      sha2_windows32: ${{ steps.sha2_file_win32.outputs.sha2_windows32 }}
-
-    needs: [build_unix, build_windows]
+      sha256_windows32: ${{ steps.sha256_file_win32.outputs.sha256_windows32 }}
+      sha512_windows32: ${{ steps.sha512_file_win32.outputs.sha512_windows32 }}
 
     steps:
     - uses: actions/checkout@v2
-    - name: Set up Python 3.4.4 32-Bit
+    # 3.7 is used for Vista support. See https://github.com/yt-dlp/yt-dlp/issues/390
+    - name: Set up Python 3.7 32-Bit
       uses: actions/setup-python@v2
       with:
-          python-version: '3.4.4'
+          python-version: '3.7'
           architecture: 'x86'
     - name: Upgrade pip and enable wheel support
-      run: python -m pip install pip==19.1.1 setuptools==43.0.0 wheel==0.33.6
-    - name: Install Requirements for 32 Bit
-      run: pip install pyinstaller==3.5 mutagen==1.42.0 pycryptodome==3.9.4
+      run: python -m pip install --upgrade pip setuptools wheel
+    - name: Install Requirements
+      run: pip install pyinstaller mutagen pycryptodome websockets
     - name: Bump version
       id: bump_version
       run: python devscripts/update-version.py
@@ -151,20 +166,28 @@ jobs:
         asset_name: yt-dlp_x86.exe
         asset_content_type: application/vnd.microsoft.portable-executable
     - name: Get SHA2-256SUMS for yt-dlp_x86.exe
-      id: sha2_file_win32
-      run: echo "::set-output name=sha2_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())"
+      id: sha256_file_win32
+      run: echo "::set-output name=sha256_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())"
+    - name: Get SHA2-512SUMS for yt-dlp_x86.exe
+      id: sha512_file_win32
+      run: echo "::set-output name=sha512_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA512).Hash.ToLower())"
+
+  finish:
+    runs-on: ubuntu-latest
+    needs: [build_unix, build_windows, build_windows32]
+
+    steps:
     - name: Make SHA2-256SUMS file
       env:
-        SHA2_WINDOWS: ${{ needs.build_windows.outputs.sha2_windows }}
-        SHA2_WINDOWS32: ${{ steps.sha2_file_win32.outputs.sha2_windows32 }}
-        SHA2_UNIX: ${{ needs.build_unix.outputs.sha2_unix }}
+        SHA256_WINDOWS: ${{ needs.build_windows.outputs.sha256_windows }}
+        SHA256_WINDOWS32: ${{ needs.build_windows32.outputs.sha256_windows32 }}
+        SHA256_UNIX: ${{ needs.build_unix.outputs.sha256_unix }}
         YTDLP_VERSION: ${{ needs.build_unix.outputs.ytdlp_version }}
       run: |
-        echo "version:${env:YTDLP_VERSION}" >> SHA2-256SUMS
-        echo "yt-dlp.exe:${env:SHA2_WINDOWS}" >> SHA2-256SUMS
-        echo "yt-dlp_x86.exe:${env:SHA2_WINDOWS32}" >> SHA2-256SUMS
-        echo "yt-dlp:${env:SHA2_UNIX}" >> SHA2-256SUMS
-
+        echo "version:${{ env.YTDLP_VERSION }}" >> SHA2-256SUMS
+        echo "yt-dlp.exe:${{ env.SHA256_WINDOWS }}" >> SHA2-256SUMS
+        echo "yt-dlp_x86.exe:${{ env.SHA256_WINDOWS32 }}" >> SHA2-256SUMS
+        echo "yt-dlp:${{ env.SHA256_UNIX }}" >> SHA2-256SUMS
     - name: Upload 256SUMS file
       id: upload-sums
       uses: actions/upload-release-asset@v1
@@ -175,3 +198,22 @@ jobs:
         asset_path: ./SHA2-256SUMS
         asset_name: SHA2-256SUMS
         asset_content_type: text/plain
+    - name: Make SHA2-512SUMS file
+      env:
+        SHA512_WINDOWS: ${{ needs.build_windows.outputs.sha512_windows }}
+        SHA512_WINDOWS32: ${{ needs.build_windows32.outputs.sha512_windows32 }}
+        SHA512_UNIX: ${{ needs.build_unix.outputs.sha512_unix }}
+      run: |
+        echo "${{ env.SHA512_WINDOWS }}  yt-dlp.exe" >> SHA2-512SUMS
+        echo "${{ env.SHA512_WINDOWS32 }}  yt-dlp_x86.exe" >> SHA2-512SUMS
+        echo "${{ env.SHA512_UNIX }}  yt-dlp" >> SHA2-512SUMS
+    - name: Upload 512SUMS file
+      id: upload-512sums
+      uses: actions/upload-release-asset@v1
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        upload_url: ${{ needs.build_unix.outputs.upload_url }}
+        asset_path: ./SHA2-512SUMS
+        asset_name: SHA2-512SUMS
+        asset_content_type: text/plain

.github/workflows/core.yml

@@ -6,49 +6,28 @@ jobs:
     if: "!contains(github.event.head_commit.message, 'ci skip')"
     runs-on: ${{ matrix.os }}
     strategy:
-      fail-fast: true
+      fail-fast: false
       matrix:
         os: [ubuntu-18.04]
-        # TODO: python 2.6
-        python-version: [2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, pypy-2.7, pypy-3.6, pypy-3.7]
-        python-impl: [cpython]
-        ytdl-test-set: [core]
+        # py3.9 is in quick-test
+        python-version: [3.7, 3.8, pypy-3.6, pypy-3.7]
         run-tests-ext: [sh]
         include:
-        # python 3.2 is only available on windows via setup-python
+        # atleast one of the tests must be in windows
         - os: windows-latest
-          python-version: 3.2
-          python-impl: cpython
-          ytdl-test-set: core
+          python-version: 3.6
           run-tests-ext: bat
-        # jython
-        - os: ubuntu-latest
-          python-impl: jython
-          ytdl-test-set: core
-          run-tests-ext: sh
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
-      if: ${{ matrix.python-impl == 'cpython' }}
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Set up Java 8
-      if: ${{ matrix.python-impl == 'jython' }}
-      uses: actions/setup-java@v1
-      with:
-        java-version: 8
-    - name: Install Jython
-      if: ${{ matrix.python-impl == 'jython' }}
-      run: |
-        wget http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar -O jython-installer.jar
-        java -jar jython-installer.jar -s -d "$HOME/jython"
-        echo "$HOME/jython/bin" >> $GITHUB_PATH
     - name: Install nose
       run: pip install nose
     - name: Run tests
-      continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
+      continue-on-error: False
       env:
-        YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
+        YTDL_TEST_SET: core
       run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}
   # Linter is in quick-test

.github/workflows/download.yml

@@ -9,45 +9,22 @@ jobs:
       fail-fast: true
       matrix:
         os: [ubuntu-18.04]
-        # TODO: python 2.6
-        python-version: [2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, pypy-2.7, pypy-3.6, pypy-3.7]
-        python-impl: [cpython]
-        ytdl-test-set: [download]
+        python-version: [3.7, 3.8, 3.9, pypy-3.6, pypy-3.7]
         run-tests-ext: [sh]
         include:
-        # python 3.2 is only available on windows via setup-python
         - os: windows-latest
-          python-version: 3.2
-          python-impl: cpython
-          ytdl-test-set: download
+          python-version: 3.6
           run-tests-ext: bat
-        # jython - disable for now since it takes too long to complete
-        # - os: ubuntu-latest
-        #   python-impl: jython
-        #   ytdl-test-set: download
-        #   run-tests-ext: sh
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
-      if: ${{ matrix.python-impl == 'cpython' }}
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Set up Java 8
-      if: ${{ matrix.python-impl == 'jython' }}
-      uses: actions/setup-java@v1
-      with:
-        java-version: 8
-    - name: Install Jython
-      if: ${{ matrix.python-impl == 'jython' }}
-      run: |
-        wget http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar -O jython-installer.jar
-        java -jar jython-installer.jar -s -d "$HOME/jython"
-        echo "$HOME/jython/bin" >> $GITHUB_PATH
     - name: Install nose
       run: pip install nose
     - name: Run tests
-      continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
+      continue-on-error: true
       env:
-        YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
+        YTDL_TEST_SET: download
       run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}

.github/workflows/quick-test.yml

@@ -11,8 +11,8 @@ jobs:
       uses: actions/setup-python@v2
       with:
         python-version: 3.9
-    - name: Install nose
-      run: pip install nose
+    - name: Install test requirements
+      run: pip install nose pycryptodome
     - name: Run tests
       env:
         YTDL_TEST_SET: core

.gitignore

@@ -1,3 +1,47 @@
+# Config
+*.conf
+*.spec
+cookies
+cookies.txt
+
+# Downloaded
+*.srt
+*.ttml
+*.sbv
+*.vtt
+*.flv
+*.mp4
+*.m4a
+*.m4v
+*.mp3
+*.3gp
+*.webm
+*.wav
+*.ape
+*.mkv
+*.swf
+*.part
+*.part-*
+*.ytdl
+*.dump
+*.frag
+*.frag.urls
+*.aria2
+*.swp
+*.ogg
+*.opus
+*.info.json
+*.live_chat.json
+*.jpg
+*.jpeg
+*.png
+*.webp
+*.annotations.xml
+*.description
+
+# Allow config/media files in testdata
+!test/testdata/**
+
 # Python
 *.pyc
 *.pyo
@@ -43,48 +87,6 @@ README.txt
 yt-dlp.zip
 *.exe
 
-# Downloaded
-*.srt
-*.ttml
-*.sbv
-*.vtt
-*.flv
-*.mp4
-*.m4a
-*.m4v
-*.mp3
-*.3gp
-*.webm
-*.wav
-*.ape
-*.mkv
-*.swf
-*.part
-*.part-*
-*.ytdl
-*.dump
-*.frag
-*.frag.urls
-*.aria2
-*.swp
-*.ogg
-*.opus
-*.info.json
-*.live_chat.json
-*.jpg
-*.png
-*.webp
-*.annotations.xml
-*.description
-
-# Config
-*.conf
-*.spec
-cookies
-cookies.txt
-
-
-
 # Text Editor / IDE
 .idea
 *.iml

CONTRIBUTING.md

@@ -3,7 +3,7 @@
 $ youtube-dl -v <your command line>
 [debug] System config: []
 [debug] User config: []
-[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj']
+[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] youtube-dl version 2015.12.06
 [debug] Git HEAD: 135392e

CONTRIBUTORS

@@ -1,6 +1,7 @@
 pukkandan (owner)
 shirt-dev (collaborator)
 colethedj (collaborator)
+Ashish0804 (collaborator)
 h-h-h-h
 pauldubois98
 nixxo
@@ -20,11 +21,9 @@ FelixFrog
 Zocker1999NET
 nao20010128nao
 kurumigi
-tsukumi
 bbepis
 animelover1984
 Pccode66
-Ashish0804
 RobinD42
 hseg
 DennyDai
@@ -35,3 +34,32 @@ damianoamatruda
 2ShedsJackson
 CXwudi
 xtkoba
+llacb47
+hheimbuerger
+B0pol
+lkho
+fstirlitz
+Lamieur
+tsukumijima
+Hadi0609
+b5eff52
+craftingmod
+tpikonen
+tripulse
+king-millez
+alex-gedeon
+hhirtz
+louie-github
+MinePlayersPE
+olifre
+rhsmachine/zenerdi0de
+nihil-admirari
+krichbanana
+ohmybahgosh
+nyuszika7h
+blackjack4494
+pyx
+TpmKranz
+mzbaulhaque
+zackmark29
+mbway

Changelog.md

@@ -6,9 +6,11 @@
 * Run `make doc`
 * Update Changelog.md and CONTRIBUTORS
 * Change "Merged with ytdl" version in Readme.md if needed
+* Add new/fixed extractors in "new features" section of Readme.md
 * Commit to master as `Release <version>`
-* Push to origin/release - build task will now run
-* Update version.py using devscripts\update-version.py
+* Push to origin/release using `git push origin master:release`
+    build task will now run
+* Update version.py using `devscripts\update-version.py`
 * Run `make issuetemplates`
 * Commit to master as `[version] update :ci skip all`
 * Push to origin/master
@@ -16,18 +18,412 @@
 
 -->
 
+### 2021.07.21
+
+* **Add option `--cookies-from-browser`** to load cookies from a browser by [mbway](https://github.com/mbway)
+    * Usage: `--cookies-from-browser BROWSER[:PROFILE_NAME_OR_PATH]`
+    * Also added `--no-cookies-from-browser`
+    * To decrypt chromium cookies, `keyring` is needed for UNIX and `pycryptodome` for Windows
+* Add option `--exec-before-download`
+* Add field `live_status`
+* [FFmpegMetadata] Add language of each stream and some refactoring
+* [douyin] Add extractor by [pukkandan](https://github.com/pukkandan), [pyx](https://github.com/pyx)
+* [pornflip] Add extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
+* **[youtube] Extract data from multiple clients** by [pukkandan](https://github.com/pukkandan), [colethedj](https://github.com/colethedj)
+    * `player_client` now accepts multiple clients
+    * Default `player_client` = `android,web`
+        * This uses twice as many requests, but avoids throttling for most videos while also not losing any formats
+    * Music clients can be specifically requested and is enabled by default if `music.youtube.com`
+    * Added `player_client=ios` (Known issue: formats from ios are not sorted correctly)
+    * Add age-gate bypass for android and ios clients
+* [youtube] Extract more thumbnails
+    * The thumbnail URLs are hard-coded and their actual existence is tested lazily
+    * Added option `--no-check-formats` to not test them
+* [youtube] Misc fixes
+    * Improve extraction of livestream metadata by [pukkandan](https://github.com/pukkandan), [krichbanana](https://github.com/krichbanana)
+    * Hide live dash formats since they can't be downloaded anyway
+    * Fix authentication when using multiple accounts by [colethedj](https://github.com/colethedj)
+    * Fix controversial videos when requested via API by [colethedj](https://github.com/colethedj)
+    * Fix session index extraction and headers for non-web player clients by [colethedj](https://github.com/colethedj)
+    * Make `--extractor-retries` work for more errors
+    * Fix sorting of 3gp format
+    * Sanity check `chapters` (and refactor related code)
+    * Make `parse_time_text` and `_extract_chapters` non-fatal
+    * Misc cleanup and bug fixes by [colethedj](https://github.com/colethedj)
+* [youtube:tab] Fix channels tab
+* [youtube:tab] Extract playlist availability by [colethedj](https://github.com/colethedj)
+* **[youtube:comments] Move comment extraction to new API** by [colethedj](https://github.com/colethedj)
+* [youtube:comments] Fix `is_favorited`, improve `like_count` parsing by [colethedj](https://github.com/colethedj)
+* [BravoTV] Improve metadata extraction by [kevinoconnor7](https://github.com/kevinoconnor7)
+* [crunchyroll:playlist] Force http
+* [yahoo:gyao:player] Relax `_VALID_URL` by [nao20010128nao](https://github.com/nao20010128nao)
+* [nebula] Authentication via tokens from cookie jar by [hheimbuerger](https://github.com/hheimbuerger), [TpmKranz](https://github.com/TpmKranz)
+* [RTP] Fix extraction and add subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [viki] Rewrite extractors and add extractor-arg `video_types` to `vikichannel` by [zackmark29](https://github.com/zackmark29), [pukkandan](https://github.com/pukkandan)
+* [vlive] Extract thumbnail directly in addition to the one from Naver
+* [generic] Extract previously missed subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [generic] Extract everything in the SMIL manifest and detect discarded subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [embedthumbnail] Fix `_get_thumbnail_resolution`
+* [metadatafromfield] Do not detect numbers as field names
+* Fix selectors `all`, `mergeall` and add tests
+* Errors in playlist extraction should obey `--ignore-errors`
+* Fix bug where `original_url` was not propagated when `_type`=`url`
+* Revert "Merge webm formats into mkv if thumbnails are to be embedded (#173)"
+    * This was wrongly checking for `write_thumbnail`
+* Improve `extractor_args` parsing
+* Rename `NOTE` in `-F` to `MORE INFO` since it's often confused to be the same as `format_note`
+* Add `only_once` param for `write_debug` and `report_warning`
+* [extractor] Allow extracting multiple groups in `_search_regex` by [fstirlitz](https://github.com/fstirlitz)
+* [utils] Improve `traverse_obj`
+* [utils] Add `variadic`
+* [utils] Improve `js_to_json` comment regex by [fstirlitz](https://github.com/fstirlitz)
+* [webtt] Fix timestamps
+* [compat] Remove unnecessary code
+* [doc] fix default of multistreams
+
+
+### 2021.07.07
+
+* Merge youtube-dl: Upto [commit/a803582](https://github.com/ytdl-org/youtube-dl/commit/a8035827177d6b59aca03bd717acb6a9bdd75ada)
+* Add `--extractor-args` to pass some extractor-specific arguments. See [readme](https://github.com/yt-dlp/yt-dlp#extractor-arguments)
+    * Add extractor option `skip` for `youtube`. Eg: `--extractor-args youtube:skip=hls,dash`
+    * Deprecates `--youtube-skip-dash-manifest`, `--youtube-skip-hls-manifest`, `--youtube-include-dash-manifest`, `--youtube-include-hls-manifest`
+* Allow `--list...` options to work with `--print`, `--quiet` and other `--list...` options
+* [youtube] Use `player` API for additional video extraction requests by [colethedj](https://github.com/colethedj)
+    * **Fixes youtube premium music** (format 141) extraction
+    * Adds extractor option `player_client` = `web`/`android`
+        * **`--extractor-args youtube:player_client=android` works around the throttling** for the time-being
+    * Adds extractor option `player_skip=config`
+    * Adds age-gate fallback using embedded client
+* [youtube] Choose correct Live chat API for upcoming streams by [krichbanana](https://github.com/krichbanana)
+* [youtube] Fix subtitle names for age-gated videos
+* [youtube:comments] Fix error handling and add `itct` to params by [colethedj](https://github.com/colethedj)
+* [youtube_live_chat] Fix download with cookies by [siikamiika](https://github.com/siikamiika)
+* [youtube_live_chat] use `clickTrackingParams` by [siikamiika](https://github.com/siikamiika)
+* [Funimation] Rewrite extractor
+    * Add `FunimationShowIE` by [Mevious](https://github.com/Mevious)
+    * **Treat the different versions of an episode as different formats of a single video**
+        * This changes the video `id` and will break break existing archives
+        * Compat option `seperate-video-versions` to fall back to old behavior including using the old video ids
+    * Support direct `/player/` URL
+    * Extractor options `language` and `version` to pre-select them during extraction
+        * These options may be removed in the future if we can extract all formats without additional network requests
+        * Do not rely on these for format selection and use `-f` filters instead
+* [AdobePass] Add Spectrum MSO by [kevinoconnor7](https://github.com/kevinoconnor7), [ohmybahgosh](https://github.com/ohmybahgosh)
+* [facebook] Extract description and fix title
+* [fancode] Fix extraction, support live and allow login with refresh token by [zenerdi0de](https://github.com/zenerdi0de)
+* [plutotv] Improve `_VALID_URL`
+* [RCTIPlus] Add extractor by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [Soundcloud] Allow login using oauth token by [blackjack4494](https://github.com/blackjack4494)
+* [TBS] Support livestreams by [llacb47](https://github.com/llacb47)
+* [videa] Fix extraction by [nyuszika7h](https://github.com/nyuszika7h)
+* [yahoo] Fix extraction by [llacb47](https://github.com/llacb47), [pukkandan](https://github.com/pukkandan)
+* Process videos when using `--ignore-no-formats-error` by [krichbanana](https://github.com/krichbanana)
+* Fix `--throttled-rate` when using `--load-info-json`
+* Fix `--flat-playlist` when entry has no `ie_key`
+* Fix `check_formats` catching `ExtractorError` instead of `DownloadError`
+* Fix deprecated option `--list-formats-old`
+* [downloader/ffmpeg] Fix `--ppa` when using simultaneous download
+* [extractor] Prevent unnecessary download of hls manifests and refactor `hls_split_discontinuity`
+* [fragment] Handle status of download and errors in threads correctly; and minor refactoring
+* [thumbnailsconvertor] Treat `jpeg` as `jpg`
+* [utils] Fix issues with `LazyList` reversal
+* [extractor] Allow extractors to set their own login hint
+* [cleanup] Simplify format selector code with `LazyList` and `yield from`
+* [cleanup] Clean `extractor.common._merge_subtitles` signature
+* [cleanup] Fix some typos
+
+
+### 2021.06.23
+
+* Merge youtube-dl: Upto [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)
+* **Add option `--throttled-rate`** below which video data is re-extracted
+* [fragment] **Merge during download for `-N`**, and refactor `hls`/`dash`
+* [websockets] Add `WebSocketFragmentFD` by [nao20010128nao](https://github.com/nao20010128nao), [pukkandan](https://github.com/pukkandan)
+* Allow `images` formats in addition to video/audio
+* [downloader/mhtml] Add new downloader for slideshows/storyboards by [fstirlitz](https://github.com/fstirlitz)
+* [youtube] Temporary **fix for age-gate**
+* [youtube] Support ongoing live chat by [siikamiika](https://github.com/siikamiika)
+* [youtube] Improve SAPISID cookie handling by [colethedj](https://github.com/colethedj)
+* [youtube] Login is not needed for `:ytrec`
+* [youtube] Non-fatal alert reporting for unavailable videos page by [colethedj](https://github.com/colethedj)
+* [twitcasting] Websocket support by [nao20010128nao](https://github.com/nao20010128nao)
+* [mediasite] Extract slides by [fstirlitz](https://github.com/fstirlitz)
+* [funimation] Extract subtitles 
+* [pornhub] Extract `cast`
+* [hotstar] Use server time for authentication instead of local time
+* [EmbedThumbnail] Fix for already downloaded thumbnail
+* [EmbedThumbnail] Add compat-option `embed-thumbnail-atomicparsley`
+* Expand `--check-formats` to thumbnails
+* Fix id sanitization in filenames
+* Skip fixup of existing files and add `--fixup force` to force it
+* Better error handling of syntax errors in `-f`
+* Use `NamedTemporaryFile` for `--check-formats`
+* [aria2c] Lower `--min-split-size` for HTTP downloads
+* [options] Rename `--add-metadata` to `--embed-metadata`
+* [utils] Improve `LazyList` and add tests
+* [build] Build Windows x86 version with py3.7 and remove redundant tests by [pukkandan](https://github.com/pukkandan), [shirt](https://github.com/shirt-dev)
+* [docs] Clarify that `--embed-metadata` embeds chapter markers
+* [cleanup] Refactor fixup
+
+
+### 2021.06.09
+
+* Fix bug where `%(field)d` in filename template throws error
+* Improve offset parsing in outtmpl
+* [test] More rigorous tests for `prepare_filename`
+
+### 2021.06.08
+
+* Remove support for obsolete Python versions: Only 3.6+ is now supported
+* Merge youtube-dl: Upto [commit/c2350ca](https://github.com/ytdl-org/youtube-dl/commit/c2350cac243ba1ec1586fe85b0d62d1b700047a2)
+* [hls] Fix decryption for multithreaded downloader
+* [extractor] Fix pre-checking archive for some extractors
+* [extractor] Fix FourCC fallback when parsing ISM by [fstirlitz](https://github.com/fstirlitz)
+* [twitcasting] Add TwitCastingUserIE, TwitCastingLiveIE by [pukkandan](https://github.com/pukkandan), [nao20010128nao](https://github.com/nao20010128nao)
+* [vidio] Add VidioPremierIE and VidioLiveIE by [MinePlayersPE](Https://github.com/MinePlayersPE)
+* [viki] Fix extraction from [ytdl-org/youtube-dl@59e583f](https://github.com/ytdl-org/youtube-dl/commit/59e583f7e8530ca92776c866897d895c072e2a82)
+* [youtube] Support shorts URL
+* [zoom] Extract transcripts as subtitles
+* Add field `original_url` with the user-inputted URL
+* Fix and refactor `prepare_outtmpl`
+* Make more fields available for `--print` when used with `--flat-playlist`
+* [utils] Generalize `traverse_dict` to `traverse_obj`
+* [downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode by [fstirlitz](https://github.com/fstirlitz)
+* [build] Release `yt-dlp.tar.gz`
+* [build,update] Add GNU-style SHA512 and prepare updater for simlar SHA256 by [nihil-admirari](https://github.com/nihil-admirari)
+* [pyinst] Show Python version in exe metadata by [nihil-admirari](https://github.com/nihil-admirari)
+* [docs] Improve documentation of dependencies
+* [cleanup] Mark unused files
+* [cleanup] Point all shebang to `python3` by [fstirlitz](https://github.com/fstirlitz)
+* [cleanup] Remove duplicate file `trovolive.py`
+
+
+### 2021.06.01
+
+* Merge youtube-dl: Upto [commit/d495292](https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf)
+* Pre-check archive and filters during playlist extraction
+* Handle Basic Auth `user:pass` in URLs by [hhirtz](https://github.com/hhirtz) and [pukkandan](https://github.com/pukkandan)
+* [archiveorg] Add YoutubeWebArchiveIE by [colethedj](https://github.com/colethedj) and [alex-gedeon](https://github.com/alex-gedeon)
+* [fancode] Add extractor by [rhsmachine](https://github.com/rhsmachine)
+* [patreon] Support vimeo embeds by [rhsmachine](https://github.com/rhsmachine)
+* [Saitosan] Add new extractor by [llacb47](https://github.com/llacb47)
+* [ShemarooMe] Add extractor by [Ashish0804](https://github.com/Ashish0804) and [pukkandan](https://github.com/pukkandan)
+* [telemundo] Add extractor by [king-millez](https://github.com/king-millez)
+* [SonyLIV] Add SonyLIVSeriesIE and subtitle support by [Ashish0804](https://github.com/Ashish0804)
+* [Hotstar] Add HotStarSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [Voot] Add VootSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [vidio] Support login and premium videos by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [fragment] When using `-N`, do not keep the fragment content in memory
+* [ffmpeg] Download and merge in a single step if possible
+* [ThumbnailsConvertor] Support conversion to `png` and make it the default by [louie-github](https://github.com/louie-github)
+* [VideoConvertor] Generalize with remuxer and allow conditional recoding
+* [EmbedThumbnail] Embed in `mp4`/`m4a` using mutagen by [tripulse](https://github.com/tripulse) and [pukkandan](https://github.com/pukkandan)
+* [EmbedThumbnail] Embed if any thumbnail was downloaded, not just the best
+* [EmbedThumbnail] Correctly escape filename
+* [update] replace self without launching a subprocess in windows
+* [update] Block further update for unsupported systems
+* Refactor `__process_playlist` by creating `LazyList`
+* Write messages to `stderr` when both `quiet` and `verbose`
+* Sanitize and sort playlist thumbnails
+* Remove `None` values from `info.json`
+* [extractor] Always prefer native hls downloader by default
+* [extractor] Skip subtitles without URI in m3u8 manifests by [hheimbuerger](https://github.com/hheimbuerger)
+* [extractor] Functions to parse `socket.io` response as `json` by [pukkandan](https://github.com/pukkandan) and [llacb47](https://github.com/llacb47)
+* [extractor] Allow `note=False` when extracting manifests
+* [utils] Escape URLs in `sanitized_Request`, not `sanitize_url`
+* [hls] Disable external downloader for `webtt`
+* [youtube] `/live` URLs should raise error if channel is not live
+* [youtube] Bug fixes
+* [zee5] Fix m3u8 formats' extension
+* [ard] Allow URLs without `-` before id by [olifre](https://github.com/olifre)
+* [cleanup] `YoutubeDL._match_entry`
+* [cleanup] Refactor updater
+* [cleanup] Refactor ffmpeg convertors
+* [cleanup] setup.py
+
+
+### 2021.05.20
+
+* **Youtube improvements**: 
+    * Support youtube music `MP`, `VL` and `browse` pages
+    * Extract more formats for youtube music by [craftingmod](https://github.com/craftingmod), [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
+    * Extract multiple subtitles in same language by [pukkandan](https://github.com/pukkandan) and [tpikonen](https://github.com/tpikonen)
+    * Redirect channels that doesn't have a `videos` tab to their `UU` playlists
+    * Support in-channel search
+    * Sort audio-only formats correctly
+    * Always extract `maxresdefault` thumbnail
+    * Extract audio language
+    * Add subtitle language names by [nixxo](https://github.com/nixxo) and [tpikonen](https://github.com/tpikonen)
+    * Show alerts only from the final webpage
+    * Add `html5=1` param to `get_video_info` page requests by [colethedj](https://github.com/colethedj)
+    * Better message when login required
+* **Add option `--print`**: to print any field/template
+    * Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
+* Field `additional_urls` to download additional videos from metadata using [`--parse-metadata`](https://github.com/yt-dlp/yt-dlp#modifying-metadata)
+* Merge youtube-dl: Upto [commit/dfbbe29](https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b)
+* Write thumbnail of playlist and add `pl_thumbnail` outtmpl key
+* [embedthumbnail] Add `flac` support and refactor `mutagen` code by [pukkandan](https://github.com/pukkandan) and [tripulse](https://github.com/tripulse)
+* [audius:artist] Add extractor by [king-millez](https://github.com/king-millez)
+* [parlview] Add extractor by [king-millez](https://github.com/king-millez)
+* [tenplay] Fix extractor by [king-millez](https://github.com/king-millez)
+* [rmcdecouverte] Generalize `_VALID_URL`
+* Add compat-option `no-attach-infojson`
+* Add field `name` for subtitles
+* Ensure `post_extract` and `pre_process` only run once
+* Fix `--check-formats` when there is network error
+* Standardize `write_debug` and `get_param`
+* [options] Alias `--write-comments`, `--no-write-comments`
+* [options] Refactor callbacks
+* [test:download] Only extract enough videos for `playlist_mincount`
+* [extractor] bugfix for when `compat_opts` is not given
+* [build] Fix x86 build by [shirt](https://github.com/shirt-dev)
+* [cleanup] code formatting, youtube tests and readme
+
+### 2021.05.11
+* **Deprecate support for python versions < 3.6**
+* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
+* **Improve output template:**
+    * Allow slicing lists/strings using `field.start:end:step`
+    * A field can also be used as offset like `field1+num+field2`
+    * A default value can be given using `field|default`
+    * Prevent invalid fields from causing errors
+* **Merge youtube-dl**: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
+* **Remove options** `-l`, `-t`, `-A` completely and disable `--auto-number`, `--title`, `--literal`, `--id`
+* [Plugins] Prioritize plugins over standard extractors and prevent plugins from overwriting the standard extractor classes
+* [downloader] Fix `quiet` and `to_stderr`
+* [fragment] Ensure the file is closed on error
+* [fragment] Make sure first segment is not skipped
+* [aria2c] Fix whitespace being stripped off
+* [embedthumbnail] Fix bug where jpeg thumbnails were converted again
+* [FormatSort] Fix for when some formats have quality and others don't
+* [utils] Add `network_exceptions`
+* [utils] Escape URL while sanitizing
+* [ukcolumn] Add Extractor
+* [whowatch] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
+* [CBS] Improve `_VALID_URL` to support movies
+* [crackle] Improve extraction
+* [curiositystream] Fix collections
+* [francetvinfo] Improve video id extraction
+* [generic] Respect the encoding in manifest
+* [limelight] Obey `allow_unplayable_formats`
+* [mediasite] Generalize URL pattern by [fstirlitz](https://github.com/fstirlitz)
+* [mxplayer] Add MxplayerShowIE by [Ashish0804](https://github.com/Ashish0804)
+* [nebula] Move to nebula.app by [Lamieur](https://github.com/Lamieur)
+* [niconico] Fix HLS formats by [CXwudi](https://github.com/CXwudi), [tsukumijima](https://github.com/tsukumijima), [nao20010128nao](https://github.com/nao20010128nao) and [pukkandan](https://github.com/pukkandan)
+* [niconico] Fix title and thumbnail extraction by [CXwudi](https://github.com/CXwudi)
+* [plutotv] Extract subtitles from manifests
+* [plutotv] Fix format extraction for some urls
+* [rmcdecouverte] Improve `_VALID_URL`
+* [sonyliv] Fix `title` and `series` extraction by [Ashish0804](https://github.com/Ashish0804)
+* [tubi] Raise "no video formats" error when video url is empty
+* [youtube:tab] Detect playlists inside community posts
+* [youtube] Add `oembed` to reserved names
+* [zee5] Fix extraction for some URLs by [Hadi0609](https://github.com/Hadi0609)
+* [zee5] Fix py2 compatibility
+* Fix `playlist_index` and add `playlist_autonumber`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details
+* Add experimental option `--check-formats` to test the URLs before format selection
+* Option `--compat-options` to revert [some of yt-dlp's changes](https://github.com/yt-dlp/yt-dlp#differences-in-default-behavior)
+    * Deprecates `--list-formats-as-table`, `--list-formats-old`
+* Fix number of digits in `%(playlist_index)s`
+* Fix case sensitivity of format selector
+* Revert "[core] be able to hand over id and title using url_result"
+* Do not strip out whitespaces in `-o` and `-P`
+* Fix `preload_download_archive` writing verbose message to `stdout`
+* Move option warnings to `YoutubeDL`so that they obey `--no-warnings` and can output colors
+* Py2 compatibility for `FileNotFoundError`
+
+
+### 2021.04.22
+* **Improve output template:**
+    * Objects can be traversed like `%(field.key1.key2)s`
+    * An offset can be added to numeric fields as `%(field+N)s`
+    * Deprecates `--autonumber-start`
+* **Improve `--sub-langs`:**
+    * Treat `--sub-langs` entries as regex
+    * `all` can be used to refer to all the subtitles
+    * language codes can be prefixed with `-` to exclude it
+    * Deprecates `--all-subs`
+* Add option `--ignore-no-formats-error` to ignore the "no video format" and similar errors
+* Add option `--skip-playlist-after-errors` to skip the rest of a playlist after a given number of errors are encountered
+* Merge youtube-dl: Upto [commit/7e8b3f9](https://github.com/ytdl-org/youtube-dl/commit/7e8b3f9439ebefb3a3a4e5da9c0bd2b595976438)
+* [downloader] Fix bug in downloader selection
+* [BilibiliChannel] Fix pagination by [nao20010128nao](https://github.com/nao20010128nao) and [pukkandan](https://github.com/pukkandan)
+* [rai] Add support for http formats by [nixxo](https://github.com/nixxo)
+* [TubiTv] Add TubiTvShowIE by [Ashish0804](https://github.com/Ashish0804)
+* [twitcasting] Fix extractor
+* [viu:ott] Fix extractor and support series by [lkho](https://github.com/lkho) and [pukkandan](https://github.com/pukkandan)
+* [youtube:tab] Show unavailable videos in playlists by [colethedj](https://github.com/colethedj)
+* [youtube:tab] Reload with unavailable videos for all playlists
+* [youtube] Ignore invalid stretch ratio
+* [youtube] Improve channel syncid extraction to support ytcfg by [colethedj](https://github.com/colethedj)
+* [youtube] Standardize API calls for tabs, mixes and search by [colethedj](https://github.com/colethedj)
+* [youtube] Bugfix in `_extract_ytcfg`
+* [mildom:user:vod] Download only necessary amount of pages
+* [mildom] Remove proxy completely by [fstirlitz](https://github.com/fstirlitz)
+* [go] Fix `_VALID_URL`
+* [MetadataFromField] Improve regex and add tests
+* [Exec] Ensure backward compatibility when the command contains `%`
+* [extractor] Fix inconsistent use of `report_warning`
+* Ensure `mergeall` selects best format when multistreams are disabled
+* Improve the yt-dlp.sh script by [fstirlitz](https://github.com/fstirlitz)
+* [lazy_extractor] Do not load plugins
+* [ci] Disable fail-fast
+* [documentation] Clarify which deprecated options still work
+* [documentation] Fix typos
+
+
+### 2021.04.11
+* Add option `--convert-thumbnails` (only jpg currently supported)
+* Format selector `mergeall` to download and merge all formats
+* Pass any field to `--exec` using similar syntax to output template
+* Choose downloader for each protocol using `--downloader PROTO:NAME`
+    * Alias `--downloader` for `--external-downloader`
+    * Added `native` as an option for the downloader
+* Merge youtube-dl: Upto [commit/4fb25ff](https://github.com/ytdl-org/youtube-dl/commit/4fb25ff5a3be5206bb72e5c4046715b1529fb2c7) (except vimeo)
+* [DiscoveryPlusIndia] Add DiscoveryPlusIndiaShowIE by [Ashish0804](https://github.com/Ashish0804)
+* [NFHSNetwork] Add extractor by [llacb47](https://github.com/llacb47)
+* [nebula] Add extractor (watchnebula.com) by [hheimbuerger](https://github.com/hheimbuerger)
+* [nitter] Fix extraction of reply tweets and update instance list by [B0pol](https://github.com/B0pol)
+* [nitter] Fix thumbnails by [B0pol](https://github.com/B0pol)
+* [youtube] Fix thumbnail URL
+* [youtube] Parse API parameters from initial webpage by [colethedj](https://github.com/colethedj)
+* [youtube] Extract comments' approximate timestamp by [colethedj](https://github.com/colethedj)
+* [youtube] Fix alert extraction
+* [bilibili] Fix uploader
+* [utils] Add `datetime_from_str` and `datetime_add_months` by [colethedj](https://github.com/colethedj)
+* Run some `postprocessors` before actual download
+* Improve argument parsing for `-P`, `-o`, `-S`
+* Fix some `m3u8` not obeying `--allow-unplayable-formats`
+* Fix default of `dynamic_mpd`
+* Deprecate `--all-formats`, `--include-ads`, `--hls-prefer-native`, `--hls-prefer-ffmpeg`
+* [documentation] Improvements
+
+### 2021.04.03
+* Merge youtube-dl: Upto [commit/654b4f4](https://github.com/ytdl-org/youtube-dl/commit/654b4f4ff2718f38b3182c1188c5d569c14cc70a)
+* Ability to set a specific field in the file's metadata using `--parse-metadata`
+* Ability to select n'th best format like `-f bv*.2`
+* [DiscoveryPlus] Add discoveryplus.in
+* [la7] Add podcasts and podcast playlists by [nixxo](https://github.com/nixxo)
+* [mildom] Update extractor with current proxy by [nao20010128nao](https://github.com/nao20010128nao)
+* [ard:mediathek] Fix video id extraction
+* [generic] Detect Invidious' link element
+* [youtube] Show premium state in `availability` by [colethedj](https://github.com/colethedj)
+* [viewsource] Add extractor to handle `view-source:`
+* [sponskrub] Run before embedding thumbnail
+* [documentation] Improve `--parse-metadata` documentation
+
 
 ### 2021.03.24.1
 * Revert [commit/8562218](https://github.com/ytdl-org/youtube-dl/commit/8562218350a79d4709da8593bb0c538aa0824acf)
 
-
 ### 2021.03.24
-* Merge youtube-dl: Upto [commit/8562218](https://github.com/ytdl-org/youtube-dl/commit/8562218350a79d4709da8593bb0c538aa0824acf)
+* Merge youtube-dl: Upto 2021.03.25 ([commit/8562218](https://github.com/ytdl-org/youtube-dl/commit/8562218350a79d4709da8593bb0c538aa0824acf))
 * Parse metadata from multiple fields using `--parse-metadata`
 * Ability to load playlist infojson using `--load-info-json`
 * Write current epoch to infojson when using `--no-clean-infojson`
 * [youtube_live_chat] fix bug when trying to set cookies
-* [niconico] Fix for when logged in by: @CXwudi and @xtkoba
+* [niconico] Fix for when logged in by [CXwudi](https://github.com/CXwudi) and [xtkoba](https://github.com/xtkoba)
 * [linuxacadamy] Fix login
 
 
@@ -102,7 +498,6 @@
 ### 2021.03.03.2
 * [build] Fix bug
 
-
 ### 2021.03.03
 * [youtube] Use new browse API for continuation page extraction by [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
 * Fix HLS playlist downloading by [shirt](https://github.com/shirt-dev)
@@ -185,7 +580,7 @@
 
 ### 2021.02.15
 * Merge youtube-dl: Upto [2021.02.10](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.02.10) (except archive.org)
-* [niconico] Improved extraction and support encrypted/SMILE movies by [kurumigi](https://github.com/kurumigi), [tsukumi](https://github.com/tsukumi), [bbepis](https://github.com/bbepis), [pukkandan](https://github.com/pukkandan)
+* [niconico] Improved extraction and support encrypted/SMILE movies by [kurumigi](https://github.com/kurumigi), [tsukumijima](https://github.com/tsukumijima), [bbepis](https://github.com/bbepis), [pukkandan](https://github.com/pukkandan)
 * Fix HLS AES-128 with multiple keys in external downloaders by [shirt](https://github.com/shirt-dev)
 * [youtube_live_chat] Fix by using POST API by [siikamiika](https://github.com/siikamiika)
 * [rumble] Add support for video page
@@ -357,7 +752,7 @@
 
 ### 2021.01.08
 * Merge youtube-dl: Upto [2021.01.08](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.01.08) except stitcher ([1](https://github.com/ytdl-org/youtube-dl/commit/bb38a1215718cdf36d73ff0a7830a64cd9fa37cc), [2](https://github.com/ytdl-org/youtube-dl/commit/a563c97c5cddf55f8989ed7ea8314ef78e30107f))
-* Moved changelog to seperate file
+* Moved changelog to separate file
 
 
 ### 2021.01.07-1
@@ -439,4 +834,4 @@
 * [generic] Extract embedded youtube and twitter videos by [diegorodriguezv](https://github.com/diegorodriguezv)
 * [ffmpeg] Ensure all streams are copied by [pukkandan](https://github.com/pukkandan)
 * [embedthumbnail] Fix for os.rename error by [pukkandan](https://github.com/pukkandan)
-* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)
+* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)

Makefile

@@ -25,6 +25,7 @@ completion-zsh: completions/zsh/_yt-dlp
 lazy-extractors: yt_dlp/extractor/lazy_extractors.py
 
 PREFIX ?= /usr/local
+DESTDIR ?= .
 BINDIR ?= $(PREFIX)/bin
 MANDIR ?= $(PREFIX)/man
 SHAREDIR ?= $(PREFIX)/share

devscripts/bash-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import os

devscripts/buildserver.py

@@ -1,3 +1,5 @@
+# UNUSED
+
 #!/usr/bin/python3
 
 import argparse

devscripts/check-porn.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 """

devscripts/create-github-release.py

@@ -1,6 +1,6 @@
 # Unused
 
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/fish-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import optparse

devscripts/generate_aes_testdata.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import codecs

devscripts/gh-pages.unused/update-copyright.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import with_statement, unicode_literals

devscripts/lazy_load_template.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

devscripts/make_contributing.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # import io

devscripts/make_issue_template.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/make_lazy_extractors.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals, print_function
 
 from inspect import getsource
@@ -14,9 +15,14 @@ lazy_extractors_filename = sys.argv[1]
 if os.path.exists(lazy_extractors_filename):
     os.remove(lazy_extractors_filename)
 
+# Block plugins from loading
+os.rename('ytdlp_plugins', 'ytdlp_plugins_blocked')
+
 from yt_dlp.extractor import _ALL_CLASSES
 from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
 
+os.rename('ytdlp_plugins_blocked', 'ytdlp_plugins')
+
 with open('devscripts/lazy_load_template.py', 'rt') as f:
     module_template = f.read()
 

devscripts/make_readme.py

@@ -1,3 +1,8 @@
+#!/usr/bin/env python3
+
+# yt-dlp --help | make_readme.py
+# This must be run in a console of correct width
+
 from __future__ import unicode_literals
 
 import io

devscripts/make_supportedsites.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/prepare_manpage.py

@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/release.sh

@@ -1,4 +1,5 @@
 # Unused
+
 #!/bin/bash
 
 # IMPORTANT: the following assumptions are made

devscripts/show-downloads-statistics.py

@@ -1,6 +1,6 @@
 # Unused
 
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import itertools

devscripts/update-version.py

@@ -1,4 +1,6 @@
+#!/usr/bin/env python3
 from __future__ import unicode_literals
+
 from datetime import datetime
 # import urllib.request
 

devscripts/wine-py2exe.sh

@@ -1,3 +1,5 @@
+# UNUSED
+
 #!/bin/bash
 
 # Run with as parameter a setup.py that works in the current directory

devscripts/zsh-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import os

pyinst.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -6,6 +6,7 @@ import sys
 # import os
 import platform
 
+from PyInstaller.utils.hooks import collect_submodules
 from PyInstaller.utils.win32.versioninfo import (
     VarStruct, VarFileInfo, StringStruct, StringTable,
     StringFileInfo, FixedFileInfo, VSVersionInfo, SetVersion,
@@ -17,7 +18,7 @@ assert arch in ('32', '64')
 print('Building %sbit version' % arch)
 _x86 = '_x86' if arch == '32' else ''
 
-FILE_DESCRIPTION = 'Media Downloader%s' % (' (32 Bit)' if _x86 else '')
+FILE_DESCRIPTION = 'yt-dlp%s' % (' (32 Bit)' if _x86 else '')
 
 # root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
 # print('Changing working directory to %s' % root_dir)
@@ -58,21 +59,23 @@ VERSION_FILE = VSVersionInfo(
                     ),
                     StringStruct('OriginalFilename', 'yt-dlp%s.exe' % _x86),
                     StringStruct('ProductName', 'yt-dlp%s' % _x86),
-                    StringStruct('ProductVersion', '%s%s' % (VERSION, _x86)),
+                    StringStruct(
+                        'ProductVersion',
+                        '%s%s on Python %s' % (VERSION, _x86, platform.python_version())),
                 ])]),
         VarFileInfo([VarStruct('Translation', [0, 1200])])
     ]
 )
 
+dependancies = ['Crypto', 'mutagen'] + collect_submodules('websockets')
+excluded_modules = ['test', 'ytdlp_plugins', 'youtube-dl', 'youtube-dlc']
+
 PyInstaller.__main__.run([
     '--name=yt-dlp%s' % _x86,
     '--onefile',
     '--icon=devscripts/cloud.ico',
-    '--exclude-module=youtube_dl',
-    '--exclude-module=test',
-    '--exclude-module=ytdlp_plugins',
-    '--hidden-import=mutagen',
-    '--hidden-import=Crypto',
+    *[f'--exclude-module={module}' for module in excluded_modules],
+    *[f'--hidden-import={module}' for module in dependancies],
     '--upx-exclude=vcruntime140.dll',
     'yt_dlp/__main__.py',
 ])

requirements.txt

@@ -1,2 +1,3 @@
 mutagen
 pycryptodome
+websockets

setup.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from setuptools import setup, Command, find_packages
@@ -9,45 +9,44 @@ from distutils.spawn import spawn
 
 
 # Get the version from yt_dlp/version.py without importing the package
-exec(compile(open('yt_dlp/version.py').read(),
-             'yt_dlp/version.py', 'exec'))
+exec(compile(open('yt_dlp/version.py').read(), 'yt_dlp/version.py', 'exec'))
 
 
 DESCRIPTION = 'Command-line program to download videos from YouTube.com and many other other video platforms.'
 
 LONG_DESCRIPTION = '\n\n'.join((
     'Official repository: <https://github.com/yt-dlp/yt-dlp>',
-    '**PS**: Many links in this document will not work since this is a copy of the README.md from Github',
-    open("README.md", "r", encoding="utf-8").read()))
+    '**PS**: Some links in this document will not work since this is a copy of the README.md from Github',
+    open('README.md', 'r', encoding='utf-8').read()))
 
-REQUIREMENTS = ['mutagen', 'pycryptodome']
+REQUIREMENTS = ['mutagen', 'pycryptodome', 'websockets']
+
+if sys.argv[1:2] == ['py2exe']:
+    raise NotImplementedError('py2exe is not currently supported; instead, use "pyinst.py" to build with pyinstaller')
 
 
-if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
-    print("inv")
-else:
-    files_spec = [
-        ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
-        ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
-        ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
-        ('share/doc/yt_dlp', ['README.txt']),
-        ('share/man/man1', ['yt-dlp.1'])
-    ]
-    root = os.path.dirname(os.path.abspath(__file__))
-    data_files = []
-    for dirname, files in files_spec:
-        resfiles = []
-        for fn in files:
-            if not os.path.exists(fn):
-                warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first.' % fn)
-            else:
-                resfiles.append(fn)
-        data_files.append((dirname, resfiles))
+files_spec = [
+    ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
+    ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
+    ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
+    ('share/doc/yt_dlp', ['README.txt']),
+    ('share/man/man1', ['yt-dlp.1'])
+]
+root = os.path.dirname(os.path.abspath(__file__))
+data_files = []
+for dirname, files in files_spec:
+    resfiles = []
+    for fn in files:
+        if not os.path.exists(fn):
+            warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first' % fn)
+        else:
+            resfiles.append(fn)
+    data_files.append((dirname, resfiles))
 
-    params = {
-        'data_files': data_files,
-    }
-    params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
+params = {
+    'data_files': data_files,
+}
+params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
 
 
 class build_lazy_extractors(Command):
@@ -61,23 +60,21 @@ class build_lazy_extractors(Command):
         pass
 
     def run(self):
-        spawn(
-            [sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
-            dry_run=self.dry_run,
-        )
+        spawn([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
+              dry_run=self.dry_run)
 
 
-packages = find_packages(exclude=("youtube_dl", "test", "ytdlp_plugins"))
+packages = find_packages(exclude=('youtube_dl', 'test', 'ytdlp_plugins'))
 
 setup(
-    name="yt-dlp",
+    name='yt-dlp',
     version=__version__,
-    maintainer="pukkandan",
-    maintainer_email="pukkandan.ytdlp@gmail.com",
+    maintainer='pukkandan',
+    maintainer_email='pukkandan.ytdlp@gmail.com',
     description=DESCRIPTION,
     long_description=LONG_DESCRIPTION,
-    long_description_content_type="text/markdown",
-    url="https://github.com/yt-dlp/yt-dlp",
+    long_description_content_type='text/markdown',
+    url='https://github.com/yt-dlp/yt-dlp',
     packages=packages,
     install_requires=REQUIREMENTS,
     project_urls={
@@ -87,30 +84,20 @@ setup(
         #'Funding': 'https://donate.pypi.org',
     },
     classifiers=[
-        "Topic :: Multimedia :: Video",
-        "Development Status :: 5 - Production/Stable",
-        "Environment :: Console",
-        "Programming Language :: Python",
-        "Programming Language :: Python :: 2",
-        "Programming Language :: Python :: 2.6",
-        "Programming Language :: Python :: 2.7",
-        "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.2",
-        "Programming Language :: Python :: 3.3",
-        "Programming Language :: Python :: 3.4",
-        "Programming Language :: Python :: 3.5",
-        "Programming Language :: Python :: 3.6",
-        "Programming Language :: Python :: 3.7",
-        "Programming Language :: Python :: 3.8",
-        "Programming Language :: Python :: Implementation",
-        "Programming Language :: Python :: Implementation :: CPython",
-        "Programming Language :: Python :: Implementation :: IronPython",
-        "Programming Language :: Python :: Implementation :: Jython",
-        "Programming Language :: Python :: Implementation :: PyPy",
-        "License :: Public Domain",
-        "Operating System :: OS Independent",
+        'Topic :: Multimedia :: Video',
+        'Development Status :: 5 - Production/Stable',
+        'Environment :: Console',
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: Implementation',
+        'Programming Language :: Python :: Implementation :: CPython',
+        'Programming Language :: Python :: Implementation :: PyPy',
+        'License :: Public Domain',
+        'Operating System :: OS Independent',
     ],
-    python_requires='>=2.6',
+    python_requires='>=3.6',
 
     cmdclass={'build_lazy_extractors': build_lazy_extractors},
     **params

supportedsites.md

@@ -3,6 +3,7 @@
  - **20min**
  - **220.ro**
  - **23video**
+ - **247sports**
  - **24video**
  - **3qsdn**: 3Q SDN
  - **3sat**
@@ -81,6 +82,7 @@
  - **audiomack**
  - **audiomack:album**
  - **Audius**: Audius.co
+ - **audius:artist**: Audius.co profile/artist pages
  - **audius:playlist**: Audius.co playlists
  - **audius:track**: Audius track ID or API link. Prepend with "audius:"
  - **AWAAN**
@@ -129,7 +131,6 @@
  - **bitwave:stream**
  - **BleacherReport**
  - **BleacherReportCMS**
- - **blinkx**
  - **Bloomberg**
  - **BokeCC**
  - **BongaCams**
@@ -171,7 +172,8 @@
  - **cbsnews**: CBS News
  - **cbsnews:embed**
  - **cbsnews:livevideo**: CBS News Live Videos
- - **CBSSports**
+ - **cbssports**
+ - **cbssports:embed**
  - **CCMA**
  - **CCTV**: 央视网
  - **CDA**
@@ -249,12 +251,15 @@
  - **DiscoveryGoPlaylist**
  - **DiscoveryNetworksDe**
  - **DiscoveryPlus**
+ - **DiscoveryPlusIndia**
+ - **DiscoveryPlusIndiaShow**
  - **DiscoveryVR**
  - **Disney**
  - **dlive:stream**
  - **dlive:vod**
  - **DoodStream**
  - **Dotsub**
+ - **Douyin**
  - **DouyuShow**
  - **DouyuTV**: 斗鱼
  - **DPlay**
@@ -302,6 +307,8 @@
  - **EyedoTV**
  - **facebook**
  - **FacebookPluginsVideo**
+ - **fancode:live**
+ - **fancode:vod**
  - **faz.net**
  - **fc2**
  - **fc2:embed**
@@ -338,6 +345,8 @@
  - **FrontendMastersLesson**
  - **FujiTVFODPlus7**
  - **Funimation**
+ - **funimation:page**
+ - **funimation:show**
  - **Funk**
  - **Fusion**
  - **Fux**
@@ -387,6 +396,7 @@
  - **HotNewHipHop**
  - **hotstar**
  - **hotstar:playlist**
+ - **hotstar:series**
  - **Howcast**
  - **HowStuffWorks**
  - **hrfernsehen**
@@ -457,6 +467,8 @@
  - **kuwo:singer**: 酷我音乐 - 歌手
  - **kuwo:song**: 酷我音乐
  - **la7.it**
+ - **la7.it:pod:episode**
+ - **la7.it:podcast**
  - **laola1tv**
  - **laola1tv:embed**
  - **lbry**
@@ -480,14 +492,14 @@
  - **limelight**
  - **limelight:channel**
  - **limelight:channel_list**
+ - **LineLive**
+ - **LineLiveChannel**
  - **LineTV**
  - **linkedin:learning**
  - **linkedin:learning:course**
  - **LinuxAcademy**
  - **LiTV**
  - **LiveJournal**
- - **LiveLeak**
- - **LiveLeakEmbed**
  - **livestream**
  - **livestream:original**
  - **LnkGo**
@@ -506,6 +518,7 @@
  - **mangomolo:live**
  - **mangomolo:video**
  - **ManyVids**
+ - **MaoriTV**
  - **Markiza**
  - **MarkizaPage**
  - **massengeschmack.tv**
@@ -575,6 +588,7 @@
  - **Mwave**
  - **MwaveMeetGreet**
  - **Mxplayer**
+ - **MxplayerShow**
  - **MyChannels**
  - **MySpace**
  - **MySpace:album**
@@ -606,6 +620,7 @@
  - **ndr:embed**
  - **ndr:embed:base**
  - **NDTV**
+ - **Nebula**
  - **NerdCubedFeed**
  - **netease:album**: 网易云音乐 - 专辑
  - **netease:djradio**: 网易云音乐 - 电台
@@ -624,6 +639,7 @@
  - **NextTV**: 壹電視
  - **Nexx**
  - **NexxEmbed**
+ - **NFHSNetwork**
  - **nfl.com** (Currently broken)
  - **nfl.com:article** (Currently broken)
  - **NhkVod**
@@ -707,9 +723,13 @@
  - **OutsideTV**
  - **PacktPub**
  - **PacktPubCourse**
+ - **PalcoMP3:artist**
+ - **PalcoMP3:song**
+ - **PalcoMP3:video**
  - **pandora.tv**: 판도라TV
  - **ParamountNetwork**
  - **parliamentlive.tv**: UK parliament videos
+ - **Parlview**
  - **Patreon**
  - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
  - **PearVideo**
@@ -733,6 +753,7 @@
  - **play.fm**
  - **player.sky.it**
  - **PlayPlusTV**
+ - **PlayStuff**
  - **PlaysTV**
  - **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
  - **Playvid**
@@ -749,6 +770,7 @@
  - **PopcornTV**
  - **PornCom**
  - **PornerBros**
+ - **PornFlip**
  - **PornHd**
  - **PornHub**: PornHub and Thumbzilla
  - **PornHubPagedVideoList**
@@ -791,6 +813,8 @@
  - **RCS**
  - **RCSEmbeds**
  - **RCSVarious**
+ - **RCTIPlus**
+ - **RCTIPlusSeries**
  - **RDS**: RDS.ca
  - **RedBull**
  - **RedBullEmbed**
@@ -841,6 +865,7 @@
  - **safari**: safaribooksonline.com online video
  - **safari:api**
  - **safari:course**: safaribooksonline.com online courses
+ - **Saitosan**
  - **SAKTV**
  - **SaltTV**
  - **SampleFocus**
@@ -865,6 +890,7 @@
  - **Shahid**
  - **ShahidShow**
  - **Shared**: shared.sx
+ - **ShemarooMe**
  - **ShowRoomLive**
  - **simplecast**
  - **simplecast:episode**
@@ -884,6 +910,7 @@
  - **Snotr**
  - **Sohu**
  - **SonyLIV**
+ - **SonyLIVSeries**
  - **soundcloud**
  - **soundcloud:playlist**
  - **soundcloud:search**: Soundcloud search
@@ -962,6 +989,7 @@
  - **Telecinco**: telecinco.es, cuatro.com and mediaset.es
  - **Telegraaf**
  - **TeleMB**
+ - **Telemundo**
  - **TeleQuebec**
  - **TeleQuebecEmission**
  - **TeleQuebecLive**
@@ -1002,6 +1030,7 @@
  - **TruTV**
  - **Tube8**
  - **TubiTv**
+ - **TubiTvShow**
  - **Tumblr**
  - **tunein:clip**
  - **tunein:program**
@@ -1044,6 +1073,8 @@
  - **TVPlayHome**
  - **Tweakers**
  - **TwitCasting**
+ - **TwitCastingLive**
+ - **TwitCastingUser**
  - **twitch:clips**
  - **twitch:stream**
  - **twitch:vod**
@@ -1061,6 +1092,7 @@
  - **UDNEmbed**: 聯合影音
  - **UFCArabia**
  - **UFCTV**
+ - **ukcolumn**
  - **UKTVPlay**
  - **umg:de**: Universal Music Deutschland
  - **Unistra**
@@ -1093,6 +1125,7 @@
  - **Vidbit**
  - **Viddler**
  - **Videa**
+ - **video.arnes.si**: Arnes Video
  - **video.google:search**: Google Video search
  - **video.sky.it**
  - **video.sky.it:live**
@@ -1103,6 +1136,8 @@
  - **videomore:video**
  - **VideoPress**
  - **Vidio**
+ - **VidioLive**
+ - **VidioPremier**
  - **VidLii**
  - **vidme**
  - **vidme:user**
@@ -1142,6 +1177,7 @@
  - **VODPlatform**
  - **VoiceRepublic**
  - **Voot**
+ - **VootSeries**
  - **VoxMedia**
  - **VoxMediaVolume**
  - **vpro**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
@@ -1171,6 +1207,7 @@
  - **wdr:mobile**
  - **WDRElefant**
  - **WDRPage**
+ - **web.archive:youtube**: web.archive.org saved youtube videos
  - **Webcaster**
  - **WebcasterFeed**
  - **WebOfStories**
@@ -1178,6 +1215,7 @@
  - **Weibo**
  - **WeiboMobile**
  - **WeiqiTV**: WQTV
+ - **whowatch**
  - **WimTV**
  - **Wistia**
  - **WistiaPlaylist**
@@ -1188,7 +1226,7 @@
  - **WWE**
  - **XBef**
  - **XboxClips**
- - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, XVideoSharing
+ - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, WolfStream, XVideoSharing
  - **XHamster**
  - **XHamsterEmbed**
  - **XHamsterUser**

test/parameters.json

@@ -1,41 +1,40 @@
 {
-    "consoletitle": false, 
-    "continuedl": true, 
-    "forcedescription": false, 
-    "forcefilename": false, 
-    "forceformat": false, 
-    "forcethumbnail": false, 
-    "forcetitle": false, 
-    "forceurl": false, 
+    "consoletitle": false,
+    "continuedl": true,
+    "forcedescription": false,
+    "forcefilename": false,
+    "forceformat": false,
+    "forcethumbnail": false,
+    "forcetitle": false,
+    "forceurl": false,
     "force_write_download_archive": false,
     "format": "best",
-    "ignoreerrors": false, 
-    "listformats": null, 
-    "logtostderr": false, 
-    "matchtitle": null, 
-    "max_downloads": null, 
-    "overwrites": null, 
-    "nopart": false, 
-    "noprogress": false, 
-    "outtmpl": "%(id)s.%(ext)s", 
-    "password": null, 
-    "playlistend": -1, 
-    "playliststart": 1, 
-    "prefer_free_formats": false, 
-    "quiet": false, 
-    "ratelimit": null, 
-    "rejecttitle": null, 
-    "retries": 10, 
-    "simulate": false, 
-    "subtitleslang": null, 
+    "ignoreerrors": false,
+    "listformats": null,
+    "logtostderr": false,
+    "matchtitle": null,
+    "max_downloads": null,
+    "overwrites": null,
+    "nopart": false,
+    "noprogress": false,
+    "outtmpl": "%(id)s.%(ext)s",
+    "password": null,
+    "playliststart": 1,
+    "prefer_free_formats": false,
+    "quiet": false,
+    "ratelimit": null,
+    "rejecttitle": null,
+    "retries": 10,
+    "simulate": false,
+    "subtitleslang": null,
     "subtitlesformat": "best",
-    "test": true, 
-    "updatetime": true, 
-    "usenetrc": false, 
-    "username": null, 
-    "verbose": true, 
-    "writedescription": false, 
-    "writeinfojson": true, 
+    "test": true,
+    "updatetime": true,
+    "usenetrc": false,
+    "username": null,
+    "verbose": true,
+    "writedescription": false,
+    "writeinfojson": true,
     "writeannotations": false,
     "writelink": false,
     "writeurllink": false,

test/test_InfoExtractor.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -450,7 +450,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': 'aud2-English',
@@ -458,7 +458,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': 'aud3-English',
@@ -466,14 +466,14 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': '530',
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -482,7 +482,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -491,7 +491,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -500,7 +500,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -509,7 +509,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -518,7 +518,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -527,7 +527,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -536,7 +536,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -545,7 +545,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -554,7 +554,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -563,7 +563,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -572,7 +572,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -581,7 +581,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -590,7 +590,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -599,7 +599,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -608,7 +608,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -617,7 +617,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -626,7 +626,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -635,7 +635,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -644,7 +644,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -653,7 +653,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -662,7 +662,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -671,7 +671,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -680,21 +680,190 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8',
+                    'protocol': 'm3u8_native',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
-                }]
+                }],
+                {}
+            ),
+            (
+                'bipbop_16x9',
+                'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                [{
+                    'format_id': 'bipbop_audio-BipBop Audio 2',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/alternate_audio_aac/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'language': 'eng',
+                    'ext': 'mp4',
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'vcodec': 'none',
+                    'audio_ext': 'mp4',
+                    'video_ext': 'none',
+                }, {
+                    'format_id': '41',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear0/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 41.457,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'vcodec': 'none',
+                    'acodec': 'mp4a.40.2',
+                    'audio_ext': 'mp4',
+                    'video_ext': 'none',
+                    'abr': 41.457,
+                }, {
+                    'format_id': '263',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear1/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 263.851,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 416,
+                    'height': 234,
+                    'vcodec': 'avc1.4d400d',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 263.851,
+                    'abr': 0,
+                }, {
+                    'format_id': '577',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear2/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 577.61,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 640,
+                    'height': 360,
+                    'vcodec': 'avc1.4d401e',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 577.61,
+                    'abr': 0,
+                }, {
+                    'format_id': '915',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear3/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 915.905,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 960,
+                    'height': 540,
+                    'vcodec': 'avc1.4d401f',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 915.905,
+                    'abr': 0,
+                }, {
+                    'format_id': '1030',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear4/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 1030.138,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 1280,
+                    'height': 720,
+                    'vcodec': 'avc1.4d401f',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 1030.138,
+                    'abr': 0,
+                }, {
+                    'format_id': '1924',
+                    'format_index': None,
+                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear5/prog_index.m3u8',
+                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
+                    'tbr': 1924.009,
+                    'ext': 'mp4',
+                    'fps': None,
+                    'protocol': 'm3u8_native',
+                    'preference': None,
+                    'quality': None,
+                    'width': 1920,
+                    'height': 1080,
+                    'vcodec': 'avc1.4d401f',
+                    'acodec': 'mp4a.40.2',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 1924.009,
+                    'abr': 0,
+                }],
+                {
+                    'en': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                    'fr': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                    'es': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                    'ja': [{
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }, {
+                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn_forced/prog_index.m3u8',
+                        'ext': 'vtt',
+                        'protocol': 'm3u8_native'
+                    }],
+                }
             ),
         ]
 
-        for m3u8_file, m3u8_url, expected_formats in _TEST_CASES:
+        for m3u8_file, m3u8_url, expected_formats, expected_subs in _TEST_CASES:
             with io.open('./test/testdata/m3u8/%s.m3u8' % m3u8_file,
                          mode='r', encoding='utf-8') as f:
-                formats = self.ie._parse_m3u8_formats(
+                formats, subs = self.ie._parse_m3u8_formats_and_subtitles(
                     f.read(), m3u8_url, ext='mp4')
                 self.ie._sort_formats(formats)
                 expect_value(self, formats, expected_formats, None)
+                expect_value(self, subs, expected_subs, None)
 
     def test_parse_mpd_formats(self):
         _TEST_CASES = [
@@ -780,7 +949,8 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'tbr': 5997.485,
                     'width': 1920,
                     'height': 1080,
-                }]
+                }],
+                {},
             ), (
                 # https://github.com/ytdl-org/youtube-dl/pull/14844
                 'urls_only',
@@ -863,7 +1033,8 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'tbr': 4400,
                     'width': 1920,
                     'height': 1080,
-                }]
+                }],
+                {},
             ), (
                 # https://github.com/ytdl-org/youtube-dl/issues/20346
                 # Media considered unfragmented even though it contains
@@ -909,18 +1080,328 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'width': 360,
                     'height': 360,
                     'fps': 30,
-                }]
+                }],
+                {},
+            ), (
+                'subtitles',
+                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/',
+                [{
+                    'format_id': 'audio=128001',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'm4a',
+                    'tbr': 128.001,
+                    'asr': 48000,
+                    'format_note': 'DASH audio',
+                    'container': 'm4a_dash',
+                    'vcodec': 'none',
+                    'acodec': 'mp4a.40.2',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'audio_ext': 'm4a',
+                    'video_ext': 'none',
+                    'abr': 128.001,
+                }, {
+                    'format_id': 'video=100000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 336,
+                    'height': 144,
+                    'tbr': 100,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 100,
+                }, {
+                    'format_id': 'video=326000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 562,
+                    'height': 240,
+                    'tbr': 326,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 326,
+                }, {
+                    'format_id': 'video=698000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 844,
+                    'height': 360,
+                    'tbr': 698,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 698,
+                }, {
+                    'format_id': 'video=1493000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 1126,
+                    'height': 480,
+                    'tbr': 1493,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 1493,
+                }, {
+                    'format_id': 'video=4482000',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'ext': 'mp4',
+                    'width': 1688,
+                    'height': 720,
+                    'tbr': 4482,
+                    'format_note': 'DASH video',
+                    'container': 'mp4_dash',
+                    'vcodec': 'avc1.4D401F',
+                    'acodec': 'none',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                    'protocol': 'http_dash_segments',
+                    'video_ext': 'mp4',
+                    'audio_ext': 'none',
+                    'vbr': 4482,
+                }],
+                {
+                    'en': [
+                        {
+                            'ext': 'mp4',
+                            'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                            'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
+                            'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
+                            'protocol': 'http_dash_segments',
+                        }
+                    ]
+                },
             )
         ]
 
-        for mpd_file, mpd_url, mpd_base_url, expected_formats in _TEST_CASES:
+        for mpd_file, mpd_url, mpd_base_url, expected_formats, expected_subtitles in _TEST_CASES:
             with io.open('./test/testdata/mpd/%s.mpd' % mpd_file,
                          mode='r', encoding='utf-8') as f:
-                formats = self.ie._parse_mpd_formats(
+                formats, subtitles = self.ie._parse_mpd_formats_and_subtitles(
                     compat_etree_fromstring(f.read().encode('utf-8')),
                     mpd_base_url=mpd_base_url, mpd_url=mpd_url)
                 self.ie._sort_formats(formats)
                 expect_value(self, formats, expected_formats, None)
+                expect_value(self, subtitles, expected_subtitles, None)
+
+    def test_parse_ism_formats(self):
+        _TEST_CASES = [
+            (
+                'sintel',
+                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                [{
+                    'format_id': 'audio-128',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'isma',
+                    'tbr': 128,
+                    'asr': 48000,
+                    'vcodec': 'none',
+                    'acodec': 'AACL',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'audio',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 0,
+                        'height': 0,
+                        'fourcc': 'AACL',
+                        'codec_private_data': '1190',
+                        'sampling_rate': 48000,
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'audio_ext': 'isma',
+                    'video_ext': 'none',
+                    'abr': 128,
+                }, {
+                    'format_id': 'video-100',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 336,
+                    'height': 144,
+                    'tbr': 100,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 336,
+                        'height': 144,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 100,
+                }, {
+                    'format_id': 'video-326',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 562,
+                    'height': 240,
+                    'tbr': 326,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 562,
+                        'height': 240,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 326,
+                }, {
+                    'format_id': 'video-698',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 844,
+                    'height': 360,
+                    'tbr': 698,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 844,
+                        'height': 360,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 698,
+                }, {
+                    'format_id': 'video-1493',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 1126,
+                    'height': 480,
+                    'tbr': 1493,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 1126,
+                        'height': 480,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 1493,
+                }, {
+                    'format_id': 'video-4482',
+                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                    'ext': 'ismv',
+                    'width': 1688,
+                    'height': 720,
+                    'tbr': 4482,
+                    'vcodec': 'AVC1',
+                    'acodec': 'none',
+                    'protocol': 'ism',
+                    '_download_params': {
+                        'stream_type': 'video',
+                        'duration': 8880746666,
+                        'timescale': 10000000,
+                        'width': 1688,
+                        'height': 720,
+                        'fourcc': 'AVC1',
+                        'codec_private_data': '00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8',
+                        'channels': 2,
+                        'bits_per_sample': 16,
+                        'nal_unit_length_field': 4
+                    },
+                    'video_ext': 'ismv',
+                    'audio_ext': 'none',
+                    'vbr': 4482,
+                }],
+                {
+                    'eng': [
+                        {
+                            'ext': 'ismt',
+                            'protocol': 'ism',
+                            'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                            'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
+                            '_download_params': {
+                                'stream_type': 'text',
+                                'duration': 8880746666,
+                                'timescale': 10000000,
+                                'fourcc': 'TTML',
+                                'codec_private_data': ''
+                            }
+                        }
+                    ]
+                },
+            ),
+        ]
+
+        for ism_file, ism_url, expected_formats, expected_subtitles in _TEST_CASES:
+            with io.open('./test/testdata/ism/%s.Manifest' % ism_file,
+                         mode='r', encoding='utf-8') as f:
+                formats, subtitles = self.ie._parse_ism_formats_and_subtitles(
+                    compat_etree_fromstring(f.read().encode('utf-8')), ism_url=ism_url)
+                self.ie._sort_formats(formats)
+                expect_value(self, formats, expected_formats, None)
+                expect_value(self, subtitles, expected_subtitles, None)
 
     def test_parse_f4m_formats(self):
         _TEST_CASES = [

test/test_YoutubeDL.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -17,7 +17,7 @@ from yt_dlp.compat import compat_str, compat_urllib_error
 from yt_dlp.extractor import YoutubeIE
 from yt_dlp.extractor.common import InfoExtractor
 from yt_dlp.postprocessor.common import PostProcessor
-from yt_dlp.utils import ExtractorError, match_filter_func
+from yt_dlp.utils import ExtractorError, int_or_none, match_filter_func
 
 TEST_URL = 'http://localhost/sample.mp4'
 
@@ -29,11 +29,15 @@ class YDL(FakeYDL):
         self.msgs = []
 
     def process_info(self, info_dict):
+        info_dict.pop('__original_infodict', None)
         self.downloaded_info_dicts.append(info_dict)
 
     def to_screen(self, msg):
         self.msgs.append(msg)
 
+    def dl(self, *args, **kwargs):
+        assert False, 'Downloader must not be invoked for test_YoutubeDL'
+
 
 def _make_result(formats, **kwargs):
     res = {
@@ -116,35 +120,24 @@ class TestFormatSelection(unittest.TestCase):
         ]
         info_dict = _make_result(formats)
 
-        ydl = YDL({'format': '20/47'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '47')
+        def test(inp, *expected, multi=False):
+            ydl = YDL({
+                'format': inp,
+                'allow_multiple_video_streams': multi,
+                'allow_multiple_audio_streams': multi,
+            })
+            ydl.process_ie_result(info_dict.copy())
+            downloaded = map(lambda x: x['format_id'], ydl.downloaded_info_dicts)
+            self.assertEqual(list(downloaded), list(expected))
 
-        ydl = YDL({'format': '20/71/worst'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '35')
-
-        ydl = YDL()
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '2')
-
-        ydl = YDL({'format': 'webm/mp4'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '47')
-
-        ydl = YDL({'format': '3gp/40/mp4'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '35')
-
-        ydl = YDL({'format': 'example-with-dashes'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], 'example-with-dashes')
+        test('20/47', '47')
+        test('20/71/worst', '35')
+        test(None, '2')
+        test('webm/mp4', '47')
+        test('3gp/40/mp4', '35')
+        test('example-with-dashes', 'example-with-dashes')
+        test('all', '35', 'example-with-dashes', '45', '47', '2')  # Order doesn't actually matter for this
+        test('mergeall', '2+47+45+example-with-dashes+35', multi=True)
 
     def test_format_selection_audio(self):
         formats = [
@@ -311,8 +304,8 @@ class TestFormatSelection(unittest.TestCase):
         self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
 
     def test_youtube_format_selection(self):
+        # FIXME: Rewrite in accordance with the new format sorting options
         return
-        # disabled for now - this needs some changes
 
         order = [
             '38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '17', '36', '13',
@@ -460,14 +453,13 @@ class TestFormatSelection(unittest.TestCase):
 
     def test_invalid_format_specs(self):
         def assert_syntax_error(format_spec):
-            ydl = YDL({'format': format_spec})
-            info_dict = _make_result([{'format_id': 'foo', 'url': TEST_URL}])
-            self.assertRaises(SyntaxError, ydl.process_ie_result, info_dict)
+            self.assertRaises(SyntaxError, YDL, {'format': format_spec})
 
         assert_syntax_error('bestvideo,,best')
         assert_syntax_error('+bestaudio')
         assert_syntax_error('bestvideo+')
         assert_syntax_error('/')
+        assert_syntax_error('[720<height]')
 
     def test_format_filtering(self):
         formats = [
@@ -601,6 +593,26 @@ class TestYoutubeDL(unittest.TestCase):
         self.assertTrue(subs)
         self.assertEqual(set(subs.keys()), set(['es', 'fr']))
 
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['all', '-en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['es', 'fr']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['en', 'fr', '-en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['fr']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['-en', 'en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['en']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['e.+']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['es', 'en']))
+
         result = get_info({'writesubtitles': True, 'writeautomaticsub': True, 'subtitleslangs': ['es', 'pt']})
         subs = result['requested_subtitles']
         self.assertTrue(subs)
@@ -627,47 +639,131 @@ class TestYoutubeDL(unittest.TestCase):
         self.assertEqual(test_dict['extractor'], 'Foo')
         self.assertEqual(test_dict['playlist'], 'funny videos')
 
-    def test_prepare_filename(self):
-        info = {
-            'id': '1234',
-            'ext': 'mp4',
-            'width': None,
-            'height': 1080,
-            'title1': '$PATH',
-            'title2': '%PATH%',
-        }
+    outtmpl_info = {
+        'id': '1234',
+        'ext': 'mp4',
+        'width': None,
+        'height': 1080,
+        'title1': '$PATH',
+        'title2': '%PATH%',
+        'title3': 'foo/bar\\test',
+        'timestamp': 1618488000,
+        'duration': 100000,
+        'playlist_index': 1,
+        '_last_playlist_index': 100,
+        'n_entries': 10,
+        'formats': [{'id': 'id1'}, {'id': 'id2'}, {'id': 'id3'}]
+    }
 
-        def fname(templ, na_placeholder='NA'):
-            params = {'outtmpl': templ}
-            if na_placeholder != 'NA':
-                params['outtmpl_na_placeholder'] = na_placeholder
+    def test_prepare_outtmpl_and_filename(self):
+        def test(tmpl, expected, *, info=None, **params):
+            params['outtmpl'] = tmpl
             ydl = YoutubeDL(params)
-            return ydl.prepare_filename(info)
-        self.assertEqual(fname('%(id)s.%(ext)s'), '1234.mp4')
-        self.assertEqual(fname('%(id)s-%(width)s.%(ext)s'), '1234-NA.mp4')
-        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(id)s.%(ext)s'
-        # Replace missing fields with 'NA' by default
-        self.assertEqual(fname(NA_TEST_OUTTMPL), 'NA-NA-1234.mp4')
-        # Or by provided placeholder
-        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder='none'), 'none-none-1234.mp4')
-        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder=''), '--1234.mp4')
-        self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
-        self.assertEqual(fname('%(height)6d.%(ext)s'), '  1080.mp4')
-        self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080  .mp4')
-        self.assertEqual(fname('%(height)06d.%(ext)s'), '001080.mp4')
-        self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)   06d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)0   6d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%(height)   0   6d.%(ext)s'), ' 01080.mp4')
-        self.assertEqual(fname('%%'), '%')
-        self.assertEqual(fname('%%%%'), '%%')
-        self.assertEqual(fname('%%(height)06d.%(ext)s'), '%(height)06d.mp4')
-        self.assertEqual(fname('%(width)06d.%(ext)s'), 'NA.mp4')
-        self.assertEqual(fname('%(width)06d.%%(ext)s'), 'NA.%(ext)s')
-        self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
-        self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
-        self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
+            ydl._num_downloads = 1
+            self.assertEqual(ydl.validate_outtmpl(tmpl), None)
+
+            outtmpl, tmpl_dict = ydl.prepare_outtmpl(tmpl, info or self.outtmpl_info)
+            out = outtmpl % tmpl_dict
+            fname = ydl.prepare_filename(info or self.outtmpl_info)
+
+            if callable(expected):
+                self.assertTrue(expected(out))
+                self.assertTrue(expected(fname))
+            elif isinstance(expected, compat_str):
+                self.assertEqual((out, fname), (expected, expected))
+            else:
+                self.assertEqual((out, fname), expected)
+
+        # Auto-generated fields
+        test('%(id)s.%(ext)s', '1234.mp4')
+        test('%(duration_string)s', ('27:46:40', '27-46-40'))
+        test('%(epoch)d', int_or_none)
+        test('%(resolution)s', '1080p')
+        test('%(playlist_index)s', '001')
+        test('%(autonumber)s', '00001')
+        test('%(autonumber+2)03d', '005', autonumber_start=3)
+        test('%(autonumber)s', '001', autonumber_size=3)
+
+        # Escaping %
+        test('%%', '%')
+        test('%%%%', '%%')
+        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+        test('%(width)06d.%(ext)s', 'NA.mp4')
+        test('%(width)06d.%%(ext)s', 'NA.%(ext)s')
+        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+
+        # ID sanitization
+        test('%(id)s', '_abcd', info={'id': '_abcd'})
+        test('%(some_id)s', '_abcd', info={'some_id': '_abcd'})
+        test('%(formats.0.id)s', '_abcd', info={'formats': [{'id': '_abcd'}]})
+        test('%(id)s', '-abcd', info={'id': '-abcd'})
+        test('%(id)s', '.abcd', info={'id': '.abcd'})
+        test('%(id)s', 'ab__cd', info={'id': 'ab__cd'})
+        test('%(id)s', ('ab:cd', 'ab -cd'), info={'id': 'ab:cd'})
+
+        # Invalid templates
+        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%'), ValueError))
+        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%(title)'), ValueError))
+        test('%(invalid@tmpl|def)s', 'none', outtmpl_na_placeholder='none')
+        test('%()s', 'NA')
+        test('%s', '%s')
+        test('%d', '%d')
+
+        # NA placeholder
+        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(x|def)s-%(id)s.%(ext)s'
+        test(NA_TEST_OUTTMPL, 'NA-NA-def-1234.mp4')
+        test(NA_TEST_OUTTMPL, 'none-none-def-1234.mp4', outtmpl_na_placeholder='none')
+        test(NA_TEST_OUTTMPL, '--def-1234.mp4', outtmpl_na_placeholder='')
+
+        # String formatting
+        FMT_TEST_OUTTMPL = '%%(height)%s.%%(ext)s'
+        test(FMT_TEST_OUTTMPL % 's', '1080.mp4')
+        test(FMT_TEST_OUTTMPL % 'd', '1080.mp4')
+        test(FMT_TEST_OUTTMPL % '6d', '  1080.mp4')
+        test(FMT_TEST_OUTTMPL % '-6d', '1080  .mp4')
+        test(FMT_TEST_OUTTMPL % '06d', '001080.mp4')
+        test(FMT_TEST_OUTTMPL % ' 06d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '   06d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '0 6d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '0   6d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '   0   6d', ' 01080.mp4')
+
+        # Type casting
+        test('%(id)d', '1234')
+        test('%(height)c', '1')
+        test('%(ext)c', 'm')
+        test('%(id)d %(id)r', "1234 '1234'")
+        test('%(id)r %(height)r', "'1234' 1080")
+        test('%(ext)s-%(ext|def)d', 'mp4-def')
+        test('%(width|0)04d', '0000')
+        test('a%(width|)d', 'a', outtmpl_na_placeholder='none')
+
+        # Internal formatting
+        FORMATS = self.outtmpl_info['formats']
+        test('%(timestamp-1000>%H-%M-%S)s', '11-43-20')
+        test('%(id+1-height+3)05d', '00158')
+        test('%(width+100)05d', 'NA')
+        test('%(formats.0) 15s', ('% 15s' % FORMATS[0], '% 15s' % str(FORMATS[0]).replace(':', ' -')))
+        test('%(formats.0)r', (repr(FORMATS[0]), repr(FORMATS[0]).replace(':', ' -')))
+        test('%(height.0)03d', '001')
+        test('%(-height.0)04d', '-001')
+        test('%(formats.-1.id)s', FORMATS[-1]['id'])
+        test('%(formats.0.id.-1)d', FORMATS[0]['id'][-1])
+        test('%(formats.3)s', 'NA')
+        test('%(formats.:2:-1)r', repr(FORMATS[:2:-1]))
+        test('%(formats.0.id.-1+id)f', '1235.000000')
+        test('%(formats.0.id.-1+formats.1.id.-1)d', '3')
+
+        # Empty filename
+        test('%(foo|)s-%(bar|)s.%(ext)s', '-.mp4')
+        # test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4'))  # fixme
+        # test('%(foo|)s', ('', '_'))  # fixme
+
+        # Path expansion and escaping
+        test('Hello %(title1)s', 'Hello $PATH')
+        test('Hello %(title2)s', 'Hello %PATH%')
+        test('%(title3)s', ('foo/bar\\test', 'foo_bar_test'))
+        test('folder/%(title3)s', ('folder/foo/bar\\test', 'folder%sfoo_bar_test' % os.path.sep))
 
     def test_format_note(self):
         ydl = YoutubeDL()
@@ -726,7 +822,7 @@ class TestYoutubeDL(unittest.TestCase):
             def process_info(self, info_dict):
                 super(YDL, self).process_info(info_dict)
 
-            def _match_entry(self, info_dict, incomplete):
+            def _match_entry(self, info_dict, incomplete=False):
                 res = super(FilterYDL, self)._match_entry(info_dict, incomplete)
                 if res is None:
                     self.downloaded_info_dicts.append(info_dict)

test/test_YoutubeDLCookieJar.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_aes.py

@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_age_restriction.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_all_urls.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -72,15 +72,6 @@ class TestAllURLsMatching(unittest.TestCase):
         self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
         self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])
 
-    def test_youtube_extract(self):
-        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
-        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
-        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
-
     def test_facebook_matching(self):
         self.assertTrue(FacebookIE.suitable('https://www.facebook.com/Shiniknoh#!/photo.php?v=10153317450565268'))
         self.assertTrue(FacebookIE.suitable('https://www.facebook.com/cindyweather?fref=ts#!/photo.php?v=10152183998945793'))

test/test_cache.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_compat.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_cookies.py

@@ -0,0 +1,96 @@
+import unittest
+from datetime import datetime, timezone
+
+from yt_dlp import cookies
+from yt_dlp.cookies import (
+    CRYPTO_AVAILABLE,
+    LinuxChromeCookieDecryptor,
+    MacChromeCookieDecryptor,
+    WindowsChromeCookieDecryptor,
+    YDLLogger,
+    parse_safari_cookies,
+    pbkdf2_sha1,
+)
+
+
+class MonkeyPatch:
+    def __init__(self, module, temporary_values):
+        self._module = module
+        self._temporary_values = temporary_values
+        self._backup_values = {}
+
+    def __enter__(self):
+        for name, temp_value in self._temporary_values.items():
+            self._backup_values[name] = getattr(self._module, name)
+            setattr(self._module, name, temp_value)
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        for name, backup_value in self._backup_values.items():
+            setattr(self._module, name, backup_value)
+
+
+class TestCookies(unittest.TestCase):
+    def test_chrome_cookie_decryptor_linux_derive_key(self):
+        key = LinuxChromeCookieDecryptor.derive_key(b'abc')
+        self.assertEqual(key, b'7\xa1\xec\xd4m\xfcA\xc7\xb19Z\xd0\x19\xdcM\x17')
+
+    def test_chrome_cookie_decryptor_mac_derive_key(self):
+        key = MacChromeCookieDecryptor.derive_key(b'abc')
+        self.assertEqual(key, b'Y\xe2\xc0\xd0P\xf6\xf4\xe1l\xc1\x8cQ\xcb|\xcdY')
+
+    def test_chrome_cookie_decryptor_linux_v10(self):
+        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b''}):
+            encrypted_value = b'v10\xccW%\xcd\xe6\xe6\x9fM" \xa7\xb0\xca\xe4\x07\xd6'
+            value = 'USD'
+            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_chrome_cookie_decryptor_linux_v11(self):
+        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b'',
+                                   'KEYRING_AVAILABLE': True}):
+            encrypted_value = b'v11#\x81\x10>`w\x8f)\xc0\xb2\xc1\r\xf4\x1al\xdd\x93\xfd\xf8\xf8N\xf2\xa9\x83\xf1\xe9o\x0elVQd'
+            value = 'tz=Europe.London'
+            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    @unittest.skipIf(not CRYPTO_AVAILABLE, 'cryptography library not available')
+    def test_chrome_cookie_decryptor_windows_v10(self):
+        with MonkeyPatch(cookies, {
+            '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&'
+        }):
+            encrypted_value = b'v10T\xb8\xf3\xb8\x01\xa7TtcV\xfc\x88\xb8\xb8\xef\x05\xb5\xfd\x18\xc90\x009\xab\xb1\x893\x85)\x87\xe1\xa9-\xa3\xad='
+            value = '32101439'
+            decryptor = WindowsChromeCookieDecryptor('', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_chrome_cookie_decryptor_mac_v10(self):
+        with MonkeyPatch(cookies, {'_get_mac_keyring_password': lambda *args, **kwargs: b'6eIDUdtKAacvlHwBVwvg/Q=='}):
+            encrypted_value = b'v10\xb3\xbe\xad\xa1[\x9fC\xa1\x98\xe0\x9a\x01\xd9\xcf\xbfc'
+            value = '2021-06-01-22'
+            decryptor = MacChromeCookieDecryptor('', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_safari_cookie_parsing(self):
+        cookies = \
+            b'cook\x00\x00\x00\x01\x00\x00\x00i\x00\x00\x01\x00\x01\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00Y' \
+            b'\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x008\x00\x00\x00B\x00\x00\x00F\x00\x00\x00H' \
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\xa5>\xc3A\x00\x00\x80\xc3\x07:\xc3A' \
+            b'localhost\x00foo\x00/\x00test%20%3Bcookie\x00\x00\x00\x054\x07\x17 \x05\x00\x00\x00Kbplist00\xd1\x01' \
+            b'\x02_\x10\x18NSHTTPCookieAcceptPolicy\x10\x02\x08\x0b&\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00' \
+            b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00('
+
+        jar = parse_safari_cookies(cookies)
+        self.assertEqual(len(jar), 1)
+        cookie = list(jar)[0]
+        self.assertEqual(cookie.domain, 'localhost')
+        self.assertEqual(cookie.port, None)
+        self.assertEqual(cookie.path, '/')
+        self.assertEqual(cookie.name, 'foo')
+        self.assertEqual(cookie.value, 'test%20%3Bcookie')
+        self.assertFalse(cookie.secure)
+        expected_expiration = datetime(2021, 6, 18, 21, 39, 19, tzinfo=timezone.utc)
+        self.assertEqual(cookie.expires, int(expected_expiration.timestamp()))
+
+    def test_pbkdf2_sha1(self):
+        key = pbkdf2_sha1(b'peanuts', b' ' * 16, 1, 16)
+        self.assertEqual(key, b'g\xe1\x8e\x0fQ\x1c\x9b\xf3\xc9`!\xaa\x90\xd9\xd34')

test/test_download.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -121,6 +121,7 @@ def generator(test_case, tname):
         params['outtmpl'] = tname + '_' + params['outtmpl']
         if is_playlist and 'playlist' not in test_case:
             params.setdefault('extract_flat', 'in_playlist')
+            params.setdefault('playlistend', test_case.get('playlist_mincount'))
             params.setdefault('skip_download', True)
 
         ydl = YoutubeDL(params, auto_init=False)

test/test_downloader_http.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_execution.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -39,6 +39,16 @@ class TestExecution(unittest.TestCase):
         _, stderr = p.communicate()
         self.assertFalse(stderr)
 
+    def test_lazy_extractors(self):
+        try:
+            subprocess.check_call([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'], cwd=rootDir, stdout=_DEV_NULL)
+            subprocess.check_call([sys.executable, 'test/test_all_urls.py'], cwd=rootDir, stdout=_DEV_NULL)
+        finally:
+            try:
+                os.remove('yt_dlp/extractor/lazy_extractors.py')
+            except (IOError, OSError):
+                pass
+
 
 if __name__ == '__main__':
     unittest.main()

test/test_http.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_iqiyi_sdk_interpreter.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_jsinterp.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_overwrites.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import os

test/test_post_hooks.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_postprocessors.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 
@@ -8,16 +8,64 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from yt_dlp.postprocessor import MetadataFromFieldPP, MetadataFromTitlePP
+from yt_dlp import YoutubeDL
+from yt_dlp.compat import compat_shlex_quote
+from yt_dlp.postprocessor import (
+    ExecAfterDownloadPP,
+    FFmpegThumbnailsConvertorPP,
+    MetadataFromFieldPP,
+    MetadataFromTitlePP,
+)
 
 
 class TestMetadataFromField(unittest.TestCase):
     def test_format_to_regex(self):
         pp = MetadataFromFieldPP(None, ['title:%(title)s - %(artist)s'])
-        self.assertEqual(pp._data[0]['regex'], r'(?P<title>[^\r\n]+)\ \-\ (?P<artist>[^\r\n]+)')
+        self.assertEqual(pp._data[0]['regex'], r'(?P<title>.+)\ \-\ (?P<artist>.+)')
+
+    def test_field_to_outtmpl(self):
+        pp = MetadataFromFieldPP(None, ['title:%(title)s : %(artist)s'])
+        self.assertEqual(pp._data[0]['tmpl'], '%(title)s')
+
+    def test_in_out_seperation(self):
+        pp = MetadataFromFieldPP(None, ['%(title)s \\: %(artist)s:%(title)s : %(artist)s'])
+        self.assertEqual(pp._data[0]['in'], '%(title)s : %(artist)s')
+        self.assertEqual(pp._data[0]['out'], '%(title)s : %(artist)s')
 
 
 class TestMetadataFromTitle(unittest.TestCase):
     def test_format_to_regex(self):
         pp = MetadataFromTitlePP(None, '%(title)s - %(artist)s')
-        self.assertEqual(pp._titleregex, r'(?P<title>[^\r\n]+)\ \-\ (?P<artist>[^\r\n]+)')
+        self.assertEqual(pp._titleregex, r'(?P<title>.+)\ \-\ (?P<artist>.+)')
+
+
+class TestConvertThumbnail(unittest.TestCase):
+    def test_escaping(self):
+        pp = FFmpegThumbnailsConvertorPP()
+        if not pp.available:
+            print('Skipping: ffmpeg not found')
+            return
+
+        file = 'test/testdata/thumbnails/foo %d bar/foo_%d.{}'
+        tests = (('webp', 'png'), ('png', 'jpg'))
+
+        for inp, out in tests:
+            out_file = file.format(out)
+            if os.path.exists(out_file):
+                os.remove(out_file)
+            pp.convert_thumbnail(file.format(inp), out)
+            assert os.path.exists(out_file)
+
+        for _, out in tests:
+            os.remove(file.format(out))
+
+
+class TestExecAfterDownload(unittest.TestCase):
+    def test_parse_cmd(self):
+        pp = ExecAfterDownloadPP(YoutubeDL(), '')
+        info = {'filepath': 'file name'}
+        quoted_filepath = compat_shlex_quote(info['filepath'])
+
+        self.assertEqual(pp.parse_cmd('echo', info), 'echo %s' % quoted_filepath)
+        self.assertEqual(pp.parse_cmd('echo.{}', info), 'echo.%s' % quoted_filepath)
+        self.assertEqual(pp.parse_cmd('echo "%(filepath)s"', info), 'echo "%s"' % info['filepath'])

test/test_socks.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_subtitles.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_swfinterp.py.disabled

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_update.py.disabled

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/test_utils.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -12,6 +12,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
 # Various small unit tests
 import io
+import itertools
 import json
 import xml.etree.ElementTree
 
@@ -23,6 +24,7 @@ from yt_dlp.utils import (
     clean_html,
     clean_podcast_url,
     date_from_str,
+    datetime_from_str,
     DateRange,
     detect_exe_version,
     determine_ext,
@@ -65,6 +67,7 @@ from yt_dlp.utils import (
     sanitize_filename,
     sanitize_path,
     sanitize_url,
+    sanitized_Request,
     expand_path,
     prepend_extension,
     replace_extension,
@@ -106,6 +109,7 @@ from yt_dlp.utils import (
     cli_bool_option,
     parse_codecs,
     iri_to_uri,
+    LazyList,
 )
 from yt_dlp.compat import (
     compat_chr,
@@ -124,6 +128,7 @@ class TestUtil(unittest.TestCase):
         self.assertTrue(timeconvert('bougrg') is None)
 
     def test_sanitize_filename(self):
+        self.assertEqual(sanitize_filename(''), '')
         self.assertEqual(sanitize_filename('abc'), 'abc')
         self.assertEqual(sanitize_filename('abc_d-e'), 'abc_d-e')
 
@@ -237,6 +242,16 @@ class TestUtil(unittest.TestCase):
         self.assertEqual(sanitize_url('httpss://foo.bar'), 'https://foo.bar')
         self.assertEqual(sanitize_url('rmtps://foo.bar'), 'rtmps://foo.bar')
         self.assertEqual(sanitize_url('https://foo.bar'), 'https://foo.bar')
+        self.assertEqual(sanitize_url('foo bar'), 'foo bar')
+
+    def test_extract_basic_auth(self):
+        auth_header = lambda url: sanitized_Request(url).get_header('Authorization')
+        self.assertFalse(auth_header('http://foo.bar'))
+        self.assertFalse(auth_header('http://:foo.bar'))
+        self.assertEqual(auth_header('http://@foo.bar'), 'Basic Og==')
+        self.assertEqual(auth_header('http://:pass@foo.bar'), 'Basic OnBhc3M=')
+        self.assertEqual(auth_header('http://user:@foo.bar'), 'Basic dXNlcjo=')
+        self.assertEqual(auth_header('http://user:pass@foo.bar'), 'Basic dXNlcjpwYXNz')
 
     def test_expand_path(self):
         def env(var):
@@ -311,8 +326,18 @@ class TestUtil(unittest.TestCase):
         self.assertEqual(date_from_str('yesterday'), date_from_str('now-1day'))
         self.assertEqual(date_from_str('now+7day'), date_from_str('now+1week'))
         self.assertEqual(date_from_str('now+14day'), date_from_str('now+2week'))
-        self.assertEqual(date_from_str('now+365day'), date_from_str('now+1year'))
-        self.assertEqual(date_from_str('now+30day'), date_from_str('now+1month'))
+        self.assertEqual(date_from_str('20200229+365day'), date_from_str('20200229+1year'))
+        self.assertEqual(date_from_str('20210131+28day'), date_from_str('20210131+1month'))
+
+    def test_datetime_from_str(self):
+        self.assertEqual(datetime_from_str('yesterday', precision='day'), datetime_from_str('now-1day', precision='auto'))
+        self.assertEqual(datetime_from_str('now+7day', precision='day'), datetime_from_str('now+1week', precision='auto'))
+        self.assertEqual(datetime_from_str('now+14day', precision='day'), datetime_from_str('now+2week', precision='auto'))
+        self.assertEqual(datetime_from_str('20200229+365day', precision='day'), datetime_from_str('20200229+1year', precision='auto'))
+        self.assertEqual(datetime_from_str('20210131+28day', precision='day'), datetime_from_str('20210131+1month', precision='auto'))
+        self.assertEqual(datetime_from_str('20210131+59day', precision='day'), datetime_from_str('20210131+2month', precision='auto'))
+        self.assertEqual(datetime_from_str('now+1day', precision='hour'), datetime_from_str('now+24hours', precision='auto'))
+        self.assertEqual(datetime_from_str('now+23hours', precision='hour'), datetime_from_str('now+23hours', precision='auto'))
 
     def test_daterange(self):
         _20century = DateRange("19000101", "20000101")
@@ -1029,6 +1054,9 @@ class TestUtil(unittest.TestCase):
         on = js_to_json('{ "040": "040" }')
         self.assertEqual(json.loads(on), {'040': '040'})
 
+        on = js_to_json('[1,//{},\n2]')
+        self.assertEqual(json.loads(on), [1, 2])
+
     def test_js_to_json_malformed(self):
         self.assertEqual(js_to_json('42a1'), '42"a1"')
         self.assertEqual(js_to_json('42a-1'), '42"a"-1')
@@ -1502,6 +1530,47 @@ Line 1
         self.assertEqual(clean_podcast_url('https://www.podtrac.com/pts/redirect.mp3/chtbl.com/track/5899E/traffic.megaphone.fm/HSW7835899191.mp3'), 'https://traffic.megaphone.fm/HSW7835899191.mp3')
         self.assertEqual(clean_podcast_url('https://play.podtrac.com/npr-344098539/edge1.pod.npr.org/anon.npr-podcasts/podcast/npr/waitwait/2020/10/20201003_waitwait_wwdtmpodcast201003-015621a5-f035-4eca-a9a1-7c118d90bc3c.mp3'), 'https://edge1.pod.npr.org/anon.npr-podcasts/podcast/npr/waitwait/2020/10/20201003_waitwait_wwdtmpodcast201003-015621a5-f035-4eca-a9a1-7c118d90bc3c.mp3')
 
+    def test_LazyList(self):
+        it = list(range(10))
+
+        self.assertEqual(list(LazyList(it)), it)
+        self.assertEqual(LazyList(it).exhaust(), it)
+        self.assertEqual(LazyList(it)[5], it[5])
+
+        self.assertEqual(LazyList(it)[::2], it[::2])
+        self.assertEqual(LazyList(it)[1::2], it[1::2])
+        self.assertEqual(LazyList(it)[6:2:-2], it[6:2:-2])
+        self.assertEqual(LazyList(it)[::-1], it[::-1])
+
+        self.assertTrue(LazyList(it))
+        self.assertFalse(LazyList(range(0)))
+        self.assertEqual(len(LazyList(it)), len(it))
+        self.assertEqual(repr(LazyList(it)), repr(it))
+        self.assertEqual(str(LazyList(it)), str(it))
+
+        self.assertEqual(list(LazyList(it).reverse()), it[::-1])
+        self.assertEqual(list(LazyList(it).reverse()[1:3:7]), it[::-1][1:3:7])
+
+    def test_LazyList_laziness(self):
+
+        def test(ll, idx, val, cache):
+            self.assertEqual(ll[idx], val)
+            self.assertEqual(getattr(ll, '_LazyList__cache'), list(cache))
+
+        ll = LazyList(range(10))
+        test(ll, 0, 0, range(1))
+        test(ll, 5, 5, range(6))
+        test(ll, -3, 7, range(10))
+
+        ll = LazyList(range(10)).reverse()
+        test(ll, -1, 0, range(1))
+        test(ll, 3, 6, range(10))
+
+        ll = LazyList(itertools.count())
+        test(ll, 10, 10, range(11))
+        ll.reverse()
+        test(ll, -15, 14, range(15))
+
 
 if __name__ == '__main__':
     unittest.main()

test/test_verbose_output.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_write_annotations.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_youtube_lists.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_youtube_misc.py

@@ -0,0 +1,26 @@
+#!/usr/bin/env python3
+from __future__ import unicode_literals
+
+# Allow direct execution
+import os
+import sys
+import unittest
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+
+from yt_dlp.extractor import YoutubeIE
+
+
+class TestYoutubeMisc(unittest.TestCase):
+    def test_youtube_extract(self):
+        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
+        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
+        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
+
+
+if __name__ == '__main__':
+    unittest.main()

test/test_youtube_signature.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import unicode_literals
 

test/testdata/ism/sintel.Manifest

@@ -0,0 +1,988 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Created with Unified Streaming Platform (version=1.10.18-20255) -->
+<SmoothStreamingMedia
+  MajorVersion="2"
+  MinorVersion="0"
+  TimeScale="10000000"
+  Duration="8880746666">
+  <StreamIndex
+    Type="audio"
+    QualityLevels="1"
+    TimeScale="10000000"
+    Name="audio"
+    Chunks="445"
+    Url="QualityLevels({bitrate})/Fragments(audio={start time})">
+    <QualityLevel
+      Index="0"
+      Bitrate="128001"
+      CodecPrivateData="1190"
+      SamplingRate="48000"
+      Channels="2"
+      BitsPerSample="16"
+      PacketSize="4"
+      AudioTag="255"
+      FourCC="AACL" />
+    <c t="0" d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="20053333" />
+    <c d="20053333" />
+    <c d="20053334" />
+    <c d="19840000" />
+    <c d="746666" />
+  </StreamIndex>
+  <StreamIndex
+    Type="text"
+    QualityLevels="1"
+    TimeScale="10000000"
+    Language="eng"
+    Subtype="CAPT"
+    Name="textstream_eng"
+    Chunks="11"
+    Url="QualityLevels({bitrate})/Fragments(textstream_eng={start time})">
+    <QualityLevel
+      Index="0"
+      Bitrate="1000"
+      CodecPrivateData=""
+      FourCC="TTML" />
+    <c t="0" d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="600000000" />
+    <c d="240000000" />
+  </StreamIndex>
+  <StreamIndex
+    Type="video"
+    QualityLevels="5"
+    TimeScale="10000000"
+    Name="video"
+    Chunks="444"
+    Url="QualityLevels({bitrate})/Fragments(video={start time})"
+    MaxWidth="1688"
+    MaxHeight="720"
+    DisplayWidth="1689"
+    DisplayHeight="720">
+    <QualityLevel
+      Index="0"
+      Bitrate="100000"
+      CodecPrivateData="00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8"
+      MaxWidth="336"
+      MaxHeight="144"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="1"
+      Bitrate="326000"
+      CodecPrivateData="00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8"
+      MaxWidth="562"
+      MaxHeight="240"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="2"
+      Bitrate="698000"
+      CodecPrivateData="00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8"
+      MaxWidth="844"
+      MaxHeight="360"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="3"
+      Bitrate="1493000"
+      CodecPrivateData="00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8"
+      MaxWidth="1126"
+      MaxHeight="480"
+      FourCC="AVC1" />
+    <QualityLevel
+      Index="4"
+      Bitrate="4482000"
+      CodecPrivateData="00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8"
+      MaxWidth="1688"
+      MaxHeight="720"
+      FourCC="AVC1" />
+    <c t="0" d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+    <c d="20000000" />
+  </StreamIndex>
+</SmoothStreamingMedia>

test/testdata/m3u8/bipbop_16x9.m3u8

@@ -0,0 +1,38 @@
+#EXTM3U
+
+#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
+#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac/prog_index.m3u8"
+
+
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="en",URI="subtitles/eng_forced/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/fra/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="fr",URI="subtitles/fra_forced/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="es",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/spa/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="es",URI="subtitles/spa_forced/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="ja",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/jpn/prog_index.m3u8"
+#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語 (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="ja",URI="subtitles/jpn_forced/prog_index.m3u8"
+
+
+#EXT-X-STREAM-INF:BANDWIDTH=263851,CODECS="mp4a.40.2, avc1.4d400d",RESOLUTION=416x234,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear1/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=28451,CODECS="avc1.4d400d",URI="gear1/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear2/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=181534,CODECS="avc1.4d401e",URI="gear2/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear3/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=297056,CODECS="avc1.4d401f",URI="gear3/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=1030138,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1280x720,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear4/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=339492,CODECS="avc1.4d401f",URI="gear4/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=1924009,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1920x1080,AUDIO="bipbop_audio",SUBTITLES="subs"
+gear5/prog_index.m3u8
+#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=669554,CODECS="avc1.4d401f",URI="gear5/iframe_index.m3u8"
+
+#EXT-X-STREAM-INF:BANDWIDTH=41457,CODECS="mp4a.40.2",AUDIO="bipbop_audio",SUBTITLES="subs"
+gear0/prog_index.m3u8

test/testdata/mpd/subtitles.mpd

@@ -0,0 +1,351 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Created with Unified Streaming Platform (version=1.10.18-20255) -->
+<MPD
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xmlns="urn:mpeg:dash:schema:mpd:2011"
+  xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
+  type="static"
+  mediaPresentationDuration="PT14M48S"
+  maxSegmentDuration="PT1M"
+  minBufferTime="PT10S"
+  profiles="urn:mpeg:dash:profile:isoff-live:2011">
+  <Period
+    id="1"
+    duration="PT14M48S">
+    <BaseURL>dash/</BaseURL>
+    <AdaptationSet
+      id="1"
+      group="1"
+      contentType="audio"
+      segmentAlignment="true"
+      audioSamplingRate="48000"
+      mimeType="audio/mp4"
+      codecs="mp4a.40.2"
+      startWithSAP="1">
+      <AudioChannelConfiguration
+        schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011"
+        value="2" />
+      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
+      <SegmentTemplate
+        timescale="48000"
+        initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
+        media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
+        <SegmentTimeline>
+          <S t="0" d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="96256" r="2" />
+          <S d="95232" />
+          <S d="3584" />
+        </SegmentTimeline>
+      </SegmentTemplate>
+      <Representation
+        id="audio=128001"
+        bandwidth="128001">
+      </Representation>
+    </AdaptationSet>
+    <AdaptationSet
+      id="2"
+      group="3"
+      contentType="text"
+      lang="en"
+      mimeType="application/mp4"
+      codecs="stpp"
+      startWithSAP="1">
+      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="subtitle" />
+      <SegmentTemplate
+        timescale="1000"
+        initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
+        media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
+        <SegmentTimeline>
+          <S t="0" d="60000" r="9" />
+          <S d="24000" />
+        </SegmentTimeline>
+      </SegmentTemplate>
+      <Representation
+        id="textstream_eng=1000"
+        bandwidth="1000">
+      </Representation>
+    </AdaptationSet>
+    <AdaptationSet
+      id="3"
+      group="2"
+      contentType="video"
+      par="960:409"
+      minBandwidth="100000"
+      maxBandwidth="4482000"
+      maxWidth="1689"
+      maxHeight="720"
+      segmentAlignment="true"
+      mimeType="video/mp4"
+      codecs="avc1.4D401F"
+      startWithSAP="1">
+      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
+      <SegmentTemplate
+        timescale="12288"
+        initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
+        media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
+        <SegmentTimeline>
+          <S t="0" d="24576" r="443" />
+        </SegmentTimeline>
+      </SegmentTemplate>
+      <Representation
+        id="video=100000"
+        bandwidth="100000"
+        width="336"
+        height="144"
+        sar="2880:2863"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=326000"
+        bandwidth="326000"
+        width="562"
+        height="240"
+        sar="115200:114929"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=698000"
+        bandwidth="698000"
+        width="844"
+        height="360"
+        sar="86400:86299"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=1493000"
+        bandwidth="1493000"
+        width="1126"
+        height="480"
+        sar="230400:230267"
+        scanType="progressive">
+      </Representation>
+      <Representation
+        id="video=4482000"
+        bandwidth="4482000"
+        width="1688"
+        height="720"
+        sar="86400:86299"
+        scanType="progressive">
+      </Representation>
+    </AdaptationSet>
+  </Period>
+</MPD>

yt-dlp.sh

@@ -1,2 +1,2 @@
-#!/bin/bash
-python3 "$(dirname $(realpath $0))/yt_dlp/__main__.py" "$@"
+#!/bin/sh
+exec python3 "$(dirname "$(realpath "$0")")/yt_dlp/__main__.py" "$@"