Release 2021.03.21

7e79ba7dd6
[vimeo:album] Fix extraction for albums with number of videos multiple to page size

.gitattributes

@@ -1,4 +1 @@
-* text=auto
-
 Makefile* text whitespace=-tab-in-indent
-*.sh text eol=lf

.github/FUNDING.yml

@@ -1,13 +0,0 @@
-# These are supported funding model platforms
-
-github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
-patreon: # Replace with a single Patreon username
-open_collective: # Replace with a single Open Collective username
-ko_fi: # Replace with a single Ko-fi username
-tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
-community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
-liberapay: # Replace with a single Liberapay username
-issuehunt: # Replace with a single IssueHunt username
-otechie: # Replace with a single Otechie username
-
-custom: ['https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md']

.github/ISSUE_TEMPLATE/1_broken_site.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.15. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support
-- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.03.15**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -42,9 +42,9 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.07.24
+ [debug] yt-dlp version 2021.03.15
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/2_site_support_request.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.15. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a new site support request
-- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.03.15**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones

.github/ISSUE_TEMPLATE/3_site_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.15. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a site feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.03.15**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE/4_bug_report.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.15. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support issue
-- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.03.15**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -44,9 +44,9 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.07.24
+ [debug] yt-dlp version 2021.03.15
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/5_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.03.15. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
+- [ ] I've verified that I'm running yt-dlp version **2021.03.15**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE_tmpl/1_broken_site.md

@@ -42,7 +42,7 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
  [debug] yt-dlp version %(version)s
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2

.github/ISSUE_TEMPLATE_tmpl/4_bug_report.md

@@ -44,7 +44,7 @@ Provide the complete verbose output of yt-dlp that clearly demonstrates the prob
 Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
  [debug] System config: []
  [debug] User config: []
- [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
+ [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
  [debug] yt-dlp version %(version)s
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2

.github/workflows/build.yml

@@ -7,13 +7,13 @@ on:
 
 jobs:
   build_unix:
+
     runs-on: ubuntu-latest
 
     outputs:
       ytdlp_version: ${{ steps.bump_version.outputs.ytdlp_version }}
       upload_url: ${{ steps.create_release.outputs.upload_url }}
-      sha256_unix: ${{ steps.sha256_file.outputs.sha256_unix }}
-      sha512_unix: ${{ steps.sha512_file.outputs.sha512_unix }}
+      sha2_unix: ${{ steps.sha2_file.outputs.sha2_unix }}
 
     steps:
     - uses: actions/checkout@v2
@@ -29,7 +29,7 @@ jobs:
     - name: Print version
       run: echo "${{ steps.bump_version.outputs.ytdlp_version }}"
     - name: Run Make
-      run: make all tar
+      run: make
     - name: Create Release
       id: create_release
       uses: actions/create-release@v1
@@ -44,7 +44,7 @@ jobs:
         draft: false
         prerelease: false
     - name: Upload yt-dlp Unix binary
-      id: upload-release-asset
+      id: upload-release-asset 
       uses: actions/upload-release-asset@v1
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -53,21 +53,9 @@ jobs:
         asset_path: ./yt-dlp
         asset_name: yt-dlp
         asset_content_type: application/octet-stream
-    - name: Upload Source tar
-      uses: actions/upload-release-asset@v1
-      env:
-        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      with:
-        upload_url: ${{ steps.create_release.outputs.upload_url }}
-        asset_path: ./yt-dlp.tar.gz
-        asset_name: yt-dlp.tar.gz
-        asset_content_type: application/gzip
     - name: Get SHA2-256SUMS for yt-dlp
-      id: sha256_file
-      run: echo "::set-output name=sha256_unix::$(sha256sum yt-dlp | awk '{print $1}')"
-    - name: Get SHA2-512SUMS for yt-dlp
-      id: sha512_file
-      run: echo "::set-output name=sha512_unix::$(sha512sum yt-dlp | awk '{print $1}')"
+      id: sha2_file
+      run: echo "::set-output name=sha2_unix::$(sha256sum yt-dlp | awk '{print $1}')"
     - name: Install dependencies for pypi
       env:
         PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
@@ -86,24 +74,24 @@ jobs:
         twine upload dist/*
 
   build_windows:
+
     runs-on: windows-latest
-    needs: build_unix
 
     outputs:
-      sha256_windows: ${{ steps.sha256_file_win.outputs.sha256_windows }}
-      sha512_windows: ${{ steps.sha512_file_win.outputs.sha512_windows }}
+      sha2_windows: ${{ steps.sha2_file_win.outputs.sha2_windows }}
+
+    needs: build_unix
 
     steps:
     - uses: actions/checkout@v2
-    # 3.8 is used for Win7 support
-    - name: Set up Python 3.8
+    - name: Set up Python
       uses: actions/setup-python@v2
       with:
           python-version: '3.8'
     - name: Upgrade pip and enable wheel support
       run: python -m pip install --upgrade pip setuptools wheel
     - name: Install Requirements
-      run: pip install pyinstaller mutagen pycryptodome websockets
+      run: pip install pyinstaller mutagen pycryptodome
     - name: Bump version
       id: bump_version
       run: python devscripts/update-version.py
@@ -122,32 +110,29 @@ jobs:
         asset_name: yt-dlp.exe
         asset_content_type: application/vnd.microsoft.portable-executable
     - name: Get SHA2-256SUMS for yt-dlp.exe
-      id: sha256_file_win
-      run: echo "::set-output name=sha256_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())"
-    - name: Get SHA2-512SUMS for yt-dlp.exe
-      id: sha512_file_win
-      run: echo "::set-output name=sha512_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA512).Hash.ToLower())"
+      id: sha2_file_win
+      run: echo "::set-output name=sha2_windows::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())"
 
   build_windows32:
+
     runs-on: windows-latest
-    needs: [build_unix, build_windows]
 
     outputs:
-      sha256_windows32: ${{ steps.sha256_file_win32.outputs.sha256_windows32 }}
-      sha512_windows32: ${{ steps.sha512_file_win32.outputs.sha512_windows32 }}
+      sha2_windows32: ${{ steps.sha2_file_win32.outputs.sha2_windows32 }}
+
+    needs: [build_unix, build_windows]
 
     steps:
     - uses: actions/checkout@v2
-    # 3.7 is used for Vista support. See https://github.com/yt-dlp/yt-dlp/issues/390
-    - name: Set up Python 3.7 32-Bit
+    - name: Set up Python 3.4.4 32-Bit
       uses: actions/setup-python@v2
       with:
-          python-version: '3.7'
+          python-version: '3.4.4'
           architecture: 'x86'
     - name: Upgrade pip and enable wheel support
-      run: python -m pip install --upgrade pip setuptools wheel
-    - name: Install Requirements
-      run: pip install pyinstaller mutagen pycryptodome websockets
+      run: python -m pip install pip==19.1.1 setuptools==43.0.0 wheel==0.33.6
+    - name: Install Requirements for 32 Bit
+      run: pip install pyinstaller==3.5 mutagen==1.42.0 pycryptodome==3.9.4
     - name: Bump version
       id: bump_version
       run: python devscripts/update-version.py
@@ -166,28 +151,20 @@ jobs:
         asset_name: yt-dlp_x86.exe
         asset_content_type: application/vnd.microsoft.portable-executable
     - name: Get SHA2-256SUMS for yt-dlp_x86.exe
-      id: sha256_file_win32
-      run: echo "::set-output name=sha256_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())"
-    - name: Get SHA2-512SUMS for yt-dlp_x86.exe
-      id: sha512_file_win32
-      run: echo "::set-output name=sha512_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA512).Hash.ToLower())"
-
-  finish:
-    runs-on: ubuntu-latest
-    needs: [build_unix, build_windows, build_windows32]
-
-    steps:
+      id: sha2_file_win32
+      run: echo "::set-output name=sha2_windows32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())"
     - name: Make SHA2-256SUMS file
       env:
-        SHA256_WINDOWS: ${{ needs.build_windows.outputs.sha256_windows }}
-        SHA256_WINDOWS32: ${{ needs.build_windows32.outputs.sha256_windows32 }}
-        SHA256_UNIX: ${{ needs.build_unix.outputs.sha256_unix }}
+        SHA2_WINDOWS: ${{ needs.build_windows.outputs.sha2_windows }}
+        SHA2_WINDOWS32: ${{ steps.sha2_file_win32.outputs.sha2_windows32 }}
+        SHA2_UNIX: ${{ needs.build_unix.outputs.sha2_unix }}
         YTDLP_VERSION: ${{ needs.build_unix.outputs.ytdlp_version }}
       run: |
-        echo "version:${{ env.YTDLP_VERSION }}" >> SHA2-256SUMS
-        echo "yt-dlp.exe:${{ env.SHA256_WINDOWS }}" >> SHA2-256SUMS
-        echo "yt-dlp_x86.exe:${{ env.SHA256_WINDOWS32 }}" >> SHA2-256SUMS
-        echo "yt-dlp:${{ env.SHA256_UNIX }}" >> SHA2-256SUMS
+        echo "version:${env:YTDLP_VERSION}" >> SHA2-256SUMS
+        echo "yt-dlp.exe:${env:SHA2_WINDOWS}" >> SHA2-256SUMS
+        echo "yt-dlp_x86.exe:${env:SHA2_WINDOWS32}" >> SHA2-256SUMS
+        echo "yt-dlp:${env:SHA2_UNIX}" >> SHA2-256SUMS
+
     - name: Upload 256SUMS file
       id: upload-sums
       uses: actions/upload-release-asset@v1
@@ -198,22 +175,3 @@ jobs:
         asset_path: ./SHA2-256SUMS
         asset_name: SHA2-256SUMS
         asset_content_type: text/plain
-    - name: Make SHA2-512SUMS file
-      env:
-        SHA512_WINDOWS: ${{ needs.build_windows.outputs.sha512_windows }}
-        SHA512_WINDOWS32: ${{ needs.build_windows32.outputs.sha512_windows32 }}
-        SHA512_UNIX: ${{ needs.build_unix.outputs.sha512_unix }}
-      run: |
-        echo "${{ env.SHA512_WINDOWS }}  yt-dlp.exe" >> SHA2-512SUMS
-        echo "${{ env.SHA512_WINDOWS32 }}  yt-dlp_x86.exe" >> SHA2-512SUMS
-        echo "${{ env.SHA512_UNIX }}  yt-dlp" >> SHA2-512SUMS
-    - name: Upload 512SUMS file
-      id: upload-512sums
-      uses: actions/upload-release-asset@v1
-      env:
-        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      with:
-        upload_url: ${{ needs.build_unix.outputs.upload_url }}
-        asset_path: ./SHA2-512SUMS
-        asset_name: SHA2-512SUMS
-        asset_content_type: text/plain

.github/workflows/core.yml

@@ -6,26 +6,49 @@ jobs:
     if: "!contains(github.event.head_commit.message, 'ci skip')"
     runs-on: ${{ matrix.os }}
     strategy:
-      fail-fast: false
+      fail-fast: true
       matrix:
         os: [ubuntu-18.04]
-        # py3.9 is in quick-test
-        python-version: [3.7, 3.8, 3.10-dev, pypy-3.6, pypy-3.7]
+        # TODO: python 2.6
+        python-version: [2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, pypy-2.7, pypy-3.6, pypy-3.7]
+        python-impl: [cpython]
+        ytdl-test-set: [core]
         run-tests-ext: [sh]
         include:
-        # atleast one of the tests must be in windows
+        # python 3.2 is only available on windows via setup-python
         - os: windows-latest
-          python-version: 3.6
+          python-version: 3.2
+          python-impl: cpython
+          ytdl-test-set: core
           run-tests-ext: bat
+        # jython
+        - os: ubuntu-latest
+          python-impl: jython
+          ytdl-test-set: core
+          run-tests-ext: sh
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
+      if: ${{ matrix.python-impl == 'cpython' }}
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Install pytest
-      run: pip install pytest
+    - name: Set up Java 8
+      if: ${{ matrix.python-impl == 'jython' }}
+      uses: actions/setup-java@v1
+      with:
+        java-version: 8
+    - name: Install Jython
+      if: ${{ matrix.python-impl == 'jython' }}
+      run: |
+        wget http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar -O jython-installer.jar
+        java -jar jython-installer.jar -s -d "$HOME/jython"
+        echo "$HOME/jython/bin" >> $GITHUB_PATH
+    - name: Install nose
+      run: pip install nose
     - name: Run tests
-      continue-on-error: False
-      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} core
+      continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
+      env:
+        YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
+      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}
   # Linter is in quick-test

.github/workflows/download.yml

@@ -9,20 +9,45 @@ jobs:
       fail-fast: true
       matrix:
         os: [ubuntu-18.04]
-        python-version: [3.7, 3.8, 3.9, 3.10-dev, pypy-3.6, pypy-3.7]
+        # TODO: python 2.6
+        python-version: [2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, pypy-2.7, pypy-3.6, pypy-3.7]
+        python-impl: [cpython]
+        ytdl-test-set: [download]
         run-tests-ext: [sh]
         include:
+        # python 3.2 is only available on windows via setup-python
         - os: windows-latest
-          python-version: 3.6
+          python-version: 3.2
+          python-impl: cpython
+          ytdl-test-set: download
           run-tests-ext: bat
+        # jython - disable for now since it takes too long to complete
+        # - os: ubuntu-latest
+        #   python-impl: jython
+        #   ytdl-test-set: download
+        #   run-tests-ext: sh
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
+      if: ${{ matrix.python-impl == 'cpython' }}
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Install pytest
-      run: pip install pytest
+    - name: Set up Java 8
+      if: ${{ matrix.python-impl == 'jython' }}
+      uses: actions/setup-java@v1
+      with:
+        java-version: 8
+    - name: Install Jython
+      if: ${{ matrix.python-impl == 'jython' }}
+      run: |
+        wget http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar -O jython-installer.jar
+        java -jar jython-installer.jar -s -d "$HOME/jython"
+        echo "$HOME/jython/bin" >> $GITHUB_PATH
+    - name: Install nose
+      run: pip install nose
     - name: Run tests
-      continue-on-error: true
-      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} download
+      continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
+      env:
+        YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
+      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}

.github/workflows/quick-test.yml

@@ -11,10 +11,12 @@ jobs:
       uses: actions/setup-python@v2
       with:
         python-version: 3.9
-    - name: Install test requirements
-      run: pip install pytest pycryptodome
+    - name: Install nose
+      run: pip install nose
     - name: Run tests
-      run: ./devscripts/run_tests.sh core
+      env:
+        YTDL_TEST_SET: core
+      run: ./devscripts/run_tests.sh
   flake8:
     name: Linter
     if: "!contains(github.event.head_commit.message, 'ci skip all')"
@@ -28,4 +30,4 @@ jobs:
     - name: Install flake8
       run: pip install flake8
     - name: Run flake8
-      run: flake8 .
+      run: flake8 .

.gitignore

@@ -1,51 +1,6 @@
-# Config
-*.conf
-*.spec
-cookies
-cookies.txt
-
-# Downloaded
-*.srt
-*.ttml
-*.sbv
-*.vtt
-*.flv
-*.mp4
-*.m4a
-*.m4v
-*.mp3
-*.3gp
-*.webm
-*.wav
-*.ape
-*.mkv
-*.swf
-*.part
-*.part-*
-*.ytdl
-*.dump
-*.frag
-*.frag.urls
-*.aria2
-*.swp
-*.ogg
-*.opus
-*.info.json
-*.live_chat.json
-*.jpg
-*.jpeg
-*.png
-*.webp
-*.annotations.xml
-*.description
-
-# Allow config/media files in testdata
-!test/testdata/**
-
 # Python
 *.pyc
 *.pyo
-.pytest_cache
 wine-py2exe/
 py2exe.log
 build/
@@ -80,7 +35,6 @@ README.txt
 *.tar.gz
 *.zsh
 *.spec
-test/testdata/player-*.js
 
 # Binary
 /youtube-dl
@@ -89,6 +43,48 @@ test/testdata/player-*.js
 yt-dlp.zip
 *.exe
 
+# Downloaded
+*.srt
+*.ttml
+*.sbv
+*.vtt
+*.flv
+*.mp4
+*.m4a
+*.m4v
+*.mp3
+*.3gp
+*.webm
+*.wav
+*.ape
+*.mkv
+*.swf
+*.part
+*.part-*
+*.ytdl
+*.dump
+*.frag
+*.frag.urls
+*.aria2
+*.swp
+*.ogg
+*.opus
+*.info.json
+*.live_chat.json
+*.jpg
+*.png
+*.webp
+*.annotations.xml
+*.description
+
+# Config
+*.conf
+*.spec
+cookies
+cookies.txt
+
+
+
 # Text Editor / IDE
 .idea
 *.iml

CONTRIBUTING.md

@@ -3,7 +3,7 @@
 $ youtube-dl -v <your command line>
 [debug] System config: []
 [debug] User config: []
-[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKc']
+[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] youtube-dl version 2015.12.06
 [debug] Git HEAD: 135392e
@@ -81,17 +81,16 @@ To run the test, simply invoke your favorite test runner, or execute a test file
     python -m unittest discover
     python test/test_download.py
     nosetests
-    pytest
 
 See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases.
 
 If you want to create a build of youtube-dl yourself, you'll need
 
-* python3
+* python
 * make (only GNU make is supported)
 * pandoc
 * zip
-* pytest
+* nosetests
 
 ### Adding support for a new site
 

CONTRIBUTORS

@@ -1,7 +1,6 @@
 pukkandan (owner)
 shirt-dev (collaborator)
-coletdjnz/colethedj (collaborator)
-Ashish0804 (collaborator)
+colethedj (collaborator)
 h-h-h-h
 pauldubois98
 nixxo
@@ -21,49 +20,14 @@ FelixFrog
 Zocker1999NET
 nao20010128nao
 kurumigi
+tsukumi
 bbepis
 animelover1984
 Pccode66
+Ashish0804
 RobinD42
 hseg
 DennyDai
 codeasashu
 teesid
 kevinoconnor7
-damianoamatruda
-2ShedsJackson
-CXwudi
-xtkoba
-llacb47
-hheimbuerger
-B0pol
-lkho
-fstirlitz
-Lamieur
-tsukumijima
-Hadi0609
-b5eff52
-craftingmod
-tpikonen
-tripulse
-king-millez
-alex-gedeon
-hhirtz
-louie-github
-MinePlayersPE
-olifre
-rhsmachine/zenerdi0de
-nihil-admirari
-krichbanana
-ohmybahgosh
-nyuszika7h
-blackjack4494
-pyx
-TpmKranz
-mzbaulhaque
-zackmark29
-mbway
-zerodytrash
-wesnm
-pento
-rigstot

Changelog.md

@@ -6,11 +6,9 @@
 * Run `make doc`
 * Update Changelog.md and CONTRIBUTORS
 * Change "Merged with ytdl" version in Readme.md if needed
-* Add new/fixed extractors in "new features" section of Readme.md
 * Commit to master as `Release <version>`
-* Push to origin/release using `git push origin master:release`
-    build task will now run
-* Update version.py using `devscripts\update-version.py`
+* Push to origin/release - build task will now run
+* Update version.py using devscripts\update-version.py
 * Run `make issuetemplates`
 * Commit to master as `[version] update :ci skip all`
 * Push to origin/master
@@ -19,475 +17,9 @@
 -->
 
 
-### 2021.08.02
-
-* Add logo, banner and donate links
-* Expand and escape environment variables correctly in output template
-* Add format types `j` (json), `l` (comma delimited list), `q` (quoted for terminal) in output template
-* [downloader] Allow streaming some unmerged formats to stdout using ffmpeg
-* [youtube] **Age-gate bypass**
-    * Add `agegate` clients by [pukkandan](https://github.com/pukkandan), [MinePlayersPE](https://github.com/MinePlayersPE)
-    * Add `thirdParty` to agegate clients to bypass more videos
-    * Simplify client definitions, expose `embedded` clients
-    * Improve age-gate detection by [coletdjnz](https://github.com/coletdjnz)
-    * Fix default global API key by [coletdjnz](https://github.com/coletdjnz)
-    * Add `creator` clients for age-gate bypass using unverified accounts by [zerodytrash](https://github.com/zerodytrash), [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
-* [adobepass] Add MSO Sling TV by [wesnm](https://github.com/wesnm)
-* [CBS] Add ParamountPlusSeriesIE by [Ashish0804](https://github.com/Ashish0804)
-* [dplay] Add `ScienceChannelIE` by [Sipherdrakon](https://github.com/Sipherdrakon)
-* [UtreonIE] Add extractor by [Ashish0804](https://github.com/Ashish0804)
-* [youtube] Add `mweb` client by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Add `player_client=all`
-* [youtube] Force `hl=en` for comments by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Fix format sorting when using alternate clients
-* [youtube] Misc cleanup by [pukkandan](https://github.com/pukkandan), [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Extract SAPISID only once
-* [CBS] Add fallback by [llacb47](https://github.com/llacb47), [pukkandan](https://github.com/pukkandan)
-* [Hotstar] Support cookies by [Ashish0804](https://github.com/Ashish0804)
-* [HotStarSeriesIE] Fix regex by [Ashish0804](https://github.com/Ashish0804)
-* [bilibili] Improve `_VALID_URL`
-* [mediaset] Fix extraction by [nixxo](https://github.com/nixxo)
-* [Mxplayer] Add h265 formats by [Ashish0804](https://github.com/Ashish0804)
-* [RCTIPlus] Remove PhantomJS dependency by [MinePlayersPE](https://github.com/MinePlayersPE)
-* [tenplay] Add MA15+ age limit by [pento](https://github.com/pento)
-* [vidio] Fix login error detection by [MinePlayersPE](https://github.com/MinePlayersPE)
-* [vimeo] Better extraction of original file by [Ashish0804](https://github.com/Ashish0804)
-* [generic] Support KVS player (replaces ThisVidIE) by [rigstot](https://github.com/rigstot)
-* Add compat-option `no-clean-infojson`
-* Remove `asr` appearing twice in `-F`
-* Set `home:` as the default key for `-P`
-* [utils] Fix slicing of reversed `LazyList`
-* [FormatSort] Fix bug for audio with unknown codec
-* [test:download] Support testing with `ignore_no_formats_error`
-* [cleanup] Refactor some code
-
-
-### 2021.07.24
-
-* [youtube:tab] Extract video duration early
-* [downloader] Pass `info_dict` to `progress_hook`s
-* [youtube] Fix age-gated videos for API clients when cookies are supplied by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Disable `get_video_info` age-gate workaround - This endpoint seems to be completely dead
-* [youtube] Try all clients even if age-gated
-* [youtube] Fix subtitles only being extracted from the first client
-* [youtube] Simplify `_get_text`
-* [cookies] bugfix for microsoft edge on macOS
-* [cookies] Handle `sqlite` `ImportError` gracefully by [mbway](https://github.com/mbway)
-* [cookies] Handle errors when importing `keyring`
-
-### 2021.07.21
-
-* **Add option `--cookies-from-browser`** to load cookies from a browser by [mbway](https://github.com/mbway)
-    * Usage: `--cookies-from-browser BROWSER[:PROFILE_NAME_OR_PATH]`
-    * Also added `--no-cookies-from-browser`
-    * To decrypt chromium cookies, `keyring` is needed for UNIX and `pycryptodome` for Windows
-* Add option `--exec-before-download`
-* Add field `live_status`
-* [FFmpegMetadata] Add language of each stream and some refactoring
-* [douyin] Add extractor by [pukkandan](https://github.com/pukkandan), [pyx](https://github.com/pyx)
-* [pornflip] Add extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
-* **[youtube] Extract data from multiple clients** by [pukkandan](https://github.com/pukkandan), [coletdjnz](https://github.com/coletdjnz)
-    * `player_client` now accepts multiple clients
-    * Default `player_client` = `android,web`
-        * This uses twice as many requests, but avoids throttling for most videos while also not losing any formats
-    * Music clients can be specifically requested and is enabled by default if `music.youtube.com`
-    * Added `player_client=ios` (Known issue: formats from ios are not sorted correctly)
-    * Add age-gate bypass for android and ios clients
-* [youtube] Extract more thumbnails
-    * The thumbnail URLs are hard-coded and their actual existence is tested lazily
-    * Added option `--no-check-formats` to not test them
-* [youtube] Misc fixes
-    * Improve extraction of livestream metadata by [pukkandan](https://github.com/pukkandan), [krichbanana](https://github.com/krichbanana)
-    * Hide live dash formats since they can't be downloaded anyway
-    * Fix authentication when using multiple accounts by [coletdjnz](https://github.com/coletdjnz)
-    * Fix controversial videos when requested via API by [coletdjnz](https://github.com/coletdjnz)
-    * Fix session index extraction and headers for non-web player clients by [coletdjnz](https://github.com/coletdjnz)
-    * Make `--extractor-retries` work for more errors
-    * Fix sorting of 3gp format
-    * Sanity check `chapters` (and refactor related code)
-    * Make `parse_time_text` and `_extract_chapters` non-fatal
-    * Misc cleanup and bug fixes by [coletdjnz](https://github.com/coletdjnz)
-* [youtube:tab] Fix channels tab
-* [youtube:tab] Extract playlist availability by [coletdjnz](https://github.com/coletdjnz)
-* **[youtube:comments] Move comment extraction to new API** by [coletdjnz](https://github.com/coletdjnz)
-    * Adds extractor-args `comment_sort` (`top`/`new`), `max_comments`, `max_comment_depth`
-* [youtube:comments] Fix `is_favorited`, improve `like_count` parsing by [coletdjnz](https://github.com/coletdjnz)
-* [BravoTV] Improve metadata extraction by [kevinoconnor7](https://github.com/kevinoconnor7)
-* [crunchyroll:playlist] Force http
-* [yahoo:gyao:player] Relax `_VALID_URL` by [nao20010128nao](https://github.com/nao20010128nao)
-* [nebula] Authentication via tokens from cookie jar by [hheimbuerger](https://github.com/hheimbuerger), [TpmKranz](https://github.com/TpmKranz)
-* [RTP] Fix extraction and add subtitles by [fstirlitz](https://github.com/fstirlitz)
-* [viki] Rewrite extractors and add extractor-arg `video_types` to `vikichannel` by [zackmark29](https://github.com/zackmark29), [pukkandan](https://github.com/pukkandan)
-* [vlive] Extract thumbnail directly in addition to the one from Naver
-* [generic] Extract previously missed subtitles by [fstirlitz](https://github.com/fstirlitz)
-* [generic] Extract everything in the SMIL manifest and detect discarded subtitles by [fstirlitz](https://github.com/fstirlitz)
-* [embedthumbnail] Fix `_get_thumbnail_resolution`
-* [metadatafromfield] Do not detect numbers as field names
-* Fix selectors `all`, `mergeall` and add tests
-* Errors in playlist extraction should obey `--ignore-errors`
-* Fix bug where `original_url` was not propagated when `_type`=`url`
-* Revert "Merge webm formats into mkv if thumbnails are to be embedded (#173)"
-    * This was wrongly checking for `write_thumbnail`
-* Improve `extractor_args` parsing
-* Rename `NOTE` in `-F` to `MORE INFO` since it's often confused to be the same as `format_note`
-* Add `only_once` param for `write_debug` and `report_warning`
-* [extractor] Allow extracting multiple groups in `_search_regex` by [fstirlitz](https://github.com/fstirlitz)
-* [utils] Improve `traverse_obj`
-* [utils] Add `variadic`
-* [utils] Improve `js_to_json` comment regex by [fstirlitz](https://github.com/fstirlitz)
-* [webtt] Fix timestamps
-* [compat] Remove unnecessary code
-* [docs] fix default of multistreams
-
-
-### 2021.07.07
-
-* Merge youtube-dl: Upto [commit/a803582](https://github.com/ytdl-org/youtube-dl/commit/a8035827177d6b59aca03bd717acb6a9bdd75ada)
-* Add `--extractor-args` to pass some extractor-specific arguments. See [readme](https://github.com/yt-dlp/yt-dlp#extractor-arguments)
-    * Add extractor option `skip` for `youtube`. Eg: `--extractor-args youtube:skip=hls,dash`
-    * Deprecates `--youtube-skip-dash-manifest`, `--youtube-skip-hls-manifest`, `--youtube-include-dash-manifest`, `--youtube-include-hls-manifest`
-* Allow `--list...` options to work with `--print`, `--quiet` and other `--list...` options
-* [youtube] Use `player` API for additional video extraction requests by [coletdjnz](https://github.com/coletdjnz)
-    * **Fixes youtube premium music** (format 141) extraction
-    * Adds extractor option `player_client` = `web`/`android`
-        * **`--extractor-args youtube:player_client=android` works around the throttling** for the time-being
-    * Adds extractor option `player_skip=config`
-    * Adds age-gate fallback using embedded client
-* [youtube] Choose correct Live chat API for upcoming streams by [krichbanana](https://github.com/krichbanana)
-* [youtube] Fix subtitle names for age-gated videos
-* [youtube:comments] Fix error handling and add `itct` to params by [coletdjnz](https://github.com/coletdjnz)
-* [youtube_live_chat] Fix download with cookies by [siikamiika](https://github.com/siikamiika)
-* [youtube_live_chat] use `clickTrackingParams` by [siikamiika](https://github.com/siikamiika)
-* [Funimation] Rewrite extractor
-    * Add `FunimationShowIE` by [Mevious](https://github.com/Mevious)
-    * **Treat the different versions of an episode as different formats of a single video**
-        * This changes the video `id` and will break break existing archives
-        * Compat option `seperate-video-versions` to fall back to old behavior including using the old video ids
-    * Support direct `/player/` URL
-    * Extractor options `language` and `version` to pre-select them during extraction
-        * These options may be removed in the future if we can extract all formats without additional network requests
-        * Do not rely on these for format selection and use `-f` filters instead
-* [AdobePass] Add Spectrum MSO by [kevinoconnor7](https://github.com/kevinoconnor7), [ohmybahgosh](https://github.com/ohmybahgosh)
-* [facebook] Extract description and fix title
-* [fancode] Fix extraction, support live and allow login with refresh token by [zenerdi0de](https://github.com/zenerdi0de)
-* [plutotv] Improve `_VALID_URL`
-* [RCTIPlus] Add extractor by [MinePlayersPE](https://github.com/MinePlayersPE)
-* [Soundcloud] Allow login using oauth token by [blackjack4494](https://github.com/blackjack4494)
-* [TBS] Support livestreams by [llacb47](https://github.com/llacb47)
-* [videa] Fix extraction by [nyuszika7h](https://github.com/nyuszika7h)
-* [yahoo] Fix extraction by [llacb47](https://github.com/llacb47), [pukkandan](https://github.com/pukkandan)
-* Process videos when using `--ignore-no-formats-error` by [krichbanana](https://github.com/krichbanana)
-* Fix `--throttled-rate` when using `--load-info-json`
-* Fix `--flat-playlist` when entry has no `ie_key`
-* Fix `check_formats` catching `ExtractorError` instead of `DownloadError`
-* Fix deprecated option `--list-formats-old`
-* [downloader/ffmpeg] Fix `--ppa` when using simultaneous download
-* [extractor] Prevent unnecessary download of hls manifests and refactor `hls_split_discontinuity`
-* [fragment] Handle status of download and errors in threads correctly; and minor refactoring
-* [thumbnailsconvertor] Treat `jpeg` as `jpg`
-* [utils] Fix issues with `LazyList` reversal
-* [extractor] Allow extractors to set their own login hint
-* [cleanup] Simplify format selector code with `LazyList` and `yield from`
-* [cleanup] Clean `extractor.common._merge_subtitles` signature
-* [cleanup] Fix some typos
-
-
-### 2021.06.23
-
-* Merge youtube-dl: Upto [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)
-* **Add option `--throttled-rate`** below which video data is re-extracted
-* [fragment] **Merge during download for `-N`**, and refactor `hls`/`dash`
-* [websockets] Add `WebSocketFragmentFD` by [nao20010128nao](https://github.com/nao20010128nao), [pukkandan](https://github.com/pukkandan)
-* Allow `images` formats in addition to video/audio
-* [downloader/mhtml] Add new downloader for slideshows/storyboards by [fstirlitz](https://github.com/fstirlitz)
-* [youtube] Temporary **fix for age-gate**
-* [youtube] Support ongoing live chat by [siikamiika](https://github.com/siikamiika)
-* [youtube] Improve SAPISID cookie handling by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Login is not needed for `:ytrec`
-* [youtube] Non-fatal alert reporting for unavailable videos page by [coletdjnz](https://github.com/coletdjnz)
-* [twitcasting] Websocket support by [nao20010128nao](https://github.com/nao20010128nao)
-* [mediasite] Extract slides by [fstirlitz](https://github.com/fstirlitz)
-* [funimation] Extract subtitles 
-* [pornhub] Extract `cast`
-* [hotstar] Use server time for authentication instead of local time
-* [EmbedThumbnail] Fix for already downloaded thumbnail
-* [EmbedThumbnail] Add compat-option `embed-thumbnail-atomicparsley`
-* Expand `--check-formats` to thumbnails
-* Fix id sanitization in filenames
-* Skip fixup of existing files and add `--fixup force` to force it
-* Better error handling of syntax errors in `-f`
-* Use `NamedTemporaryFile` for `--check-formats`
-* [aria2c] Lower `--min-split-size` for HTTP downloads
-* [options] Rename `--add-metadata` to `--embed-metadata`
-* [utils] Improve `LazyList` and add tests
-* [build] Build Windows x86 version with py3.7 and remove redundant tests by [pukkandan](https://github.com/pukkandan), [shirt](https://github.com/shirt-dev)
-* [docs] Clarify that `--embed-metadata` embeds chapter markers
-* [cleanup] Refactor fixup
-
-
-### 2021.06.09
-
-* Fix bug where `%(field)d` in filename template throws error
-* Improve offset parsing in outtmpl
-* [test] More rigorous tests for `prepare_filename`
-
-### 2021.06.08
-
-* Remove support for obsolete Python versions: Only 3.6+ is now supported
-* Merge youtube-dl: Upto [commit/c2350ca](https://github.com/ytdl-org/youtube-dl/commit/c2350cac243ba1ec1586fe85b0d62d1b700047a2)
-* [hls] Fix decryption for multithreaded downloader
-* [extractor] Fix pre-checking archive for some extractors
-* [extractor] Fix FourCC fallback when parsing ISM by [fstirlitz](https://github.com/fstirlitz)
-* [twitcasting] Add TwitCastingUserIE, TwitCastingLiveIE by [pukkandan](https://github.com/pukkandan), [nao20010128nao](https://github.com/nao20010128nao)
-* [vidio] Add VidioPremierIE and VidioLiveIE by [MinePlayersPE](Https://github.com/MinePlayersPE)
-* [viki] Fix extraction from [ytdl-org/youtube-dl@59e583f](https://github.com/ytdl-org/youtube-dl/commit/59e583f7e8530ca92776c866897d895c072e2a82)
-* [youtube] Support shorts URL
-* [zoom] Extract transcripts as subtitles
-* Add field `original_url` with the user-inputted URL
-* Fix and refactor `prepare_outtmpl`
-* Make more fields available for `--print` when used with `--flat-playlist`
-* [utils] Generalize `traverse_dict` to `traverse_obj`
-* [downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode by [fstirlitz](https://github.com/fstirlitz)
-* [build] Release `yt-dlp.tar.gz`
-* [build,update] Add GNU-style SHA512 and prepare updater for simlar SHA256 by [nihil-admirari](https://github.com/nihil-admirari)
-* [pyinst] Show Python version in exe metadata by [nihil-admirari](https://github.com/nihil-admirari)
-* [docs] Improve documentation of dependencies
-* [cleanup] Mark unused files
-* [cleanup] Point all shebang to `python3` by [fstirlitz](https://github.com/fstirlitz)
-* [cleanup] Remove duplicate file `trovolive.py`
-
-
-### 2021.06.01
-
-* Merge youtube-dl: Upto [commit/d495292](https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf)
-* Pre-check archive and filters during playlist extraction
-* Handle Basic Auth `user:pass` in URLs by [hhirtz](https://github.com/hhirtz) and [pukkandan](https://github.com/pukkandan)
-* [archiveorg] Add YoutubeWebArchiveIE by [coletdjnz](https://github.com/coletdjnz) and [alex-gedeon](https://github.com/alex-gedeon)
-* [fancode] Add extractor by [rhsmachine](https://github.com/rhsmachine)
-* [patreon] Support vimeo embeds by [rhsmachine](https://github.com/rhsmachine)
-* [Saitosan] Add new extractor by [llacb47](https://github.com/llacb47)
-* [ShemarooMe] Add extractor by [Ashish0804](https://github.com/Ashish0804) and [pukkandan](https://github.com/pukkandan)
-* [telemundo] Add extractor by [king-millez](https://github.com/king-millez)
-* [SonyLIV] Add SonyLIVSeriesIE and subtitle support by [Ashish0804](https://github.com/Ashish0804)
-* [Hotstar] Add HotStarSeriesIE by [Ashish0804](https://github.com/Ashish0804)
-* [Voot] Add VootSeriesIE by [Ashish0804](https://github.com/Ashish0804)
-* [vidio] Support login and premium videos by [MinePlayersPE](https://github.com/MinePlayersPE)
-* [fragment] When using `-N`, do not keep the fragment content in memory
-* [ffmpeg] Download and merge in a single step if possible
-* [ThumbnailsConvertor] Support conversion to `png` and make it the default by [louie-github](https://github.com/louie-github)
-* [VideoConvertor] Generalize with remuxer and allow conditional recoding
-* [EmbedThumbnail] Embed in `mp4`/`m4a` using mutagen by [tripulse](https://github.com/tripulse) and [pukkandan](https://github.com/pukkandan)
-* [EmbedThumbnail] Embed if any thumbnail was downloaded, not just the best
-* [EmbedThumbnail] Correctly escape filename
-* [update] replace self without launching a subprocess in windows
-* [update] Block further update for unsupported systems
-* Refactor `__process_playlist` by creating `LazyList`
-* Write messages to `stderr` when both `quiet` and `verbose`
-* Sanitize and sort playlist thumbnails
-* Remove `None` values from `info.json`
-* [extractor] Always prefer native hls downloader by default
-* [extractor] Skip subtitles without URI in m3u8 manifests by [hheimbuerger](https://github.com/hheimbuerger)
-* [extractor] Functions to parse `socket.io` response as `json` by [pukkandan](https://github.com/pukkandan) and [llacb47](https://github.com/llacb47)
-* [extractor] Allow `note=False` when extracting manifests
-* [utils] Escape URLs in `sanitized_Request`, not `sanitize_url`
-* [hls] Disable external downloader for `webtt`
-* [youtube] `/live` URLs should raise error if channel is not live
-* [youtube] Bug fixes
-* [zee5] Fix m3u8 formats' extension
-* [ard] Allow URLs without `-` before id by [olifre](https://github.com/olifre)
-* [cleanup] `YoutubeDL._match_entry`
-* [cleanup] Refactor updater
-* [cleanup] Refactor ffmpeg convertors
-* [cleanup] setup.py
-
-
-### 2021.05.20
-
-* **Youtube improvements**: 
-    * Support youtube music `MP`, `VL` and `browse` pages
-    * Extract more formats for youtube music by [craftingmod](https://github.com/craftingmod), [coletdjnz](https://github.com/coletdjnz) and [pukkandan](https://github.com/pukkandan)
-    * Extract multiple subtitles in same language by [pukkandan](https://github.com/pukkandan) and [tpikonen](https://github.com/tpikonen)
-    * Redirect channels that doesn't have a `videos` tab to their `UU` playlists
-    * Support in-channel search
-    * Sort audio-only formats correctly
-    * Always extract `maxresdefault` thumbnail
-    * Extract audio language
-    * Add subtitle language names by [nixxo](https://github.com/nixxo) and [tpikonen](https://github.com/tpikonen)
-    * Show alerts only from the final webpage
-    * Add `html5=1` param to `get_video_info` page requests by [coletdjnz](https://github.com/coletdjnz)
-    * Better message when login required
-* **Add option `--print`**: to print any field/template
-    * Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
-* Field `additional_urls` to download additional videos from metadata using [`--parse-metadata`](https://github.com/yt-dlp/yt-dlp#modifying-metadata)
-* Merge youtube-dl: Upto [commit/dfbbe29](https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b)
-* Write thumbnail of playlist and add `pl_thumbnail` outtmpl key
-* [embedthumbnail] Add `flac` support and refactor `mutagen` code by [pukkandan](https://github.com/pukkandan) and [tripulse](https://github.com/tripulse)
-* [audius:artist] Add extractor by [king-millez](https://github.com/king-millez)
-* [parlview] Add extractor by [king-millez](https://github.com/king-millez)
-* [tenplay] Fix extractor by [king-millez](https://github.com/king-millez)
-* [rmcdecouverte] Generalize `_VALID_URL`
-* Add compat-option `no-attach-infojson`
-* Add field `name` for subtitles
-* Ensure `post_extract` and `pre_process` only run once
-* Fix `--check-formats` when there is network error
-* Standardize `write_debug` and `get_param`
-* [options] Alias `--write-comments`, `--no-write-comments`
-* [options] Refactor callbacks
-* [test:download] Only extract enough videos for `playlist_mincount`
-* [extractor] bugfix for when `compat_opts` is not given
-* [build] Fix x86 build by [shirt](https://github.com/shirt-dev)
-* [cleanup] code formatting, youtube tests and readme
-
-### 2021.05.11
-* **Deprecate support for python versions < 3.6**
-* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
-* **Improve output template:**
-    * Allow slicing lists/strings using `field.start:end:step`
-    * A field can also be used as offset like `field1+num+field2`
-    * A default value can be given using `field|default`
-    * Prevent invalid fields from causing errors
-* **Merge youtube-dl**: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
-* **Remove options** `-l`, `-t`, `-A` completely and disable `--auto-number`, `--title`, `--literal`, `--id`
-* [Plugins] Prioritize plugins over standard extractors and prevent plugins from overwriting the standard extractor classes
-* [downloader] Fix `quiet` and `to_stderr`
-* [fragment] Ensure the file is closed on error
-* [fragment] Make sure first segment is not skipped
-* [aria2c] Fix whitespace being stripped off
-* [embedthumbnail] Fix bug where jpeg thumbnails were converted again
-* [FormatSort] Fix for when some formats have quality and others don't
-* [utils] Add `network_exceptions`
-* [utils] Escape URL while sanitizing
-* [ukcolumn] Add Extractor
-* [whowatch] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
-* [CBS] Improve `_VALID_URL` to support movies
-* [crackle] Improve extraction
-* [curiositystream] Fix collections
-* [francetvinfo] Improve video id extraction
-* [generic] Respect the encoding in manifest
-* [limelight] Obey `allow_unplayable_formats`
-* [mediasite] Generalize URL pattern by [fstirlitz](https://github.com/fstirlitz)
-* [mxplayer] Add MxplayerShowIE by [Ashish0804](https://github.com/Ashish0804)
-* [nebula] Move to nebula.app by [Lamieur](https://github.com/Lamieur)
-* [niconico] Fix HLS formats by [CXwudi](https://github.com/CXwudi), [tsukumijima](https://github.com/tsukumijima), [nao20010128nao](https://github.com/nao20010128nao) and [pukkandan](https://github.com/pukkandan)
-* [niconico] Fix title and thumbnail extraction by [CXwudi](https://github.com/CXwudi)
-* [plutotv] Extract subtitles from manifests
-* [plutotv] Fix format extraction for some urls
-* [rmcdecouverte] Improve `_VALID_URL`
-* [sonyliv] Fix `title` and `series` extraction by [Ashish0804](https://github.com/Ashish0804)
-* [tubi] Raise "no video formats" error when video url is empty
-* [youtube:tab] Detect playlists inside community posts
-* [youtube] Add `oembed` to reserved names
-* [zee5] Fix extraction for some URLs by [Hadi0609](https://github.com/Hadi0609)
-* [zee5] Fix py2 compatibility
-* Fix `playlist_index` and add `playlist_autonumber`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details
-* Add experimental option `--check-formats` to test the URLs before format selection
-* Option `--compat-options` to revert [some of yt-dlp's changes](https://github.com/yt-dlp/yt-dlp#differences-in-default-behavior)
-    * Deprecates `--list-formats-as-table`, `--list-formats-old`
-* Fix number of digits in `%(playlist_index)s`
-* Fix case sensitivity of format selector
-* Revert "[core] be able to hand over id and title using url_result"
-* Do not strip out whitespaces in `-o` and `-P`
-* Fix `preload_download_archive` writing verbose message to `stdout`
-* Move option warnings to `YoutubeDL`so that they obey `--no-warnings` and can output colors
-* Py2 compatibility for `FileNotFoundError`
-
-
-### 2021.04.22
-* **Improve output template:**
-    * Objects can be traversed like `%(field.key1.key2)s`
-    * An offset can be added to numeric fields as `%(field+N)s`
-    * Deprecates `--autonumber-start`
-* **Improve `--sub-langs`:**
-    * Treat `--sub-langs` entries as regex
-    * `all` can be used to refer to all the subtitles
-    * language codes can be prefixed with `-` to exclude it
-    * Deprecates `--all-subs`
-* Add option `--ignore-no-formats-error` to ignore the "no video format" and similar errors
-* Add option `--skip-playlist-after-errors` to skip the rest of a playlist after a given number of errors are encountered
-* Merge youtube-dl: Upto [commit/7e8b3f9](https://github.com/ytdl-org/youtube-dl/commit/7e8b3f9439ebefb3a3a4e5da9c0bd2b595976438)
-* [downloader] Fix bug in downloader selection
-* [BilibiliChannel] Fix pagination by [nao20010128nao](https://github.com/nao20010128nao) and [pukkandan](https://github.com/pukkandan)
-* [rai] Add support for http formats by [nixxo](https://github.com/nixxo)
-* [TubiTv] Add TubiTvShowIE by [Ashish0804](https://github.com/Ashish0804)
-* [twitcasting] Fix extractor
-* [viu:ott] Fix extractor and support series by [lkho](https://github.com/lkho) and [pukkandan](https://github.com/pukkandan)
-* [youtube:tab] Show unavailable videos in playlists by [coletdjnz](https://github.com/coletdjnz)
-* [youtube:tab] Reload with unavailable videos for all playlists
-* [youtube] Ignore invalid stretch ratio
-* [youtube] Improve channel syncid extraction to support ytcfg by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Standardize API calls for tabs, mixes and search by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Bugfix in `_extract_ytcfg`
-* [mildom:user:vod] Download only necessary amount of pages
-* [mildom] Remove proxy completely by [fstirlitz](https://github.com/fstirlitz)
-* [go] Fix `_VALID_URL`
-* [MetadataFromField] Improve regex and add tests
-* [Exec] Ensure backward compatibility when the command contains `%`
-* [extractor] Fix inconsistent use of `report_warning`
-* Ensure `mergeall` selects best format when multistreams are disabled
-* Improve the yt-dlp.sh script by [fstirlitz](https://github.com/fstirlitz)
-* [lazy_extractor] Do not load plugins
-* [ci] Disable fail-fast
-* [docs] Clarify which deprecated options still work
-* [docs] Fix typos
-
-
-### 2021.04.11
-* Add option `--convert-thumbnails` (only jpg currently supported)
-* Format selector `mergeall` to download and merge all formats
-* Pass any field to `--exec` using similar syntax to output template
-* Choose downloader for each protocol using `--downloader PROTO:NAME`
-    * Alias `--downloader` for `--external-downloader`
-    * Added `native` as an option for the downloader
-* Merge youtube-dl: Upto [commit/4fb25ff](https://github.com/ytdl-org/youtube-dl/commit/4fb25ff5a3be5206bb72e5c4046715b1529fb2c7) (except vimeo)
-* [DiscoveryPlusIndia] Add DiscoveryPlusIndiaShowIE by [Ashish0804](https://github.com/Ashish0804)
-* [NFHSNetwork] Add extractor by [llacb47](https://github.com/llacb47)
-* [nebula] Add extractor (watchnebula.com) by [hheimbuerger](https://github.com/hheimbuerger)
-* [nitter] Fix extraction of reply tweets and update instance list by [B0pol](https://github.com/B0pol)
-* [nitter] Fix thumbnails by [B0pol](https://github.com/B0pol)
-* [youtube] Fix thumbnail URL
-* [youtube] Parse API parameters from initial webpage by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Extract comments' approximate timestamp by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Fix alert extraction
-* [bilibili] Fix uploader
-* [utils] Add `datetime_from_str` and `datetime_add_months` by [coletdjnz](https://github.com/coletdjnz)
-* Run some `postprocessors` before actual download
-* Improve argument parsing for `-P`, `-o`, `-S`
-* Fix some `m3u8` not obeying `--allow-unplayable-formats`
-* Fix default of `dynamic_mpd`
-* Deprecate `--all-formats`, `--include-ads`, `--hls-prefer-native`, `--hls-prefer-ffmpeg`
-* [docs] Improvements
-
-### 2021.04.03
-* Merge youtube-dl: Upto [commit/654b4f4](https://github.com/ytdl-org/youtube-dl/commit/654b4f4ff2718f38b3182c1188c5d569c14cc70a)
-* Ability to set a specific field in the file's metadata using `--parse-metadata`
-* Ability to select n'th best format like `-f bv*.2`
-* [DiscoveryPlus] Add discoveryplus.in
-* [la7] Add podcasts and podcast playlists by [nixxo](https://github.com/nixxo)
-* [mildom] Update extractor with current proxy by [nao20010128nao](https://github.com/nao20010128nao)
-* [ard:mediathek] Fix video id extraction
-* [generic] Detect Invidious' link element
-* [youtube] Show premium state in `availability` by [coletdjnz](https://github.com/coletdjnz)
-* [viewsource] Add extractor to handle `view-source:`
-* [sponskrub] Run before embedding thumbnail
-* [docs] Improve `--parse-metadata` documentation
-
-
-### 2021.03.24.1
-* Revert [commit/8562218](https://github.com/ytdl-org/youtube-dl/commit/8562218350a79d4709da8593bb0c538aa0824acf)
-
-### 2021.03.24
-* Merge youtube-dl: Upto 2021.03.25 ([commit/8562218](https://github.com/ytdl-org/youtube-dl/commit/8562218350a79d4709da8593bb0c538aa0824acf))
-* Parse metadata from multiple fields using `--parse-metadata`
-* Ability to load playlist infojson using `--load-info-json`
-* Write current epoch to infojson when using `--no-clean-infojson`
-* [youtube_live_chat] fix bug when trying to set cookies
-* [niconico] Fix for when logged in by [CXwudi](https://github.com/CXwudi) and [xtkoba](https://github.com/xtkoba)
-* [linuxacadamy] Fix login
-
-
 ### 2021.03.21
 * Merge youtube-dl: Upto [commit/7e79ba7](https://github.com/ytdl-org/youtube-dl/commit/7e79ba7dd6e6649dd2ce3a74004b2044f2182881)
-* Option `--no-clean-infojson` to keep private keys in the infojson
+* Option `--clean-infojson` to keep private keys in the infojson
 * [aria2c] Support retry/abort unavailable fragments by [damianoamatruda](https://github.com/damianoamatruda)
 * [aria2c] Better default arguments
 * [movefiles] Fix bugs and make more robust
@@ -501,8 +33,8 @@
 * Use headers and cookies when downloading subtitles by [damianoamatruda](https://github.com/damianoamatruda)
 * Parse resolution in info dictionary by [damianoamatruda](https://github.com/damianoamatruda)
 * More consistent warning messages by [damianoamatruda](https://github.com/damianoamatruda) and [pukkandan](https://github.com/pukkandan)
-* [docs] Add deprecated options and aliases in readme
-* [docs] Fix some minor mistakes
+* [documentation] Add deprecated options and aliases in readme
+* [documentation] Fix some minor mistakes
 
 * [niconico] Partial fix adapted from [animelover1984/youtube-dl@b5eff52](https://github.com/animelover1984/youtube-dl/commit/b5eff52dd9ed5565672ea1694b38c9296db3fade) (login and smile formats still don't work)
 * [niconico] Add user extractor by [animelover1984](https://github.com/animelover1984)
@@ -511,7 +43,7 @@
 * [stitcher] Merge from youtube-dl by [nixxo](https://github.com/nixxo)
 * [rcs] Improved extraction by [nixxo](https://github.com/nixxo)
 * [linuxacadamy] Improve regex
-* [youtube] Show if video is `private`, `unlisted` etc in info (`availability`) by [coletdjnz](https://github.com/coletdjnz) and [pukkandan](https://github.com/pukkandan)
+* [youtube] Show if video is `private`, `unlisted` etc in info (`availability`) by [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
 * [youtube] bugfix for channel playlist extraction
 * [nbc] Improve metadata extraction by [2ShedsJackson](https://github.com/2ShedsJackson)
 
@@ -528,15 +60,15 @@
 * [wimtv] Add extractor by [nixxo](https://github.com/nixxo)
 * [mtv] Add mtv.it and extract series metadata by [nixxo](https://github.com/nixxo)
 * [pluto.tv] Add extractor by [kevinoconnor7](https://github.com/kevinoconnor7)
-* [youtube] Rewrite comment extraction by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Rewrite comment extraction by [colethedj](https://github.com/colethedj)
 * [embedthumbnail] Set mtime correctly
 * Refactor some postprocessor/downloader code by [pukkandan](https://github.com/pukkandan) and [shirt](https://github.com/shirt-dev)
 
 
 ### 2021.03.07
-* [youtube] Fix history, mixes, community pages and trending by [pukkandan](https://github.com/pukkandan) and [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Fix private feeds/playlists on multi-channel accounts by [coletdjnz](https://github.com/coletdjnz)
-* [youtube] Extract alerts from continuation by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Fix history, mixes, community pages and trending by [pukkandan](https://github.com/pukkandan) and [colethedj](https://github.com/colethedj)
+* [youtube] Fix private feeds/playlists on multi-channel accounts by [colethedj](https://github.com/colethedj)
+* [youtube] Extract alerts from continuation by [colethedj](https://github.com/colethedj)
 * [cbs] Add support for ParamountPlus by [shirt](https://github.com/shirt-dev)
 * [mxplayer] Rewrite extractor with show support by [pukkandan](https://github.com/pukkandan) and [Ashish0804](https://github.com/Ashish0804)
 * [gedi] Improvements from youtube-dl by [nixxo](https://github.com/nixxo)
@@ -548,7 +80,7 @@
 * [downloader] Fix bug for `ffmpeg`/`httpie`
 * [update] Fix updater removing the executable bit on some UNIX distros
 * [update] Fix current build hash for UNIX
-* [docs] Include wget/curl/aria2c install instructions for Unix by [Ashish0804](https://github.com/Ashish0804)
+* [documentation] Include wget/curl/aria2c install instructions for Unix by [Ashish0804](https://github.com/Ashish0804)
 * Fix some videos downloading with `m3u8` extension
 * Remove "fixup is ignored" warning when fixup wasn't passed by user
 
@@ -556,8 +88,9 @@
 ### 2021.03.03.2
 * [build] Fix bug
 
+
 ### 2021.03.03
-* [youtube] Use new browse API for continuation page extraction by [coletdjnz](https://github.com/coletdjnz) and [pukkandan](https://github.com/pukkandan)
+* [youtube] Use new browse API for continuation page extraction by [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
 * Fix HLS playlist downloading by [shirt](https://github.com/shirt-dev)
 * Merge youtube-dl: Upto [2021.03.03](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.03.03)
 * [mtv] Fix extractor
@@ -605,7 +138,7 @@
 * [ffmpeg] Allow passing custom arguments before -i using `--ppa "ffmpeg_i1:ARGS"` syntax
 * Fix `--windows-filenames` removing `/` from UNIX paths
 * [hls] Show warning if pycryptodome is not found
-* [docs] Improvements
+* [documentation] Improvements
     * Fix documentation of `Extractor Options`
     * Document `all` in format selection
     * Document `playable_in_embed` in output templates
@@ -633,12 +166,12 @@
     * Exclude `vcruntime140.dll` from UPX by [jbruchon](https://github.com/jbruchon)
     * Set version number based on UTC time, not local time
     * Publish on PyPi only if token is set
-* [docs] Better document `--prefer-free-formats` and add `--no-prefer-free-format`
+* [documentation] Better document `--prefer-free-formats` and add `--no-prefer-free-format`
 
 
 ### 2021.02.15
 * Merge youtube-dl: Upto [2021.02.10](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.02.10) (except archive.org)
-* [niconico] Improved extraction and support encrypted/SMILE movies by [kurumigi](https://github.com/kurumigi), [tsukumijima](https://github.com/tsukumijima), [bbepis](https://github.com/bbepis), [pukkandan](https://github.com/pukkandan)
+* [niconico] Improved extraction and support encrypted/SMILE movies by [kurumigi](https://github.com/kurumigi), [tsukumi](https://github.com/tsukumi), [bbepis](https://github.com/bbepis), [pukkandan](https://github.com/pukkandan)
 * Fix HLS AES-128 with multiple keys in external downloaders by [shirt](https://github.com/shirt-dev)
 * [youtube_live_chat] Fix by using POST API by [siikamiika](https://github.com/siikamiika)
 * [rumble] Add support for video page
@@ -676,7 +209,7 @@
 * [movefiles] Fix compatibility with python2
 * [remuxvideo] Fix validation of conditional remux
 * [sponskrub] Don't raise error when the video does not exist
-* [docs] Crypto is an optional dependency
+* [documentation] Crypto is an optional dependency
 
 
 ### 2021.02.04
@@ -737,10 +270,10 @@
 * Merge youtube-dl: Upto [2021.01.24](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.01.16)
 * Plugin support ([documentation](https://github.com/yt-dlp/yt-dlp#plugins))
 * **Multiple paths**: New option `-P`/`--paths` to give different paths for different types of files
-    * The syntax is `-P "type:path" -P "type:path"`
+    * The syntax is `-P "type:path" -P "type:path"` ([documentation](https://github.com/yt-dlp/yt-dlp#:~:text=-P,%20--paths%20TYPE:PATH))
     * Valid types are: home, temp, description, annotation, subtitle, infojson, thumbnail
-    * Additionally, configuration file is taken from home directory or current directory
-* Allow passing different arguments to different external downloaders
+    * Additionally, configuration file is taken from home directory or current directory ([documentation](https://github.com/yt-dlp/yt-dlp#:~:text=Home%20Configuration))
+* Allow passing different arguments to different external downloaders ([documentation](https://github.com/yt-dlp/yt-dlp#:~:text=--downloader-args%20NAME:ARGS))
 * [mildom] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
 * Warn when using old style `--external-downloader-args` and `--post-processor-args`
 * Fix `--no-overwrite` when using `--write-link`
@@ -775,9 +308,9 @@
 * [roosterteeth.com] Fix for bonus episodes by [Zocker1999NET](https://github.com/Zocker1999NET)
 * [tiktok] Fix for when share_info is empty
 * [EmbedThumbnail] Fix bug due to incorrect function name
-* [docs] Changed sponskrub links to point to [yt-dlp/SponSkrub](https://github.com/yt-dlp/SponSkrub) since I am now providing both linux and windows releases
-* [docs] Change all links to correctly point to new fork URL
-* [docs] Fixes typos
+* [documentation] Changed sponskrub links to point to [yt-dlp/SponSkrub](https://github.com/yt-dlp/SponSkrub) since I am now providing both linux and windows releases
+* [documentation] Change all links to correctly point to new fork URL
+* [documentation] Fixes typos
 
 
 ### 2021.01.12
@@ -810,7 +343,7 @@
 
 ### 2021.01.08
 * Merge youtube-dl: Upto [2021.01.08](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.01.08) except stitcher ([1](https://github.com/ytdl-org/youtube-dl/commit/bb38a1215718cdf36d73ff0a7830a64cd9fa37cc), [2](https://github.com/ytdl-org/youtube-dl/commit/a563c97c5cddf55f8989ed7ea8314ef78e30107f))
-* Moved changelog to separate file
+* Moved changelog to seperate file
 
 
 ### 2021.01.07-1
@@ -873,7 +406,7 @@
     * Redirect channel home to /video
     * Print youtube's warning message
     * Handle Multiple pages for feeds better
-* [youtube] Fix ytsearch not returning results sometimes due to promoted content by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Fix ytsearch not returning results sometimes due to promoted content by [colethedj](https://github.com/colethedj)
 * [youtube] Temporary fix for automatic captions - disable json3 by [blackjack4494](https://github.com/blackjack4494)
 * Add --break-on-existing by [gergesh](https://github.com/gergesh)
 * Pre-check video IDs in the archive before downloading by [pukkandan](https://github.com/pukkandan)
@@ -892,4 +425,4 @@
 * [generic] Extract embedded youtube and twitter videos by [diegorodriguezv](https://github.com/diegorodriguezv)
 * [ffmpeg] Ensure all streams are copied by [pukkandan](https://github.com/pukkandan)
 * [embedthumbnail] Fix for os.rename error by [pukkandan](https://github.com/pukkandan)
-* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)
+* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)

Collaborators.md

@@ -1,37 +0,0 @@
-# Collaborators
-
-This is a list of the collaborators of the project and their major contributions. See the [Changelog](Changelog.md) for more details.
-
-You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [authors of youtube-dl](https://github.com/ytdl-org/youtube-dl/blob/master/AUTHORS)
-
-
-## [pukkandan](https://github.com/pukkandan)
-
-[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/pukkandan)
-
-* Owner of the fork
-
-
-
-## [shirt](https://github.com/shirt-dev)
-
-* Multithreading (`-N`) and aria2c support for fragment downloads
-* Support for media initialization and discontinuity in HLS
-* The self-updater (`-U`)
-
-
-
-## [coletdjnz](https://github.com/coletdjnz)
-
-[![gh-sponsor](https://img.shields.io/badge/_-Sponsor-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/coletdjnz)
-
-* YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements
-
-
-
-## [Ashish0804](https://github.com/Ashish0804)
-
-[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/ashish0804)
-
-* Added support for new websites Zee5, MXPlayer, DiscoveryPlusIndia, ShemarooMe, Utreon etc
-* Added playlist/series downloads for TubiTv, SonyLIV, Voot, HotStar etc

Makefile

@@ -13,7 +13,7 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites com
 .PHONY: all clean install test tar pypi-files completions ot offlinetest codetest supportedsites
 
 clean-test:
-	rm -rf *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.frag *.frag.urls *.frag.aria2 test/testdata/player-*.js
+	rm -rf *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.frag *.frag.urls *.frag.aria2
 clean-dist:
 	rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap
 clean-cache:
@@ -25,7 +25,6 @@ completion-zsh: completions/zsh/_yt-dlp
 lazy-extractors: yt_dlp/extractor/lazy_extractors.py
 
 PREFIX ?= /usr/local
-DESTDIR ?= .
 BINDIR ?= $(PREFIX)/bin
 MANDIR ?= $(PREFIX)/man
 SHAREDIR ?= $(PREFIX)/share
@@ -49,11 +48,23 @@ codetest:
 	flake8 .
 
 test:
-	$(PYTHON) -m pytest
+	#nosetests --with-coverage --cover-package=yt_dlp --cover-html --verbose --processes 4 test
+	nosetests --verbose test
 	$(MAKE) codetest
 
+# Keep this list in sync with devscripts/run_tests.sh
 offlinetest: codetest
-	$(PYTHON) -m pytest -k "not download"
+	$(PYTHON) -m nose --verbose test \
+		--exclude test_age_restriction.py \
+		--exclude test_download.py \
+		--exclude test_iqiyi_sdk_interpreter.py \
+		--exclude test_overwrites.py \
+		--exclude test_socks.py \
+		--exclude test_subtitles.py \
+		--exclude test_write_annotations.py \
+		--exclude test_youtube_lists.py \
+		--exclude test_youtube_signature.py \
+		--exclude test_post_hooks.py
 
 yt-dlp: yt_dlp/*.py yt_dlp/*/*.py
 	mkdir -p zip

devscripts/bash-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import os

devscripts/buildserver.py

@@ -1,5 +1,3 @@
-# UNUSED
-
 #!/usr/bin/python3
 
 import argparse

devscripts/check-porn.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 """

devscripts/create-github-release.py

@@ -1,6 +1,6 @@
 # Unused
 
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import io

devscripts/fish-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import optparse

devscripts/generate_aes_testdata.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import codecs

devscripts/gh-pages/update-copyright.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import with_statement, unicode_literals

devscripts/lazy_load_template.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import unicode_literals
 

devscripts/make_contributing.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 # import io

devscripts/make_issue_template.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import io

devscripts/make_lazy_extractors.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
 from __future__ import unicode_literals, print_function
 
 from inspect import getsource
@@ -15,14 +14,9 @@ lazy_extractors_filename = sys.argv[1]
 if os.path.exists(lazy_extractors_filename):
     os.remove(lazy_extractors_filename)
 
-# Block plugins from loading
-os.rename('ytdlp_plugins', 'ytdlp_plugins_blocked')
-
 from yt_dlp.extractor import _ALL_CLASSES
 from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
 
-os.rename('ytdlp_plugins_blocked', 'ytdlp_plugins')
-
 with open('devscripts/lazy_load_template.py', 'rt') as f:
     module_template = f.read()
 

devscripts/make_readme.py

@@ -1,8 +1,3 @@
-#!/usr/bin/env python3
-
-# yt-dlp --help | make_readme.py
-# This must be run in a console of correct width
-
 from __future__ import unicode_literals
 
 import io

devscripts/make_supportedsites.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import io

devscripts/prepare_manpage.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
 from __future__ import unicode_literals
 
 import io

devscripts/release.sh

@@ -1,5 +1,4 @@
 # Unused
-
 #!/bin/bash
 
 # IMPORTANT: the following assumptions are made

devscripts/run_tests.bat

@@ -1,16 +1,17 @@
-@setlocal
 @echo off
-cd /d %~dp0..
 
-if ["%~1"]==[""] (
-    set "test_set="
-) else if ["%~1"]==["core"] (
-    set "test_set=-k "not download""
-) else if ["%~1"]==["download"] (
-    set "test_set=-k download"
+rem Keep this list in sync with the `offlinetest` target in Makefile
+set DOWNLOAD_TESTS="age_restriction^|download^|iqiyi_sdk_interpreter^|socks^|subtitles^|write_annotations^|youtube_lists^|youtube_signature^|post_hooks"
+
+if "%YTDL_TEST_SET%" == "core" (
+    set test_set="-I test_("%DOWNLOAD_TESTS%")\.py"
+    set multiprocess_args=""
+) else if "%YTDL_TEST_SET%" == "download" (
+    set test_set="-I test_(?!"%DOWNLOAD_TESTS%").+\.py"
+    set multiprocess_args="--processes=4 --process-timeout=540"
 ) else (
-    echo.Invalid test type "%~1". Use "core" ^| "download"
+    echo YTDL_TEST_SET is not set or invalid
     exit /b 1
 )
 
-pytest %test_set%
+nosetests test --verbose %test_set:"=% %multiprocess_args:"=%

devscripts/run_tests.sh

@@ -1,15 +1,22 @@
-#!/bin/sh
+#!/bin/bash
 
-if [ -z $1 ]; then
-    test_set='test'
-elif [ $1 = 'core' ]; then
-    test_set='not download'
-elif [ $1 = 'download' ]; then
-    test_set='download'
-else
-    echo 'Invalid test type "'$1'". Use "core" | "download"'
-    exit 1
-fi
+# Keep this list in sync with the `offlinetest` target in Makefile
+DOWNLOAD_TESTS="age_restriction|download|iqiyi_sdk_interpreter|overwrites|socks|subtitles|write_annotations|youtube_lists|youtube_signature|post_hooks"
 
-echo python3 -m pytest -k $test_set
-python3 -m pytest -k "$test_set"
+test_set=""
+multiprocess_args=""
+
+case "$YTDL_TEST_SET" in
+    core)
+        test_set="-I test_($DOWNLOAD_TESTS)\.py"
+    ;;
+    download)
+        test_set="-I test_(?!$DOWNLOAD_TESTS).+\.py"
+        multiprocess_args="--processes=4 --process-timeout=540"
+    ;;
+    *)
+        break
+    ;;
+esac
+
+nosetests test --verbose $test_set $multiprocess_args

devscripts/show-downloads-statistics.py

@@ -1,6 +1,6 @@
 # Unused
 
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import itertools

devscripts/update-version.py

@@ -1,6 +1,4 @@
-#!/usr/bin/env python3
 from __future__ import unicode_literals
-
 from datetime import datetime
 # import urllib.request
 

devscripts/wine-py2exe.sh

@@ -1,5 +1,3 @@
-# UNUSED
-
 #!/bin/bash
 
 # Run with as parameter a setup.py that works in the current directory

devscripts/zsh-completion.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import os

docs/Collaborators.md

@@ -1,5 +0,0 @@
----
-orphan: true
----
-```{include} ../Collaborators.md
-```

pyinst.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -6,7 +6,6 @@ import sys
 # import os
 import platform
 
-from PyInstaller.utils.hooks import collect_submodules
 from PyInstaller.utils.win32.versioninfo import (
     VarStruct, VarFileInfo, StringStruct, StringTable,
     StringFileInfo, FixedFileInfo, VSVersionInfo, SetVersion,
@@ -18,7 +17,7 @@ assert arch in ('32', '64')
 print('Building %sbit version' % arch)
 _x86 = '_x86' if arch == '32' else ''
 
-FILE_DESCRIPTION = 'yt-dlp%s' % (' (32 Bit)' if _x86 else '')
+FILE_DESCRIPTION = 'Media Downloader%s' % (' (32 Bit)' if _x86 else '')
 
 # root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
 # print('Changing working directory to %s' % root_dir)
@@ -59,23 +58,21 @@ VERSION_FILE = VSVersionInfo(
                     ),
                     StringStruct('OriginalFilename', 'yt-dlp%s.exe' % _x86),
                     StringStruct('ProductName', 'yt-dlp%s' % _x86),
-                    StringStruct(
-                        'ProductVersion',
-                        '%s%s on Python %s' % (VERSION, _x86, platform.python_version())),
+                    StringStruct('ProductVersion', '%s%s' % (VERSION, _x86)),
                 ])]),
         VarFileInfo([VarStruct('Translation', [0, 1200])])
     ]
 )
 
-dependancies = ['Crypto', 'mutagen'] + collect_submodules('websockets')
-excluded_modules = ['test', 'ytdlp_plugins', 'youtube-dl', 'youtube-dlc']
-
 PyInstaller.__main__.run([
     '--name=yt-dlp%s' % _x86,
     '--onefile',
-    '--icon=devscripts/logo.ico',
-    *[f'--exclude-module={module}' for module in excluded_modules],
-    *[f'--hidden-import={module}' for module in dependancies],
+    '--icon=devscripts/cloud.ico',
+    '--exclude-module=youtube_dl',
+    '--exclude-module=test',
+    '--exclude-module=ytdlp_plugins',
+    '--hidden-import=mutagen',
+    '--hidden-import=Crypto',
     '--upx-exclude=vcruntime140.dll',
     'yt_dlp/__main__.py',
 ])

pytest.ini

@@ -1,4 +0,0 @@
-[pytest]
-addopts = -ra -v --strict-markers
-markers =
-    download

requirements.txt

@@ -1,3 +1,2 @@
 mutagen
 pycryptodome
-websockets

setup.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from setuptools import setup, Command, find_packages
@@ -9,44 +9,45 @@ from distutils.spawn import spawn
 
 
 # Get the version from yt_dlp/version.py without importing the package
-exec(compile(open('yt_dlp/version.py').read(), 'yt_dlp/version.py', 'exec'))
+exec(compile(open('yt_dlp/version.py').read(),
+             'yt_dlp/version.py', 'exec'))
 
 
 DESCRIPTION = 'Command-line program to download videos from YouTube.com and many other other video platforms.'
 
 LONG_DESCRIPTION = '\n\n'.join((
     'Official repository: <https://github.com/yt-dlp/yt-dlp>',
-    '**PS**: Some links in this document will not work since this is a copy of the README.md from Github',
-    open('README.md', 'r', encoding='utf-8').read()))
+    '**PS**: Many links in this document will not work since this is a copy of the README.md from Github',
+    open("README.md", "r", encoding="utf-8").read()))
 
-REQUIREMENTS = ['mutagen', 'pycryptodome', 'websockets']
-
-if sys.argv[1:2] == ['py2exe']:
-    raise NotImplementedError('py2exe is not currently supported; instead, use "pyinst.py" to build with pyinstaller')
+REQUIREMENTS = ['mutagen', 'pycryptodome']
 
 
-files_spec = [
-    ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
-    ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
-    ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
-    ('share/doc/yt_dlp', ['README.txt']),
-    ('share/man/man1', ['yt-dlp.1'])
-]
-root = os.path.dirname(os.path.abspath(__file__))
-data_files = []
-for dirname, files in files_spec:
-    resfiles = []
-    for fn in files:
-        if not os.path.exists(fn):
-            warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first' % fn)
-        else:
-            resfiles.append(fn)
-    data_files.append((dirname, resfiles))
+if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
+    print("inv")
+else:
+    files_spec = [
+        ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
+        ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
+        ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
+        ('share/doc/yt_dlp', ['README.txt']),
+        ('share/man/man1', ['yt-dlp.1'])
+    ]
+    root = os.path.dirname(os.path.abspath(__file__))
+    data_files = []
+    for dirname, files in files_spec:
+        resfiles = []
+        for fn in files:
+            if not os.path.exists(fn):
+                warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first.' % fn)
+            else:
+                resfiles.append(fn)
+        data_files.append((dirname, resfiles))
 
-params = {
-    'data_files': data_files,
-}
-params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
+    params = {
+        'data_files': data_files,
+    }
+    params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
 
 
 class build_lazy_extractors(Command):
@@ -60,21 +61,23 @@ class build_lazy_extractors(Command):
         pass
 
     def run(self):
-        spawn([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
-              dry_run=self.dry_run)
+        spawn(
+            [sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
+            dry_run=self.dry_run,
+        )
 
 
-packages = find_packages(exclude=('youtube_dl', 'test', 'ytdlp_plugins'))
+packages = find_packages(exclude=("youtube_dl", "test", "ytdlp_plugins"))
 
 setup(
-    name='yt-dlp',
+    name="yt-dlp",
     version=__version__,
-    maintainer='pukkandan',
-    maintainer_email='pukkandan.ytdlp@gmail.com',
+    maintainer="pukkandan",
+    maintainer_email="pukkandan.ytdlp@gmail.com",
     description=DESCRIPTION,
     long_description=LONG_DESCRIPTION,
-    long_description_content_type='text/markdown',
-    url='https://github.com/yt-dlp/yt-dlp',
+    long_description_content_type="text/markdown",
+    url="https://github.com/yt-dlp/yt-dlp",
     packages=packages,
     install_requires=REQUIREMENTS,
     project_urls={
@@ -84,20 +87,30 @@ setup(
         #'Funding': 'https://donate.pypi.org',
     },
     classifiers=[
-        'Topic :: Multimedia :: Video',
-        'Development Status :: 5 - Production/Stable',
-        'Environment :: Console',
-        'Programming Language :: Python',
-        'Programming Language :: Python :: 3.6',
-        'Programming Language :: Python :: 3.7',
-        'Programming Language :: Python :: 3.8',
-        'Programming Language :: Python :: Implementation',
-        'Programming Language :: Python :: Implementation :: CPython',
-        'Programming Language :: Python :: Implementation :: PyPy',
-        'License :: Public Domain',
-        'Operating System :: OS Independent',
+        "Topic :: Multimedia :: Video",
+        "Development Status :: 5 - Production/Stable",
+        "Environment :: Console",
+        "Programming Language :: Python",
+        "Programming Language :: Python :: 2",
+        "Programming Language :: Python :: 2.6",
+        "Programming Language :: Python :: 2.7",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.2",
+        "Programming Language :: Python :: 3.3",
+        "Programming Language :: Python :: 3.4",
+        "Programming Language :: Python :: 3.5",
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: Implementation",
+        "Programming Language :: Python :: Implementation :: CPython",
+        "Programming Language :: Python :: Implementation :: IronPython",
+        "Programming Language :: Python :: Implementation :: Jython",
+        "Programming Language :: Python :: Implementation :: PyPy",
+        "License :: Public Domain",
+        "Operating System :: OS Independent",
     ],
-    python_requires='>=3.6',
+    python_requires='>=2.6',
 
     cmdclass={'build_lazy_extractors': build_lazy_extractors},
     **params

supportedsites.md

@@ -3,7 +3,6 @@
  - **20min**
  - **220.ro**
  - **23video**
- - **247sports**
  - **24video**
  - **3qsdn**: 3Q SDN
  - **3sat**
@@ -82,7 +81,6 @@
  - **audiomack**
  - **audiomack:album**
  - **Audius**: Audius.co
- - **audius:artist**: Audius.co profile/artist pages
  - **audius:playlist**: Audius.co playlists
  - **audius:track**: Audius track ID or API link. Prepend with "audius:"
  - **AWAAN**
@@ -99,8 +97,7 @@
  - **bbc**: BBC
  - **bbc.co.uk**: BBC iPlayer
  - **bbc.co.uk:article**: BBC articles
- - **bbc.co.uk:iplayer:episodes**
- - **bbc.co.uk:iplayer:group**
+ - **bbc.co.uk:iplayer:playlist**
  - **bbc.co.uk:playlist**
  - **BBVTV**
  - **Beatport**
@@ -131,6 +128,7 @@
  - **bitwave:stream**
  - **BleacherReport**
  - **BleacherReportCMS**
+ - **blinkx**
  - **Bloomberg**
  - **BokeCC**
  - **BongaCams**
@@ -172,8 +170,7 @@
  - **cbsnews**: CBS News
  - **cbsnews:embed**
  - **cbsnews:livevideo**: CBS News Live Videos
- - **cbssports**
- - **cbssports:embed**
+ - **CBSSports**
  - **CCMA**
  - **CCTV**: 央视网
  - **CDA**
@@ -251,15 +248,12 @@
  - **DiscoveryGoPlaylist**
  - **DiscoveryNetworksDe**
  - **DiscoveryPlus**
- - **DiscoveryPlusIndia**
- - **DiscoveryPlusIndiaShow**
  - **DiscoveryVR**
  - **Disney**
  - **dlive:stream**
  - **dlive:vod**
  - **DoodStream**
  - **Dotsub**
- - **Douyin**
  - **DouyuShow**
  - **DouyuTV**: 斗鱼
  - **DPlay**
@@ -307,8 +301,6 @@
  - **EyedoTV**
  - **facebook**
  - **FacebookPluginsVideo**
- - **fancode:live**
- - **fancode:vod**
  - **faz.net**
  - **fc2**
  - **fc2:embed**
@@ -345,8 +337,6 @@
  - **FrontendMastersLesson**
  - **FujiTVFODPlus7**
  - **Funimation**
- - **funimation:page**
- - **funimation:show**
  - **Funk**
  - **Fusion**
  - **Fux**
@@ -396,7 +386,6 @@
  - **HotNewHipHop**
  - **hotstar**
  - **hotstar:playlist**
- - **hotstar:series**
  - **Howcast**
  - **HowStuffWorks**
  - **hrfernsehen**
@@ -467,8 +456,6 @@
  - **kuwo:singer**: 酷我音乐 - 歌手
  - **kuwo:song**: 酷我音乐
  - **la7.it**
- - **la7.it:pod:episode**
- - **la7.it:podcast**
  - **laola1tv**
  - **laola1tv:embed**
  - **lbry**
@@ -492,14 +479,14 @@
  - **limelight**
  - **limelight:channel**
  - **limelight:channel_list**
- - **LineLive**
- - **LineLiveChannel**
  - **LineTV**
  - **linkedin:learning**
  - **linkedin:learning:course**
  - **LinuxAcademy**
  - **LiTV**
  - **LiveJournal**
+ - **LiveLeak**
+ - **LiveLeakEmbed**
  - **livestream**
  - **livestream:original**
  - **LnkGo**
@@ -518,7 +505,6 @@
  - **mangomolo:live**
  - **mangomolo:video**
  - **ManyVids**
- - **MaoriTV**
  - **Markiza**
  - **MarkizaPage**
  - **massengeschmack.tv**
@@ -588,7 +574,6 @@
  - **Mwave**
  - **MwaveMeetGreet**
  - **Mxplayer**
- - **MxplayerShow**
  - **MyChannels**
  - **MySpace**
  - **MySpace:album**
@@ -620,7 +605,6 @@
  - **ndr:embed**
  - **ndr:embed:base**
  - **NDTV**
- - **Nebula**
  - **NerdCubedFeed**
  - **netease:album**: 网易云音乐 - 专辑
  - **netease:djradio**: 网易云音乐 - 电台
@@ -639,7 +623,6 @@
  - **NextTV**: 壹電視
  - **Nexx**
  - **NexxEmbed**
- - **NFHSNetwork**
  - **nfl.com** (Currently broken)
  - **nfl.com:article** (Currently broken)
  - **NhkVod**
@@ -723,14 +706,9 @@
  - **OutsideTV**
  - **PacktPub**
  - **PacktPubCourse**
- - **PalcoMP3:artist**
- - **PalcoMP3:song**
- - **PalcoMP3:video**
  - **pandora.tv**: 판도라TV
  - **ParamountNetwork**
- - **ParamountPlusSeries**
  - **parliamentlive.tv**: UK parliament videos
- - **Parlview**
  - **Patreon**
  - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
  - **PearVideo**
@@ -754,7 +732,6 @@
  - **play.fm**
  - **player.sky.it**
  - **PlayPlusTV**
- - **PlayStuff**
  - **PlaysTV**
  - **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
  - **Playvid**
@@ -771,7 +748,6 @@
  - **PopcornTV**
  - **PornCom**
  - **PornerBros**
- - **PornFlip**
  - **PornHd**
  - **PornHub**: PornHub and Thumbzilla
  - **PornHubPagedVideoList**
@@ -814,8 +790,6 @@
  - **RCS**
  - **RCSEmbeds**
  - **RCSVarious**
- - **RCTIPlus**
- - **RCTIPlusSeries**
  - **RDS**: RDS.ca
  - **RedBull**
  - **RedBullEmbed**
@@ -866,7 +840,6 @@
  - **safari**: safaribooksonline.com online video
  - **safari:api**
  - **safari:course**: safaribooksonline.com online courses
- - **Saitosan**
  - **SAKTV**
  - **SaltTV**
  - **SampleFocus**
@@ -874,7 +847,6 @@
  - **savefrom.net**
  - **SBS**: sbs.com.au
  - **schooltv**
- - **ScienceChannel**
  - **screen.yahoo:search**: Yahoo screen search
  - **Screencast**
  - **ScreencastOMatic**
@@ -892,7 +864,6 @@
  - **Shahid**
  - **ShahidShow**
  - **Shared**: shared.sx
- - **ShemarooMe**
  - **ShowRoomLive**
  - **simplecast**
  - **simplecast:episode**
@@ -912,7 +883,6 @@
  - **Snotr**
  - **Sohu**
  - **SonyLIV**
- - **SonyLIVSeries**
  - **soundcloud**
  - **soundcloud:playlist**
  - **soundcloud:search**: Soundcloud search
@@ -991,7 +961,6 @@
  - **Telecinco**: telecinco.es, cuatro.com and mediaset.es
  - **Telegraaf**
  - **TeleMB**
- - **Telemundo**
  - **TeleQuebec**
  - **TeleQuebecEmission**
  - **TeleQuebecLive**
@@ -1013,6 +982,7 @@
  - **ThisAmericanLife**
  - **ThisAV**
  - **ThisOldHouse**
+ - **ThisVid**
  - **TikTok**
  - **tinypic**: tinypic.com videos
  - **TMZ**
@@ -1031,7 +1001,6 @@
  - **TruTV**
  - **Tube8**
  - **TubiTv**
- - **TubiTvShow**
  - **Tumblr**
  - **tunein:clip**
  - **tunein:program**
@@ -1074,8 +1043,6 @@
  - **TVPlayHome**
  - **Tweakers**
  - **TwitCasting**
- - **TwitCastingLive**
- - **TwitCastingUser**
  - **twitch:clips**
  - **twitch:stream**
  - **twitch:vod**
@@ -1093,7 +1060,6 @@
  - **UDNEmbed**: 聯合影音
  - **UFCArabia**
  - **UFCTV**
- - **ukcolumn**
  - **UKTVPlay**
  - **umg:de**: Universal Music Deutschland
  - **Unistra**
@@ -1109,7 +1075,6 @@
  - **ustream:channel**
  - **ustudio**
  - **ustudio:embed**
- - **Utreon**
  - **Varzesh3**
  - **Vbox7**
  - **VeeHD**
@@ -1127,7 +1092,6 @@
  - **Vidbit**
  - **Viddler**
  - **Videa**
- - **video.arnes.si**: Arnes Video
  - **video.google:search**: Google Video search
  - **video.sky.it**
  - **video.sky.it:live**
@@ -1138,8 +1102,6 @@
  - **videomore:video**
  - **VideoPress**
  - **Vidio**
- - **VidioLive**
- - **VidioPremier**
  - **VidLii**
  - **vidme**
  - **vidme:user**
@@ -1179,7 +1141,6 @@
  - **VODPlatform**
  - **VoiceRepublic**
  - **Voot**
- - **VootSeries**
  - **VoxMedia**
  - **VoxMediaVolume**
  - **vpro**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
@@ -1209,7 +1170,6 @@
  - **wdr:mobile**
  - **WDRElefant**
  - **WDRPage**
- - **web.archive:youtube**: web.archive.org saved youtube videos
  - **Webcaster**
  - **WebcasterFeed**
  - **WebOfStories**
@@ -1217,7 +1177,6 @@
  - **Weibo**
  - **WeiboMobile**
  - **WeiqiTV**: WQTV
- - **whowatch**
  - **WimTV**
  - **Wistia**
  - **WistiaPlaylist**
@@ -1228,7 +1187,7 @@
  - **WWE**
  - **XBef**
  - **XboxClips**
- - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, WolfStream, XVideoSharing
+ - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, XVideoSharing
  - **XHamster**
  - **XHamsterEmbed**
  - **XHamsterUser**
@@ -1292,6 +1251,5 @@
  - **zee5:series**
  - **Zhihu**
  - **zingmp3**: mp3.zing.vn
- - **zingmp3:album**
  - **zoom**
  - **Zype**

test/helper.py

@@ -22,14 +22,6 @@ from yt_dlp.utils import (
 )
 
 
-if "pytest" in sys.modules:
-    import pytest
-    is_download_test = pytest.mark.download
-else:
-    def is_download_test(testClass):
-        return testClass
-
-
 def get_params(override=None):
     PARAMETERS_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)),
                                    "parameters.json")
@@ -198,10 +190,7 @@ def expect_info_dict(self, got_dict, expected_dict):
     expect_dict(self, got_dict, expected_dict)
     # Check for the presence of mandatory fields
     if got_dict.get('_type') not in ('playlist', 'multi_video'):
-        mandatory_fields = ['id', 'title']
-        if expected_dict.get('ext'):
-            mandatory_fields.extend(('url', 'ext'))
-        for key in mandatory_fields:
+        for key in ('id', 'url', 'title', 'ext'):
             self.assertTrue(got_dict.get(key), 'Missing mandatory field %s' % key)
     # Check for mandatory fields that are automatically set by YoutubeDL
     for key in ['webpage_url', 'extractor', 'extractor_key']:

test/parameters.json

@@ -1,41 +1,41 @@
 {
-    "check_formats": false,
-    "consoletitle": false,
-    "continuedl": true,
-    "forcedescription": false,
-    "forcefilename": false,
-    "forceformat": false,
-    "forcethumbnail": false,
-    "forcetitle": false,
-    "forceurl": false,
+    "consoletitle": false, 
+    "continuedl": true, 
+    "forcedescription": false, 
+    "forcefilename": false, 
+    "forceformat": false, 
+    "forcethumbnail": false, 
+    "forcetitle": false, 
+    "forceurl": false, 
     "force_write_download_archive": false,
     "format": "best",
-    "ignoreerrors": false,
-    "listformats": null,
-    "logtostderr": false,
-    "matchtitle": null,
-    "max_downloads": null,
-    "overwrites": null,
-    "nopart": false,
-    "noprogress": false,
-    "outtmpl": "%(id)s.%(ext)s",
-    "password": null,
-    "playliststart": 1,
-    "prefer_free_formats": false,
-    "quiet": false,
-    "ratelimit": null,
-    "rejecttitle": null,
-    "retries": 10,
-    "simulate": false,
-    "subtitleslang": null,
+    "ignoreerrors": false, 
+    "listformats": null, 
+    "logtostderr": false, 
+    "matchtitle": null, 
+    "max_downloads": null, 
+    "overwrites": null, 
+    "nopart": false, 
+    "noprogress": false, 
+    "outtmpl": "%(id)s.%(ext)s", 
+    "password": null, 
+    "playlistend": -1, 
+    "playliststart": 1, 
+    "prefer_free_formats": false, 
+    "quiet": false, 
+    "ratelimit": null, 
+    "rejecttitle": null, 
+    "retries": 10, 
+    "simulate": false, 
+    "subtitleslang": null, 
     "subtitlesformat": "best",
-    "test": true,
-    "updatetime": true,
-    "usenetrc": false,
-    "username": null,
-    "verbose": true,
-    "writedescription": false,
-    "writeinfojson": true,
+    "test": true, 
+    "updatetime": true, 
+    "usenetrc": false, 
+    "username": null, 
+    "verbose": true, 
+    "writedescription": false, 
+    "writeinfojson": true, 
     "writeannotations": false,
     "writelink": false,
     "writeurllink": false,

test/test_InfoExtractor.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 
@@ -35,13 +35,13 @@ class InfoExtractorTestRequestHandler(compat_http_server.BaseHTTPRequestHandler)
             assert False
 
 
-class DummyIE(InfoExtractor):
+class TestIE(InfoExtractor):
     pass
 
 
 class TestInfoExtractor(unittest.TestCase):
     def setUp(self):
-        self.ie = DummyIE(FakeYDL())
+        self.ie = TestIE(FakeYDL())
 
     def test_ie_key(self):
         self.assertEqual(get_info_extractor(YoutubeIE.ie_key()), YoutubeIE)
@@ -450,7 +450,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': 'aud2-English',
@@ -458,7 +458,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': 'aud3-English',
@@ -466,14 +466,14 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'language': 'en',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'audio_ext': 'mp4',
                 }, {
                     'format_id': '530',
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -482,7 +482,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -491,7 +491,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 480,
                     'height': 270,
                     'vcodec': 'avc1.640015',
@@ -500,7 +500,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -509,7 +509,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -518,7 +518,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v3/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 640,
                     'height': 360,
                     'vcodec': 'avc1.64001e',
@@ -527,7 +527,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -536,7 +536,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -545,7 +545,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v4/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 768,
                     'height': 432,
                     'vcodec': 'avc1.64001e',
@@ -554,7 +554,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -563,7 +563,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -572,7 +572,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v5/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 960,
                     'height': 540,
                     'vcodec': 'avc1.640020',
@@ -581,7 +581,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -590,7 +590,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -599,7 +599,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v6/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1280,
                     'height': 720,
                     'vcodec': 'avc1.640020',
@@ -608,7 +608,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -617,7 +617,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -626,7 +626,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v7/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -635,7 +635,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -644,7 +644,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -653,7 +653,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v8/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -662,7 +662,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -671,7 +671,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
@@ -680,190 +680,21 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v9/prog_index.m3u8',
                     'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/master.m3u8',
                     'ext': 'mp4',
-                    'protocol': 'm3u8_native',
+                    'protocol': 'm3u8',
                     'width': 1920,
                     'height': 1080,
                     'vcodec': 'avc1.64002a',
-                }],
-                {}
-            ),
-            (
-                'bipbop_16x9',
-                'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                [{
-                    'format_id': 'bipbop_audio-BipBop Audio 2',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/alternate_audio_aac/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'language': 'eng',
-                    'ext': 'mp4',
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'vcodec': 'none',
-                    'audio_ext': 'mp4',
-                    'video_ext': 'none',
-                }, {
-                    'format_id': '41',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear0/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'tbr': 41.457,
-                    'ext': 'mp4',
-                    'fps': None,
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'vcodec': 'none',
-                    'acodec': 'mp4a.40.2',
-                    'audio_ext': 'mp4',
-                    'video_ext': 'none',
-                    'abr': 41.457,
-                }, {
-                    'format_id': '263',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear1/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'tbr': 263.851,
-                    'ext': 'mp4',
-                    'fps': None,
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'width': 416,
-                    'height': 234,
-                    'vcodec': 'avc1.4d400d',
-                    'acodec': 'mp4a.40.2',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 263.851,
-                    'abr': 0,
-                }, {
-                    'format_id': '577',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear2/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'tbr': 577.61,
-                    'ext': 'mp4',
-                    'fps': None,
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'width': 640,
-                    'height': 360,
-                    'vcodec': 'avc1.4d401e',
-                    'acodec': 'mp4a.40.2',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 577.61,
-                    'abr': 0,
-                }, {
-                    'format_id': '915',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear3/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'tbr': 915.905,
-                    'ext': 'mp4',
-                    'fps': None,
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'width': 960,
-                    'height': 540,
-                    'vcodec': 'avc1.4d401f',
-                    'acodec': 'mp4a.40.2',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 915.905,
-                    'abr': 0,
-                }, {
-                    'format_id': '1030',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear4/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'tbr': 1030.138,
-                    'ext': 'mp4',
-                    'fps': None,
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'width': 1280,
-                    'height': 720,
-                    'vcodec': 'avc1.4d401f',
-                    'acodec': 'mp4a.40.2',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 1030.138,
-                    'abr': 0,
-                }, {
-                    'format_id': '1924',
-                    'format_index': None,
-                    'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/gear5/prog_index.m3u8',
-                    'manifest_url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8',
-                    'tbr': 1924.009,
-                    'ext': 'mp4',
-                    'fps': None,
-                    'protocol': 'm3u8_native',
-                    'preference': None,
-                    'quality': None,
-                    'width': 1920,
-                    'height': 1080,
-                    'vcodec': 'avc1.4d401f',
-                    'acodec': 'mp4a.40.2',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 1924.009,
-                    'abr': 0,
-                }],
-                {
-                    'en': [{
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }, {
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng_forced/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }],
-                    'fr': [{
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }, {
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra_forced/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }],
-                    'es': [{
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }, {
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa_forced/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }],
-                    'ja': [{
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }, {
-                        'url': 'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn_forced/prog_index.m3u8',
-                        'ext': 'vtt',
-                        'protocol': 'm3u8_native'
-                    }],
-                }
+                }]
             ),
         ]
 
-        for m3u8_file, m3u8_url, expected_formats, expected_subs in _TEST_CASES:
+        for m3u8_file, m3u8_url, expected_formats in _TEST_CASES:
             with io.open('./test/testdata/m3u8/%s.m3u8' % m3u8_file,
                          mode='r', encoding='utf-8') as f:
-                formats, subs = self.ie._parse_m3u8_formats_and_subtitles(
+                formats = self.ie._parse_m3u8_formats(
                     f.read(), m3u8_url, ext='mp4')
                 self.ie._sort_formats(formats)
                 expect_value(self, formats, expected_formats, None)
-                expect_value(self, subs, expected_subs, None)
 
     def test_parse_mpd_formats(self):
         _TEST_CASES = [
@@ -949,8 +780,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'tbr': 5997.485,
                     'width': 1920,
                     'height': 1080,
-                }],
-                {},
+                }]
             ), (
                 # https://github.com/ytdl-org/youtube-dl/pull/14844
                 'urls_only',
@@ -1033,8 +863,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'tbr': 4400,
                     'width': 1920,
                     'height': 1080,
-                }],
-                {},
+                }]
             ), (
                 # https://github.com/ytdl-org/youtube-dl/issues/20346
                 # Media considered unfragmented even though it contains
@@ -1080,328 +909,18 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                     'width': 360,
                     'height': 360,
                     'fps': 30,
-                }],
-                {},
-            ), (
-                'subtitles',
-                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/',
-                [{
-                    'format_id': 'audio=128001',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'ext': 'm4a',
-                    'tbr': 128.001,
-                    'asr': 48000,
-                    'format_note': 'DASH audio',
-                    'container': 'm4a_dash',
-                    'vcodec': 'none',
-                    'acodec': 'mp4a.40.2',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                    'protocol': 'http_dash_segments',
-                    'audio_ext': 'm4a',
-                    'video_ext': 'none',
-                    'abr': 128.001,
-                }, {
-                    'format_id': 'video=100000',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'ext': 'mp4',
-                    'width': 336,
-                    'height': 144,
-                    'tbr': 100,
-                    'format_note': 'DASH video',
-                    'container': 'mp4_dash',
-                    'vcodec': 'avc1.4D401F',
-                    'acodec': 'none',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                    'protocol': 'http_dash_segments',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 100,
-                }, {
-                    'format_id': 'video=326000',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'ext': 'mp4',
-                    'width': 562,
-                    'height': 240,
-                    'tbr': 326,
-                    'format_note': 'DASH video',
-                    'container': 'mp4_dash',
-                    'vcodec': 'avc1.4D401F',
-                    'acodec': 'none',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                    'protocol': 'http_dash_segments',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 326,
-                }, {
-                    'format_id': 'video=698000',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'ext': 'mp4',
-                    'width': 844,
-                    'height': 360,
-                    'tbr': 698,
-                    'format_note': 'DASH video',
-                    'container': 'mp4_dash',
-                    'vcodec': 'avc1.4D401F',
-                    'acodec': 'none',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                    'protocol': 'http_dash_segments',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 698,
-                }, {
-                    'format_id': 'video=1493000',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'ext': 'mp4',
-                    'width': 1126,
-                    'height': 480,
-                    'tbr': 1493,
-                    'format_note': 'DASH video',
-                    'container': 'mp4_dash',
-                    'vcodec': 'avc1.4D401F',
-                    'acodec': 'none',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                    'protocol': 'http_dash_segments',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 1493,
-                }, {
-                    'format_id': 'video=4482000',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'ext': 'mp4',
-                    'width': 1688,
-                    'height': 720,
-                    'tbr': 4482,
-                    'format_note': 'DASH video',
-                    'container': 'mp4_dash',
-                    'vcodec': 'avc1.4D401F',
-                    'acodec': 'none',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                    'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                    'protocol': 'http_dash_segments',
-                    'video_ext': 'mp4',
-                    'audio_ext': 'none',
-                    'vbr': 4482,
-                }],
-                {
-                    'en': [
-                        {
-                            'ext': 'mp4',
-                            'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                            'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/manifest.mpd',
-                            'fragment_base_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/dash/',
-                            'protocol': 'http_dash_segments',
-                        }
-                    ]
-                },
+                }]
             )
         ]
 
-        for mpd_file, mpd_url, mpd_base_url, expected_formats, expected_subtitles in _TEST_CASES:
+        for mpd_file, mpd_url, mpd_base_url, expected_formats in _TEST_CASES:
             with io.open('./test/testdata/mpd/%s.mpd' % mpd_file,
                          mode='r', encoding='utf-8') as f:
-                formats, subtitles = self.ie._parse_mpd_formats_and_subtitles(
+                formats = self.ie._parse_mpd_formats(
                     compat_etree_fromstring(f.read().encode('utf-8')),
                     mpd_base_url=mpd_base_url, mpd_url=mpd_url)
                 self.ie._sort_formats(formats)
                 expect_value(self, formats, expected_formats, None)
-                expect_value(self, subtitles, expected_subtitles, None)
-
-    def test_parse_ism_formats(self):
-        _TEST_CASES = [
-            (
-                'sintel',
-                'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                [{
-                    'format_id': 'audio-128',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'ext': 'isma',
-                    'tbr': 128,
-                    'asr': 48000,
-                    'vcodec': 'none',
-                    'acodec': 'AACL',
-                    'protocol': 'ism',
-                    '_download_params': {
-                        'stream_type': 'audio',
-                        'duration': 8880746666,
-                        'timescale': 10000000,
-                        'width': 0,
-                        'height': 0,
-                        'fourcc': 'AACL',
-                        'codec_private_data': '1190',
-                        'sampling_rate': 48000,
-                        'channels': 2,
-                        'bits_per_sample': 16,
-                        'nal_unit_length_field': 4
-                    },
-                    'audio_ext': 'isma',
-                    'video_ext': 'none',
-                    'abr': 128,
-                }, {
-                    'format_id': 'video-100',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'ext': 'ismv',
-                    'width': 336,
-                    'height': 144,
-                    'tbr': 100,
-                    'vcodec': 'AVC1',
-                    'acodec': 'none',
-                    'protocol': 'ism',
-                    '_download_params': {
-                        'stream_type': 'video',
-                        'duration': 8880746666,
-                        'timescale': 10000000,
-                        'width': 336,
-                        'height': 144,
-                        'fourcc': 'AVC1',
-                        'codec_private_data': '00000001674D401FDA0544EFFC2D002CBC40000003004000000C03C60CA80000000168EF32C8',
-                        'channels': 2,
-                        'bits_per_sample': 16,
-                        'nal_unit_length_field': 4
-                    },
-                    'video_ext': 'ismv',
-                    'audio_ext': 'none',
-                    'vbr': 100,
-                }, {
-                    'format_id': 'video-326',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'ext': 'ismv',
-                    'width': 562,
-                    'height': 240,
-                    'tbr': 326,
-                    'vcodec': 'AVC1',
-                    'acodec': 'none',
-                    'protocol': 'ism',
-                    '_download_params': {
-                        'stream_type': 'video',
-                        'duration': 8880746666,
-                        'timescale': 10000000,
-                        'width': 562,
-                        'height': 240,
-                        'fourcc': 'AVC1',
-                        'codec_private_data': '00000001674D401FDA0241FE23FFC3BC83BA44000003000400000300C03C60CA800000000168EF32C8',
-                        'channels': 2,
-                        'bits_per_sample': 16,
-                        'nal_unit_length_field': 4
-                    },
-                    'video_ext': 'ismv',
-                    'audio_ext': 'none',
-                    'vbr': 326,
-                }, {
-                    'format_id': 'video-698',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'ext': 'ismv',
-                    'width': 844,
-                    'height': 360,
-                    'tbr': 698,
-                    'vcodec': 'AVC1',
-                    'acodec': 'none',
-                    'protocol': 'ism',
-                    '_download_params': {
-                        'stream_type': 'video',
-                        'duration': 8880746666,
-                        'timescale': 10000000,
-                        'width': 844,
-                        'height': 360,
-                        'fourcc': 'AVC1',
-                        'codec_private_data': '00000001674D401FDA0350BFB97FF06AF06AD1000003000100000300300F1832A00000000168EF32C8',
-                        'channels': 2,
-                        'bits_per_sample': 16,
-                        'nal_unit_length_field': 4
-                    },
-                    'video_ext': 'ismv',
-                    'audio_ext': 'none',
-                    'vbr': 698,
-                }, {
-                    'format_id': 'video-1493',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'ext': 'ismv',
-                    'width': 1126,
-                    'height': 480,
-                    'tbr': 1493,
-                    'vcodec': 'AVC1',
-                    'acodec': 'none',
-                    'protocol': 'ism',
-                    '_download_params': {
-                        'stream_type': 'video',
-                        'duration': 8880746666,
-                        'timescale': 10000000,
-                        'width': 1126,
-                        'height': 480,
-                        'fourcc': 'AVC1',
-                        'codec_private_data': '00000001674D401FDA011C3DE6FFF0D890D871000003000100000300300F1832A00000000168EF32C8',
-                        'channels': 2,
-                        'bits_per_sample': 16,
-                        'nal_unit_length_field': 4
-                    },
-                    'video_ext': 'ismv',
-                    'audio_ext': 'none',
-                    'vbr': 1493,
-                }, {
-                    'format_id': 'video-4482',
-                    'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                    'ext': 'ismv',
-                    'width': 1688,
-                    'height': 720,
-                    'tbr': 4482,
-                    'vcodec': 'AVC1',
-                    'acodec': 'none',
-                    'protocol': 'ism',
-                    '_download_params': {
-                        'stream_type': 'video',
-                        'duration': 8880746666,
-                        'timescale': 10000000,
-                        'width': 1688,
-                        'height': 720,
-                        'fourcc': 'AVC1',
-                        'codec_private_data': '00000001674D401FDA01A816F97FFC1ABC1AB440000003004000000C03C60CA80000000168EF32C8',
-                        'channels': 2,
-                        'bits_per_sample': 16,
-                        'nal_unit_length_field': 4
-                    },
-                    'video_ext': 'ismv',
-                    'audio_ext': 'none',
-                    'vbr': 4482,
-                }],
-                {
-                    'eng': [
-                        {
-                            'ext': 'ismt',
-                            'protocol': 'ism',
-                            'url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                            'manifest_url': 'https://sdn-global-streaming-cache-3qsdn.akamaized.net/stream/3144/files/17/07/672975/3144-kZT4LWMQw6Rh7Kpd.ism/Manifest',
-                            '_download_params': {
-                                'stream_type': 'text',
-                                'duration': 8880746666,
-                                'timescale': 10000000,
-                                'fourcc': 'TTML',
-                                'codec_private_data': ''
-                            }
-                        }
-                    ]
-                },
-            ),
-        ]
-
-        for ism_file, ism_url, expected_formats, expected_subtitles in _TEST_CASES:
-            with io.open('./test/testdata/ism/%s.Manifest' % ism_file,
-                         mode='r', encoding='utf-8') as f:
-                formats, subtitles = self.ie._parse_ism_formats_and_subtitles(
-                    compat_etree_fromstring(f.read().encode('utf-8')), ism_url=ism_url)
-                self.ie._sort_formats(formats)
-                expect_value(self, formats, expected_formats, None)
-                expect_value(self, subtitles, expected_subtitles, None)
 
     def test_parse_f4m_formats(self):
         _TEST_CASES = [

test/test_YoutubeDL.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -10,15 +10,14 @@ import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
 import copy
-import json
 
 from test.helper import FakeYDL, assertRegexpMatches
 from yt_dlp import YoutubeDL
-from yt_dlp.compat import compat_os_name, compat_setenv, compat_str, compat_urllib_error
+from yt_dlp.compat import compat_str, compat_urllib_error
 from yt_dlp.extractor import YoutubeIE
 from yt_dlp.extractor.common import InfoExtractor
 from yt_dlp.postprocessor.common import PostProcessor
-from yt_dlp.utils import ExtractorError, int_or_none, match_filter_func
+from yt_dlp.utils import ExtractorError, match_filter_func
 
 TEST_URL = 'http://localhost/sample.mp4'
 
@@ -30,15 +29,11 @@ class YDL(FakeYDL):
         self.msgs = []
 
     def process_info(self, info_dict):
-        info_dict.pop('__original_infodict', None)
         self.downloaded_info_dicts.append(info_dict)
 
     def to_screen(self, msg):
         self.msgs.append(msg)
 
-    def dl(self, *args, **kwargs):
-        assert False, 'Downloader must not be invoked for test_YoutubeDL'
-
 
 def _make_result(formats, **kwargs):
     res = {
@@ -121,24 +116,35 @@ class TestFormatSelection(unittest.TestCase):
         ]
         info_dict = _make_result(formats)
 
-        def test(inp, *expected, multi=False):
-            ydl = YDL({
-                'format': inp,
-                'allow_multiple_video_streams': multi,
-                'allow_multiple_audio_streams': multi,
-            })
-            ydl.process_ie_result(info_dict.copy())
-            downloaded = map(lambda x: x['format_id'], ydl.downloaded_info_dicts)
-            self.assertEqual(list(downloaded), list(expected))
+        ydl = YDL({'format': '20/47'})
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], '47')
 
-        test('20/47', '47')
-        test('20/71/worst', '35')
-        test(None, '2')
-        test('webm/mp4', '47')
-        test('3gp/40/mp4', '35')
-        test('example-with-dashes', 'example-with-dashes')
-        test('all', '35', 'example-with-dashes', '45', '47', '2')  # Order doesn't actually matter for this
-        test('mergeall', '2+47+45+example-with-dashes+35', multi=True)
+        ydl = YDL({'format': '20/71/worst'})
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], '35')
+
+        ydl = YDL()
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], '2')
+
+        ydl = YDL({'format': 'webm/mp4'})
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], '47')
+
+        ydl = YDL({'format': '3gp/40/mp4'})
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], '35')
+
+        ydl = YDL({'format': 'example-with-dashes'})
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], 'example-with-dashes')
 
     def test_format_selection_audio(self):
         formats = [
@@ -305,8 +311,8 @@ class TestFormatSelection(unittest.TestCase):
         self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
 
     def test_youtube_format_selection(self):
-        # FIXME: Rewrite in accordance with the new format sorting options
         return
+        # disabled for now - this needs some changes
 
         order = [
             '38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '17', '36', '13',
@@ -454,13 +460,14 @@ class TestFormatSelection(unittest.TestCase):
 
     def test_invalid_format_specs(self):
         def assert_syntax_error(format_spec):
-            self.assertRaises(SyntaxError, YDL, {'format': format_spec})
+            ydl = YDL({'format': format_spec})
+            info_dict = _make_result([{'format_id': 'foo', 'url': TEST_URL}])
+            self.assertRaises(SyntaxError, ydl.process_ie_result, info_dict)
 
         assert_syntax_error('bestvideo,,best')
         assert_syntax_error('+bestaudio')
         assert_syntax_error('bestvideo+')
         assert_syntax_error('/')
-        assert_syntax_error('[720<height]')
 
     def test_format_filtering(self):
         formats = [
@@ -594,26 +601,6 @@ class TestYoutubeDL(unittest.TestCase):
         self.assertTrue(subs)
         self.assertEqual(set(subs.keys()), set(['es', 'fr']))
 
-        result = get_info({'writesubtitles': True, 'subtitleslangs': ['all', '-en']})
-        subs = result['requested_subtitles']
-        self.assertTrue(subs)
-        self.assertEqual(set(subs.keys()), set(['es', 'fr']))
-
-        result = get_info({'writesubtitles': True, 'subtitleslangs': ['en', 'fr', '-en']})
-        subs = result['requested_subtitles']
-        self.assertTrue(subs)
-        self.assertEqual(set(subs.keys()), set(['fr']))
-
-        result = get_info({'writesubtitles': True, 'subtitleslangs': ['-en', 'en']})
-        subs = result['requested_subtitles']
-        self.assertTrue(subs)
-        self.assertEqual(set(subs.keys()), set(['en']))
-
-        result = get_info({'writesubtitles': True, 'subtitleslangs': ['e.+']})
-        subs = result['requested_subtitles']
-        self.assertTrue(subs)
-        self.assertEqual(set(subs.keys()), set(['es', 'en']))
-
         result = get_info({'writesubtitles': True, 'writeautomaticsub': True, 'subtitleslangs': ['es', 'pt']})
         subs = result['requested_subtitles']
         self.assertTrue(subs)
@@ -640,155 +627,47 @@ class TestYoutubeDL(unittest.TestCase):
         self.assertEqual(test_dict['extractor'], 'Foo')
         self.assertEqual(test_dict['playlist'], 'funny videos')
 
-    outtmpl_info = {
-        'id': '1234',
-        'ext': 'mp4',
-        'width': None,
-        'height': 1080,
-        'title1': '$PATH',
-        'title2': '%PATH%',
-        'title3': 'foo/bar\\test',
-        'title4': 'foo "bar" test',
-        'timestamp': 1618488000,
-        'duration': 100000,
-        'playlist_index': 1,
-        '_last_playlist_index': 100,
-        'n_entries': 10,
-        'formats': [{'id': 'id1'}, {'id': 'id2'}, {'id': 'id3'}]
-    }
+    def test_prepare_filename(self):
+        info = {
+            'id': '1234',
+            'ext': 'mp4',
+            'width': None,
+            'height': 1080,
+            'title1': '$PATH',
+            'title2': '%PATH%',
+        }
 
-    def test_prepare_outtmpl_and_filename(self):
-        def test(tmpl, expected, *, info=None, **params):
-            params['outtmpl'] = tmpl
+        def fname(templ, na_placeholder='NA'):
+            params = {'outtmpl': templ}
+            if na_placeholder != 'NA':
+                params['outtmpl_na_placeholder'] = na_placeholder
             ydl = YoutubeDL(params)
-            ydl._num_downloads = 1
-            self.assertEqual(ydl.validate_outtmpl(tmpl), None)
-
-            outtmpl, tmpl_dict = ydl.prepare_outtmpl(tmpl, info or self.outtmpl_info)
-            out = ydl.escape_outtmpl(outtmpl) % tmpl_dict
-            fname = ydl.prepare_filename(info or self.outtmpl_info)
-
-            if callable(expected):
-                self.assertTrue(expected(out))
-                self.assertTrue(expected(fname))
-            elif isinstance(expected, str):
-                self.assertEqual(out, expected)
-                self.assertEqual(fname, expected)
-            else:
-                self.assertEqual(out, expected[0])
-                self.assertEqual(fname, expected[1])
-
-        # Auto-generated fields
-        test('%(id)s.%(ext)s', '1234.mp4')
-        test('%(duration_string)s', ('27:46:40', '27-46-40'))
-        test('%(epoch)d', int_or_none)
-        test('%(resolution)s', '1080p')
-        test('%(playlist_index)s', '001')
-        test('%(autonumber)s', '00001')
-        test('%(autonumber+2)03d', '005', autonumber_start=3)
-        test('%(autonumber)s', '001', autonumber_size=3)
-
-        # Escaping %
-        test('%', '%')
-        test('%%', '%')
-        test('%%%%', '%%')
-        test('%s', '%s')
-        test('%%%s', '%%s')
-        test('%d', '%d')
-        test('%abc%', '%abc%')
-        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
-        test('%%%(height)s', '%1080')
-        test('%(width)06d.%(ext)s', 'NA.mp4')
-        test('%(width)06d.%%(ext)s', 'NA.%(ext)s')
-        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
-
-        # ID sanitization
-        test('%(id)s', '_abcd', info={'id': '_abcd'})
-        test('%(some_id)s', '_abcd', info={'some_id': '_abcd'})
-        test('%(formats.0.id)s', '_abcd', info={'formats': [{'id': '_abcd'}]})
-        test('%(id)s', '-abcd', info={'id': '-abcd'})
-        test('%(id)s', '.abcd', info={'id': '.abcd'})
-        test('%(id)s', 'ab__cd', info={'id': 'ab__cd'})
-        test('%(id)s', ('ab:cd', 'ab -cd'), info={'id': 'ab:cd'})
-
-        # Invalid templates
-        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%(title)'), ValueError))
-        test('%(invalid@tmpl|def)s', 'none', outtmpl_na_placeholder='none')
-        test('%()s', 'NA')
-
-        # NA placeholder
-        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(x|def)s-%(id)s.%(ext)s'
-        test(NA_TEST_OUTTMPL, 'NA-NA-def-1234.mp4')
-        test(NA_TEST_OUTTMPL, 'none-none-def-1234.mp4', outtmpl_na_placeholder='none')
-        test(NA_TEST_OUTTMPL, '--def-1234.mp4', outtmpl_na_placeholder='')
-
-        # String formatting
-        FMT_TEST_OUTTMPL = '%%(height)%s.%%(ext)s'
-        test(FMT_TEST_OUTTMPL % 's', '1080.mp4')
-        test(FMT_TEST_OUTTMPL % 'd', '1080.mp4')
-        test(FMT_TEST_OUTTMPL % '6d', '  1080.mp4')
-        test(FMT_TEST_OUTTMPL % '-6d', '1080  .mp4')
-        test(FMT_TEST_OUTTMPL % '06d', '001080.mp4')
-        test(FMT_TEST_OUTTMPL % ' 06d', ' 01080.mp4')
-        test(FMT_TEST_OUTTMPL % '   06d', ' 01080.mp4')
-        test(FMT_TEST_OUTTMPL % '0 6d', ' 01080.mp4')
-        test(FMT_TEST_OUTTMPL % '0   6d', ' 01080.mp4')
-        test(FMT_TEST_OUTTMPL % '   0   6d', ' 01080.mp4')
-
-        # Type casting
-        test('%(id)d', '1234')
-        test('%(height)c', '1')
-        test('%(ext)c', 'm')
-        test('%(id)d %(id)r', "1234 '1234'")
-        test('%(id)r %(height)r', "'1234' 1080")
-        test('%(ext)s-%(ext|def)d', 'mp4-def')
-        test('%(width|0)04d', '0000')
-        test('a%(width|)d', 'a', outtmpl_na_placeholder='none')
-
-        FORMATS = self.outtmpl_info['formats']
-        sanitize = lambda x: x.replace(':', ' -').replace('"', "'")
-
-        # Custom type casting
-        test('%(formats.:.id)l', 'id1, id2, id3')
-        test('%(ext)l', 'mp4')
-        test('%(formats.:.id) 15l', '  id1, id2, id3')
-        test('%(formats)j', (json.dumps(FORMATS), sanitize(json.dumps(FORMATS))))
-        if compat_os_name == 'nt':
-            test('%(title4)q', ('"foo \\"bar\\" test"', "'foo _'bar_' test'"))
-        else:
-            test('%(title4)q', ('\'foo "bar" test\'', "'foo 'bar' test'"))
-
-        # Internal formatting
-        test('%(timestamp-1000>%H-%M-%S)s', '11-43-20')
-        test('%(title|%)s %(title|%%)s', '% %%')
-        test('%(id+1-height+3)05d', '00158')
-        test('%(width+100)05d', 'NA')
-        test('%(formats.0) 15s', ('% 15s' % FORMATS[0], '% 15s' % sanitize(str(FORMATS[0]))))
-        test('%(formats.0)r', (repr(FORMATS[0]), sanitize(repr(FORMATS[0]))))
-        test('%(height.0)03d', '001')
-        test('%(-height.0)04d', '-001')
-        test('%(formats.-1.id)s', FORMATS[-1]['id'])
-        test('%(formats.0.id.-1)d', FORMATS[0]['id'][-1])
-        test('%(formats.3)s', 'NA')
-        test('%(formats.:2:-1)r', repr(FORMATS[:2:-1]))
-        test('%(formats.0.id.-1+id)f', '1235.000000')
-        test('%(formats.0.id.-1+formats.1.id.-1)d', '3')
-
-        # Empty filename
-        test('%(foo|)s-%(bar|)s.%(ext)s', '-.mp4')
-        # test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4'))  # fixme
-        # test('%(foo|)s', ('', '_'))  # fixme
-
-        # Environment variable expansion for prepare_filename
-        compat_setenv('__yt_dlp_var', 'expanded')
-        envvar = '%__yt_dlp_var%' if compat_os_name == 'nt' else '$__yt_dlp_var'
-        test(envvar, (envvar, 'expanded'))
-
-        # Path expansion and escaping
-        test('Hello %(title1)s', 'Hello $PATH')
-        test('Hello %(title2)s', 'Hello %PATH%')
-        test('%(title3)s', ('foo/bar\\test', 'foo_bar_test'))
-        test('folder/%(title3)s', ('folder/foo/bar\\test', 'folder%sfoo_bar_test' % os.path.sep))
+            return ydl.prepare_filename(info)
+        self.assertEqual(fname('%(id)s.%(ext)s'), '1234.mp4')
+        self.assertEqual(fname('%(id)s-%(width)s.%(ext)s'), '1234-NA.mp4')
+        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(id)s.%(ext)s'
+        # Replace missing fields with 'NA' by default
+        self.assertEqual(fname(NA_TEST_OUTTMPL), 'NA-NA-1234.mp4')
+        # Or by provided placeholder
+        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder='none'), 'none-none-1234.mp4')
+        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder=''), '--1234.mp4')
+        self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
+        self.assertEqual(fname('%(height)6d.%(ext)s'), '  1080.mp4')
+        self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080  .mp4')
+        self.assertEqual(fname('%(height)06d.%(ext)s'), '001080.mp4')
+        self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
+        self.assertEqual(fname('%(height)   06d.%(ext)s'), ' 01080.mp4')
+        self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
+        self.assertEqual(fname('%(height)0   6d.%(ext)s'), ' 01080.mp4')
+        self.assertEqual(fname('%(height)   0   6d.%(ext)s'), ' 01080.mp4')
+        self.assertEqual(fname('%%'), '%')
+        self.assertEqual(fname('%%%%'), '%%')
+        self.assertEqual(fname('%%(height)06d.%(ext)s'), '%(height)06d.mp4')
+        self.assertEqual(fname('%(width)06d.%(ext)s'), 'NA.mp4')
+        self.assertEqual(fname('%(width)06d.%%(ext)s'), 'NA.%(ext)s')
+        self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
+        self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
+        self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
 
     def test_format_note(self):
         ydl = YoutubeDL()
@@ -847,7 +726,7 @@ class TestYoutubeDL(unittest.TestCase):
             def process_info(self, info_dict):
                 super(YDL, self).process_info(info_dict)
 
-            def _match_entry(self, info_dict, incomplete=False):
+            def _match_entry(self, info_dict, incomplete):
                 res = super(FilterYDL, self)._match_entry(info_dict, incomplete)
                 if res is None:
                     self.downloaded_info_dicts.append(info_dict)

test/test_YoutubeDLCookieJar.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_aes.py

@@ -1,4 +1,5 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
+
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_age_restriction.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 # Allow direct execution
@@ -7,7 +7,8 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from test.helper import try_rm, is_download_test
+from test.helper import try_rm
+
 
 from yt_dlp import YoutubeDL
 
@@ -31,7 +32,6 @@ def _download_restricted(url, filename, age):
     return res
 
 
-@is_download_test
 class TestAgeRestriction(unittest.TestCase):
     def _assert_restricted(self, url, filename, age, old_age=None):
         self.assertTrue(_download_restricted(url, filename, old_age))

test/test_all_urls.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 
@@ -72,6 +72,15 @@ class TestAllURLsMatching(unittest.TestCase):
         self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
         self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])
 
+    def test_youtube_extract(self):
+        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
+        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
+        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
+
     def test_facebook_matching(self):
         self.assertTrue(FacebookIE.suitable('https://www.facebook.com/Shiniknoh#!/photo.php?v=10153317450565268'))
         self.assertTrue(FacebookIE.suitable('https://www.facebook.com/cindyweather?fref=ts#!/photo.php?v=10152183998945793'))

test/test_cache.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_compat.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals

test/test_cookies.py

@@ -1,96 +0,0 @@
-import unittest
-from datetime import datetime, timezone
-
-from yt_dlp import cookies
-from yt_dlp.cookies import (
-    CRYPTO_AVAILABLE,
-    LinuxChromeCookieDecryptor,
-    MacChromeCookieDecryptor,
-    WindowsChromeCookieDecryptor,
-    YDLLogger,
-    parse_safari_cookies,
-    pbkdf2_sha1,
-)
-
-
-class MonkeyPatch:
-    def __init__(self, module, temporary_values):
-        self._module = module
-        self._temporary_values = temporary_values
-        self._backup_values = {}
-
-    def __enter__(self):
-        for name, temp_value in self._temporary_values.items():
-            self._backup_values[name] = getattr(self._module, name)
-            setattr(self._module, name, temp_value)
-
-    def __exit__(self, exc_type, exc_val, exc_tb):
-        for name, backup_value in self._backup_values.items():
-            setattr(self._module, name, backup_value)
-
-
-class TestCookies(unittest.TestCase):
-    def test_chrome_cookie_decryptor_linux_derive_key(self):
-        key = LinuxChromeCookieDecryptor.derive_key(b'abc')
-        self.assertEqual(key, b'7\xa1\xec\xd4m\xfcA\xc7\xb19Z\xd0\x19\xdcM\x17')
-
-    def test_chrome_cookie_decryptor_mac_derive_key(self):
-        key = MacChromeCookieDecryptor.derive_key(b'abc')
-        self.assertEqual(key, b'Y\xe2\xc0\xd0P\xf6\xf4\xe1l\xc1\x8cQ\xcb|\xcdY')
-
-    def test_chrome_cookie_decryptor_linux_v10(self):
-        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b''}):
-            encrypted_value = b'v10\xccW%\xcd\xe6\xe6\x9fM" \xa7\xb0\xca\xe4\x07\xd6'
-            value = 'USD'
-            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
-            self.assertEqual(decryptor.decrypt(encrypted_value), value)
-
-    def test_chrome_cookie_decryptor_linux_v11(self):
-        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b'',
-                                   'KEYRING_AVAILABLE': True}):
-            encrypted_value = b'v11#\x81\x10>`w\x8f)\xc0\xb2\xc1\r\xf4\x1al\xdd\x93\xfd\xf8\xf8N\xf2\xa9\x83\xf1\xe9o\x0elVQd'
-            value = 'tz=Europe.London'
-            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
-            self.assertEqual(decryptor.decrypt(encrypted_value), value)
-
-    @unittest.skipIf(not CRYPTO_AVAILABLE, 'cryptography library not available')
-    def test_chrome_cookie_decryptor_windows_v10(self):
-        with MonkeyPatch(cookies, {
-            '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&'
-        }):
-            encrypted_value = b'v10T\xb8\xf3\xb8\x01\xa7TtcV\xfc\x88\xb8\xb8\xef\x05\xb5\xfd\x18\xc90\x009\xab\xb1\x893\x85)\x87\xe1\xa9-\xa3\xad='
-            value = '32101439'
-            decryptor = WindowsChromeCookieDecryptor('', YDLLogger())
-            self.assertEqual(decryptor.decrypt(encrypted_value), value)
-
-    def test_chrome_cookie_decryptor_mac_v10(self):
-        with MonkeyPatch(cookies, {'_get_mac_keyring_password': lambda *args, **kwargs: b'6eIDUdtKAacvlHwBVwvg/Q=='}):
-            encrypted_value = b'v10\xb3\xbe\xad\xa1[\x9fC\xa1\x98\xe0\x9a\x01\xd9\xcf\xbfc'
-            value = '2021-06-01-22'
-            decryptor = MacChromeCookieDecryptor('', YDLLogger())
-            self.assertEqual(decryptor.decrypt(encrypted_value), value)
-
-    def test_safari_cookie_parsing(self):
-        cookies = \
-            b'cook\x00\x00\x00\x01\x00\x00\x00i\x00\x00\x01\x00\x01\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00Y' \
-            b'\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x008\x00\x00\x00B\x00\x00\x00F\x00\x00\x00H' \
-            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\xa5>\xc3A\x00\x00\x80\xc3\x07:\xc3A' \
-            b'localhost\x00foo\x00/\x00test%20%3Bcookie\x00\x00\x00\x054\x07\x17 \x05\x00\x00\x00Kbplist00\xd1\x01' \
-            b'\x02_\x10\x18NSHTTPCookieAcceptPolicy\x10\x02\x08\x0b&\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00' \
-            b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00('
-
-        jar = parse_safari_cookies(cookies)
-        self.assertEqual(len(jar), 1)
-        cookie = list(jar)[0]
-        self.assertEqual(cookie.domain, 'localhost')
-        self.assertEqual(cookie.port, None)
-        self.assertEqual(cookie.path, '/')
-        self.assertEqual(cookie.name, 'foo')
-        self.assertEqual(cookie.value, 'test%20%3Bcookie')
-        self.assertFalse(cookie.secure)
-        expected_expiration = datetime(2021, 6, 18, 21, 39, 19, tzinfo=timezone.utc)
-        self.assertEqual(cookie.expires, int(expected_expiration.timestamp()))
-
-    def test_pbkdf2_sha1(self):
-        key = pbkdf2_sha1(b'peanuts', b' ' * 16, 1, 16)
-        self.assertEqual(key, b'g\xe1\x8e\x0fQ\x1c\x9b\xf3\xc9`!\xaa\x90\xd9\xd34')

test/test_download.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 
@@ -10,13 +10,12 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
 from test.helper import (
     assertGreaterEqual,
-    expect_info_dict,
     expect_warnings,
     get_params,
     gettestcases,
-    is_download_test,
-    report_warning,
+    expect_info_dict,
     try_rm,
+    report_warning,
 )
 
 
@@ -65,7 +64,6 @@ def _file_md5(fn):
 defs = gettestcases()
 
 
-@is_download_test
 class TestDownload(unittest.TestCase):
     # Parallel testing in nosetests. See
     # http://nose.readthedocs.org/en/latest/doc_tests/test_multiprocess/multiprocess.html
@@ -108,13 +106,8 @@ def generator(test_case, tname):
 
         for tc in test_cases:
             info_dict = tc.get('info_dict', {})
-            params = tc.get('params', {})
-            if not info_dict.get('id'):
-                raise Exception('Test definition incorrect. \'id\' key is not present')
-            elif not info_dict.get('ext'):
-                if params.get('skip_download') and params.get('ignore_no_formats_error'):
-                    continue
-                raise Exception('Test definition incorrect. The output file cannot be known. \'ext\' key is not present')
+            if not (info_dict.get('id') and info_dict.get('ext')):
+                raise Exception('Test definition incorrect. The output file cannot be known. Are both \'id\' and \'ext\' keys present?')
 
         if 'skip' in test_case:
             print_skipping(test_case['skip'])
@@ -128,7 +121,6 @@ def generator(test_case, tname):
         params['outtmpl'] = tname + '_' + params['outtmpl']
         if is_playlist and 'playlist' not in test_case:
             params.setdefault('extract_flat', 'in_playlist')
-            params.setdefault('playlistend', test_case.get('playlist_mincount'))
             params.setdefault('skip_download', True)
 
         ydl = YoutubeDL(params, auto_init=False)

test/test_downloader_http.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_execution.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -39,16 +39,6 @@ class TestExecution(unittest.TestCase):
         _, stderr = p.communicate()
         self.assertFalse(stderr)
 
-    def test_lazy_extractors(self):
-        try:
-            subprocess.check_call([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'], cwd=rootDir, stdout=_DEV_NULL)
-            subprocess.check_call([sys.executable, 'test/test_all_urls.py'], cwd=rootDir, stdout=_DEV_NULL)
-        finally:
-            try:
-                os.remove('yt_dlp/extractor/lazy_extractors.py')
-            except (IOError, OSError):
-                pass
-
 
 if __name__ == '__main__':
     unittest.main()

test/test_http.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 from __future__ import unicode_literals
 

test/test_iqiyi_sdk_interpreter.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 
@@ -8,7 +8,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from test.helper import FakeYDL, is_download_test
+from test.helper import FakeYDL
 from yt_dlp.extractor import IqiyiIE
 
 
@@ -31,7 +31,6 @@ class WarningLogger(object):
         pass
 
 
-@is_download_test
 class TestIqiyiSDKInterpreter(unittest.TestCase):
     def test_iqiyi_sdk_interpreter(self):
         '''

test/test_jsinterp.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 

test/test_overwrites.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 import os

test/test_post_hooks.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 
@@ -7,7 +7,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from test.helper import get_params, try_rm, is_download_test
+from test.helper import get_params, try_rm
 import yt_dlp.YoutubeDL
 from yt_dlp.utils import DownloadError
 
@@ -22,7 +22,6 @@ TEST_ID = 'gr51aVj-mLg'
 EXPECTED_NAME = 'gr51aVj-mLg'
 
 
-@is_download_test
 class TestPostHooks(unittest.TestCase):
     def setUp(self):
         self.stored_name_1 = None

test/test_postprocessors.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 
@@ -8,64 +8,16 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from yt_dlp import YoutubeDL
-from yt_dlp.compat import compat_shlex_quote
-from yt_dlp.postprocessor import (
-    ExecAfterDownloadPP,
-    FFmpegThumbnailsConvertorPP,
-    MetadataFromFieldPP,
-    MetadataFromTitlePP,
-)
+from yt_dlp.postprocessor import MetadataFromFieldPP, MetadataFromTitlePP
 
 
 class TestMetadataFromField(unittest.TestCase):
     def test_format_to_regex(self):
         pp = MetadataFromFieldPP(None, ['title:%(title)s - %(artist)s'])
-        self.assertEqual(pp._data[0]['regex'], r'(?P<title>.+)\ \-\ (?P<artist>.+)')
-
-    def test_field_to_outtmpl(self):
-        pp = MetadataFromFieldPP(None, ['title:%(title)s : %(artist)s'])
-        self.assertEqual(pp._data[0]['tmpl'], '%(title)s')
-
-    def test_in_out_seperation(self):
-        pp = MetadataFromFieldPP(None, ['%(title)s \\: %(artist)s:%(title)s : %(artist)s'])
-        self.assertEqual(pp._data[0]['in'], '%(title)s : %(artist)s')
-        self.assertEqual(pp._data[0]['out'], '%(title)s : %(artist)s')
+        self.assertEqual(pp._data[0]['regex'], r'(?P<title>[^\r\n]+)\ \-\ (?P<artist>[^\r\n]+)')
 
 
 class TestMetadataFromTitle(unittest.TestCase):
     def test_format_to_regex(self):
         pp = MetadataFromTitlePP(None, '%(title)s - %(artist)s')
-        self.assertEqual(pp._titleregex, r'(?P<title>.+)\ \-\ (?P<artist>.+)')
-
-
-class TestConvertThumbnail(unittest.TestCase):
-    def test_escaping(self):
-        pp = FFmpegThumbnailsConvertorPP()
-        if not pp.available:
-            print('Skipping: ffmpeg not found')
-            return
-
-        file = 'test/testdata/thumbnails/foo %d bar/foo_%d.{}'
-        tests = (('webp', 'png'), ('png', 'jpg'))
-
-        for inp, out in tests:
-            out_file = file.format(out)
-            if os.path.exists(out_file):
-                os.remove(out_file)
-            pp.convert_thumbnail(file.format(inp), out)
-            assert os.path.exists(out_file)
-
-        for _, out in tests:
-            os.remove(file.format(out))
-
-
-class TestExecAfterDownload(unittest.TestCase):
-    def test_parse_cmd(self):
-        pp = ExecAfterDownloadPP(YoutubeDL(), '')
-        info = {'filepath': 'file name'}
-        quoted_filepath = compat_shlex_quote(info['filepath'])
-
-        self.assertEqual(pp.parse_cmd('echo', info), 'echo %s' % quoted_filepath)
-        self.assertEqual(pp.parse_cmd('echo.{}', info), 'echo.%s' % quoted_filepath)
-        self.assertEqual(pp.parse_cmd('echo "%(filepath)s"', info), 'echo "%s"' % info['filepath'])
+        self.assertEqual(pp._titleregex, r'(?P<title>[^\r\n]+)\ \-\ (?P<artist>[^\r\n]+)')

test/test_socks.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 from __future__ import unicode_literals
 
@@ -14,7 +14,6 @@ import subprocess
 from test.helper import (
     FakeYDL,
     get_params,
-    is_download_test,
 )
 from yt_dlp.compat import (
     compat_str,
@@ -22,7 +21,6 @@ from yt_dlp.compat import (
 )
 
 
-@is_download_test
 class TestMultipleSocks(unittest.TestCase):
     @staticmethod
     def _check_params(attrs):
@@ -78,7 +76,6 @@ class TestMultipleSocks(unittest.TestCase):
             params['secondary_server_ip'])
 
 
-@is_download_test
 class TestSocks(unittest.TestCase):
     _SKIP_SOCKS_TEST = True
 

test/test_subtitles.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 # Allow direct execution
@@ -7,7 +7,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from test.helper import FakeYDL, md5, is_download_test
+from test.helper import FakeYDL, md5
 
 
 from yt_dlp.extractor import (
@@ -30,7 +30,6 @@ from yt_dlp.extractor import (
 )
 
 
-@is_download_test
 class BaseTestSubtitles(unittest.TestCase):
     url = None
     IE = None
@@ -56,7 +55,6 @@ class BaseTestSubtitles(unittest.TestCase):
         return dict((l, sub_info['data']) for l, sub_info in subtitles.items())
 
 
-@is_download_test
 class TestYoutubeSubtitles(BaseTestSubtitles):
     url = 'QRS8MkLhQmM'
     IE = YoutubeIE
@@ -113,7 +111,6 @@ class TestYoutubeSubtitles(BaseTestSubtitles):
         self.assertFalse(subtitles)
 
 
-@is_download_test
 class TestDailymotionSubtitles(BaseTestSubtitles):
     url = 'http://www.dailymotion.com/video/xczg00'
     IE = DailymotionIE
@@ -137,7 +134,6 @@ class TestDailymotionSubtitles(BaseTestSubtitles):
         self.assertFalse(subtitles)
 
 
-@is_download_test
 class TestTedSubtitles(BaseTestSubtitles):
     url = 'http://www.ted.com/talks/dan_dennett_on_our_consciousness.html'
     IE = TEDIE
@@ -153,7 +149,6 @@ class TestTedSubtitles(BaseTestSubtitles):
             self.assertTrue(subtitles.get(lang) is not None, 'Subtitles for \'%s\' not extracted' % lang)
 
 
-@is_download_test
 class TestVimeoSubtitles(BaseTestSubtitles):
     url = 'http://vimeo.com/76979871'
     IE = VimeoIE
@@ -175,7 +170,6 @@ class TestVimeoSubtitles(BaseTestSubtitles):
         self.assertFalse(subtitles)
 
 
-@is_download_test
 class TestWallaSubtitles(BaseTestSubtitles):
     url = 'http://vod.walla.co.il/movie/2705958/the-yes-men'
     IE = WallaIE
@@ -197,7 +191,6 @@ class TestWallaSubtitles(BaseTestSubtitles):
         self.assertFalse(subtitles)
 
 
-@is_download_test
 class TestCeskaTelevizeSubtitles(BaseTestSubtitles):
     url = 'http://www.ceskatelevize.cz/ivysilani/10600540290-u6-uzasny-svet-techniky'
     IE = CeskaTelevizeIE
@@ -219,7 +212,6 @@ class TestCeskaTelevizeSubtitles(BaseTestSubtitles):
         self.assertFalse(subtitles)
 
 
-@is_download_test
 class TestLyndaSubtitles(BaseTestSubtitles):
     url = 'http://www.lynda.com/Bootstrap-tutorials/Using-exercise-files/110885/114408-4.html'
     IE = LyndaIE
@@ -232,7 +224,6 @@ class TestLyndaSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['en']), '09bbe67222259bed60deaa26997d73a7')
 
 
-@is_download_test
 class TestNPOSubtitles(BaseTestSubtitles):
     url = 'http://www.npo.nl/nos-journaal/28-08-2014/POW_00722860'
     IE = NPOIE
@@ -245,7 +236,6 @@ class TestNPOSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['nl']), 'fc6435027572b63fb4ab143abd5ad3f4')
 
 
-@is_download_test
 class TestMTVSubtitles(BaseTestSubtitles):
     url = 'http://www.cc.com/video-clips/p63lk0/adam-devine-s-house-party-chasing-white-swans'
     IE = ComedyCentralIE
@@ -261,7 +251,6 @@ class TestMTVSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['en']), '78206b8d8a0cfa9da64dc026eea48961')
 
 
-@is_download_test
 class TestNRKSubtitles(BaseTestSubtitles):
     url = 'http://tv.nrk.no/serie/ikke-gjoer-dette-hjemme/DMPV73000411/sesong-2/episode-1'
     IE = NRKTVIE
@@ -274,7 +263,6 @@ class TestNRKSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['no']), '544fa917d3197fcbee64634559221cc2')
 
 
-@is_download_test
 class TestRaiPlaySubtitles(BaseTestSubtitles):
     IE = RaiPlayIE
 
@@ -295,7 +283,6 @@ class TestRaiPlaySubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['it']), '4b3264186fbb103508abe5311cfcb9cd')
 
 
-@is_download_test
 class TestVikiSubtitles(BaseTestSubtitles):
     url = 'http://www.viki.com/videos/1060846v-punch-episode-18'
     IE = VikiIE
@@ -308,7 +295,6 @@ class TestVikiSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['en']), '53cb083a5914b2d84ef1ab67b880d18a')
 
 
-@is_download_test
 class TestThePlatformSubtitles(BaseTestSubtitles):
     # from http://www.3playmedia.com/services-features/tools/integrations/theplatform/
     # (see http://theplatform.com/about/partners/type/subtitles-closed-captioning/)
@@ -323,7 +309,6 @@ class TestThePlatformSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['en']), '97e7670cbae3c4d26ae8bcc7fdd78d4b')
 
 
-@is_download_test
 class TestThePlatformFeedSubtitles(BaseTestSubtitles):
     url = 'http://feed.theplatform.com/f/7wvmTC/msnbc_video-p-test?form=json&pretty=true&range=-40&byGuid=n_hardball_5biden_140207'
     IE = ThePlatformFeedIE
@@ -336,7 +321,6 @@ class TestThePlatformFeedSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['en']), '48649a22e82b2da21c9a67a395eedade')
 
 
-@is_download_test
 class TestRtveSubtitles(BaseTestSubtitles):
     url = 'http://www.rtve.es/alacarta/videos/los-misterios-de-laura/misterios-laura-capitulo-32-misterio-del-numero-17-2-parte/2428621/'
     IE = RTVEALaCartaIE
@@ -351,7 +335,6 @@ class TestRtveSubtitles(BaseTestSubtitles):
         self.assertEqual(md5(subtitles['es']), '69e70cae2d40574fb7316f31d6eb7fca')
 
 
-@is_download_test
 class TestDemocracynowSubtitles(BaseTestSubtitles):
     url = 'http://www.democracynow.org/shows/2015/7/3'
     IE = DemocracynowIE

test/test_swfinterp.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 from __future__ import unicode_literals
 
 # Allow direct execution

test/test_update.py.disabled

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 from __future__ import unicode_literals
 

test/test_utils.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals
@@ -12,7 +12,6 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
 # Various small unit tests
 import io
-import itertools
 import json
 import xml.etree.ElementTree
 
@@ -24,7 +23,6 @@ from yt_dlp.utils import (
     clean_html,
     clean_podcast_url,
     date_from_str,
-    datetime_from_str,
     DateRange,
     detect_exe_version,
     determine_ext,
@@ -67,7 +65,6 @@ from yt_dlp.utils import (
     sanitize_filename,
     sanitize_path,
     sanitize_url,
-    sanitized_Request,
     expand_path,
     prepend_extension,
     replace_extension,
@@ -109,7 +106,6 @@ from yt_dlp.utils import (
     cli_bool_option,
     parse_codecs,
     iri_to_uri,
-    LazyList,
 )
 from yt_dlp.compat import (
     compat_chr,
@@ -128,7 +124,6 @@ class TestUtil(unittest.TestCase):
         self.assertTrue(timeconvert('bougrg') is None)
 
     def test_sanitize_filename(self):
-        self.assertEqual(sanitize_filename(''), '')
         self.assertEqual(sanitize_filename('abc'), 'abc')
         self.assertEqual(sanitize_filename('abc_d-e'), 'abc_d-e')
 
@@ -242,16 +237,6 @@ class TestUtil(unittest.TestCase):
         self.assertEqual(sanitize_url('httpss://foo.bar'), 'https://foo.bar')
         self.assertEqual(sanitize_url('rmtps://foo.bar'), 'rtmps://foo.bar')
         self.assertEqual(sanitize_url('https://foo.bar'), 'https://foo.bar')
-        self.assertEqual(sanitize_url('foo bar'), 'foo bar')
-
-    def test_extract_basic_auth(self):
-        auth_header = lambda url: sanitized_Request(url).get_header('Authorization')
-        self.assertFalse(auth_header('http://foo.bar'))
-        self.assertFalse(auth_header('http://:foo.bar'))
-        self.assertEqual(auth_header('http://@foo.bar'), 'Basic Og==')
-        self.assertEqual(auth_header('http://:pass@foo.bar'), 'Basic OnBhc3M=')
-        self.assertEqual(auth_header('http://user:@foo.bar'), 'Basic dXNlcjo=')
-        self.assertEqual(auth_header('http://user:pass@foo.bar'), 'Basic dXNlcjpwYXNz')
 
     def test_expand_path(self):
         def env(var):
@@ -326,18 +311,8 @@ class TestUtil(unittest.TestCase):
         self.assertEqual(date_from_str('yesterday'), date_from_str('now-1day'))
         self.assertEqual(date_from_str('now+7day'), date_from_str('now+1week'))
         self.assertEqual(date_from_str('now+14day'), date_from_str('now+2week'))
-        self.assertEqual(date_from_str('20200229+365day'), date_from_str('20200229+1year'))
-        self.assertEqual(date_from_str('20210131+28day'), date_from_str('20210131+1month'))
-
-    def test_datetime_from_str(self):
-        self.assertEqual(datetime_from_str('yesterday', precision='day'), datetime_from_str('now-1day', precision='auto'))
-        self.assertEqual(datetime_from_str('now+7day', precision='day'), datetime_from_str('now+1week', precision='auto'))
-        self.assertEqual(datetime_from_str('now+14day', precision='day'), datetime_from_str('now+2week', precision='auto'))
-        self.assertEqual(datetime_from_str('20200229+365day', precision='day'), datetime_from_str('20200229+1year', precision='auto'))
-        self.assertEqual(datetime_from_str('20210131+28day', precision='day'), datetime_from_str('20210131+1month', precision='auto'))
-        self.assertEqual(datetime_from_str('20210131+59day', precision='day'), datetime_from_str('20210131+2month', precision='auto'))
-        self.assertEqual(datetime_from_str('now+1day', precision='hour'), datetime_from_str('now+24hours', precision='auto'))
-        self.assertEqual(datetime_from_str('now+23hours', precision='hour'), datetime_from_str('now+23hours', precision='auto'))
+        self.assertEqual(date_from_str('now+365day'), date_from_str('now+1year'))
+        self.assertEqual(date_from_str('now+30day'), date_from_str('now+1month'))
 
     def test_daterange(self):
         _20century = DateRange("19000101", "20000101")
@@ -1054,9 +1029,6 @@ class TestUtil(unittest.TestCase):
         on = js_to_json('{ "040": "040" }')
         self.assertEqual(json.loads(on), {'040': '040'})
 
-        on = js_to_json('[1,//{},\n2]')
-        self.assertEqual(json.loads(on), [1, 2])
-
     def test_js_to_json_malformed(self):
         self.assertEqual(js_to_json('42a1'), '42"a1"')
         self.assertEqual(js_to_json('42a-1'), '42"a"-1')
@@ -1530,51 +1502,6 @@ Line 1
         self.assertEqual(clean_podcast_url('https://www.podtrac.com/pts/redirect.mp3/chtbl.com/track/5899E/traffic.megaphone.fm/HSW7835899191.mp3'), 'https://traffic.megaphone.fm/HSW7835899191.mp3')
         self.assertEqual(clean_podcast_url('https://play.podtrac.com/npr-344098539/edge1.pod.npr.org/anon.npr-podcasts/podcast/npr/waitwait/2020/10/20201003_waitwait_wwdtmpodcast201003-015621a5-f035-4eca-a9a1-7c118d90bc3c.mp3'), 'https://edge1.pod.npr.org/anon.npr-podcasts/podcast/npr/waitwait/2020/10/20201003_waitwait_wwdtmpodcast201003-015621a5-f035-4eca-a9a1-7c118d90bc3c.mp3')
 
-    def test_LazyList(self):
-        it = list(range(10))
-
-        self.assertEqual(list(LazyList(it)), it)
-        self.assertEqual(LazyList(it).exhaust(), it)
-        self.assertEqual(LazyList(it)[5], it[5])
-
-        self.assertEqual(LazyList(it)[5:], it[5:])
-        self.assertEqual(LazyList(it)[:5], it[:5])
-        self.assertEqual(LazyList(it)[::2], it[::2])
-        self.assertEqual(LazyList(it)[1::2], it[1::2])
-        self.assertEqual(LazyList(it)[5::-1], it[5::-1])
-        self.assertEqual(LazyList(it)[6:2:-2], it[6:2:-2])
-        self.assertEqual(LazyList(it)[::-1], it[::-1])
-
-        self.assertTrue(LazyList(it))
-        self.assertFalse(LazyList(range(0)))
-        self.assertEqual(len(LazyList(it)), len(it))
-        self.assertEqual(repr(LazyList(it)), repr(it))
-        self.assertEqual(str(LazyList(it)), str(it))
-
-        self.assertEqual(list(LazyList(it).reverse()), it[::-1])
-        self.assertEqual(list(LazyList(it).reverse()[1:3:7]), it[::-1][1:3:7])
-        self.assertEqual(list(LazyList(it).reverse()[::-1]), it)
-
-    def test_LazyList_laziness(self):
-
-        def test(ll, idx, val, cache):
-            self.assertEqual(ll[idx], val)
-            self.assertEqual(getattr(ll, '_LazyList__cache'), list(cache))
-
-        ll = LazyList(range(10))
-        test(ll, 0, 0, range(1))
-        test(ll, 5, 5, range(6))
-        test(ll, -3, 7, range(10))
-
-        ll = LazyList(range(10)).reverse()
-        test(ll, -1, 0, range(1))
-        test(ll, 3, 6, range(10))
-
-        ll = LazyList(itertools.count())
-        test(ll, 10, 10, range(11))
-        ll.reverse()
-        test(ll, -15, 14, range(15))
-
 
 if __name__ == '__main__':
     unittest.main()

test/test_verbose_output.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 # coding: utf-8
 
 from __future__ import unicode_literals