Release 2021.06.09

All tests of `prepare_outtmpl` is now also run on `prepare_filename`

.github/ISSUE_TEMPLATE/1_broken_site.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.08. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support
-- [ ] I've verified that I'm running yt-dlp version **2021.06.01**
+- [ ] I've verified that I'm running yt-dlp version **2021.06.08**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
  [debug] User config: []
  [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.06.01
+ [debug] yt-dlp version 2021.06.08
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/2_site_support_request.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.08. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a new site support request
-- [ ] I've verified that I'm running yt-dlp version **2021.06.01**
+- [ ] I've verified that I'm running yt-dlp version **2021.06.08**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones

.github/ISSUE_TEMPLATE/3_site_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.08. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a site feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.06.01**
+- [ ] I've verified that I'm running yt-dlp version **2021.06.08**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones
 
 

.github/ISSUE_TEMPLATE/4_bug_report.md

@@ -21,7 +21,7 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.08. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->
 
 - [ ] I'm reporting a broken site support issue
-- [ ] I've verified that I'm running yt-dlp version **2021.06.01**
+- [ ] I've verified that I'm running yt-dlp version **2021.06.08**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -46,7 +46,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
  [debug] User config: []
  [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
  [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.06.01
+ [debug] yt-dlp version 2021.06.08
  [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
  [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
  [debug] Proxy map: {}

.github/ISSUE_TEMPLATE/5_feature_request.md

@@ -21,13 +21,13 @@ assignees: ''
 
 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
-- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.08. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->
 
 - [ ] I'm reporting a feature request
-- [ ] I've verified that I'm running yt-dlp version **2021.06.01**
+- [ ] I've verified that I'm running yt-dlp version **2021.06.08**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones
 
 

Changelog.md

@@ -19,29 +19,35 @@
 -->
 
 
+### 2021.06.09
+
+* Fix bug where `%(field)d` in filename template throws error
+* Improve offset parsing in outtmpl
+* [test] More rigorous tests for `prepare_filename`
+
 ### 2021.06.08
 
 * Remove support for obsolete Python versions: Only 3.6+ is now supported
 * Merge youtube-dl: Upto [commit/c2350ca](https://github.com/ytdl-org/youtube-dl/commit/c2350cac243ba1ec1586fe85b0d62d1b700047a2)
 * [hls] Fix decryption for multithreaded downloader
 * [extractor] Fix pre-checking archive for some extractors
-* [extractor] Fix FourCC fallback when parsing ISM [fstirlitz](https://github.com/fstirlitz)
-* [twitcasting] Add TwitCastingUserIE, TwitCastingLiveIE [pukkandan](https://github.com/pukkandan), [nao20010128nao](https://github.com/nao20010128nao)
-* [vidio] Add VidioPremierIE and VidioLiveIE [minEplaYerspe](Https://github.com/MinePlayersPE)
-* [viki] Fix extraction from [ytdl-org/youtube-dl@59e583f](https://github.com/ytdl-org/youtube-dl/commit/59e583f7e8530ca92776c866897d895c072e2a82)
+* [extractor] Fix FourCC fallback when parsing ISM by [fstirlitz](https://github.com/fstirlitz)
+* [twitcasting] Add TwitCastingUserIE, TwitCastingLiveIE by [pukkandan](https://github.com/pukkandan), [nao20010128nao](https://github.com/nao20010128nao)
+* [vidio] Add VidioPremierIE and VidioLiveIE by [MinePlayersPE](Https://github.com/MinePlayersPE)
+* [viki] Fix extraction from by [ytdl-org/youtube-dl@59e583f](https://github.com/ytdl-org/youtube-dl/commit/59e583f7e8530ca92776c866897d895c072e2a82)
 * [youtube] Support shorts URL
 * [zoom] Extract transcripts as subtitles
 * Add field `original_url` with the user-inputted URL
 * Fix and refactor `prepare_outtmpl`
 * Make more fields available for `--print` when used with `--flat-playlist`
 * [utils] Generalize `traverse_dict` to `traverse_obj`
-* [downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode [fstirlitz](https://github.com/fstirlitz)
+* [downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode by [fstirlitz](https://github.com/fstirlitz)
 * [build] Release `yt-dlp.tar.gz`
-* [build,update] Add GNU-style SHA512 and prepare updater for simlar SHA256 [nihil-admirari](https://github.com/nihil-admirari)
-* [pyinst] Show Python version in exe metadata [nihil-admirari](https://github.com/nihil-admirari)
+* [build,update] Add GNU-style SHA512 and prepare updater for simlar SHA256 by [nihil-admirari](https://github.com/nihil-admirari)
+* [pyinst] Show Python version in exe metadata by [nihil-admirari](https://github.com/nihil-admirari)
 * [docs] Improve documentation of dependencies
 * [cleanup] Mark unused files
-* [cleanup] Point all shebang to `python3` [fstirlitz](https://github.com/fstirlitz)
+* [cleanup] Point all shebang to `python3` by [fstirlitz](https://github.com/fstirlitz)
 * [cleanup] Remove duplicate file `trovolive.py`
 
 

test/test_YoutubeDL.py

@@ -17,7 +17,7 @@ from yt_dlp.compat import compat_str, compat_urllib_error
 from yt_dlp.extractor import YoutubeIE
 from yt_dlp.extractor.common import InfoExtractor
 from yt_dlp.postprocessor.common import PostProcessor
-from yt_dlp.utils import ExtractorError, float_or_none, match_filter_func
+from yt_dlp.utils import ExtractorError, int_or_none, match_filter_func
 
 TEST_URL = 'http://localhost/sample.mp4'
 
@@ -664,92 +664,106 @@ class TestYoutubeDL(unittest.TestCase):
         'formats': [{'id': 'id1'}, {'id': 'id2'}, {'id': 'id3'}]
     }
 
-    def test_prepare_outtmpl(self):
-        def out(tmpl, **params):
+    def test_prepare_outtmpl_and_filename(self):
+        def test(tmpl, expected, **params):
             params['outtmpl'] = tmpl
             ydl = YoutubeDL(params)
             ydl._num_downloads = 1
-            err = ydl.validate_outtmpl(tmpl)
-            if err:
-                raise err
+            self.assertEqual(ydl.validate_outtmpl(tmpl), None)
+
             outtmpl, tmpl_dict = ydl.prepare_outtmpl(tmpl, self.outtmpl_info)
-            return outtmpl % tmpl_dict
+            out = outtmpl % tmpl_dict
+            fname = ydl.prepare_filename(self.outtmpl_info)
 
-        self.assertEqual(out('%(id)s.%(ext)s'), '1234.mp4')
-        self.assertEqual(out('%(duration_string)s'), '27:46:40')
-        self.assertTrue(float_or_none(out('%(epoch)d')))
-        self.assertEqual(out('%(resolution)s'), '1080p')
-        self.assertEqual(out('%(playlist_index)s'), '001')
-        self.assertEqual(out('%(autonumber)s'), '00001')
-        self.assertEqual(out('%(autonumber+2)03d', autonumber_start=3), '005')
-        self.assertEqual(out('%(autonumber)s', autonumber_size=3), '001')
+            if callable(expected):
+                self.assertTrue(expected(out))
+                self.assertTrue(expected(fname))
+            elif isinstance(expected, compat_str):
+                self.assertEqual((out, fname), (expected, expected))
+            else:
+                self.assertEqual((out, fname), expected)
 
-        self.assertEqual(out('%%'), '%')
-        self.assertEqual(out('%%%%'), '%%')
-        self.assertEqual(out('%(invalid@tmpl|def)s', outtmpl_na_placeholder='none'), 'none')
-        self.assertEqual(out('%()s'), 'NA')
-        self.assertEqual(out('%s'), '%s')
-        self.assertEqual(out('%d'), '%d')
-        self.assertRaises(ValueError, out, '%')
-        self.assertRaises(ValueError, out, '%(title)')
+        # Auto-generated fields
+        test('%(id)s.%(ext)s', '1234.mp4')
+        test('%(duration_string)s', ('27:46:40', '27-46-40'))
+        test('%(epoch)d', int_or_none)
+        test('%(resolution)s', '1080p')
+        test('%(playlist_index)s', '001')
+        test('%(autonumber)s', '00001')
+        test('%(autonumber+2)03d', '005', autonumber_start=3)
+        test('%(autonumber)s', '001', autonumber_size=3)
 
+        # Escaping %
+        test('%%', '%')
+        test('%%%%', '%%')
+        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+        test('%(width)06d.%(ext)s', 'NA.mp4')
+        test('%(width)06d.%%(ext)s', 'NA.%(ext)s')
+        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+
+        # Invalid templates
+        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%'), ValueError))
+        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%(title)'), ValueError))
+        test('%(invalid@tmpl|def)s', 'none', outtmpl_na_placeholder='none')
+        test('%()s', 'NA')
+        test('%s', '%s')
+        test('%d', '%d')
+
+        # NA placeholder
         NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(x|def)s-%(id)s.%(ext)s'
-        self.assertEqual(out(NA_TEST_OUTTMPL), 'NA-NA-def-1234.mp4')
-        self.assertEqual(out(NA_TEST_OUTTMPL, outtmpl_na_placeholder='none'), 'none-none-def-1234.mp4')
-        self.assertEqual(out(NA_TEST_OUTTMPL, outtmpl_na_placeholder=''), '--def-1234.mp4')
+        test(NA_TEST_OUTTMPL, 'NA-NA-def-1234.mp4')
+        test(NA_TEST_OUTTMPL, 'none-none-def-1234.mp4', outtmpl_na_placeholder='none')
+        test(NA_TEST_OUTTMPL, '--def-1234.mp4', outtmpl_na_placeholder='')
 
+        # String formatting
         FMT_TEST_OUTTMPL = '%%(height)%s.%%(ext)s'
-        self.assertEqual(out(FMT_TEST_OUTTMPL % 's'), '1080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % 'd'), '1080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '6d'), '  1080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '-6d'), '1080  .mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '06d'), '001080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % ' 06d'), ' 01080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '   06d'), ' 01080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '0 6d'), ' 01080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '0   6d'), ' 01080.mp4')
-        self.assertEqual(out(FMT_TEST_OUTTMPL % '   0   6d'), ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % 's', '1080.mp4')
+        test(FMT_TEST_OUTTMPL % 'd', '1080.mp4')
+        test(FMT_TEST_OUTTMPL % '6d', '  1080.mp4')
+        test(FMT_TEST_OUTTMPL % '-6d', '1080  .mp4')
+        test(FMT_TEST_OUTTMPL % '06d', '001080.mp4')
+        test(FMT_TEST_OUTTMPL % ' 06d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '   06d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '0 6d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '0   6d', ' 01080.mp4')
+        test(FMT_TEST_OUTTMPL % '   0   6d', ' 01080.mp4')
 
-        self.assertEqual(out('%(id)d'), '1234')
-        self.assertEqual(out('%(height)c'), '1')
-        self.assertEqual(out('%(ext)c'), 'm')
-        self.assertEqual(out('%(id)d %(id)r'), "1234 '1234'")
-        self.assertEqual(out('%(ext)s-%(ext|def)d'), 'mp4-def')
-        self.assertEqual(out('%(width|0)04d'), '0000')
-        self.assertEqual(out('%(width|)d', outtmpl_na_placeholder='none'), '')
+        # Type casting
+        test('%(id)d', '1234')
+        test('%(height)c', '1')
+        test('%(ext)c', 'm')
+        test('%(id)d %(id)r', "1234 '1234'")
+        test('%(id)r %(height)r', "'1234' 1080")
+        test('%(ext)s-%(ext|def)d', 'mp4-def')
+        test('%(width|0)04d', '0000')
+        test('a%(width|)d', 'a', outtmpl_na_placeholder='none')
 
+        # Internal formatting
         FORMATS = self.outtmpl_info['formats']
-        self.assertEqual(out('%(timestamp+-1000>%H-%M-%S)s'), '11-43-20')
-        self.assertEqual(out('%(id+1-height+3)05d'), '00158')
-        self.assertEqual(out('%(width+100)05d'), 'NA')
-        self.assertEqual(out('%(formats.0)s'), str(FORMATS[0]))
-        self.assertEqual(out('%(height.0)03d'), '001')
-        self.assertEqual(out('%(formats.-1.id)s'), str(FORMATS[-1]['id']))
-        self.assertEqual(out('%(formats.3)s'), 'NA')
-        self.assertEqual(out('%(formats.:2:-1)r'), repr(FORMATS[:2:-1]))
-        self.assertEqual(out('%(formats.0.id.-1+id)f'), '1235.000000')
+        test('%(timestamp-1000>%H-%M-%S)s', '11-43-20')
+        test('%(id+1-height+3)05d', '00158')
+        test('%(width+100)05d', 'NA')
+        test('%(formats.0) 15s', ('% 15s' % FORMATS[0], '% 15s' % str(FORMATS[0]).replace(':', ' -')))
+        test('%(formats.0)r', (repr(FORMATS[0]), repr(FORMATS[0]).replace(':', ' -')))
+        test('%(height.0)03d', '001')
+        test('%(-height.0)04d', '-001')
+        test('%(formats.-1.id)s', FORMATS[-1]['id'])
+        test('%(formats.0.id.-1)d', FORMATS[0]['id'][-1])
+        test('%(formats.3)s', 'NA')
+        test('%(formats.:2:-1)r', repr(FORMATS[:2:-1]))
+        test('%(formats.0.id.-1+id)f', '1235.000000')
+        test('%(formats.0.id.-1+formats.1.id.-1)d', '3')
 
-    def test_prepare_filename(self):
-        def fname(templ):
-            params = {'outtmpl': templ}
-            ydl = YoutubeDL(params)
-            return ydl.prepare_filename(self.outtmpl_info)
+        # Empty filename
+        test('%(foo|)s-%(bar|)s.%(ext)s', '-.mp4')
+        # test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4'))  # fixme
+        # test('%(foo|)s', ('', '_'))  # fixme
 
-        self.assertEqual(fname('%%'), '%')
-        self.assertEqual(fname('%%%%'), '%%')
-        self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
-        self.assertEqual(fname('%(width)06d.%(ext)s'), 'NA.mp4')
-        self.assertEqual(fname('%(width)06d.%%(ext)s'), 'NA.%(ext)s')
-        self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
-
-        self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
-        self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
-
-        self.assertEqual(fname('%(title3)s'), 'foo_bar_test')
-        self.assertEqual(fname('%(formats.0)s'), "{'id' - 'id1'}")
-
-        self.assertEqual(fname('%(id)r %(height)r'), "'1234' 1080")
-        self.assertEqual(fname('%(formats.0)r'), "{'id' - 'id1'}")
+        # Path expansion and escaping
+        test('Hello %(title1)s', 'Hello $PATH')
+        test('Hello %(title2)s', 'Hello %PATH%')
+        test('%(title3)s', ('foo/bar\\test', 'foo_bar_test'))
+        test('folder/%(title3)s', ('folder/foo/bar\\test', 'folder%sfoo_bar_test' % os.path.sep))
 
     def test_format_note(self):
         ydl = YoutubeDL()

test/test_utils.py

@@ -126,6 +126,7 @@ class TestUtil(unittest.TestCase):
         self.assertTrue(timeconvert('bougrg') is None)
 
     def test_sanitize_filename(self):
+        self.assertEqual(sanitize_filename(''), '')
         self.assertEqual(sanitize_filename('abc'), 'abc')
         self.assertEqual(sanitize_filename('abc_d-e'), 'abc_d-e')
 

yt_dlp/YoutubeDL.py

@@ -847,23 +847,24 @@ class YoutubeDL(object):
             'autonumber': self.params.get('autonumber_size') or 5,
         }
 
-        EXTERNAL_FORMAT_RE = STR_FORMAT_RE.format('[^)]*')
-        # Field is of the form key1.key2...
-        # where keys (except first) can be string, int or slice
-        FIELD_RE = r'\w+(?:\.(?:\w+|[-\d]*(?::[-\d]*){0,2}))*'
-        INTERNAL_FORMAT_RE = re.compile(r'''(?x)
-            (?P<negate>-)?
-            (?P<fields>{0})
-            (?P<maths>(?:[-+]-?(?:\d+(?:\.\d+)?|{0}))*)
-            (?:>(?P<strf_format>.+?))?
-            (?:\|(?P<default>.*?))?
-            $'''.format(FIELD_RE))
-        MATH_OPERATORS_RE = re.compile(r'(?<![-+])([-+])')
+        TMPL_DICT = {}
+        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE.format('[^)]*'))
         MATH_FUNCTIONS = {
             '+': float.__add__,
             '-': float.__sub__,
         }
-        tmpl_dict = {}
+        # Field is of the form key1.key2...
+        # where keys (except first) can be string, int or slice
+        FIELD_RE = r'\w+(?:\.(?:\w+|{num}|{num}?(?::{num}?){{1,2}}))*'.format(num=r'(?:-?\d+)')
+        MATH_FIELD_RE = r'''{field}|{num}'''.format(field=FIELD_RE, num=r'-?\d+(?:.\d+)?')
+        MATH_OPERATORS_RE = r'(?:%s)' % '|'.join(map(re.escape, MATH_FUNCTIONS.keys()))
+        INTERNAL_FORMAT_RE = re.compile(r'''(?x)
+            (?P<negate>-)?
+            (?P<fields>{field})
+            (?P<maths>(?:{math_op}{math_field})*)
+            (?:>(?P<strf_format>.+?))?
+            (?:\|(?P<default>.*?))?
+            $'''.format(field=FIELD_RE, math_op=MATH_OPERATORS_RE, math_field=MATH_FIELD_RE))
 
         get_key = lambda k: traverse_obj(
             info_dict, k.split('.'), is_user_input=True, traverse_string=True)
@@ -877,24 +878,27 @@ class YoutubeDL(object):
                 if value is not None:
                     value *= -1
             # Do maths
-            if mdict['maths']:
+            offset_key = mdict['maths']
+            if offset_key:
                 value = float_or_none(value)
                 operator = None
-                for item in MATH_OPERATORS_RE.split(mdict['maths'])[1:]:
-                    if item == '' or value is None:
-                        return None
-                    if operator:
-                        item, multiplier = (item[1:], -1) if item[0] == '-' else (item, 1)
-                        offset = float_or_none(item)
-                        if offset is None:
-                            offset = float_or_none(get_key(item))
-                        try:
-                            value = operator(value, multiplier * offset)
-                        except (TypeError, ZeroDivisionError):
-                            return None
-                        operator = None
-                    else:
+                while offset_key:
+                    item = re.match(
+                        MATH_FIELD_RE if operator else MATH_OPERATORS_RE,
+                        offset_key).group(0)
+                    offset_key = offset_key[len(item):]
+                    if operator is None:
                         operator = MATH_FUNCTIONS[item]
+                        continue
+                    item, multiplier = (item[1:], -1) if item[0] == '-' else (item, 1)
+                    offset = float_or_none(item)
+                    if offset is None:
+                        offset = float_or_none(get_key(item))
+                    try:
+                        value = operator(value, multiplier * offset)
+                    except (TypeError, ZeroDivisionError):
+                        return None
+                    operator = None
             # Datetime formatting
             if mdict['strf_format']:
                 value = strftime_or_none(value, mdict['strf_format'])
@@ -936,11 +940,12 @@ class YoutubeDL(object):
                     # If value is an object, sanitize might convert it to a string
                     # So we convert it to repr first
                     value, fmt = repr(value), '%ss' % fmt[:-1]
-                value = sanitize(key, value)
-            tmpl_dict[key] = value
+                if fmt[-1] in 'csr':
+                    value = sanitize(key, value)
+            TMPL_DICT[key] = value
             return '%({key}){fmt}'.format(key=key, fmt=fmt)
 
-        return re.sub(EXTERNAL_FORMAT_RE, create_key, outtmpl), tmpl_dict
+        return EXTERNAL_FORMAT_RE.sub(create_key, outtmpl), TMPL_DICT
 
     def _prepare_filename(self, info_dict, tmpl_type='default'):
         try:

yt_dlp/utils.py

@@ -2107,6 +2107,8 @@ def sanitize_filename(s, restricted=False, is_id=False):
             return '_'
         return char
 
+    if s == '':
+        return ''
     # Handle timestamps
     s = re.sub(r'[0-9]+(?::[0-9]+)+', lambda m: m.group(0).replace(':', '_'), s)
     result = ''.join(map(replace_insane, s))

yt_dlp/version.py

@@ -1,3 +1,3 @@
 from __future__ import unicode_literals
 
-__version__ = '2021.06.01'
+__version__ = '2021.06.08'