We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[senate.gov] Support request for senate.gov website
## Please follow the guide below

- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
- Use *Preview* tab to see how your issue will actually look like

---

### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.03.20*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2017.03.20**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [ ] Bug report (encountered problems with youtube-dl)
- [x] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---

### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*

---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
```
$ youtube-dl -v https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.03.20
[debug] Python version 2.7.12 - Linux-4.4.0-64-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: none
[debug] Proxy map: {}
[generic] canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security: Requesting header
WARNING: Falling back on generic information extractor.
[generic] canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security: Downloading webpage
[generic] canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security: Extracting information
ERROR: Unsupported URL: https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1779, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2526, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2515, in _XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
    raise err
ParseError: syntax error: line 208, column 0
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 761, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 428, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 2642, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security

...
<end of log>
```

---

### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
- Single video: https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security

Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

---

### Description of your *issue*, suggested solution and other information

Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
If work on your *issue* requires account credentials please provide them or explain how one can obtain them.


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `youtube_dl/extractor/senateisvp.py`
Content:
```
1 # coding: utf-8
2 from __future__ import unicode_literals
3 
4 import re
5 from .common import InfoExtractor
6 from ..utils import (
7     ExtractorError,
8     unsmuggle_url,
9 )
10 from ..compat import (
11     compat_parse_qs,
12     compat_urlparse,
13 )
14 
15 
16 class SenateISVPIE(InfoExtractor):
17     _COMM_MAP = [
18         ['ag', '76440', 'http://ag-f.akamaihd.net'],
19         ['aging', '76442', 'http://aging-f.akamaihd.net'],
20         ['approps', '76441', 'http://approps-f.akamaihd.net'],
21         ['armed', '76445', 'http://armed-f.akamaihd.net'],
22         ['banking', '76446', 'http://banking-f.akamaihd.net'],
23         ['budget', '76447', 'http://budget-f.akamaihd.net'],
24         ['cecc', '76486', 'http://srs-f.akamaihd.net'],
25         ['commerce', '80177', 'http://commerce1-f.akamaihd.net'],
26         ['csce', '75229', 'http://srs-f.akamaihd.net'],
27         ['dpc', '76590', 'http://dpc-f.akamaihd.net'],
28         ['energy', '76448', 'http://energy-f.akamaihd.net'],
29         ['epw', '76478', 'http://epw-f.akamaihd.net'],
30         ['ethics', '76449', 'http://ethics-f.akamaihd.net'],
31         ['finance', '76450', 'http://finance-f.akamaihd.net'],
32         ['foreign', '76451', 'http://foreign-f.akamaihd.net'],
33         ['govtaff', '76453', 'http://govtaff-f.akamaihd.net'],
34         ['help', '76452', 'http://help-f.akamaihd.net'],
35         ['indian', '76455', 'http://indian-f.akamaihd.net'],
36         ['intel', '76456', 'http://intel-f.akamaihd.net'],
37         ['intlnarc', '76457', 'http://intlnarc-f.akamaihd.net'],
38         ['jccic', '85180', 'http://jccic-f.akamaihd.net'],
39         ['jec', '76458', 'http://jec-f.akamaihd.net'],
40         ['judiciary', '76459', 'http://judiciary-f.akamaihd.net'],
41         ['rpc', '76591', 'http://rpc-f.akamaihd.net'],
42         ['rules', '76460', 'http://rules-f.akamaihd.net'],
43         ['saa', '76489', 'http://srs-f.akamaihd.net'],
44         ['smbiz', '76461', 'http://smbiz-f.akamaihd.net'],
45         ['srs', '75229', 'http://srs-f.akamaihd.net'],
46         ['uscc', '76487', 'http://srs-f.akamaihd.net'],
47         ['vetaff', '76462', 'http://vetaff-f.akamaihd.net'],
48         ['arch', '', 'http://ussenate-f.akamaihd.net/']
49     ]
50     _IE_NAME = 'senate.gov'
51     _VALID_URL = r'https?://(?:www\.)?senate\.gov/isvp/?\?(?P<qs>.+)'
52     _TESTS = [{
53         'url': 'http://www.senate.gov/isvp/?comm=judiciary&type=live&stt=&filename=judiciary031715&auto_play=false&wmode=transparent&poster=http%3A%2F%2Fwww.judiciary.senate.gov%2Fthemes%2Fjudiciary%2Fimages%2Fvideo-poster-flash-fit.png',
54         'info_dict': {
55             'id': 'judiciary031715',
56             'ext': 'mp4',
57             'title': 'Integrated Senate Video Player',
58             'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
59         },
60         'params': {
61             # m3u8 download
62             'skip_download': True,
63         },
64     }, {
65         'url': 'http://www.senate.gov/isvp/?type=live&comm=commerce&filename=commerce011514.mp4&auto_play=false',
66         'info_dict': {
67             'id': 'commerce011514',
68             'ext': 'mp4',
69             'title': 'Integrated Senate Video Player'
70         },
71         'params': {
72             # m3u8 download
73             'skip_download': True,
74         },
75     }, {
76         'url': 'http://www.senate.gov/isvp/?type=arch&comm=intel&filename=intel090613&hc_location=ufi',
77         # checksum differs each time
78         'info_dict': {
79             'id': 'intel090613',
80             'ext': 'mp4',
81             'title': 'Integrated Senate Video Player'
82         }
83     }, {
84         # From http://www.c-span.org/video/?96791-1
85         'url': 'http://www.senate.gov/isvp?type=live&comm=banking&filename=banking012715',
86         'only_matching': True,
87     }]
88 
89     @staticmethod
90     def _search_iframe_url(webpage):
91         mobj = re.search(
92             r"<iframe[^>]+src=['\"](?P<url>http://www\.senate\.gov/isvp/?\?[^'\"]+)['\"]",
93             webpage)
94         if mobj:
95             return mobj.group('url')
96 
97     def _get_info_for_comm(self, committee):
98         for entry in self._COMM_MAP:
99             if entry[0] == committee:
100                 return entry[1:]
101 
102     def _real_extract(self, url):
103         url, smuggled_data = unsmuggle_url(url, {})
104 
105         qs = compat_parse_qs(re.match(self._VALID_URL, url).group('qs'))
106         if not qs.get('filename') or not qs.get('type') or not qs.get('comm'):
107             raise ExtractorError('Invalid URL', expected=True)
108 
109         video_id = re.sub(r'.mp4$', '', qs['filename'][0])
110 
111         webpage = self._download_webpage(url, video_id)
112 
113         if smuggled_data.get('force_title'):
114             title = smuggled_data['force_title']
115         else:
116             title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, video_id)
117         poster = qs.get('poster')
118         thumbnail = poster[0] if poster else None
119 
120         video_type = qs['type'][0]
121         committee = video_type if video_type == 'arch' else qs['comm'][0]
122         stream_num, domain = self._get_info_for_comm(committee)
123 
124         formats = []
125         if video_type == 'arch':
126             filename = video_id if '.' in video_id else video_id + '.mp4'
127             formats = [{
128                 # All parameters in the query string are necessary to prevent a 403 error
129                 'url': compat_urlparse.urljoin(domain, filename) + '?v=3.1.0&fp=&r=&g=',
130             }]
131         else:
132             hdcore_sign = 'hdcore=3.1.0'
133             url_params = (domain, video_id, stream_num)
134             f4m_url = '%s/z/%s_1@%s/manifest.f4m?' % url_params + hdcore_sign
135             m3u8_url = '%s/i/%s_1@%s/master.m3u8' % url_params
136             for entry in self._extract_f4m_formats(f4m_url, video_id, f4m_id='f4m'):
137                 # URLs without the extra param induce an 404 error
138                 entry.update({'extra_param_to_segment_url': hdcore_sign})
139                 formats.append(entry)
140             for entry in self._extract_m3u8_formats(m3u8_url, video_id, ext='mp4', m3u8_id='m3u8'):
141                 mobj = re.search(r'(?P<tag>(?:-p|-b)).m3u8', entry['url'])
142                 if mobj:
143                     entry['format_id'] += mobj.group('tag')
144                 formats.append(entry)
145 
146             self._sort_formats(formats)
147 
148         return {
149             'id': video_id,
150             'title': title,
151             'formats': formats,
152             'thumbnail': thumbnail,
153         }
154
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/youtube_dl/extractor/senateisvp.py b/youtube_dl/extractor/senateisvp.py
--- a/youtube_dl/extractor/senateisvp.py
+++ b/youtube_dl/extractor/senateisvp.py
@@ -89,7 +89,7 @@
     @staticmethod
     def _search_iframe_url(webpage):
         mobj = re.search(
-            r"<iframe[^>]+src=['\"](?P<url>http://www\.senate\.gov/isvp/?\?[^'\"]+)['\"]",
+            r"<iframe[^>]+src=['\"](?P<url>https?://www\.senate\.gov/isvp/?\?[^'\"]+)['\"]",
             webpage)
         if mobj:
             return mobj.group('url')

{"golden_diff": "diff --git a/youtube_dl/extractor/senateisvp.py b/youtube_dl/extractor/senateisvp.py\n--- a/youtube_dl/extractor/senateisvp.py\n+++ b/youtube_dl/extractor/senateisvp.py\n@@ -89,7 +89,7 @@\n     @staticmethod\n     def _search_iframe_url(webpage):\n         mobj = re.search(\n-            r\"<iframe[^>]+src=['\\\"](?P<url>http://www\\.senate\\.gov/isvp/?\\?[^'\\\"]+)['\\\"]\",\n+            r\"<iframe[^>]+src=['\\\"](?P<url>https?://www\\.senate\\.gov/isvp/?\\?[^'\\\"]+)['\\\"]\",\n             webpage)\n         if mobj:\n             return mobj.group('url')\n", "issue": "[senate.gov] Support request for senate.gov website\n## Please follow the guide below\r\n\r\n- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly\r\n- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])\r\n- Use *Preview* tab to see how your issue will actually look like\r\n\r\n---\r\n\r\n### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.03.20*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.\r\n- [x] I've **verified** and **I assure** that I'm running youtube-dl **2017.03.20**\r\n\r\n### Before submitting an *issue* make sure you have:\r\n- [x] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections\r\n- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones\r\n\r\n### What is the purpose of your *issue*?\r\n- [ ] Bug report (encountered problems with youtube-dl)\r\n- [x] Site support request (request for adding support for a new site)\r\n- [ ] Feature request (request for a new functionality)\r\n- [ ] Question\r\n- [ ] Other\r\n\r\n---\r\n\r\n### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*\r\n\r\n---\r\n\r\n### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:\r\n\r\nAdd `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):\r\n```\r\n$ youtube-dl -v https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security\r\n[debug] System config: []\r\n[debug] User config: []\r\n[debug] Custom config: []\r\n[debug] Command-line args: [u'-v', u'https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security']\r\n[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8\r\n[debug] youtube-dl version 2017.03.20\r\n[debug] Python version 2.7.12 - Linux-4.4.0-64-generic-x86_64-with-Ubuntu-16.04-xenial\r\n[debug] exe versions: none\r\n[debug] Proxy map: {}\r\n[generic] canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security: Requesting header\r\nWARNING: Falling back on generic information extractor.\r\n[generic] canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security: Downloading webpage\r\n[generic] canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security: Extracting information\r\nERROR: Unsupported URL: https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security\r\nTraceback (most recent call last):\r\n  File \"/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py\", line 1779, in _real_extract\r\n    doc = compat_etree_fromstring(webpage.encode('utf-8'))\r\n  File \"/usr/local/bin/youtube-dl/youtube_dl/compat.py\", line 2526, in compat_etree_fromstring\r\n    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))\r\n  File \"/usr/local/bin/youtube-dl/youtube_dl/compat.py\", line 2515, in _XML\r\n    parser.feed(text)\r\n  File \"/usr/lib/python2.7/xml/etree/ElementTree.py\", line 1653, in feed\r\n    self._raiseerror(v)\r\n  File \"/usr/lib/python2.7/xml/etree/ElementTree.py\", line 1517, in _raiseerror\r\n    raise err\r\nParseError: syntax error: line 208, column 0\r\nTraceback (most recent call last):\r\n  File \"/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py\", line 761, in extract_info\r\n    ie_result = ie.extract(url)\r\n  File \"/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py\", line 428, in extract\r\n    ie_result = self._real_extract(url)\r\n  File \"/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py\", line 2642, in _real_extract\r\n    raise UnsupportedError(url)\r\nUnsupportedError: Unsupported URL: https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security\r\n\r\n...\r\n<end of log>\r\n```\r\n\r\n---\r\n\r\n### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):\r\n- Single video: https://www.hsgac.senate.gov/hearings/canadas-fast-track-refugee-plan-unanswered-questions-and-implications-for-us-national-security\r\n\r\nNote that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.\r\n\r\n---\r\n\r\n### Description of your *issue*, suggested solution and other information\r\n\r\nExplanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.\r\nIf work on your *issue* requires account credentials please provide them or explain how one can obtain them.\r\n\n", "before_files": [{"content": "# coding: utf-8\nfrom __future__ import unicode_literals\n\nimport re\nfrom .common import InfoExtractor\nfrom ..utils import (\n    ExtractorError,\n    unsmuggle_url,\n)\nfrom ..compat import (\n    compat_parse_qs,\n    compat_urlparse,\n)\n\n\nclass SenateISVPIE(InfoExtractor):\n    _COMM_MAP = [\n        ['ag', '76440', 'http://ag-f.akamaihd.net'],\n        ['aging', '76442', 'http://aging-f.akamaihd.net'],\n        ['approps', '76441', 'http://approps-f.akamaihd.net'],\n        ['armed', '76445', 'http://armed-f.akamaihd.net'],\n        ['banking', '76446', 'http://banking-f.akamaihd.net'],\n        ['budget', '76447', 'http://budget-f.akamaihd.net'],\n        ['cecc', '76486', 'http://srs-f.akamaihd.net'],\n        ['commerce', '80177', 'http://commerce1-f.akamaihd.net'],\n        ['csce', '75229', 'http://srs-f.akamaihd.net'],\n        ['dpc', '76590', 'http://dpc-f.akamaihd.net'],\n        ['energy', '76448', 'http://energy-f.akamaihd.net'],\n        ['epw', '76478', 'http://epw-f.akamaihd.net'],\n        ['ethics', '76449', 'http://ethics-f.akamaihd.net'],\n        ['finance', '76450', 'http://finance-f.akamaihd.net'],\n        ['foreign', '76451', 'http://foreign-f.akamaihd.net'],\n        ['govtaff', '76453', 'http://govtaff-f.akamaihd.net'],\n        ['help', '76452', 'http://help-f.akamaihd.net'],\n        ['indian', '76455', 'http://indian-f.akamaihd.net'],\n        ['intel', '76456', 'http://intel-f.akamaihd.net'],\n        ['intlnarc', '76457', 'http://intlnarc-f.akamaihd.net'],\n        ['jccic', '85180', 'http://jccic-f.akamaihd.net'],\n        ['jec', '76458', 'http://jec-f.akamaihd.net'],\n        ['judiciary', '76459', 'http://judiciary-f.akamaihd.net'],\n        ['rpc', '76591', 'http://rpc-f.akamaihd.net'],\n        ['rules', '76460', 'http://rules-f.akamaihd.net'],\n        ['saa', '76489', 'http://srs-f.akamaihd.net'],\n        ['smbiz', '76461', 'http://smbiz-f.akamaihd.net'],\n        ['srs', '75229', 'http://srs-f.akamaihd.net'],\n        ['uscc', '76487', 'http://srs-f.akamaihd.net'],\n        ['vetaff', '76462', 'http://vetaff-f.akamaihd.net'],\n        ['arch', '', 'http://ussenate-f.akamaihd.net/']\n    ]\n    _IE_NAME = 'senate.gov'\n    _VALID_URL = r'https?://(?:www\\.)?senate\\.gov/isvp/?\\?(?P<qs>.+)'\n    _TESTS = [{\n        'url': 'http://www.senate.gov/isvp/?comm=judiciary&type=live&stt=&filename=judiciary031715&auto_play=false&wmode=transparent&poster=http%3A%2F%2Fwww.judiciary.senate.gov%2Fthemes%2Fjudiciary%2Fimages%2Fvideo-poster-flash-fit.png',\n        'info_dict': {\n            'id': 'judiciary031715',\n            'ext': 'mp4',\n            'title': 'Integrated Senate Video Player',\n            'thumbnail': r're:^https?://.*\\.(?:jpg|png)$',\n        },\n        'params': {\n            # m3u8 download\n            'skip_download': True,\n        },\n    }, {\n        'url': 'http://www.senate.gov/isvp/?type=live&comm=commerce&filename=commerce011514.mp4&auto_play=false',\n        'info_dict': {\n            'id': 'commerce011514',\n            'ext': 'mp4',\n            'title': 'Integrated Senate Video Player'\n        },\n        'params': {\n            # m3u8 download\n            'skip_download': True,\n        },\n    }, {\n        'url': 'http://www.senate.gov/isvp/?type=arch&comm=intel&filename=intel090613&hc_location=ufi',\n        # checksum differs each time\n        'info_dict': {\n            'id': 'intel090613',\n            'ext': 'mp4',\n            'title': 'Integrated Senate Video Player'\n        }\n    }, {\n        # From http://www.c-span.org/video/?96791-1\n        'url': 'http://www.senate.gov/isvp?type=live&comm=banking&filename=banking012715',\n        'only_matching': True,\n    }]\n\n    @staticmethod\n    def _search_iframe_url(webpage):\n        mobj = re.search(\n            r\"<iframe[^>]+src=['\\\"](?P<url>http://www\\.senate\\.gov/isvp/?\\?[^'\\\"]+)['\\\"]\",\n            webpage)\n        if mobj:\n            return mobj.group('url')\n\n    def _get_info_for_comm(self, committee):\n        for entry in self._COMM_MAP:\n            if entry[0] == committee:\n                return entry[1:]\n\n    def _real_extract(self, url):\n        url, smuggled_data = unsmuggle_url(url, {})\n\n        qs = compat_parse_qs(re.match(self._VALID_URL, url).group('qs'))\n        if not qs.get('filename') or not qs.get('type') or not qs.get('comm'):\n            raise ExtractorError('Invalid URL', expected=True)\n\n        video_id = re.sub(r'.mp4$', '', qs['filename'][0])\n\n        webpage = self._download_webpage(url, video_id)\n\n        if smuggled_data.get('force_title'):\n            title = smuggled_data['force_title']\n        else:\n            title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, video_id)\n        poster = qs.get('poster')\n        thumbnail = poster[0] if poster else None\n\n        video_type = qs['type'][0]\n        committee = video_type if video_type == 'arch' else qs['comm'][0]\n        stream_num, domain = self._get_info_for_comm(committee)\n\n        formats = []\n        if video_type == 'arch':\n            filename = video_id if '.' in video_id else video_id + '.mp4'\n            formats = [{\n                # All parameters in the query string are necessary to prevent a 403 error\n                'url': compat_urlparse.urljoin(domain, filename) + '?v=3.1.0&fp=&r=&g=',\n            }]\n        else:\n            hdcore_sign = 'hdcore=3.1.0'\n            url_params = (domain, video_id, stream_num)\n            f4m_url = '%s/z/%s_1@%s/manifest.f4m?' % url_params + hdcore_sign\n            m3u8_url = '%s/i/%s_1@%s/master.m3u8' % url_params\n            for entry in self._extract_f4m_formats(f4m_url, video_id, f4m_id='f4m'):\n                # URLs without the extra param induce an 404 error\n                entry.update({'extra_param_to_segment_url': hdcore_sign})\n                formats.append(entry)\n            for entry in self._extract_m3u8_formats(m3u8_url, video_id, ext='mp4', m3u8_id='m3u8'):\n                mobj = re.search(r'(?P<tag>(?:-p|-b)).m3u8', entry['url'])\n                if mobj:\n                    entry['format_id'] += mobj.group('tag')\n                formats.append(entry)\n\n            self._sort_formats(formats)\n\n        return {\n            'id': video_id,\n            'title': title,\n            'formats': formats,\n            'thumbnail': thumbnail,\n        }\n", "path": "youtube_dl/extractor/senateisvp.py"}], "after_files": [{"content": "# coding: utf-8\nfrom __future__ import unicode_literals\n\nimport re\nfrom .common import InfoExtractor\nfrom ..utils import (\n    ExtractorError,\n    unsmuggle_url,\n)\nfrom ..compat import (\n    compat_parse_qs,\n    compat_urlparse,\n)\n\n\nclass SenateISVPIE(InfoExtractor):\n    _COMM_MAP = [\n        ['ag', '76440', 'http://ag-f.akamaihd.net'],\n        ['aging', '76442', 'http://aging-f.akamaihd.net'],\n        ['approps', '76441', 'http://approps-f.akamaihd.net'],\n        ['armed', '76445', 'http://armed-f.akamaihd.net'],\n        ['banking', '76446', 'http://banking-f.akamaihd.net'],\n        ['budget', '76447', 'http://budget-f.akamaihd.net'],\n        ['cecc', '76486', 'http://srs-f.akamaihd.net'],\n        ['commerce', '80177', 'http://commerce1-f.akamaihd.net'],\n        ['csce', '75229', 'http://srs-f.akamaihd.net'],\n        ['dpc', '76590', 'http://dpc-f.akamaihd.net'],\n        ['energy', '76448', 'http://energy-f.akamaihd.net'],\n        ['epw', '76478', 'http://epw-f.akamaihd.net'],\n        ['ethics', '76449', 'http://ethics-f.akamaihd.net'],\n        ['finance', '76450', 'http://finance-f.akamaihd.net'],\n        ['foreign', '76451', 'http://foreign-f.akamaihd.net'],\n        ['govtaff', '76453', 'http://govtaff-f.akamaihd.net'],\n        ['help', '76452', 'http://help-f.akamaihd.net'],\n        ['indian', '76455', 'http://indian-f.akamaihd.net'],\n        ['intel', '76456', 'http://intel-f.akamaihd.net'],\n        ['intlnarc', '76457', 'http://intlnarc-f.akamaihd.net'],\n        ['jccic', '85180', 'http://jccic-f.akamaihd.net'],\n        ['jec', '76458', 'http://jec-f.akamaihd.net'],\n        ['judiciary', '76459', 'http://judiciary-f.akamaihd.net'],\n        ['rpc', '76591', 'http://rpc-f.akamaihd.net'],\n        ['rules', '76460', 'http://rules-f.akamaihd.net'],\n        ['saa', '76489', 'http://srs-f.akamaihd.net'],\n        ['smbiz', '76461', 'http://smbiz-f.akamaihd.net'],\n        ['srs', '75229', 'http://srs-f.akamaihd.net'],\n        ['uscc', '76487', 'http://srs-f.akamaihd.net'],\n        ['vetaff', '76462', 'http://vetaff-f.akamaihd.net'],\n        ['arch', '', 'http://ussenate-f.akamaihd.net/']\n    ]\n    _IE_NAME = 'senate.gov'\n    _VALID_URL = r'https?://(?:www\\.)?senate\\.gov/isvp/?\\?(?P<qs>.+)'\n    _TESTS = [{\n        'url': 'http://www.senate.gov/isvp/?comm=judiciary&type=live&stt=&filename=judiciary031715&auto_play=false&wmode=transparent&poster=http%3A%2F%2Fwww.judiciary.senate.gov%2Fthemes%2Fjudiciary%2Fimages%2Fvideo-poster-flash-fit.png',\n        'info_dict': {\n            'id': 'judiciary031715',\n            'ext': 'mp4',\n            'title': 'Integrated Senate Video Player',\n            'thumbnail': r're:^https?://.*\\.(?:jpg|png)$',\n        },\n        'params': {\n            # m3u8 download\n            'skip_download': True,\n        },\n    }, {\n        'url': 'http://www.senate.gov/isvp/?type=live&comm=commerce&filename=commerce011514.mp4&auto_play=false',\n        'info_dict': {\n            'id': 'commerce011514',\n            'ext': 'mp4',\n            'title': 'Integrated Senate Video Player'\n        },\n        'params': {\n            # m3u8 download\n            'skip_download': True,\n        },\n    }, {\n        'url': 'http://www.senate.gov/isvp/?type=arch&comm=intel&filename=intel090613&hc_location=ufi',\n        # checksum differs each time\n        'info_dict': {\n            'id': 'intel090613',\n            'ext': 'mp4',\n            'title': 'Integrated Senate Video Player'\n        }\n    }, {\n        # From http://www.c-span.org/video/?96791-1\n        'url': 'http://www.senate.gov/isvp?type=live&comm=banking&filename=banking012715',\n        'only_matching': True,\n    }]\n\n    @staticmethod\n    def _search_iframe_url(webpage):\n        mobj = re.search(\n            r\"<iframe[^>]+src=['\\\"](?P<url>https?://www\\.senate\\.gov/isvp/?\\?[^'\\\"]+)['\\\"]\",\n            webpage)\n        if mobj:\n            return mobj.group('url')\n\n    def _get_info_for_comm(self, committee):\n        for entry in self._COMM_MAP:\n            if entry[0] == committee:\n                return entry[1:]\n\n    def _real_extract(self, url):\n        url, smuggled_data = unsmuggle_url(url, {})\n\n        qs = compat_parse_qs(re.match(self._VALID_URL, url).group('qs'))\n        if not qs.get('filename') or not qs.get('type') or not qs.get('comm'):\n            raise ExtractorError('Invalid URL', expected=True)\n\n        video_id = re.sub(r'.mp4$', '', qs['filename'][0])\n\n        webpage = self._download_webpage(url, video_id)\n\n        if smuggled_data.get('force_title'):\n            title = smuggled_data['force_title']\n        else:\n            title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, video_id)\n        poster = qs.get('poster')\n        thumbnail = poster[0] if poster else None\n\n        video_type = qs['type'][0]\n        committee = video_type if video_type == 'arch' else qs['comm'][0]\n        stream_num, domain = self._get_info_for_comm(committee)\n\n        formats = []\n        if video_type == 'arch':\n            filename = video_id if '.' in video_id else video_id + '.mp4'\n            formats = [{\n                # All parameters in the query string are necessary to prevent a 403 error\n                'url': compat_urlparse.urljoin(domain, filename) + '?v=3.1.0&fp=&r=&g=',\n            }]\n        else:\n            hdcore_sign = 'hdcore=3.1.0'\n            url_params = (domain, video_id, stream_num)\n            f4m_url = '%s/z/%s_1@%s/manifest.f4m?' % url_params + hdcore_sign\n            m3u8_url = '%s/i/%s_1@%s/master.m3u8' % url_params\n            for entry in self._extract_f4m_formats(f4m_url, video_id, f4m_id='f4m'):\n                # URLs without the extra param induce an 404 error\n                entry.update({'extra_param_to_segment_url': hdcore_sign})\n                formats.append(entry)\n            for entry in self._extract_m3u8_formats(m3u8_url, video_id, ext='mp4', m3u8_id='m3u8'):\n                mobj = re.search(r'(?P<tag>(?:-p|-b)).m3u8', entry['url'])\n                if mobj:\n                    entry['format_id'] += mobj.group('tag')\n                formats.append(entry)\n\n            self._sort_formats(formats)\n\n        return {\n            'id': video_id,\n            'title': title,\n            'formats': formats,\n            'thumbnail': thumbnail,\n        }\n", "path": "youtube_dl/extractor/senateisvp.py"}]}