{
    "file_diffs": [
        {
            "old_file_content": "from tornado.escape import _unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPResponse,\n    HTTPError,\n    AsyncHTTPClient,\n    main,\n    _RequestProxy,\n    HTTPRequest,\n)\nfrom tornado import httputil\nfrom tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import StreamClosedError, IOStream\nfrom tornado.netutil import (\n    Resolver,\n    OverrideResolver,\n    _client_ssl_defaults,\n    is_valid_ip,\n)\nfrom tornado.log import gen_log\nfrom tornado.tcpclient import TCPClient\n\nimport base64\nimport collections\nimport copy\nimport functools\nimport re\nimport socket\nimport ssl\nimport sys\nimport time\nfrom io import BytesIO\nimport urllib.parse\n\nfrom typing import Dict, Any, Callable, Optional, Type, Union\nfrom types import TracebackType\nimport typing\n\nif typing.TYPE_CHECKING:\n    from typing import Deque, Tuple, List  # noqa: F401\n\n\nclass HTTPTimeoutError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient on timeout.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPTimeoutError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Timeout\"\n\n\nclass HTTPStreamClosedError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.\n\n    When a more specific exception is available (such as `ConnectionResetError`),\n    it may be raised instead of this one.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPStreamClosedError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Stream closed\"\n\n\nclass SimpleAsyncHTTPClient(AsyncHTTPClient):\n    \"\"\"Non-blocking HTTP client with no external dependencies.\n\n    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.\n    Some features found in the curl-based AsyncHTTPClient are not yet\n    supported.  In particular, proxies are not supported, connections\n    are not reused, and callers cannot select the network interface to be\n    used.\n    \"\"\"\n\n    def initialize(  # type: ignore\n        self,\n        max_clients: int = 10,\n        hostname_mapping: Dict[str, str] = None,\n        max_buffer_size: int = 104857600,\n        resolver: Resolver = None,\n        defaults: Dict[str, Any] = None,\n        max_header_size: int = None,\n        max_body_size: int = None,\n    ) -> None:\n        \"\"\"Creates a AsyncHTTPClient.\n\n        Only a single AsyncHTTPClient instance exists per IOLoop\n        in order to provide limitations on the number of pending connections.\n        ``force_instance=True`` may be used to suppress this behavior.\n\n        Note that because of this implicit reuse, unless ``force_instance``\n        is used, only the first call to the constructor actually uses\n        its arguments. It is recommended to use the ``configure`` method\n        instead of the constructor to ensure that arguments take effect.\n\n        ``max_clients`` is the number of concurrent requests that can be\n        in progress; when this limit is reached additional requests will be\n        queued. Note that time spent waiting in this queue still counts\n        against the ``request_timeout``.\n\n        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.\n        It can be used to make local DNS changes when modifying system-wide\n        settings like ``/etc/hosts`` is not possible or desirable (e.g. in\n        unittests).\n\n        ``max_buffer_size`` (default 100MB) is the number of bytes\n        that can be read into memory at once. ``max_body_size``\n        (defaults to ``max_buffer_size``) is the largest response body\n        that the client will accept.  Without a\n        ``streaming_callback``, the smaller of these two limits\n        applies; with a ``streaming_callback`` only ``max_body_size``\n        does.\n\n        .. versionchanged:: 4.2\n           Added the ``max_body_size`` argument.\n        \"\"\"\n        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)\n        self.max_clients = max_clients\n        self.queue = (\n            collections.deque()\n        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.active = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.waiting = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]\n        self.max_buffer_size = max_buffer_size\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        # TCPClient could create a Resolver for us, but we have to do it\n        # ourselves to support hostname_mapping.\n        if resolver:\n            self.resolver = resolver\n            self.own_resolver = False\n        else:\n            self.resolver = Resolver()\n            self.own_resolver = True\n        if hostname_mapping is not None:\n            self.resolver = OverrideResolver(\n                resolver=self.resolver, mapping=hostname_mapping\n            )\n        self.tcp_client = TCPClient(resolver=self.resolver)\n\n    def close(self) -> None:\n        super(SimpleAsyncHTTPClient, self).close()\n        if self.own_resolver:\n            self.resolver.close()\n        self.tcp_client.close()\n\n    def fetch_impl(\n        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]\n    ) -> None:\n        key = object()\n        self.queue.append((key, request, callback))\n        if not len(self.active) < self.max_clients:\n            assert request.connect_timeout is not None\n            assert request.request_timeout is not None\n            timeout_handle = self.io_loop.add_timeout(\n                self.io_loop.time()\n                + min(request.connect_timeout, request.request_timeout),\n                functools.partial(self._on_timeout, key, \"in request queue\"),\n            )\n        else:\n            timeout_handle = None\n        self.waiting[key] = (request, callback, timeout_handle)\n        self._process_queue()\n        if self.queue:\n            gen_log.debug(\n                \"max_clients limit reached, request queued. \"\n                \"%d active, %d queued requests.\" % (len(self.active), len(self.queue))\n            )\n\n    def _process_queue(self) -> None:\n        while self.queue and len(self.active) < self.max_clients:\n            key, request, callback = self.queue.popleft()\n            if key not in self.waiting:\n                continue\n            self._remove_timeout(key)\n            self.active[key] = (request, callback)\n            release_callback = functools.partial(self._release_fetch, key)\n            self._handle_request(request, release_callback, callback)\n\n    def _connection_class(self) -> type:\n        return _HTTPConnection\n\n    def _handle_request(\n        self,\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n    ) -> None:\n        self._connection_class()(\n            self,\n            request,\n            release_callback,\n            final_callback,\n            self.max_buffer_size,\n            self.tcp_client,\n            self.max_header_size,\n            self.max_body_size,\n        )\n\n    def _release_fetch(self, key: object) -> None:\n        del self.active[key]\n        self._process_queue()\n\n    def _remove_timeout(self, key: object) -> None:\n        if key in self.waiting:\n            request, callback, timeout_handle = self.waiting[key]\n            if timeout_handle is not None:\n                self.io_loop.remove_timeout(timeout_handle)\n            del self.waiting[key]\n\n    def _on_timeout(self, key: object, info: str = None) -> None:\n        \"\"\"Timeout callback of request.\n\n        Construct a timeout HTTPResponse when a timeout occurs.\n\n        :arg object key: A simple object to mark the request.\n        :info string key: More detailed timeout information.\n        \"\"\"\n        request, callback, timeout_handle = self.waiting[key]\n        self.queue.remove((key, request, callback))\n\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        timeout_response = HTTPResponse(\n            request,\n            599,\n            error=HTTPTimeoutError(error_message),\n            request_time=self.io_loop.time() - request.start_time,\n        )\n        self.io_loop.add_callback(callback, timeout_response)\n        del self.waiting[key]\n\n\nclass _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n                and self.headers is not None\n                and self.headers.get(\"Location\") is not None\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)\n\n\nif __name__ == \"__main__\":\n    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)\n    main()\n",
            "new_file_content": "from tornado.escape import _unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPResponse,\n    HTTPError,\n    AsyncHTTPClient,\n    main,\n    _RequestProxy,\n    HTTPRequest,\n)\nfrom tornado import httputil\nfrom tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import StreamClosedError, IOStream\nfrom tornado.netutil import (\n    Resolver,\n    OverrideResolver,\n    _client_ssl_defaults,\n    is_valid_ip,\n)\nfrom tornado.log import gen_log\nfrom tornado.tcpclient import TCPClient\n\nimport base64\nimport collections\nimport copy\nimport functools\nimport re\nimport socket\nimport ssl\nimport sys\nimport time\nfrom io import BytesIO\nimport urllib.parse\n\nfrom typing import Dict, Any, Callable, Optional, Type, Union\nfrom types import TracebackType\nimport typing\n\nif typing.TYPE_CHECKING:\n    from typing import Deque, Tuple, List  # noqa: F401\n\n\nclass HTTPTimeoutError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient on timeout.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPTimeoutError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Timeout\"\n\n\nclass HTTPStreamClosedError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.\n\n    When a more specific exception is available (such as `ConnectionResetError`),\n    it may be raised instead of this one.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPStreamClosedError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Stream closed\"\n\n\nclass SimpleAsyncHTTPClient(AsyncHTTPClient):\n    \"\"\"Non-blocking HTTP client with no external dependencies.\n\n    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.\n    Some features found in the curl-based AsyncHTTPClient are not yet\n    supported.  In particular, proxies are not supported, connections\n    are not reused, and callers cannot select the network interface to be\n    used.\n    \"\"\"\n\n    def initialize(  # type: ignore\n        self,\n        max_clients: int = 10,\n        hostname_mapping: Dict[str, str] = None,\n        max_buffer_size: int = 104857600,\n        resolver: Resolver = None,\n        defaults: Dict[str, Any] = None,\n        max_header_size: int = None,\n        max_body_size: int = None,\n    ) -> None:\n        \"\"\"Creates a AsyncHTTPClient.\n\n        Only a single AsyncHTTPClient instance exists per IOLoop\n        in order to provide limitations on the number of pending connections.\n        ``force_instance=True`` may be used to suppress this behavior.\n\n        Note that because of this implicit reuse, unless ``force_instance``\n        is used, only the first call to the constructor actually uses\n        its arguments. It is recommended to use the ``configure`` method\n        instead of the constructor to ensure that arguments take effect.\n\n        ``max_clients`` is the number of concurrent requests that can be\n        in progress; when this limit is reached additional requests will be\n        queued. Note that time spent waiting in this queue still counts\n        against the ``request_timeout``.\n\n        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.\n        It can be used to make local DNS changes when modifying system-wide\n        settings like ``/etc/hosts`` is not possible or desirable (e.g. in\n        unittests).\n\n        ``max_buffer_size`` (default 100MB) is the number of bytes\n        that can be read into memory at once. ``max_body_size``\n        (defaults to ``max_buffer_size``) is the largest response body\n        that the client will accept.  Without a\n        ``streaming_callback``, the smaller of these two limits\n        applies; with a ``streaming_callback`` only ``max_body_size``\n        does.\n\n        .. versionchanged:: 4.2\n           Added the ``max_body_size`` argument.\n        \"\"\"\n        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)\n        self.max_clients = max_clients\n        self.queue = (\n            collections.deque()\n        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.active = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.waiting = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]\n        self.max_buffer_size = max_buffer_size\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        # TCPClient could create a Resolver for us, but we have to do it\n        # ourselves to support hostname_mapping.\n        if resolver:\n            self.resolver = resolver\n            self.own_resolver = False\n        else:\n            self.resolver = Resolver()\n            self.own_resolver = True\n        if hostname_mapping is not None:\n            self.resolver = OverrideResolver(\n                resolver=self.resolver, mapping=hostname_mapping\n            )\n        self.tcp_client = TCPClient(resolver=self.resolver)\n\n    def close(self) -> None:\n        super(SimpleAsyncHTTPClient, self).close()\n        if self.own_resolver:\n            self.resolver.close()\n        self.tcp_client.close()\n\n    def fetch_impl(\n        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]\n    ) -> None:\n        key = object()\n        self.queue.append((key, request, callback))\n        if not len(self.active) < self.max_clients:\n            assert request.connect_timeout is not None\n            assert request.request_timeout is not None\n            timeout_handle = self.io_loop.add_timeout(\n                self.io_loop.time()\n                + min(request.connect_timeout, request.request_timeout),\n                functools.partial(self._on_timeout, key, \"in request queue\"),\n            )\n        else:\n            timeout_handle = None\n        self.waiting[key] = (request, callback, timeout_handle)\n        self._process_queue()\n        if self.queue:\n            gen_log.debug(\n                \"max_clients limit reached, request queued. \"\n                \"%d active, %d queued requests.\" % (len(self.active), len(self.queue))\n            )\n\n    def _process_queue(self) -> None:\n        while self.queue and len(self.active) < self.max_clients:\n            key, request, callback = self.queue.popleft()\n            if key not in self.waiting:\n                continue\n            self._remove_timeout(key)\n            self.active[key] = (request, callback)\n            release_callback = functools.partial(self._release_fetch, key)\n            self._handle_request(request, release_callback, callback)\n\n    def _connection_class(self) -> type:\n        return _HTTPConnection\n\n    def _handle_request(\n        self,\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n    ) -> None:\n        self._connection_class()(\n            self,\n            request,\n            release_callback,\n            final_callback,\n            self.max_buffer_size,\n            self.tcp_client,\n            self.max_header_size,\n            self.max_body_size,\n        )\n\n    def _release_fetch(self, key: object) -> None:\n        del self.active[key]\n        self._process_queue()\n\n    def _remove_timeout(self, key: object) -> None:\n        if key in self.waiting:\n            request, callback, timeout_handle = self.waiting[key]\n            if timeout_handle is not None:\n                self.io_loop.remove_timeout(timeout_handle)\n            del self.waiting[key]\n\n    def _on_timeout(self, key: object, info: str = None) -> None:\n        \"\"\"Timeout callback of request.\n\n        Construct a timeout HTTPResponse when a timeout occurs.\n\n        :arg object key: A simple object to mark the request.\n        :info string key: More detailed timeout information.\n        \"\"\"\n        request, callback, timeout_handle = self.waiting[key]\n        self.queue.remove((key, request, callback))\n\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        timeout_response = HTTPResponse(\n            request,\n            599,\n            error=HTTPTimeoutError(error_message),\n            request_time=self.io_loop.time() - request.start_time,\n        )\n        self.io_loop.add_callback(callback, timeout_response)\n        del self.waiting[key]\n\n\nclass _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n                and self.headers is not None\n                and self.headers.get(\"Location\") is not None\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # https://tools.ietf.org/html/rfc7231#section-6.4\n            #\n            # The original HTTP spec said that after a 301 or 302\n            # redirect, the request method should be preserved.\n            # However, browsers implemented this by changing the\n            # method to GET, and the behavior stuck. 303 redirects\n            # always specified this POST-to-GET behavior (arguably 303\n            # redirects should change *all* requests to GET, but\n            # libcurl only does this for POST so we follow their\n            # example).\n            if self.code in (301, 302, 303) and self.request.method == \"POST\":\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)\n\n\nif __name__ == \"__main__\":\n    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)\n    main()\n",
            "header": {
                "file": {
                    "path": "tornado/simple_httpclient.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "6af9dd15",
                "new_commit_hash": "6e075aa6",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/tornado/simple_httpclient.py"
            },
            "plus_file": {
                "path": "b/tornado/simple_httpclient.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 626,
                            "length": 14
                        },
                        "new_range": {
                            "start": 626,
                            "length": 17
                        },
                        "section": "class _HTTPConnection(httputil.HTTPMessageDelegate):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "            )",
                                "type": "context"
                            },
                            {
                                "content": "            new_request.max_redirects = self.request.max_redirects - 1",
                                "type": "context"
                            },
                            {
                                "content": "            del new_request.headers[\"Host\"]",
                                "type": "context"
                            },
                            {
                                "content": "            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4",
                                "type": "deleted"
                            },
                            {
                                "content": "            # Client SHOULD make a GET request after a 303.",
                                "type": "deleted"
                            },
                            {
                                "content": "            # According to the spec, 302 should be followed by the same",
                                "type": "deleted"
                            },
                            {
                                "content": "            # method as the original request, but in practice browsers",
                                "type": "deleted"
                            },
                            {
                                "content": "            # treat 302 the same as 303, and many servers use 302 for",
                                "type": "deleted"
                            },
                            {
                                "content": "            # compatibility with pre-HTTP/1.1 user agents which don't",
                                "type": "deleted"
                            },
                            {
                                "content": "            # understand the 303 status.",
                                "type": "deleted"
                            },
                            {
                                "content": "            if self.code in (302, 303):",
                                "type": "deleted"
                            },
                            {
                                "content": "            # https://tools.ietf.org/html/rfc7231#section-6.4",
                                "type": "added"
                            },
                            {
                                "content": "            #",
                                "type": "added"
                            },
                            {
                                "content": "            # The original HTTP spec said that after a 301 or 302",
                                "type": "added"
                            },
                            {
                                "content": "            # redirect, the request method should be preserved.",
                                "type": "added"
                            },
                            {
                                "content": "            # However, browsers implemented this by changing the",
                                "type": "added"
                            },
                            {
                                "content": "            # method to GET, and the behavior stuck. 303 redirects",
                                "type": "added"
                            },
                            {
                                "content": "            # always specified this POST-to-GET behavior (arguably 303",
                                "type": "added"
                            },
                            {
                                "content": "            # redirects should change *all* requests to GET, but",
                                "type": "added"
                            },
                            {
                                "content": "            # libcurl only does this for POST so we follow their",
                                "type": "added"
                            },
                            {
                                "content": "            # example).",
                                "type": "added"
                            },
                            {
                                "content": "            if self.code in (301, 302, 303) and self.request.method == \"POST\":",
                                "type": "added"
                            },
                            {
                                "content": "                new_request.method = \"GET\"",
                                "type": "context"
                            },
                            {
                                "content": "                new_request.body = None",
                                "type": "context"
                            },
                            {
                                "content": "                for h in [",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "class",
                            "name": "_HTTPConnection",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n                and self.headers is not None\n                and self.headers.get(\"Location\") is not None\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 251,
                            "end_lineno": 684,
                            "parent": null
                        },
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "method",
                            "name": "_HTTPConnection.finish",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n                and self.headers is not None\n                and self.headers.get(\"Location\") is not None\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 616,
                            "end_lineno": 672,
                            "parent": {
                                "file_name": "tornado/simple_httpclient.py",
                                "type": "class",
                                "name": "_HTTPConnection",
                                "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n                and self.headers is not None\n                and self.headers.get(\"Location\") is not None\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 251,
                                "end_lineno": 684,
                                "parent": null
                            }
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": "# -*- coding: utf-8 -*-\nimport base64\nimport binascii\nfrom contextlib import closing\nimport copy\nimport threading\nimport datetime\nfrom io import BytesIO\nimport subprocess\nimport sys\nimport time\nimport typing  # noqa: F401\nimport unicodedata\nimport unittest\n\nfrom tornado.escape import utf8, native_str, to_unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPRequest,\n    HTTPResponse,\n    _RequestProxy,\n    HTTPError,\n    HTTPClient,\n)\nfrom tornado.httpserver import HTTPServer\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import IOStream\nfrom tornado.log import gen_log, app_log\nfrom tornado import netutil\nfrom tornado.testing import AsyncHTTPTestCase, bind_unused_port, gen_test, ExpectLog\nfrom tornado.test.util import skipOnTravis\nfrom tornado.web import Application, RequestHandler, url\nfrom tornado.httputil import format_timestamp, HTTPHeaders\n\n\nclass HelloWorldHandler(RequestHandler):\n    def get(self):\n        name = self.get_argument(\"name\", \"world\")\n        self.set_header(\"Content-Type\", \"text/plain\")\n        self.finish(\"Hello %s!\" % name)\n\n\nclass PostHandler(RequestHandler):\n    def post(self):\n        self.finish(\n            \"Post arg1: %s, arg2: %s\"\n            % (self.get_argument(\"arg1\"), self.get_argument(\"arg2\"))\n        )\n\n\nclass PutHandler(RequestHandler):\n    def put(self):\n        self.write(\"Put body: \")\n        self.write(self.request.body)\n\n\nclass RedirectHandler(RequestHandler):\n    def prepare(self):\n        self.write(\"redirects can have bodies too\")\n        self.redirect(\n            self.get_argument(\"url\"), status=int(self.get_argument(\"status\", \"302\"))\n        )\n\n\nclass RedirectWithoutLocationHandler(RequestHandler):\n    def prepare(self):\n        # For testing error handling of a redirect with no location header.\n        self.set_status(301)\n        self.finish()\n\n\nclass ChunkHandler(RequestHandler):\n    @gen.coroutine\n    def get(self):\n        self.write(\"asdf\")\n        self.flush()\n        # Wait a bit to ensure the chunks are sent and received separately.\n        yield gen.sleep(0.01)\n        self.write(\"qwer\")\n\n\nclass AuthHandler(RequestHandler):\n    def get(self):\n        self.finish(self.request.headers[\"Authorization\"])\n\n\nclass CountdownHandler(RequestHandler):\n    def get(self, count):\n        count = int(count)\n        if count > 0:\n            self.redirect(self.reverse_url(\"countdown\", count - 1))\n        else:\n            self.write(\"Zero\")\n\n\nclass EchoPostHandler(RequestHandler):\n    def post(self):\n        self.write(self.request.body)\n\n\nclass UserAgentHandler(RequestHandler):\n    def get(self):\n        self.write(self.request.headers.get(\"User-Agent\", \"User agent not set\"))\n\n\nclass ContentLength304Handler(RequestHandler):\n    def get(self):\n        self.set_status(304)\n        self.set_header(\"Content-Length\", 42)\n\n    def _clear_headers_for_304(self):\n        # Tornado strips content-length from 304 responses, but here we\n        # want to simulate servers that include the headers anyway.\n        pass\n\n\nclass PatchHandler(RequestHandler):\n    def patch(self):\n        \"Return the request payload - so we can check it is being kept\"\n        self.write(self.request.body)\n\n\nclass AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = post = put = delete = options = patch = other = method  # type: ignore\n\n\nclass SetHeaderHandler(RequestHandler):\n    def get(self):\n        # Use get_arguments for keys to get strings, but\n        # request.arguments for values to get bytes.\n        for k, v in zip(self.get_arguments(\"k\"), self.request.arguments[\"v\"]):\n            self.set_header(k, v)\n\n\n# These tests end up getting run redundantly: once here with the default\n# HTTPClient implementation, and then again in each implementation's own\n# test suite.\n\n\nclass HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/redirect_without_location\", RedirectWithoutLocationHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_redirect_without_location(self):\n        response = self.fetch(\"/redirect_without_location\", follow_redirects=True)\n        # If there is no location header, the redirect response should\n        # just be returned as-is. (This should arguably raise an\n        # error, but libcurl doesn't treat this as an error, so we\n        # don't either).\n        self.assertEqual(301, response.code)\n\n    def test_redirect_put_with_body(self):\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\", method=\"PUT\", body=\"hello\"\n        )\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_redirect_put_without_body(self):\n        # This \"without body\" edge case is similar to what happens with body_producer.\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\",\n            method=\"PUT\",\n            allow_nonstandard_methods=True,\n        )\n        self.assertEqual(response.body, b\"Put body: \")\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n    @gen_test\n    def test_error_after_cancel(self):\n        fut = self.http_client.fetch(self.get_url(\"/404\"))\n        self.assertTrue(fut.cancel())\n        with ExpectLog(app_log, \"Exception after Future was cancelled\") as el:\n            # We can't wait on the cancelled Future any more, so just\n            # let the IOLoop run until the exception gets logged (or\n            # not, in which case we exit the loop and ExpectLog will\n            # raise).\n            for i in range(100):\n                yield gen.sleep(0.01)\n                if el.logged_stack:\n                    break\n\n\nclass RequestProxyTest(unittest.TestCase):\n    def test_request_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", user_agent=\"foo\"), dict()\n        )\n        self.assertEqual(proxy.user_agent, \"foo\")\n\n    def test_default_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\"), dict(network_interface=\"foo\")\n        )\n        self.assertEqual(proxy.network_interface, \"foo\")\n\n    def test_both_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", proxy_host=\"foo\"), dict(proxy_host=\"bar\")\n        )\n        self.assertEqual(proxy.proxy_host, \"foo\")\n\n    def test_neither_set(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        self.assertIs(proxy.auth_username, None)\n\n    def test_bad_attribute(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        with self.assertRaises(AttributeError):\n            proxy.foo\n\n    def test_defaults_none(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), None)\n        self.assertIs(proxy.auth_username, None)\n\n\nclass HTTPResponseTestCase(unittest.TestCase):\n    def test_str(self):\n        response = HTTPResponse(  # type: ignore\n            HTTPRequest(\"http://example.com\"), 200, headers={}, buffer=BytesIO()\n        )\n        s = str(response)\n        self.assertTrue(s.startswith(\"HTTPResponse(\"))\n        self.assertIn(\"code=200\", s)\n\n\nclass SyncHTTPClientTest(unittest.TestCase):\n    def setUp(self):\n        self.server_ioloop = IOLoop()\n        event = threading.Event()\n\n        @gen.coroutine\n        def init_server():\n            sock, self.port = bind_unused_port()\n            app = Application([(\"/\", HelloWorldHandler)])\n            self.server = HTTPServer(app)\n            self.server.add_socket(sock)\n            event.set()\n\n        def start():\n            self.server_ioloop.run_sync(init_server)\n            self.server_ioloop.start()\n\n        self.server_thread = threading.Thread(target=start)\n        self.server_thread.start()\n        event.wait()\n\n        self.http_client = HTTPClient()\n\n    def tearDown(self):\n        def stop_server():\n            self.server.stop()\n            # Delay the shutdown of the IOLoop by several iterations because\n            # the server may still have some cleanup work left when\n            # the client finishes with the response (this is noticeable\n            # with http/2, which leaves a Future with an unexamined\n            # StreamClosedError on the loop).\n\n            @gen.coroutine\n            def slow_stop():\n                yield self.server.close_all_connections()\n                # The number of iterations is difficult to predict. Typically,\n                # one is sufficient, although sometimes it needs more.\n                for i in range(5):\n                    yield\n                self.server_ioloop.stop()\n\n            self.server_ioloop.add_callback(slow_stop)\n\n        self.server_ioloop.add_callback(stop_server)\n        self.server_thread.join()\n        self.http_client.close()\n        self.server_ioloop.close(all_fds=True)\n\n    def get_url(self, path):\n        return \"http://127.0.0.1:%d%s\" % (self.port, path)\n\n    def test_sync_client(self):\n        response = self.http_client.fetch(self.get_url(\"/\"))\n        self.assertEqual(b\"Hello world!\", response.body)\n\n    def test_sync_client_error(self):\n        # Synchronous HTTPClient raises errors directly; no need for\n        # response.rethrow()\n        with self.assertRaises(HTTPError) as assertion:\n            self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(assertion.exception.code, 404)\n\n\nclass SyncHTTPClientSubprocessTest(unittest.TestCase):\n    def test_destructor_log(self):\n        # Regression test for\n        # https://github.com/tornadoweb/tornado/issues/2539\n        #\n        # In the past, the following program would log an\n        # \"inconsistent AsyncHTTPClient cache\" error from a destructor\n        # when the process is shutting down. The shutdown process is\n        # subtle and I don't fully understand it; the failure does not\n        # manifest if that lambda isn't there or is a simpler object\n        # like an int (nor does it manifest in the tornado test suite\n        # as a whole, which is why we use this subprocess).\n        proc = subprocess.run(\n            [\n                sys.executable,\n                \"-c\",\n                \"from tornado.httpclient import HTTPClient; f = lambda: None; c = HTTPClient()\",\n            ],\n            stdout=subprocess.PIPE,\n            stderr=subprocess.STDOUT,\n            check=True,\n        )\n        if proc.stdout:\n            print(\"STDOUT:\")\n            print(to_unicode(proc.stdout))\n        if proc.stdout:\n            self.fail(\"subprocess produced unexpected output\")\n\n\nclass HTTPRequestTestCase(unittest.TestCase):\n    def test_headers(self):\n        request = HTTPRequest(\"http://example.com\", headers={\"foo\": \"bar\"})\n        self.assertEqual(request.headers, {\"foo\": \"bar\"})\n\n    def test_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = {\"bar\": \"baz\"}  # type: ignore\n        self.assertEqual(request.headers, {\"bar\": \"baz\"})\n\n    def test_null_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = None  # type: ignore\n        self.assertEqual(request.headers, {})\n\n    def test_body(self):\n        request = HTTPRequest(\"http://example.com\", body=\"foo\")\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_body_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.body = \"foo\"  # type: ignore\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_if_modified_since(self):\n        http_date = datetime.datetime.utcnow()\n        request = HTTPRequest(\"http://example.com\", if_modified_since=http_date)\n        self.assertEqual(\n            request.headers, {\"If-Modified-Since\": format_timestamp(http_date)}\n        )\n\n\nclass HTTPErrorTestCase(unittest.TestCase):\n    def test_copy(self):\n        e = HTTPError(403)\n        e2 = copy.copy(e)\n        self.assertIsNot(e, e2)\n        self.assertEqual(e.code, e2.code)\n\n    def test_plain_error(self):\n        e = HTTPError(403)\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n\n    def test_error_with_response(self):\n        resp = HTTPResponse(HTTPRequest(\"http://example.com/\"), 403)\n        with self.assertRaises(HTTPError) as cm:\n            resp.rethrow()\n        e = cm.exception\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n",
            "new_file_content": "# -*- coding: utf-8 -*-\nimport base64\nimport binascii\nfrom contextlib import closing\nimport copy\nimport threading\nimport datetime\nfrom io import BytesIO\nimport subprocess\nimport sys\nimport time\nimport typing  # noqa: F401\nimport unicodedata\nimport unittest\n\nfrom tornado.escape import utf8, native_str, to_unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPRequest,\n    HTTPResponse,\n    _RequestProxy,\n    HTTPError,\n    HTTPClient,\n)\nfrom tornado.httpserver import HTTPServer\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import IOStream\nfrom tornado.log import gen_log, app_log\nfrom tornado import netutil\nfrom tornado.testing import AsyncHTTPTestCase, bind_unused_port, gen_test, ExpectLog\nfrom tornado.test.util import skipOnTravis\nfrom tornado.web import Application, RequestHandler, url\nfrom tornado.httputil import format_timestamp, HTTPHeaders\n\n\nclass HelloWorldHandler(RequestHandler):\n    def get(self):\n        name = self.get_argument(\"name\", \"world\")\n        self.set_header(\"Content-Type\", \"text/plain\")\n        self.finish(\"Hello %s!\" % name)\n\n\nclass PostHandler(RequestHandler):\n    def post(self):\n        self.finish(\n            \"Post arg1: %s, arg2: %s\"\n            % (self.get_argument(\"arg1\"), self.get_argument(\"arg2\"))\n        )\n\n\nclass PutHandler(RequestHandler):\n    def put(self):\n        self.write(\"Put body: \")\n        self.write(self.request.body)\n\n\nclass RedirectHandler(RequestHandler):\n    def prepare(self):\n        self.write(\"redirects can have bodies too\")\n        self.redirect(\n            self.get_argument(\"url\"), status=int(self.get_argument(\"status\", \"302\"))\n        )\n\n\nclass RedirectWithoutLocationHandler(RequestHandler):\n    def prepare(self):\n        # For testing error handling of a redirect with no location header.\n        self.set_status(301)\n        self.finish()\n\n\nclass ChunkHandler(RequestHandler):\n    @gen.coroutine\n    def get(self):\n        self.write(\"asdf\")\n        self.flush()\n        # Wait a bit to ensure the chunks are sent and received separately.\n        yield gen.sleep(0.01)\n        self.write(\"qwer\")\n\n\nclass AuthHandler(RequestHandler):\n    def get(self):\n        self.finish(self.request.headers[\"Authorization\"])\n\n\nclass CountdownHandler(RequestHandler):\n    def get(self, count):\n        count = int(count)\n        if count > 0:\n            self.redirect(self.reverse_url(\"countdown\", count - 1))\n        else:\n            self.write(\"Zero\")\n\n\nclass EchoPostHandler(RequestHandler):\n    def post(self):\n        self.write(self.request.body)\n\n\nclass UserAgentHandler(RequestHandler):\n    def get(self):\n        self.write(self.request.headers.get(\"User-Agent\", \"User agent not set\"))\n\n\nclass ContentLength304Handler(RequestHandler):\n    def get(self):\n        self.set_status(304)\n        self.set_header(\"Content-Length\", 42)\n\n    def _clear_headers_for_304(self):\n        # Tornado strips content-length from 304 responses, but here we\n        # want to simulate servers that include the headers anyway.\n        pass\n\n\nclass PatchHandler(RequestHandler):\n    def patch(self):\n        \"Return the request payload - so we can check it is being kept\"\n        self.write(self.request.body)\n\n\nclass AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = head = post = put = delete = options = patch = other = method  # type: ignore\n\n\nclass SetHeaderHandler(RequestHandler):\n    def get(self):\n        # Use get_arguments for keys to get strings, but\n        # request.arguments for values to get bytes.\n        for k, v in zip(self.get_arguments(\"k\"), self.request.arguments[\"v\"]):\n            self.set_header(k, v)\n\n\n# These tests end up getting run redundantly: once here with the default\n# HTTPClient implementation, and then again in each implementation's own\n# test suite.\n\n\nclass HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/redirect_without_location\", RedirectWithoutLocationHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_redirect_without_location(self):\n        response = self.fetch(\"/redirect_without_location\", follow_redirects=True)\n        # If there is no location header, the redirect response should\n        # just be returned as-is. (This should arguably raise an\n        # error, but libcurl doesn't treat this as an error, so we\n        # don't either).\n        self.assertEqual(301, response.code)\n\n    def test_redirect_put_with_body(self):\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\", method=\"PUT\", body=\"hello\"\n        )\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_redirect_put_without_body(self):\n        # This \"without body\" edge case is similar to what happens with body_producer.\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\",\n            method=\"PUT\",\n            allow_nonstandard_methods=True,\n        )\n        self.assertEqual(response.body, b\"Put body: \")\n\n    def test_method_after_redirect(self):\n        # Legacy redirect codes (301, 302) convert POST requests to GET.\n        for status in [301, 302, 303]:\n            url = \"/redirect?url=/all_methods&status=%d\" % status\n            resp = self.fetch(url, method=\"POST\", body=b\"\")\n            self.assertEqual(b\"GET\", resp.body)\n\n            # Other methods are left alone.\n            for method in [\"GET\", \"OPTIONS\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(utf8(method), resp.body)\n            # HEAD is different so check it separately.\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n        # Newer redirects always preserve the original method.\n        for status in [307, 308]:\n            url = \"/redirect?url=/all_methods&status=307\"\n            for method in [\"GET\", \"OPTIONS\", \"POST\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(method, to_unicode(resp.body))\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n    @gen_test\n    def test_error_after_cancel(self):\n        fut = self.http_client.fetch(self.get_url(\"/404\"))\n        self.assertTrue(fut.cancel())\n        with ExpectLog(app_log, \"Exception after Future was cancelled\") as el:\n            # We can't wait on the cancelled Future any more, so just\n            # let the IOLoop run until the exception gets logged (or\n            # not, in which case we exit the loop and ExpectLog will\n            # raise).\n            for i in range(100):\n                yield gen.sleep(0.01)\n                if el.logged_stack:\n                    break\n\n\nclass RequestProxyTest(unittest.TestCase):\n    def test_request_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", user_agent=\"foo\"), dict()\n        )\n        self.assertEqual(proxy.user_agent, \"foo\")\n\n    def test_default_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\"), dict(network_interface=\"foo\")\n        )\n        self.assertEqual(proxy.network_interface, \"foo\")\n\n    def test_both_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", proxy_host=\"foo\"), dict(proxy_host=\"bar\")\n        )\n        self.assertEqual(proxy.proxy_host, \"foo\")\n\n    def test_neither_set(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        self.assertIs(proxy.auth_username, None)\n\n    def test_bad_attribute(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        with self.assertRaises(AttributeError):\n            proxy.foo\n\n    def test_defaults_none(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), None)\n        self.assertIs(proxy.auth_username, None)\n\n\nclass HTTPResponseTestCase(unittest.TestCase):\n    def test_str(self):\n        response = HTTPResponse(  # type: ignore\n            HTTPRequest(\"http://example.com\"), 200, headers={}, buffer=BytesIO()\n        )\n        s = str(response)\n        self.assertTrue(s.startswith(\"HTTPResponse(\"))\n        self.assertIn(\"code=200\", s)\n\n\nclass SyncHTTPClientTest(unittest.TestCase):\n    def setUp(self):\n        self.server_ioloop = IOLoop()\n        event = threading.Event()\n\n        @gen.coroutine\n        def init_server():\n            sock, self.port = bind_unused_port()\n            app = Application([(\"/\", HelloWorldHandler)])\n            self.server = HTTPServer(app)\n            self.server.add_socket(sock)\n            event.set()\n\n        def start():\n            self.server_ioloop.run_sync(init_server)\n            self.server_ioloop.start()\n\n        self.server_thread = threading.Thread(target=start)\n        self.server_thread.start()\n        event.wait()\n\n        self.http_client = HTTPClient()\n\n    def tearDown(self):\n        def stop_server():\n            self.server.stop()\n            # Delay the shutdown of the IOLoop by several iterations because\n            # the server may still have some cleanup work left when\n            # the client finishes with the response (this is noticeable\n            # with http/2, which leaves a Future with an unexamined\n            # StreamClosedError on the loop).\n\n            @gen.coroutine\n            def slow_stop():\n                yield self.server.close_all_connections()\n                # The number of iterations is difficult to predict. Typically,\n                # one is sufficient, although sometimes it needs more.\n                for i in range(5):\n                    yield\n                self.server_ioloop.stop()\n\n            self.server_ioloop.add_callback(slow_stop)\n\n        self.server_ioloop.add_callback(stop_server)\n        self.server_thread.join()\n        self.http_client.close()\n        self.server_ioloop.close(all_fds=True)\n\n    def get_url(self, path):\n        return \"http://127.0.0.1:%d%s\" % (self.port, path)\n\n    def test_sync_client(self):\n        response = self.http_client.fetch(self.get_url(\"/\"))\n        self.assertEqual(b\"Hello world!\", response.body)\n\n    def test_sync_client_error(self):\n        # Synchronous HTTPClient raises errors directly; no need for\n        # response.rethrow()\n        with self.assertRaises(HTTPError) as assertion:\n            self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(assertion.exception.code, 404)\n\n\nclass SyncHTTPClientSubprocessTest(unittest.TestCase):\n    def test_destructor_log(self):\n        # Regression test for\n        # https://github.com/tornadoweb/tornado/issues/2539\n        #\n        # In the past, the following program would log an\n        # \"inconsistent AsyncHTTPClient cache\" error from a destructor\n        # when the process is shutting down. The shutdown process is\n        # subtle and I don't fully understand it; the failure does not\n        # manifest if that lambda isn't there or is a simpler object\n        # like an int (nor does it manifest in the tornado test suite\n        # as a whole, which is why we use this subprocess).\n        proc = subprocess.run(\n            [\n                sys.executable,\n                \"-c\",\n                \"from tornado.httpclient import HTTPClient; f = lambda: None; c = HTTPClient()\",\n            ],\n            stdout=subprocess.PIPE,\n            stderr=subprocess.STDOUT,\n            check=True,\n        )\n        if proc.stdout:\n            print(\"STDOUT:\")\n            print(to_unicode(proc.stdout))\n        if proc.stdout:\n            self.fail(\"subprocess produced unexpected output\")\n\n\nclass HTTPRequestTestCase(unittest.TestCase):\n    def test_headers(self):\n        request = HTTPRequest(\"http://example.com\", headers={\"foo\": \"bar\"})\n        self.assertEqual(request.headers, {\"foo\": \"bar\"})\n\n    def test_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = {\"bar\": \"baz\"}  # type: ignore\n        self.assertEqual(request.headers, {\"bar\": \"baz\"})\n\n    def test_null_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = None  # type: ignore\n        self.assertEqual(request.headers, {})\n\n    def test_body(self):\n        request = HTTPRequest(\"http://example.com\", body=\"foo\")\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_body_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.body = \"foo\"  # type: ignore\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_if_modified_since(self):\n        http_date = datetime.datetime.utcnow()\n        request = HTTPRequest(\"http://example.com\", if_modified_since=http_date)\n        self.assertEqual(\n            request.headers, {\"If-Modified-Since\": format_timestamp(http_date)}\n        )\n\n\nclass HTTPErrorTestCase(unittest.TestCase):\n    def test_copy(self):\n        e = HTTPError(403)\n        e2 = copy.copy(e)\n        self.assertIsNot(e, e2)\n        self.assertEqual(e.code, e2.code)\n\n    def test_plain_error(self):\n        e = HTTPError(403)\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n\n    def test_error_with_response(self):\n        resp = HTTPResponse(HTTPRequest(\"http://example.com/\"), 403)\n        with self.assertRaises(HTTPError) as cm:\n            resp.rethrow()\n        e = cm.exception\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n",
            "header": {
                "file": {
                    "path": "tornado/test/httpclient_test.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "9adbcea4",
                "new_commit_hash": "26a1ad7f",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/tornado/test/httpclient_test.py"
            },
            "plus_file": {
                "path": "b/tornado/test/httpclient_test.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 126,
                            "length": 7
                        },
                        "new_range": {
                            "start": 126,
                            "length": 7
                        },
                        "section": "class AllMethodsHandler(RequestHandler):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "    def method(self):",
                                "type": "context"
                            },
                            {
                                "content": "        self.write(self.request.method)",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "    get = post = put = delete = options = patch = other = method  # type: ignore",
                                "type": "deleted"
                            },
                            {
                                "content": "    get = head = post = put = delete = options = patch = other = method  # type: ignore",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "class SetHeaderHandler(RequestHandler):",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/test/httpclient_test.py",
                            "type": "class",
                            "name": "AllMethodsHandler",
                            "content": "class AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = post = put = delete = options = patch = other = method",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 123,
                            "end_lineno": 129,
                            "parent": null
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 322,
                            "length": 6
                        },
                        "new_range": {
                            "start": 322,
                            "length": 32
                        },
                        "section": "Transfer-Encoding: chunked"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "        )",
                                "type": "context"
                            },
                            {
                                "content": "        self.assertEqual(response.body, b\"Put body: \")",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "    def test_method_after_redirect(self):",
                                "type": "added"
                            },
                            {
                                "content": "        # Legacy redirect codes (301, 302) convert POST requests to GET.",
                                "type": "added"
                            },
                            {
                                "content": "        for status in [301, 302, 303]:",
                                "type": "added"
                            },
                            {
                                "content": "            url = \"/redirect?url=/all_methods&status=%d\" % status",
                                "type": "added"
                            },
                            {
                                "content": "            resp = self.fetch(url, method=\"POST\", body=b\"\")",
                                "type": "added"
                            },
                            {
                                "content": "            self.assertEqual(b\"GET\", resp.body)",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "            # Other methods are left alone.",
                                "type": "added"
                            },
                            {
                                "content": "            for method in [\"GET\", \"OPTIONS\", \"PUT\", \"DELETE\"]:",
                                "type": "added"
                            },
                            {
                                "content": "                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)",
                                "type": "added"
                            },
                            {
                                "content": "                self.assertEqual(utf8(method), resp.body)",
                                "type": "added"
                            },
                            {
                                "content": "            # HEAD is different so check it separately.",
                                "type": "added"
                            },
                            {
                                "content": "            resp = self.fetch(url, method=\"HEAD\")",
                                "type": "added"
                            },
                            {
                                "content": "            self.assertEqual(200, resp.code)",
                                "type": "added"
                            },
                            {
                                "content": "            self.assertEqual(b\"\", resp.body)",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "        # Newer redirects always preserve the original method.",
                                "type": "added"
                            },
                            {
                                "content": "        for status in [307, 308]:",
                                "type": "added"
                            },
                            {
                                "content": "            url = \"/redirect?url=/all_methods&status=307\"",
                                "type": "added"
                            },
                            {
                                "content": "            for method in [\"GET\", \"OPTIONS\", \"POST\", \"PUT\", \"DELETE\"]:",
                                "type": "added"
                            },
                            {
                                "content": "                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)",
                                "type": "added"
                            },
                            {
                                "content": "                self.assertEqual(method, to_unicode(resp.body))",
                                "type": "added"
                            },
                            {
                                "content": "            resp = self.fetch(url, method=\"HEAD\")",
                                "type": "added"
                            },
                            {
                                "content": "            self.assertEqual(200, resp.code)",
                                "type": "added"
                            },
                            {
                                "content": "            self.assertEqual(b\"\", resp.body)",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "    def test_credentials_in_url(self):",
                                "type": "context"
                            },
                            {
                                "content": "        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")",
                                "type": "context"
                            },
                            {
                                "content": "        response = self.fetch(url)",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/test/httpclient_test.py",
                            "type": "class",
                            "name": "HTTPClientCommonTestCase",
                            "content": "class HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/redirect_without_location\", RedirectWithoutLocationHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_redirect_without_location(self):\n        response = self.fetch(\"/redirect_without_location\", follow_redirects=True)\n        # If there is no location header, the redirect response should\n        # just be returned as-is. (This should arguably raise an\n        # error, but libcurl doesn't treat this as an error, so we\n        # don't either).\n        self.assertEqual(301, response.code)\n\n    def test_redirect_put_with_body(self):\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\", method=\"PUT\", body=\"hello\"\n        )\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_redirect_put_without_body(self):\n        # This \"without body\" edge case is similar to what happens with body_producer.\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\",\n            method=\"PUT\",\n            allow_nonstandard_methods=True,\n        )\n        self.assertEqual(response.body, b\"Put body: \")\n\n    def test_method_after_redirect(self):\n        # Legacy redirect codes (301, 302) convert POST requests to GET.\n        for status in [301, 302, 303]:\n            url = \"/redirect?url=/all_methods&status=%d\" % status\n            resp = self.fetch(url, method=\"POST\", body=b\"\")\n            self.assertEqual(b\"GET\", resp.body)\n\n            # Other methods are left alone.\n            for method in [\"GET\", \"OPTIONS\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(utf8(method), resp.body)\n            # HEAD is different so check it separately.\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n        # Newer redirects always preserve the original method.\n        for status in [307, 308]:\n            url = \"/redirect?url=/all_methods&status=307\"\n            for method in [\"GET\", \"OPTIONS\", \"POST\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(method, to_unicode(resp.body))\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n    @gen_test\n    def test_error_after_cancel(self):\n        fut = self.http_client.fetch(self.get_url(\"/404\"))\n        self.assertTrue(fut.cancel())\n        with ExpectLog(app_log, \"Exception after Future was cancelled\") as el:\n            # We can't wait on the cancelled Future any more, so just\n            # let the IOLoop run until the exception gets logged (or\n            # not, in which case we exit the loop and ExpectLog will\n            # raise).\n            for i in range(100):\n                yield gen.sleep(0.01)\n                if el.logged_stack:\n                    break",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 145,
                            "end_lineno": 644,
                            "parent": null
                        }
                    ],
                    "added_entities": [
                        {
                            "file_name": "tornado/test/httpclient_test.py",
                            "type": "method",
                            "name": "HTTPClientCommonTestCase.test_method_after_redirect",
                            "content": "class HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/redirect_without_location\", RedirectWithoutLocationHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_redirect_without_location(self):\n        response = self.fetch(\"/redirect_without_location\", follow_redirects=True)\n        # If there is no location header, the redirect response should\n        # just be returned as-is. (This should arguably raise an\n        # error, but libcurl doesn't treat this as an error, so we\n        # don't either).\n        self.assertEqual(301, response.code)\n\n    def test_redirect_put_with_body(self):\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\", method=\"PUT\", body=\"hello\"\n        )\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_redirect_put_without_body(self):\n        # This \"without body\" edge case is similar to what happens with body_producer.\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\",\n            method=\"PUT\",\n            allow_nonstandard_methods=True,\n        )\n        self.assertEqual(response.body, b\"Put body: \")\n\n    def test_method_after_redirect(self):\n        # Legacy redirect codes (301, 302) convert POST requests to GET.\n        for status in [301, 302, 303]:\n            url = \"/redirect?url=/all_methods&status=%d\" % status\n            resp = self.fetch(url, method=\"POST\", body=b\"\")\n            self.assertEqual(b\"GET\", resp.body)\n\n            # Other methods are left alone.\n            for method in [\"GET\", \"OPTIONS\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(utf8(method), resp.body)\n            # HEAD is different so check it separately.\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n        # Newer redirects always preserve the original method.\n        for status in [307, 308]:\n            url = \"/redirect?url=/all_methods&status=307\"\n            for method in [\"GET\", \"OPTIONS\", \"POST\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(method, to_unicode(resp.body))\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n    @gen_test\n    def test_error_after_cancel(self):\n        fut = self.http_client.fetch(self.get_url(\"/404\"))\n        self.assertTrue(fut.cancel())\n        with ExpectLog(app_log, \"Exception after Future was cancelled\") as el:\n            # We can't wait on the cancelled Future any more, so just\n            # let the IOLoop run until the exception gets logged (or\n            # not, in which case we exit the loop and ExpectLog will\n            # raise).\n            for i in range(100):\n                yield gen.sleep(0.01)\n                if el.logged_stack:\n                    break",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 325,
                            "end_lineno": 349,
                            "parent": {
                                "file_name": "tornado/test/httpclient_test.py",
                                "type": "class",
                                "name": "HTTPClientCommonTestCase",
                                "content": "class HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/redirect_without_location\", RedirectWithoutLocationHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_redirect_without_location(self):\n        response = self.fetch(\"/redirect_without_location\", follow_redirects=True)\n        # If there is no location header, the redirect response should\n        # just be returned as-is. (This should arguably raise an\n        # error, but libcurl doesn't treat this as an error, so we\n        # don't either).\n        self.assertEqual(301, response.code)\n\n    def test_redirect_put_with_body(self):\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\", method=\"PUT\", body=\"hello\"\n        )\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_redirect_put_without_body(self):\n        # This \"without body\" edge case is similar to what happens with body_producer.\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\",\n            method=\"PUT\",\n            allow_nonstandard_methods=True,\n        )\n        self.assertEqual(response.body, b\"Put body: \")\n\n    def test_method_after_redirect(self):\n        # Legacy redirect codes (301, 302) convert POST requests to GET.\n        for status in [301, 302, 303]:\n            url = \"/redirect?url=/all_methods&status=%d\" % status\n            resp = self.fetch(url, method=\"POST\", body=b\"\")\n            self.assertEqual(b\"GET\", resp.body)\n\n            # Other methods are left alone.\n            for method in [\"GET\", \"OPTIONS\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(utf8(method), resp.body)\n            # HEAD is different so check it separately.\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n        # Newer redirects always preserve the original method.\n        for status in [307, 308]:\n            url = \"/redirect?url=/all_methods&status=307\"\n            for method in [\"GET\", \"OPTIONS\", \"POST\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(method, to_unicode(resp.body))\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n    @gen_test\n    def test_error_after_cancel(self):\n        fut = self.http_client.fetch(self.get_url(\"/404\"))\n        self.assertTrue(fut.cancel())\n        with ExpectLog(app_log, \"Exception after Future was cancelled\") as el:\n            # We can't wait on the cancelled Future any more, so just\n            # let the IOLoop run until the exception gets logged (or\n            # not, in which case we exit the loop and ExpectLog will\n            # raise).\n            for i in range(100):\n                yield gen.sleep(0.01)\n                if el.logged_stack:\n                    break",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 145,
                                "end_lineno": 644,
                                "parent": null
                            }
                        }
                    ],
                    "deleted_entities": []
                }
            ]
        }
    ],
    "old_commit_hash": "c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d^",
    "new_commit_hash": "c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d",
    "commit_message": "simple_httpclient: Only convert POST requests to GETs on redirect\n\nConvert POST to GET on 301 in addition to 302 and 303.",
    "commit_date": "2019-01-01T12:55:58-05:00",
    "metadata": {}
}
{
    "repo_name": "tornado",
    "new_commit_hash": "c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d",
    "test_file_codes": [
        "# -*- coding: utf-8 -*-\nimport base64\nimport binascii\nfrom contextlib import closing\nimport copy\nimport threading\nimport datetime\nfrom io import BytesIO\nimport subprocess\nimport sys\nimport time\nimport typing  # noqa: F401\nimport unicodedata\nimport unittest\n\nfrom tornado.escape import utf8, native_str, to_unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPRequest,\n    HTTPResponse,\n    _RequestProxy,\n    HTTPError,\n    HTTPClient,\n)\nfrom tornado.httpserver import HTTPServer\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import IOStream\nfrom tornado.log import gen_log, app_log\nfrom tornado import netutil\nfrom tornado.testing import AsyncHTTPTestCase, bind_unused_port, gen_test, ExpectLog\nfrom tornado.test.util import skipOnTravis\nfrom tornado.web import Application, RequestHandler, url\nfrom tornado.httputil import format_timestamp, HTTPHeaders\n\n\nclass HelloWorldHandler(RequestHandler):\n    def get(self):\n        name = self.get_argument(\"name\", \"world\")\n        self.set_header(\"Content-Type\", \"text/plain\")\n        self.finish(\"Hello %s!\" % name)\n\n\nclass PostHandler(RequestHandler):\n    def post(self):\n        self.finish(\n            \"Post arg1: %s, arg2: %s\"\n            % (self.get_argument(\"arg1\"), self.get_argument(\"arg2\"))\n        )\n\n\nclass PutHandler(RequestHandler):\n    def put(self):\n        self.write(\"Put body: \")\n        self.write(self.request.body)\n\n\nclass RedirectHandler(RequestHandler):\n    def prepare(self):\n        self.write(\"redirects can have bodies too\")\n        self.redirect(\n            self.get_argument(\"url\"), status=int(self.get_argument(\"status\", \"302\"))\n        )\n\n\nclass RedirectWithoutLocationHandler(RequestHandler):\n    def prepare(self):\n        # For testing error handling of a redirect with no location header.\n        self.set_status(301)\n        self.finish()\n\n\nclass ChunkHandler(RequestHandler):\n    @gen.coroutine\n    def get(self):\n        self.write(\"asdf\")\n        self.flush()\n        # Wait a bit to ensure the chunks are sent and received separately.\n        yield gen.sleep(0.01)\n        self.write(\"qwer\")\n\n\nclass AuthHandler(RequestHandler):\n    def get(self):\n        self.finish(self.request.headers[\"Authorization\"])\n\n\nclass CountdownHandler(RequestHandler):\n    def get(self, count):\n        count = int(count)\n        if count > 0:\n            self.redirect(self.reverse_url(\"countdown\", count - 1))\n        else:\n            self.write(\"Zero\")\n\n\nclass EchoPostHandler(RequestHandler):\n    def post(self):\n        self.write(self.request.body)\n\n\nclass UserAgentHandler(RequestHandler):\n    def get(self):\n        self.write(self.request.headers.get(\"User-Agent\", \"User agent not set\"))\n\n\nclass ContentLength304Handler(RequestHandler):\n    def get(self):\n        self.set_status(304)\n        self.set_header(\"Content-Length\", 42)\n\n    def _clear_headers_for_304(self):\n        # Tornado strips content-length from 304 responses, but here we\n        # want to simulate servers that include the headers anyway.\n        pass\n\n\nclass PatchHandler(RequestHandler):\n    def patch(self):\n        \"Return the request payload - so we can check it is being kept\"\n        self.write(self.request.body)\n\n\nclass AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = head = post = put = delete = options = patch = other = method  # type: ignore\n\n\nclass SetHeaderHandler(RequestHandler):\n    def get(self):\n        # Use get_arguments for keys to get strings, but\n        # request.arguments for values to get bytes.\n        for k, v in zip(self.get_arguments(\"k\"), self.request.arguments[\"v\"]):\n            self.set_header(k, v)\n\n\n# These tests end up getting run redundantly: once here with the default\n# HTTPClient implementation, and then again in each implementation's own\n# test suite.\n\n\nclass HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/redirect_without_location\", RedirectWithoutLocationHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_redirect_without_location(self):\n        response = self.fetch(\"/redirect_without_location\", follow_redirects=True)\n        # If there is no location header, the redirect response should\n        # just be returned as-is. (This should arguably raise an\n        # error, but libcurl doesn't treat this as an error, so we\n        # don't either).\n        self.assertEqual(301, response.code)\n\n    def test_redirect_put_with_body(self):\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\", method=\"PUT\", body=\"hello\"\n        )\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_redirect_put_without_body(self):\n        # This \"without body\" edge case is similar to what happens with body_producer.\n        response = self.fetch(\n            \"/redirect?url=/put&status=307\",\n            method=\"PUT\",\n            allow_nonstandard_methods=True,\n        )\n        self.assertEqual(response.body, b\"Put body: \")\n\n    def test_method_after_redirect(self):\n        # Legacy redirect codes (301, 302) convert POST requests to GET.\n        for status in [301, 302, 303]:\n            url = \"/redirect?url=/all_methods&status=%d\" % status\n            resp = self.fetch(url, method=\"POST\", body=b\"\")\n            self.assertEqual(b\"GET\", resp.body)\n\n            # Other methods are left alone.\n            for method in [\"GET\", \"OPTIONS\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(utf8(method), resp.body)\n            # HEAD is different so check it separately.\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n        # Newer redirects always preserve the original method.\n        for status in [307, 308]:\n            url = \"/redirect?url=/all_methods&status=307\"\n            for method in [\"GET\", \"OPTIONS\", \"POST\", \"PUT\", \"DELETE\"]:\n                resp = self.fetch(url, method=method, allow_nonstandard_methods=True)\n                self.assertEqual(method, to_unicode(resp.body))\n            resp = self.fetch(url, method=\"HEAD\")\n            self.assertEqual(200, resp.code)\n            self.assertEqual(b\"\", resp.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n    @gen_test\n    def test_error_after_cancel(self):\n        fut = self.http_client.fetch(self.get_url(\"/404\"))\n        self.assertTrue(fut.cancel())\n        with ExpectLog(app_log, \"Exception after Future was cancelled\") as el:\n            # We can't wait on the cancelled Future any more, so just\n            # let the IOLoop run until the exception gets logged (or\n            # not, in which case we exit the loop and ExpectLog will\n            # raise).\n            for i in range(100):\n                yield gen.sleep(0.01)\n                if el.logged_stack:\n                    break\n\n\nclass RequestProxyTest(unittest.TestCase):\n    def test_request_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", user_agent=\"foo\"), dict()\n        )\n        self.assertEqual(proxy.user_agent, \"foo\")\n\n    def test_default_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\"), dict(network_interface=\"foo\")\n        )\n        self.assertEqual(proxy.network_interface, \"foo\")\n\n    def test_both_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", proxy_host=\"foo\"), dict(proxy_host=\"bar\")\n        )\n        self.assertEqual(proxy.proxy_host, \"foo\")\n\n    def test_neither_set(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        self.assertIs(proxy.auth_username, None)\n\n    def test_bad_attribute(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        with self.assertRaises(AttributeError):\n            proxy.foo\n\n    def test_defaults_none(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), None)\n        self.assertIs(proxy.auth_username, None)\n\n\nclass HTTPResponseTestCase(unittest.TestCase):\n    def test_str(self):\n        response = HTTPResponse(  # type: ignore\n            HTTPRequest(\"http://example.com\"), 200, headers={}, buffer=BytesIO()\n        )\n        s = str(response)\n        self.assertTrue(s.startswith(\"HTTPResponse(\"))\n        self.assertIn(\"code=200\", s)\n\n\nclass SyncHTTPClientTest(unittest.TestCase):\n    def setUp(self):\n        self.server_ioloop = IOLoop()\n        event = threading.Event()\n\n        @gen.coroutine\n        def init_server():\n            sock, self.port = bind_unused_port()\n            app = Application([(\"/\", HelloWorldHandler)])\n            self.server = HTTPServer(app)\n            self.server.add_socket(sock)\n            event.set()\n\n        def start():\n            self.server_ioloop.run_sync(init_server)\n            self.server_ioloop.start()\n\n        self.server_thread = threading.Thread(target=start)\n        self.server_thread.start()\n        event.wait()\n\n        self.http_client = HTTPClient()\n\n    def tearDown(self):\n        def stop_server():\n            self.server.stop()\n            # Delay the shutdown of the IOLoop by several iterations because\n            # the server may still have some cleanup work left when\n            # the client finishes with the response (this is noticeable\n            # with http/2, which leaves a Future with an unexamined\n            # StreamClosedError on the loop).\n\n            @gen.coroutine\n            def slow_stop():\n                yield self.server.close_all_connections()\n                # The number of iterations is difficult to predict. Typically,\n                # one is sufficient, although sometimes it needs more.\n                for i in range(5):\n                    yield\n                self.server_ioloop.stop()\n\n            self.server_ioloop.add_callback(slow_stop)\n\n        self.server_ioloop.add_callback(stop_server)\n        self.server_thread.join()\n        self.http_client.close()\n        self.server_ioloop.close(all_fds=True)\n\n    def get_url(self, path):\n        return \"http://127.0.0.1:%d%s\" % (self.port, path)\n\n    def test_sync_client(self):\n        response = self.http_client.fetch(self.get_url(\"/\"))\n        self.assertEqual(b\"Hello world!\", response.body)\n\n    def test_sync_client_error(self):\n        # Synchronous HTTPClient raises errors directly; no need for\n        # response.rethrow()\n        with self.assertRaises(HTTPError) as assertion:\n            self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(assertion.exception.code, 404)\n\n\nclass SyncHTTPClientSubprocessTest(unittest.TestCase):\n    def test_destructor_log(self):\n        # Regression test for\n        # https://github.com/tornadoweb/tornado/issues/2539\n        #\n        # In the past, the following program would log an\n        # \"inconsistent AsyncHTTPClient cache\" error from a destructor\n        # when the process is shutting down. The shutdown process is\n        # subtle and I don't fully understand it; the failure does not\n        # manifest if that lambda isn't there or is a simpler object\n        # like an int (nor does it manifest in the tornado test suite\n        # as a whole, which is why we use this subprocess).\n        proc = subprocess.run(\n            [\n                sys.executable,\n                \"-c\",\n                \"from tornado.httpclient import HTTPClient; f = lambda: None; c = HTTPClient()\",\n            ],\n            stdout=subprocess.PIPE,\n            stderr=subprocess.STDOUT,\n            check=True,\n        )\n        if proc.stdout:\n            print(\"STDOUT:\")\n            print(to_unicode(proc.stdout))\n        if proc.stdout:\n            self.fail(\"subprocess produced unexpected output\")\n\n\nclass HTTPRequestTestCase(unittest.TestCase):\n    def test_headers(self):\n        request = HTTPRequest(\"http://example.com\", headers={\"foo\": \"bar\"})\n        self.assertEqual(request.headers, {\"foo\": \"bar\"})\n\n    def test_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = {\"bar\": \"baz\"}  # type: ignore\n        self.assertEqual(request.headers, {\"bar\": \"baz\"})\n\n    def test_null_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = None  # type: ignore\n        self.assertEqual(request.headers, {})\n\n    def test_body(self):\n        request = HTTPRequest(\"http://example.com\", body=\"foo\")\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_body_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.body = \"foo\"  # type: ignore\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_if_modified_since(self):\n        http_date = datetime.datetime.utcnow()\n        request = HTTPRequest(\"http://example.com\", if_modified_since=http_date)\n        self.assertEqual(\n            request.headers, {\"If-Modified-Since\": format_timestamp(http_date)}\n        )\n\n\nclass HTTPErrorTestCase(unittest.TestCase):\n    def test_copy(self):\n        e = HTTPError(403)\n        e2 = copy.copy(e)\n        self.assertIsNot(e, e2)\n        self.assertEqual(e.code, e2.code)\n\n    def test_plain_error(self):\n        e = HTTPError(403)\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n\n    def test_error_with_response(self):\n        resp = HTTPResponse(HTTPRequest(\"http://example.com/\"), 403)\n        with self.assertRaises(HTTPError) as cm:\n            resp.rethrow()\n        e = cm.exception\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n",
        "#!/usr/bin/env python\nimport unittest\nimport sys\nimport time\nimport traceback\nimport platform\nimport os\n\n\nclass PytestLikeResult(unittest.TestResult):\n    def __init__(self, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n        self.start_time = None\n        # Store (test, outcome, info) for each test:\n        #   outcome ∈ { \"passed\", \"failed\", \"error\", \"skipped\" }\n        #   info can hold the exception or skip reason, etc.\n        self.results = []\n\n    def startTestRun(self):\n        \"\"\"Called once before all tests.\"\"\"\n        self.start_time = time.time()\n        # Print a header similar to pytest\n        print(\"=\" * 29 + \" test session starts \" + \"=\" * 29)\n        # You can customize the platform/versions/paths as you like:\n        print(\n            f\"platform {platform.system().lower()} -- Python {platform.python_version()}\"\n        )\n        print(f\"rootdir: {os.path.abspath(os.curdir)}\")\n\n    def stopTestRun(self):\n        \"\"\"Called once after all tests are done.\"\"\"\n        elapsed = time.time() - self.start_time\n        print()  # blank line after final dot\n\n        # Summarize\n        ntests = len(self.results)\n        print(f\"collected {ntests} items\")\n\n        # Show details of failures/errors\n        failures = [r for r in self.results if r[1] == \"failed\"]\n        errors = [r for r in self.results if r[1] == \"error\"]\n\n        if failures:\n            print(\"\\n\" + \"=\" * 35 + \" FAILURES \" + \"=\" * 35)\n            for test, outcome, err_info in failures:\n                test = test.split(\"::\")\n                if len(test) == 3:\n                    test = f\"{test[1]}.{test[2]}\"\n                else:\n                    test = f\"{test[1]}\"\n                print(f\"__________ {test} __________\")\n                self._print_traceback(err_info)\n        if errors:\n            print(\"\\n\" + \"=\" * 35 + \" ERRORS \" + \"=\" * 37)\n            for test, outcome, err_info in errors:\n                test = test.split(\"::\")\n                if len(test) == 3:\n                    test = f\"{test[1]}.{test[2]}\"\n                else:\n                    test = f\"{test[1]}\"\n                print(f\"__________ {test} __________\")\n                self._print_traceback(err_info)\n\n        # Optional short summary lines for passes/fails/skips\n        print(\"\\n==================== short test summary info ====================\")\n        for test, outcome, info in self.results:\n            if outcome in (\"passed\", \"failed\", \"error\", \"skipped\"):\n                print(f\"{outcome.upper()} {test}\")\n\n        # Final one-line summary: “1 failed, 1 errors, 10 passed in 2.04s”\n        passed_count = sum(1 for r in self.results if r[1] == \"passed\")\n        failed_count = len(failures)\n        error_count = len(errors)\n        skipped_count = sum(1 for r in self.results if r[1] == \"skipped\")\n\n        summary_bits = []\n        if failed_count:\n            summary_bits.append(f\"{failed_count} failed\")\n        if error_count:\n            summary_bits.append(f\"{error_count} error{'s' if error_count != 1 else ''}\")\n        if passed_count:\n            summary_bits.append(f\"{passed_count} passed\")\n        if skipped_count:\n            summary_bits.append(f\"{skipped_count} skipped\")\n\n        summary_str = \", \".join(summary_bits) if summary_bits else \"no tests run\"\n        print(\n            f\"=================== {summary_str} in {elapsed:.2f}s ===================\"\n        )\n\n    def startTest(self, test):\n        \"\"\"Called right before each test method.\"\"\"\n        super().startTest(test)\n        # Print a dot or some indicator\n        print(\".\", end=\"\", flush=True)\n\n    def addSuccess(self, test):\n        super().addSuccess(test)\n        self.results.append((self._test_id(test), \"passed\", None))\n\n    def addFailure(self, test, err):\n        super().addFailure(test, err)\n        self.results.append((self._test_id(test), \"failed\", err))\n\n    def addError(self, test, err):\n        super().addError(test, err)\n        self.results.append((self._test_id(test), \"error\", err))\n\n    def addSkip(self, test, reason):\n        super().addSkip(test, reason)\n        self.results.append((self._test_id(test), \"skipped\", reason))\n\n    def _print_traceback(self, err_info):\n        \"\"\"Helper to print the traceback like pytest does.\"\"\"\n        if isinstance(err_info, tuple) and len(err_info) == 3:\n            exc_type, exc_value, tb = err_info\n            traceback.print_exception(exc_type, exc_value, tb, file=sys.stdout)\n        else:\n            print(str(err_info))\n\n    def _test_id(self, test):\n        try:\n            return f\"{test.__class__.__module__}::{test.__class__.__name__}::{test._testMethodName}\"\n        except AttributeError:\n            # Fallback for _ErrorHolder objects\n            return f\"{test.__class__.__module__}::{test.__class__.__name__}\"\n\n\nclass PytestLikeRunner(unittest.TextTestRunner):\n    \"\"\"A custom TextTestRunner that uses the PytestLikeResult.\"\"\"\n\n    resultclass = PytestLikeResult\n\n    def run(self, test):\n        result = self._makeResult()\n        # Overriding run to ensure we call startTestRun/stopTestRun\n        result.startTestRun()\n        test(result)\n        result.stopTestRun()\n        return result\n\n    def tearDown(self):\n        # Reset signal handlers\n        from twisted.internet import reactor\n\n        reactor.removeAll()\n        reactor.stop()\n        super().tearDown()\n\n\ndef main():\n    loader = unittest.TestLoader()\n    suite = loader.discover(\".\", pattern=\"test_*.py\")  # discover tests in r2e_tests/\n    runner = PytestLikeRunner(verbosity=0)\n    result = runner.run(suite)\n    # Exit with code 0 if all tests passed/skipped, 1 if there were fails/errors\n    sys.exit(not result.wasSuccessful())\n\n\nif __name__ == \"__main__\":\n    main()\n"
    ],
    "test_file_names": [
        "test_1.py",
        "tornado_unittest_runner.py"
    ],
    "setup_res_code": 0,
    "setup_res_stdout": "",
    "setup_res_stderr": "Using CPython 3.9.21\nCreating virtual environment at: .venv\nActivate with: source .venv/bin/activate\nResolved 1 package in 795ms\nPrepared 1 package in 1.04s\nInstalled 1 package in 0.93ms\n + tornado==6.0.dev1 (from file:///home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d)\nResolved 29 packages in 13ms\nInstalled 29 packages in 157ms\n + attrs==24.3.0\n + automat==24.8.1\n + cachetools==5.5.0\n + cffi==1.17.1\n + chardet==5.2.0\n + colorama==0.4.6\n + constantly==23.10.4\n + distlib==0.3.9\n + exceptiongroup==1.2.2\n + filelock==3.16.1\n + hyperlink==21.0.0\n + idna==3.10\n + incremental==24.7.2\n + iniconfig==2.0.0\n + packaging==24.2\n + platformdirs==4.3.6\n + pluggy==1.5.0\n + pycares==4.5.0\n + pycparser==2.22\n + pycurl==7.45.4\n + pyproject-api==1.8.0\n + pytest==8.3.4\n + setuptools==75.6.0\n + tomli==2.2.1\n + tox==4.23.2\n + twisted==24.11.0\n + typing-extensions==4.12.2\n + virtualenv==20.28.0\n + zope-interface==7.2\n",
    "new_commit_res_code": 0,
    "new_commit_res_stdout": "============================= test session starts =============================\nplatform linux -- Python 3.9.21\nrootdir: /home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d\n.....................................................\ncollected 53 items\n\n==================== short test summary info ====================\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_304_with_content_length\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_all_methods\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_explicit_mode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_unicode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_bind_source_ip\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_encoding\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_sanity_checks\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked_close\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_configure_defaults\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_credentials_in_url\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_error_after_cancel\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_follow_redirect\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error_no_raise\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_interface\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_hello_world\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_method_after_redirect\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_multi_line_headers\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_non_ascii_header\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_patch_receives_payload\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_post\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_put_307\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_redirect_put_with_body\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_redirect_put_without_body\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_redirect_without_location\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_response_times\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_reuse_request_from_response\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_streaming_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_unsupported_auth_mode\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_copy\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_error_with_response\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_plain_error\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_if_modified_since\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_null_headers_setter\nPASSED r2e_tests.test_1::HTTPResponseTestCase::test_str\nPASSED r2e_tests.test_1::RequestProxyTest::test_bad_attribute\nPASSED r2e_tests.test_1::RequestProxyTest::test_both_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_default_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_defaults_none\nPASSED r2e_tests.test_1::RequestProxyTest::test_neither_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_request_set\nPASSED r2e_tests.test_1::SyncHTTPClientSubprocessTest::test_destructor_log\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client_error\n=================== 53 passed in 0.39s ===================\n",
    "new_commit_res_stderr": "WARNING:tornado.access:404 GET /404 (127.0.0.1) 0.40ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.41ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.36ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.79ms\n",
    "old_commit_res_code": 1,
    "old_commit_res_stdout": "============================= test session starts =============================\nplatform linux -- Python 3.9.21\nrootdir: /home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d\n.....................................................\ncollected 53 items\n\n=================================== FAILURES ===================================\n__________ HTTPClientCommonTestCase.test_method_after_redirect __________\nTraceback (most recent call last):\n  File \"/home/gcpuser/.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/unittest/case.py\", line 59, in testPartExecutor\n    yield\n  File \"/home/gcpuser/.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/unittest/case.py\", line 592, in run\n    self._callTestMethod(testMethod)\n  File \"/home/gcpuser/.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/unittest/case.py\", line 550, in _callTestMethod\n    method()\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d/tornado/testing.py\", line 94, in __call__\n    result = self.orig_method(*args, **kwargs)\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_c443fb7bf8a87ba8ab02b9a6af9e140cabc0ab0d/r2e_tests/test_1.py\", line 330, in test_method_after_redirect\n    self.assertEqual(b\"GET\", resp.body)\nAssertionError: b'GET' != b'POST'\n\n==================== short test summary info ====================\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_304_with_content_length\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_all_methods\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_explicit_mode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_unicode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_bind_source_ip\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_encoding\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_sanity_checks\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked_close\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_configure_defaults\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_credentials_in_url\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_error_after_cancel\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_follow_redirect\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error_no_raise\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_interface\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_hello_world\nFAILED r2e_tests.test_1::HTTPClientCommonTestCase::test_method_after_redirect\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_multi_line_headers\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_non_ascii_header\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_patch_receives_payload\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_post\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_put_307\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_redirect_put_with_body\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_redirect_put_without_body\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_redirect_without_location\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_response_times\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_reuse_request_from_response\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_streaming_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_unsupported_auth_mode\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_copy\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_error_with_response\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_plain_error\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_if_modified_since\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_null_headers_setter\nPASSED r2e_tests.test_1::HTTPResponseTestCase::test_str\nPASSED r2e_tests.test_1::RequestProxyTest::test_bad_attribute\nPASSED r2e_tests.test_1::RequestProxyTest::test_both_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_default_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_defaults_none\nPASSED r2e_tests.test_1::RequestProxyTest::test_neither_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_request_set\nPASSED r2e_tests.test_1::SyncHTTPClientSubprocessTest::test_destructor_log\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client_error\n=================== 1 failed, 52 passed in 0.27s ===================\n",
    "old_commit_res_stderr": "WARNING:tornado.access:404 GET /404 (127.0.0.1) 0.26ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.23ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.23ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.68ms\n"
}
tornado/simple_httpclient.py tornado/test/httpclient_test.py
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[ISSUE]
**Title:** HTTP client does not convert POST to GET on 301 redirects

**Description:**
When a POST request receives a 301 redirect response, the HTTP client fails to convert the request method to GET. While the client correctly changes POST to GET for 302 and 303 redirects, it leaves the method as POST for 301 redirects. This leads to unexpected behavior when handling redirects.

**Example Code:**
```python
# Simulate a POST request that gets a 301 redirect
url = "/redirect?url=/all_methods&status=301"
response = http_client.fetch(url, method="POST", body=b"")
print(response.body)  # Expected output: b'GET', but actual output: b'POST'
```

**Expected Behavior:**
After receiving a 301 redirect response, the HTTP client should convert the request method from POST to GET and remove the request body, ensuring consistency with how 302 and 303 redirects are handled.

**Actual Behavior:**
The HTTP client retains the request method as POST and preserves the request body, resulting in the response body being `b'POST'` instead of the expected `b'GET'`.

[/ISSUE]
--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.

--- BEGIN FILE ---
```
### tornado/simple_httpclient.py
from tornado.escape import _unicode
from tornado import gen
from tornado.httpclient import (
    HTTPResponse,
    HTTPError,
    AsyncHTTPClient,
    main,
    _RequestProxy,
    HTTPRequest,
)
from tornado import httputil
from tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters
from tornado.ioloop import IOLoop
from tornado.iostream import StreamClosedError, IOStream
from tornado.netutil import (
    Resolver,
    OverrideResolver,
    _client_ssl_defaults,
    is_valid_ip,
)
from tornado.log import gen_log
from tornado.tcpclient import TCPClient

import base64
import collections
import copy
import functools
import re
import socket
import ssl
import sys
import time
from io import BytesIO
import urllib.parse

from typing import Dict, Any, Callable, Optional, Type, Union
from types import TracebackType
import typing

if typing.TYPE_CHECKING:
    from typing import Deque, Tuple, List  # noqa: F401


class HTTPTimeoutError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient on timeout.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPTimeoutError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Timeout"


class HTTPStreamClosedError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.

    When a more specific exception is available (such as `ConnectionResetError`),
    it may be raised instead of this one.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPStreamClosedError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Stream closed"


class SimpleAsyncHTTPClient(AsyncHTTPClient):
    """Non-blocking HTTP client with no external dependencies.

    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.
    Some features found in the curl-based AsyncHTTPClient are not yet
    supported.  In particular, proxies are not supported, connections
    are not reused, and callers cannot select the network interface to be
    used.
    """

    def initialize(  # type: ignore
        self,
        max_clients: int = 10,
        hostname_mapping: Dict[str, str] = None,
        max_buffer_size: int = 104857600,
        resolver: Resolver = None,
        defaults: Dict[str, Any] = None,
        max_header_size: int = None,
        max_body_size: int = None,
    ) -> None:
        """Creates a AsyncHTTPClient.

        Only a single AsyncHTTPClient instance exists per IOLoop
        in order to provide limitations on the number of pending connections.
        ``force_instance=True`` may be used to suppress this behavior.

        Note that because of this implicit reuse, unless ``force_instance``
        is used, only the first call to the constructor actually uses
        its arguments. It is recommended to use the ``configure`` method
        instead of the constructor to ensure that arguments take effect.

        ``max_clients`` is the number of concurrent requests that can be
        in progress; when this limit is reached additional requests will be
        queued. Note that time spent waiting in this queue still counts
        against the ``request_timeout``.

        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.
        It can be used to make local DNS changes when modifying system-wide
        settings like ``/etc/hosts`` is not possible or desirable (e.g. in
        unittests).

        ``max_buffer_size`` (default 100MB) is the number of bytes
        that can be read into memory at once. ``max_body_size``
        (defaults to ``max_buffer_size``) is the largest response body
        that the client will accept.  Without a
        ``streaming_callback``, the smaller of these two limits
        applies; with a ``streaming_callback`` only ``max_body_size``
        does.

        .. versionchanged:: 4.2
           Added the ``max_body_size`` argument.
        """
        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)
        self.max_clients = max_clients
        self.queue = (
            collections.deque()
        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]
        self.active = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]
        self.waiting = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]
        self.max_buffer_size = max_buffer_size
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        # TCPClient could create a Resolver for us, but we have to do it
        # ourselves to support hostname_mapping.
        if resolver:
            self.resolver = resolver
            self.own_resolver = False
        else:
            self.resolver = Resolver()
            self.own_resolver = True
        if hostname_mapping is not None:
            self.resolver = OverrideResolver(
                resolver=self.resolver, mapping=hostname_mapping
            )
        self.tcp_client = TCPClient(resolver=self.resolver)

    def close(self) -> None:
        super(SimpleAsyncHTTPClient, self).close()
        if self.own_resolver:
            self.resolver.close()
        self.tcp_client.close()

    def fetch_impl(
        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]
    ) -> None:
        key = object()
        self.queue.append((key, request, callback))
        if not len(self.active) < self.max_clients:
            assert request.connect_timeout is not None
            assert request.request_timeout is not None
            timeout_handle = self.io_loop.add_timeout(
                self.io_loop.time()
                + min(request.connect_timeout, request.request_timeout),
                functools.partial(self._on_timeout, key, "in request queue"),
            )
        else:
            timeout_handle = None
        self.waiting[key] = (request, callback, timeout_handle)
        self._process_queue()
        if self.queue:
            gen_log.debug(
                "max_clients limit reached, request queued. "
                "%d active, %d queued requests." % (len(self.active), len(self.queue))
            )

    def _process_queue(self) -> None:
        while self.queue and len(self.active) < self.max_clients:
            key, request, callback = self.queue.popleft()
            if key not in self.waiting:
                continue
            self._remove_timeout(key)
            self.active[key] = (request, callback)
            release_callback = functools.partial(self._release_fetch, key)
            self._handle_request(request, release_callback, callback)

    def _connection_class(self) -> type:
        return _HTTPConnection

    def _handle_request(
        self,
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
    ) -> None:
        self._connection_class()(
            self,
            request,
            release_callback,
            final_callback,
            self.max_buffer_size,
            self.tcp_client,
            self.max_header_size,
            self.max_body_size,
        )

    def _release_fetch(self, key: object) -> None:
        del self.active[key]
        self._process_queue()

    def _remove_timeout(self, key: object) -> None:
        if key in self.waiting:
            request, callback, timeout_handle = self.waiting[key]
            if timeout_handle is not None:
                self.io_loop.remove_timeout(timeout_handle)
            del self.waiting[key]

    def _on_timeout(self, key: object, info: str = None) -> None:
        """Timeout callback of request.

        Construct a timeout HTTPResponse when a timeout occurs.

        :arg object key: A simple object to mark the request.
        :info string key: More detailed timeout information.
        """
        request, callback, timeout_handle = self.waiting[key]
        self.queue.remove((key, request, callback))

        error_message = "Timeout {0}".format(info) if info else "Timeout"
        timeout_response = HTTPResponse(
            request,
            599,
            error=HTTPTimeoutError(error_message),
            request_time=self.io_loop.time() - request.start_time,
        )
        self.io_loop.add_callback(callback, timeout_response)
        del self.waiting[key]


class _HTTPConnection(httputil.HTTPMessageDelegate):
    _SUPPORTED_METHODS = set(
        ["GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"]
    )

    def __init__(
        self,
        client: Optional[SimpleAsyncHTTPClient],
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
        max_buffer_size: int,
        tcp_client: TCPClient,
        max_header_size: int,
        max_body_size: int,
    ) -> None:
        self.io_loop = IOLoop.current()
        self.start_time = self.io_loop.time()
        self.start_wall_time = time.time()
        self.client = client
        self.request = request
        self.release_callback = release_callback
        self.final_callback = final_callback
        self.max_buffer_size = max_buffer_size
        self.tcp_client = tcp_client
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        self.code = None  # type: Optional[int]
        self.headers = None  # type: Optional[httputil.HTTPHeaders]
        self.chunks = []  # type: List[bytes]
        self._decompressor = None
        # Timeout handle returned by IOLoop.add_timeout
        self._timeout = None  # type: object
        self._sockaddr = None
        IOLoop.current().add_future(
            gen.convert_yielded(self.run()), lambda f: f.result()
        )

    async def run(self) -> None:
        try:
            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))
            if self.parsed.scheme not in ("http", "https"):
                raise ValueError("Unsupported url scheme: %s" % self.request.url)
            # urlsplit results have hostname and port results, but they
            # didn't support ipv6 literals until python 2.7.
            netloc = self.parsed.netloc
            if "@" in netloc:
                userpass, _, netloc = netloc.rpartition("@")
            host, port = httputil.split_host_and_port(netloc)
            if port is None:
                port = 443 if self.parsed.scheme == "https" else 80
            if re.match(r"^\[.*\]$", host):
                # raw ipv6 addresses in urls are enclosed in brackets
                host = host[1:-1]
            self.parsed_hostname = host  # save final host for _on_connect

            if self.request.allow_ipv6 is False:
                af = socket.AF_INET
            else:
                af = socket.AF_UNSPEC

            ssl_options = self._get_ssl_options(self.parsed.scheme)

            source_ip = None
            if self.request.network_interface:
                if is_valid_ip(self.request.network_interface):
                    source_ip = self.request.network_interface
                else:
                    raise ValueError(
                        "Unrecognized IPv4 or IPv6 address for network_interface, got %r"
                        % (self.request.network_interface,)
                    )

            timeout = min(self.request.connect_timeout, self.request.request_timeout)
            if timeout:
                self._timeout = self.io_loop.add_timeout(
                    self.start_time + timeout,
                    functools.partial(self._on_timeout, "while connecting"),
                )
                stream = await self.tcp_client.connect(
                    host,
                    port,
                    af=af,
                    ssl_options=ssl_options,
                    max_buffer_size=self.max_buffer_size,
                    source_ip=source_ip,
                )

                if self.final_callback is None:
                    # final_callback is cleared if we've hit our timeout.
                    stream.close()
                    return
                self.stream = stream
                self.stream.set_close_callback(self.on_connection_close)
                self._remove_timeout()
                if self.final_callback is None:
                    return
                if self.request.request_timeout:
                    self._timeout = self.io_loop.add_timeout(
                        self.start_time + self.request.request_timeout,
                        functools.partial(self._on_timeout, "during request"),
                    )
                if (
                    self.request.method not in self._SUPPORTED_METHODS
                    and not self.request.allow_nonstandard_methods
                ):
                    raise KeyError("unknown method %s" % self.request.method)
                for key in (
                    "proxy_host",
                    "proxy_port",
                    "proxy_username",
                    "proxy_password",
                    "proxy_auth_mode",
                ):
                    if getattr(self.request, key, None):
                        raise NotImplementedError("%s not supported" % key)
                if "Connection" not in self.request.headers:
                    self.request.headers["Connection"] = "close"
                if "Host" not in self.request.headers:
                    if "@" in self.parsed.netloc:
                        self.request.headers["Host"] = self.parsed.netloc.rpartition(
                            "@"
                        )[-1]
                    else:
                        self.request.headers["Host"] = self.parsed.netloc
                username, password = None, None
                if self.parsed.username is not None:
                    username, password = self.parsed.username, self.parsed.password
                elif self.request.auth_username is not None:
                    username = self.request.auth_username
                    password = self.request.auth_password or ""
                if username is not None:
                    assert password is not None
                    if self.request.auth_mode not in (None, "basic"):
                        raise ValueError(
                            "unsupported auth_mode %s", self.request.auth_mode
                        )
                    self.request.headers["Authorization"] = "Basic " + _unicode(
                        base64.b64encode(
                            httputil.encode_username_password(username, password)
                        )
                    )
                if self.request.user_agent:
                    self.request.headers["User-Agent"] = self.request.user_agent
                if not self.request.allow_nonstandard_methods:
                    # Some HTTP methods nearly always have bodies while others
                    # almost never do. Fail in this case unless the user has
                    # opted out of sanity checks with allow_nonstandard_methods.
                    body_expected = self.request.method in ("POST", "PATCH", "PUT")
                    body_present = (
                        self.request.body is not None
                        or self.request.body_producer is not None
                    )
                    if (body_expected and not body_present) or (
                        body_present and not body_expected
                    ):
                        raise ValueError(
                            "Body must %sbe None for method %s (unless "
                            "allow_nonstandard_methods is true)"
                            % ("not " if body_expected else "", self.request.method)
                        )
                if self.request.expect_100_continue:
                    self.request.headers["Expect"] = "100-continue"
                if self.request.body is not None:
                    # When body_producer is used the caller is responsible for
                    # setting Content-Length (or else chunked encoding will be used).
                    self.request.headers["Content-Length"] = str(len(self.request.body))
                if (
                    self.request.method == "POST"
                    and "Content-Type" not in self.request.headers
                ):
                    self.request.headers[
                        "Content-Type"
                    ] = "application/x-www-form-urlencoded"
                if self.request.decompress_response:
                    self.request.headers["Accept-Encoding"] = "gzip"
                req_path = (self.parsed.path or "/") + (
                    ("?" + self.parsed.query) if self.parsed.query else ""
                )
                self.connection = self._create_connection(stream)
                start_line = httputil.RequestStartLine(
                    self.request.method, req_path, ""
                )
                self.connection.write_headers(start_line, self.request.headers)
                if self.request.expect_100_continue:
                    await self.connection.read_response(self)
                else:
                    await self._write_body(True)
        except Exception:
            if not self._handle_exception(*sys.exc_info()):
                raise

    def _get_ssl_options(
        self, scheme: str
    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:
        if scheme == "https":
            if self.request.ssl_options is not None:
                return self.request.ssl_options
            # If we are using the defaults, don't construct a
            # new SSLContext.
            if (
                self.request.validate_cert
                and self.request.ca_certs is None
                and self.request.client_cert is None
                and self.request.client_key is None
            ):
                return _client_ssl_defaults
            ssl_ctx = ssl.create_default_context(
                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs
            )
            if not self.request.validate_cert:
                ssl_ctx.check_hostname = False
                ssl_ctx.verify_mode = ssl.CERT_NONE
            if self.request.client_cert is not None:
                ssl_ctx.load_cert_chain(
                    self.request.client_cert, self.request.client_key
                )
            if hasattr(ssl, "OP_NO_COMPRESSION"):
                # See netutil.ssl_options_to_context
                ssl_ctx.options |= ssl.OP_NO_COMPRESSION
            return ssl_ctx
        return None

    def _on_timeout(self, info: str = None) -> None:
        """Timeout callback of _HTTPConnection instance.

        Raise a `HTTPTimeoutError` when a timeout occurs.

        :info string key: More detailed timeout information.
        """
        self._timeout = None
        error_message = "Timeout {0}".format(info) if info else "Timeout"
        if self.final_callback is not None:
            self._handle_exception(
                HTTPTimeoutError, HTTPTimeoutError(error_message), None
            )

    def _remove_timeout(self) -> None:
        if self._timeout is not None:
            self.io_loop.remove_timeout(self._timeout)
            self._timeout = None

    def _create_connection(self, stream: IOStream) -> HTTP1Connection:
        stream.set_nodelay(True)
        connection = HTTP1Connection(
            stream,
            True,
            HTTP1ConnectionParameters(
                no_keep_alive=True,
                max_header_size=self.max_header_size,
                max_body_size=self.max_body_size,
                decompress=bool(self.request.decompress_response),
            ),
            self._sockaddr,
        )
        return connection

    async def _write_body(self, start_read: bool) -> None:
        if self.request.body is not None:
            self.connection.write(self.request.body)
        elif self.request.body_producer is not None:
            fut = self.request.body_producer(self.connection.write)
            if fut is not None:
                await fut
        self.connection.finish()
        if start_read:
            try:
                await self.connection.read_response(self)
            except StreamClosedError:
                if not self._handle_exception(*sys.exc_info()):
                    raise

    def _release(self) -> None:
        if self.release_callback is not None:
            release_callback = self.release_callback
            self.release_callback = None  # type: ignore
            release_callback()

    def _run_callback(self, response: HTTPResponse) -> None:
        self._release()
        if self.final_callback is not None:
            final_callback = self.final_callback
            self.final_callback = None  # type: ignore
            self.io_loop.add_callback(final_callback, response)

    def _handle_exception(
        self,
        typ: Optional[Type[BaseException]],
        value: Optional[BaseException],
        tb: Optional[TracebackType],
    ) -> bool:
        if self.final_callback:
            self._remove_timeout()
            if isinstance(value, StreamClosedError):
                if value.real_error is None:
                    value = HTTPStreamClosedError("Stream closed")
                else:
                    value = value.real_error
            self._run_callback(
                HTTPResponse(
                    self.request,
                    599,
                    error=value,
                    request_time=self.io_loop.time() - self.start_time,
                    start_time=self.start_wall_time,
                )
            )

            if hasattr(self, "stream"):
                # TODO: this may cause a StreamClosedError to be raised
                # by the connection's Future.  Should we cancel the
                # connection more gracefully?
                self.stream.close()
            return True
        else:
            # If our callback has already been called, we are probably
            # catching an exception that is not caused by us but rather
            # some child of our callback. Rather than drop it on the floor,
            # pass it along, unless it's just the stream being closed.
            return isinstance(value, StreamClosedError)

    def on_connection_close(self) -> None:
        if self.final_callback is not None:
            message = "Connection closed"
            if self.stream.error:
                raise self.stream.error
            try:
                raise HTTPStreamClosedError(message)
            except HTTPStreamClosedError:
                self._handle_exception(*sys.exc_info())

    async def headers_received(
        self,
        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],
        headers: httputil.HTTPHeaders,
    ) -> None:
        assert isinstance(first_line, httputil.ResponseStartLine)
        if self.request.expect_100_continue and first_line.code == 100:
            await self._write_body(False)
            return
        self.code = first_line.code
        self.reason = first_line.reason
        self.headers = headers

        if self._should_follow_redirect():
            return

        if self.request.header_callback is not None:
            # Reassemble the start line.
            self.request.header_callback("%s %s %s\r\n" % first_line)
            for k, v in self.headers.get_all():
                self.request.header_callback("%s: %s\r\n" % (k, v))
            self.request.header_callback("\r\n")

    def _should_follow_redirect(self) -> bool:
        if self.request.follow_redirects:
            assert self.request.max_redirects is not None
            return (
                self.code in (301, 302, 303, 307, 308)
                and self.request.max_redirects > 0
                and self.headers is not None
                and self.headers.get("Location") is not None
            )
        return False

    def finish(self) -> None:
        assert self.code is not None
        data = b"".join(self.chunks)
        self._remove_timeout()
        original_request = getattr(self.request, "original_request", self.request)
        if self._should_follow_redirect():
            assert isinstance(self.request, _RequestProxy)
            new_request = copy.copy(self.request.request)
            new_request.url = urllib.parse.urljoin(
                self.request.url, self.headers["Location"]
            )
            new_request.max_redirects = self.request.max_redirects - 1
            del new_request.headers["Host"]
            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4
            # Client SHOULD make a GET request after a 303.
            # According to the spec, 302 should be followed by the same
            # method as the original request, but in practice browsers
            # treat 302 the same as 303, and many servers use 302 for
            # compatibility with pre-HTTP/1.1 user agents which don't
            # understand the 303 status.
            if self.code in (302, 303):
                new_request.method = "GET"
                new_request.body = None
                for h in [
                    "Content-Length",
                    "Content-Type",
                    "Content-Encoding",
                    "Transfer-Encoding",
                ]:
                    try:
                        del self.request.headers[h]
                    except KeyError:
                        pass
            new_request.original_request = original_request
            final_callback = self.final_callback
            self.final_callback = None
            self._release()
            fut = self.client.fetch(new_request, raise_error=False)
            fut.add_done_callback(lambda f: final_callback(f.result()))
            self._on_end_request()
            return
        if self.request.streaming_callback:
            buffer = BytesIO()
        else:
            buffer = BytesIO(data)  # TODO: don't require one big string?
        response = HTTPResponse(
            original_request,
            self.code,
            reason=getattr(self, "reason", None),
            headers=self.headers,
            request_time=self.io_loop.time() - self.start_time,
            start_time=self.start_wall_time,
            buffer=buffer,
            effective_url=self.request.url,
        )
        self._run_callback(response)
        self._on_end_request()

    def _on_end_request(self) -> None:
        self.stream.close()

    def data_received(self, chunk: bytes) -> None:
        if self._should_follow_redirect():
            # We're going to follow a redirect so just discard the body.
            return
        if self.request.streaming_callback is not None:
            self.request.streaming_callback(chunk)
        else:
            self.chunks.append(chunk)


if __name__ == "__main__":
    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)
    main()

```
--- END FILE ---

Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits to fix the issue.

Every *SEARCH/REPLACE* edit must use this format:
1. The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
4. The dividing line: =======
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE

Here is an example:

```python
### mathweb/flask/app.py
<<<<<<< SEARCH
from flask import Flask
=======
import math
from flask import Flask
>>>>>>> REPLACE
```
</solution>

Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If you would like to add the line '        print(x)', you must fully write that out, with all those spaces before the code!
Wrap each *SEARCH/REPLACE* edit in a code block as shown in the example above. If you have multiple *SEARCH/REPLACE* edits, use a separate code block for each one.
[ISSUE]
**Title:** HTTP client does not convert POST to GET on 301 redirects

**Description:**
When a POST request receives a 301 redirect response, the HTTP client fails to convert the request method to GET. While the client correctly changes POST to GET for 302 and 303 redirects, it leaves the method as POST for 301 redirects. This leads to unexpected behavior when handling redirects.

**Example Code:**
```python
# Simulate a POST request that gets a 301 redirect
url = "/redirect?url=/all_methods&status=301"
response = http_client.fetch(url, method="POST", body=b"")
print(response.body)  # Expected output: b'GET', but actual output: b'POST'
```

**Expected Behavior:**
After receiving a 301 redirect response, the HTTP client should convert the request method from POST to GET and remove the request body, ensuring consistency with how 302 and 303 redirects are handled.

**Actual Behavior:**
The HTTP client retains the request method as POST and preserves the request body, resulting in the response body being `b'POST'` instead of the expected `b'GET'`.

[/ISSUE]
{
    "HTTPClientCommonTestCase.test_304_with_content_length": "PASSED",
    "HTTPClientCommonTestCase.test_all_methods": "PASSED",
    "HTTPClientCommonTestCase.test_basic_auth": "PASSED",
    "HTTPClientCommonTestCase.test_basic_auth_explicit_mode": "PASSED",
    "HTTPClientCommonTestCase.test_basic_auth_unicode": "PASSED",
    "HTTPClientCommonTestCase.test_bind_source_ip": "PASSED",
    "HTTPClientCommonTestCase.test_body_encoding": "PASSED",
    "HTTPClientCommonTestCase.test_body_sanity_checks": "PASSED",
    "HTTPClientCommonTestCase.test_chunked": "PASSED",
    "HTTPClientCommonTestCase.test_chunked_close": "PASSED",
    "HTTPClientCommonTestCase.test_configure_defaults": "PASSED",
    "HTTPClientCommonTestCase.test_credentials_in_url": "PASSED",
    "HTTPClientCommonTestCase.test_error_after_cancel": "PASSED",
    "HTTPClientCommonTestCase.test_follow_redirect": "PASSED",
    "HTTPClientCommonTestCase.test_future_http_error": "PASSED",
    "HTTPClientCommonTestCase.test_future_http_error_no_raise": "PASSED",
    "HTTPClientCommonTestCase.test_future_interface": "PASSED",
    "HTTPClientCommonTestCase.test_header_callback": "PASSED",
    "HTTPClientCommonTestCase.test_header_types": "PASSED",
    "HTTPClientCommonTestCase.test_hello_world": "PASSED",
    "HTTPClientCommonTestCase.test_method_after_redirect": "PASSED",
    "HTTPClientCommonTestCase.test_multi_line_headers": "PASSED",
    "HTTPClientCommonTestCase.test_non_ascii_header": "PASSED",
    "HTTPClientCommonTestCase.test_patch_receives_payload": "PASSED",
    "HTTPClientCommonTestCase.test_post": "PASSED",
    "HTTPClientCommonTestCase.test_put_307": "PASSED",
    "HTTPClientCommonTestCase.test_redirect_put_with_body": "PASSED",
    "HTTPClientCommonTestCase.test_redirect_put_without_body": "PASSED",
    "HTTPClientCommonTestCase.test_redirect_without_location": "PASSED",
    "HTTPClientCommonTestCase.test_response_times": "PASSED",
    "HTTPClientCommonTestCase.test_reuse_request_from_response": "PASSED",
    "HTTPClientCommonTestCase.test_streaming_callback": "PASSED",
    "HTTPClientCommonTestCase.test_types": "PASSED",
    "HTTPClientCommonTestCase.test_unsupported_auth_mode": "PASSED",
    "HTTPErrorTestCase.test_copy": "PASSED",
    "HTTPErrorTestCase.test_error_with_response": "PASSED",
    "HTTPErrorTestCase.test_plain_error": "PASSED",
    "HTTPRequestTestCase.test_body": "PASSED",
    "HTTPRequestTestCase.test_body_setter": "PASSED",
    "HTTPRequestTestCase.test_headers": "PASSED",
    "HTTPRequestTestCase.test_headers_setter": "PASSED",
    "HTTPRequestTestCase.test_if_modified_since": "PASSED",
    "HTTPRequestTestCase.test_null_headers_setter": "PASSED",
    "HTTPResponseTestCase.test_str": "PASSED",
    "RequestProxyTest.test_bad_attribute": "PASSED",
    "RequestProxyTest.test_both_set": "PASSED",
    "RequestProxyTest.test_default_set": "PASSED",
    "RequestProxyTest.test_defaults_none": "PASSED",
    "RequestProxyTest.test_neither_set": "PASSED",
    "RequestProxyTest.test_request_set": "PASSED",
    "SyncHTTPClientSubprocessTest.test_destructor_log": "PASSED",
    "SyncHTTPClientTest.test_sync_client": "PASSED",
    "SyncHTTPClientTest.test_sync_client_error": "PASSED"
}
diff --git a/tornado/simple_httpclient.py b/tornado/simple_httpclient.py
index 6af9dd15..6e075aa6 100644
--- a/tornado/simple_httpclient.py
+++ b/tornado/simple_httpclient.py
@@ -626,14 +626,17 @@ class _HTTPConnection(httputil.HTTPMessageDelegate):
             )
             new_request.max_redirects = self.request.max_redirects - 1
             del new_request.headers["Host"]
-            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4
-            # Client SHOULD make a GET request after a 303.
-            # According to the spec, 302 should be followed by the same
-            # method as the original request, but in practice browsers
-            # treat 302 the same as 303, and many servers use 302 for
-            # compatibility with pre-HTTP/1.1 user agents which don't
-            # understand the 303 status.
-            if self.code in (302, 303):
+            # https://tools.ietf.org/html/rfc7231#section-6.4
+            #
+            # The original HTTP spec said that after a 301 or 302
+            # redirect, the request method should be preserved.
+            # However, browsers implemented this by changing the
+            # method to GET, and the behavior stuck. 303 redirects
+            # always specified this POST-to-GET behavior (arguably 303
+            # redirects should change *all* requests to GET, but
+            # libcurl only does this for POST so we follow their
+            # example).
+            if self.code in (301, 302, 303) and self.request.method == "POST":
                 new_request.method = "GET"
                 new_request.body = None
                 for h in [

{"tornado/simple_httpclient.py": "from tornado.escape import _unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPResponse,\n    HTTPError,\n    AsyncHTTPClient,\n    main,\n    _RequestProxy,\n    HTTPRequest,\n)\nfrom tornado import httputil\nfrom tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import StreamClosedError, IOStream\nfrom tornado.netutil import (\n    Resolver,\n    OverrideResolver,\n    _client_ssl_defaults,\n    is_valid_ip,\n)\nfrom tornado.log import gen_log\nfrom tornado.tcpclient import TCPClient\n\nimport base64\nimport collections\nimport copy\nimport functools\nimport re\nimport socket\nimport ssl\nimport sys\nimport time\nfrom io import BytesIO\nimport urllib.parse\n\nfrom typing import Dict, Any, Callable, Optional, Type, Union\nfrom types import TracebackType\nimport typing\n\nif typing.TYPE_CHECKING:\n    from typing import Deque, Tuple, List  # noqa: F401\n\n\nclass HTTPTimeoutError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient on timeout.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPTimeoutError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Timeout\"\n\n\nclass HTTPStreamClosedError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.\n\n    When a more specific exception is available (such as `ConnectionResetError`),\n    it may be raised instead of this one.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPStreamClosedError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Stream closed\"\n\n\nclass SimpleAsyncHTTPClient(AsyncHTTPClient):\n    \"\"\"Non-blocking HTTP client with no external dependencies.\n\n    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.\n    Some features found in the curl-based AsyncHTTPClient are not yet\n    supported.  In particular, proxies are not supported, connections\n    are not reused, and callers cannot select the network interface to be\n    used.\n    \"\"\"\n\n    def initialize(  # type: ignore\n        self,\n        max_clients: int = 10,\n        hostname_mapping: Dict[str, str] = None,\n        max_buffer_size: int = 104857600,\n        resolver: Resolver = None,\n        defaults: Dict[str, Any] = None,\n        max_header_size: int = None,\n        max_body_size: int = None,\n    ) -> None:\n        \"\"\"Creates a AsyncHTTPClient.\n\n        Only a single AsyncHTTPClient instance exists per IOLoop\n        in order to provide limitations on the number of pending connections.\n        ``force_instance=True`` may be used to suppress this behavior.\n\n        Note that because of this implicit reuse, unless ``force_instance``\n        is used, only the first call to the constructor actually uses\n        its arguments. It is recommended to use the ``configure`` method\n        instead of the constructor to ensure that arguments take effect.\n\n        ``max_clients`` is the number of concurrent requests that can be\n        in progress; when this limit is reached additional requests will be\n        queued. Note that time spent waiting in this queue still counts\n        against the ``request_timeout``.\n\n        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.\n        It can be used to make local DNS changes when modifying system-wide\n        settings like ``/etc/hosts`` is not possible or desirable (e.g. in\n        unittests).\n\n        ``max_buffer_size`` (default 100MB) is the number of bytes\n        that can be read into memory at once. ``max_body_size``\n        (defaults to ``max_buffer_size``) is the largest response body\n        that the client will accept.  Without a\n        ``streaming_callback``, the smaller of these two limits\n        applies; with a ``streaming_callback`` only ``max_body_size``\n        does.\n\n        .. versionchanged:: 4.2\n           Added the ``max_body_size`` argument.\n        \"\"\"\n        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)\n        self.max_clients = max_clients\n        self.queue = (\n            collections.deque()\n        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.active = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.waiting = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]\n        self.max_buffer_size = max_buffer_size\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        # TCPClient could create a Resolver for us, but we have to do it\n        # ourselves to support hostname_mapping.\n        if resolver:\n            self.resolver = resolver\n            self.own_resolver = False\n        else:\n            self.resolver = Resolver()\n            self.own_resolver = True\n        if hostname_mapping is not None:\n            self.resolver = OverrideResolver(\n                resolver=self.resolver, mapping=hostname_mapping\n            )\n        self.tcp_client = TCPClient(resolver=self.resolver)\n\n    def close(self) -> None:\n        super(SimpleAsyncHTTPClient, self).close()\n        if self.own_resolver:\n            self.resolver.close()\n        self.tcp_client.close()\n\n    def fetch_impl(\n        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]\n    ) -> None:\n        key = object()\n        self.queue.append((key, request, callback))\n        if not len(self.active) < self.max_clients:\n            assert request.connect_timeout is not None\n            assert request.request_timeout is not None\n            timeout_handle = self.io_loop.add_timeout(\n                self.io_loop.time()\n                + min(request.connect_timeout, request.request_timeout),\n                functools.partial(self._on_timeout, key, \"in request queue\"),\n            )\n        else:\n            timeout_handle = None\n        self.waiting[key] = (request, callback, timeout_handle)\n        self._process_queue()\n        if self.queue:\n            gen_log.debug(\n                \"max_clients limit reached, request queued. \"\n                \"%d active, %d queued requests.\" % (len(self.active), len(self.queue))\n            )\n\n    def _process_queue(self) -> None:\n        while self.queue and len(self.active) < self.max_clients:\n            key, request, callback = self.queue.popleft()\n            if key not in self.waiting:\n                continue\n            self._remove_timeout(key)\n            self.active[key] = (request, callback)\n            release_callback = functools.partial(self._release_fetch, key)\n            self._handle_request(request, release_callback, callback)\n\n    def _connection_class(self) -> type:\n        return _HTTPConnection\n\n    def _handle_request(\n        self,\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n    ) -> None:\n        self._connection_class()(\n            self,\n            request,\n            release_callback,\n            final_callback,\n            self.max_buffer_size,\n            self.tcp_client,\n            self.max_header_size,\n            self.max_body_size,\n        )\n\n    def _release_fetch(self, key: object) -> None:\n        del self.active[key]\n        self._process_queue()\n\n    def _remove_timeout(self, key: object) -> None:\n        if key in self.waiting:\n            request, callback, timeout_handle = self.waiting[key]\n            if timeout_handle is not None:\n                self.io_loop.remove_timeout(timeout_handle)\n            del self.waiting[key]\n\n    def _on_timeout(self, key: object, info: str = None) -> None:\n        \"\"\"Timeout callback of request.\n\n        Construct a timeout HTTPResponse when a timeout occurs.\n\n        :arg object key: A simple object to mark the request.\n        :info string key: More detailed timeout information.\n        \"\"\"\n        request, callback, timeout_handle = self.waiting[key]\n        self.queue.remove((key, request, callback))\n\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        timeout_response = HTTPResponse(\n            request,\n            599,\n            error=HTTPTimeoutError(error_message),\n            request_time=self.io_loop.time() - request.start_time,\n        )\n        self.io_loop.add_callback(callback, timeout_response)\n        del self.waiting[key]\n\n\nclass _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n                and self.headers is not None\n                and self.headers.get(\"Location\") is not None\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)\n\n\nif __name__ == \"__main__\":\n    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)\n    main()\n"}
A user will ask you to solve a task. You should first draft your thinking process (inner monologue). Then, generate the solution.

Your response format must follow the template below:
<think>
Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct solution.
</think>
<solution>
Final solution presented to the user.
</solution>
system
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[ISSUE]
**Title:** HTTP client does not convert POST to GET on 301 redirects

**Description:**
When a POST request receives a 301 redirect response, the HTTP client fails to convert the request method to GET. While the client correctly changes POST to GET for 302 and 303 redirects, it leaves the method as POST for 301 redirects. This leads to unexpected behavior when handling redirects.

**Example Code:**
```python
# Simulate a POST request that gets a 301 redirect
url = "/redirect?url=/all_methods&status=301"
response = http_client.fetch(url, method="POST", body=b"")
print(response.body)  # Expected output: b'GET', but actual output: b'POST'
```

**Expected Behavior:**
After receiving a 301 redirect response, the HTTP client should convert the request method from POST to GET and remove the request body, ensuring consistency with how 302 and 303 redirects are handled.

**Actual Behavior:**
The HTTP client retains the request method as POST and preserves the request body, resulting in the response body being `b'POST'` instead of the expected `b'GET'`.

[/ISSUE]
--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.

--- BEGIN FILE ---
```
### tornado/simple_httpclient.py
from tornado.escape import _unicode
from tornado import gen
from tornado.httpclient import (
    HTTPResponse,
    HTTPError,
    AsyncHTTPClient,
    main,
    _RequestProxy,
    HTTPRequest,
)
from tornado import httputil
from tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters
from tornado.ioloop import IOLoop
from tornado.iostream import StreamClosedError, IOStream
from tornado.netutil import (
    Resolver,
    OverrideResolver,
    _client_ssl_defaults,
    is_valid_ip,
)
from tornado.log import gen_log
from tornado.tcpclient import TCPClient

import base64
import collections
import copy
import functools
import re
import socket
import ssl
import sys
import time
from io import BytesIO
import urllib.parse

from typing import Dict, Any, Callable, Optional, Type, Union
from types import TracebackType
import typing

if typing.TYPE_CHECKING:
    from typing import Deque, Tuple, List  # noqa: F401


class HTTPTimeoutError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient on timeout.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPTimeoutError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Timeout"


class HTTPStreamClosedError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.

    When a more specific exception is available (such as `ConnectionResetError`),
    it may be raised instead of this one.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPStreamClosedError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Stream closed"


class SimpleAsyncHTTPClient(AsyncHTTPClient):
    """Non-blocking HTTP client with no external dependencies.

    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.
    Some features found in the curl-based AsyncHTTPClient are not yet
    supported.  In particular, proxies are not supported, connections
    are not reused, and callers cannot select the network interface to be
    used.
    """

    def initialize(  # type: ignore
        self,
        max_clients: int = 10,
        hostname_mapping: Dict[str, str] = None,
        max_buffer_size: int = 104857600,
        resolver: Resolver = None,
        defaults: Dict[str, Any] = None,
        max_header_size: int = None,
        max_body_size: int = None,
    ) -> None:
        """Creates a AsyncHTTPClient.

        Only a single AsyncHTTPClient instance exists per IOLoop
        in order to provide limitations on the number of pending connections.
        ``force_instance=True`` may be used to suppress this behavior.

        Note that because of this implicit reuse, unless ``force_instance``
        is used, only the first call to the constructor actually uses
        its arguments. It is recommended to use the ``configure`` method
        instead of the constructor to ensure that arguments take effect.

        ``max_clients`` is the number of concurrent requests that can be
        in progress; when this limit is reached additional requests will be
        queued. Note that time spent waiting in this queue still counts
        against the ``request_timeout``.

        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.
        It can be used to make local DNS changes when modifying system-wide
        settings like ``/etc/hosts`` is not possible or desirable (e.g. in
        unittests).

        ``max_buffer_size`` (default 100MB) is the number of bytes
        that can be read into memory at once. ``max_body_size``
        (defaults to ``max_buffer_size``) is the largest response body
        that the client will accept.  Without a
        ``streaming_callback``, the smaller of these two limits
        applies; with a ``streaming_callback`` only ``max_body_size``
        does.

        .. versionchanged:: 4.2
           Added the ``max_body_size`` argument.
        """
        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)
        self.max_clients = max_clients
        self.queue = (
            collections.deque()
        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]
        self.active = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]
        self.waiting = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]
        self.max_buffer_size = max_buffer_size
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        # TCPClient could create a Resolver for us, but we have to do it
        # ourselves to support hostname_mapping.
        if resolver:
            self.resolver = resolver
            self.own_resolver = False
        else:
            self.resolver = Resolver()
            self.own_resolver = True
        if hostname_mapping is not None:
            self.resolver = OverrideResolver(
                resolver=self.resolver, mapping=hostname_mapping
            )
        self.tcp_client = TCPClient(resolver=self.resolver)

    def close(self) -> None:
        super(SimpleAsyncHTTPClient, self).close()
        if self.own_resolver:
            self.resolver.close()
        self.tcp_client.close()

    def fetch_impl(
        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]
    ) -> None:
        key = object()
        self.queue.append((key, request, callback))
        if not len(self.active) < self.max_clients:
            assert request.connect_timeout is not None
            assert request.request_timeout is not None
            timeout_handle = self.io_loop.add_timeout(
                self.io_loop.time()
                + min(request.connect_timeout, request.request_timeout),
                functools.partial(self._on_timeout, key, "in request queue"),
            )
        else:
            timeout_handle = None
        self.waiting[key] = (request, callback, timeout_handle)
        self._process_queue()
        if self.queue:
            gen_log.debug(
                "max_clients limit reached, request queued. "
                "%d active, %d queued requests." % (len(self.active), len(self.queue))
            )

    def _process_queue(self) -> None:
        while self.queue and len(self.active) < self.max_clients:
            key, request, callback = self.queue.popleft()
            if key not in self.waiting:
                continue
            self._remove_timeout(key)
            self.active[key] = (request, callback)
            release_callback = functools.partial(self._release_fetch, key)
            self._handle_request(request, release_callback, callback)

    def _connection_class(self) -> type:
        return _HTTPConnection

    def _handle_request(
        self,
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
    ) -> None:
        self._connection_class()(
            self,
            request,
            release_callback,
            final_callback,
            self.max_buffer_size,
            self.tcp_client,
            self.max_header_size,
            self.max_body_size,
        )

    def _release_fetch(self, key: object) -> None:
        del self.active[key]
        self._process_queue()

    def _remove_timeout(self, key: object) -> None:
        if key in self.waiting:
            request, callback, timeout_handle = self.waiting[key]
            if timeout_handle is not None:
                self.io_loop.remove_timeout(timeout_handle)
            del self.waiting[key]

    def _on_timeout(self, key: object, info: str = None) -> None:
        """Timeout callback of request.

        Construct a timeout HTTPResponse when a timeout occurs.

        :arg object key: A simple object to mark the request.
        :info string key: More detailed timeout information.
        """
        request, callback, timeout_handle = self.waiting[key]
        self.queue.remove((key, request, callback))

        error_message = "Timeout {0}".format(info) if info else "Timeout"
        timeout_response = HTTPResponse(
            request,
            599,
            error=HTTPTimeoutError(error_message),
            request_time=self.io_loop.time() - request.start_time,
        )
        self.io_loop.add_callback(callback, timeout_response)
        del self.waiting[key]


class _HTTPConnection(httputil.HTTPMessageDelegate):
    _SUPPORTED_METHODS = set(
        ["GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"]
    )

    def __init__(
        self,
        client: Optional[SimpleAsyncHTTPClient],
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
        max_buffer_size: int,
        tcp_client: TCPClient,
        max_header_size: int,
        max_body_size: int,
    ) -> None:
        self.io_loop = IOLoop.current()
        self.start_time = self.io_loop.time()
        self.start_wall_time = time.time()
        self.client = client
        self.request = request
        self.release_callback = release_callback
        self.final_callback = final_callback
        self.max_buffer_size = max_buffer_size
        self.tcp_client = tcp_client
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        self.code = None  # type: Optional[int]
        self.headers = None  # type: Optional[httputil.HTTPHeaders]
        self.chunks = []  # type: List[bytes]
        self._decompressor = None
        # Timeout handle returned by IOLoop.add_timeout
        self._timeout = None  # type: object
        self._sockaddr = None
        IOLoop.current().add_future(
            gen.convert_yielded(self.run()), lambda f: f.result()
        )

    async def run(self) -> None:
        try:
            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))
            if self.parsed.scheme not in ("http", "https"):
                raise ValueError("Unsupported url scheme: %s" % self.request.url)
            # urlsplit results have hostname and port results, but they
            # didn't support ipv6 literals until python 2.7.
            netloc = self.parsed.netloc
            if "@" in netloc:
                userpass, _, netloc = netloc.rpartition("@")
            host, port = httputil.split_host_and_port(netloc)
            if port is None:
                port = 443 if self.parsed.scheme == "https" else 80
            if re.match(r"^\[.*\]$", host):
                # raw ipv6 addresses in urls are enclosed in brackets
                host = host[1:-1]
            self.parsed_hostname = host  # save final host for _on_connect

            if self.request.allow_ipv6 is False:
                af = socket.AF_INET
            else:
                af = socket.AF_UNSPEC

            ssl_options = self._get_ssl_options(self.parsed.scheme)

            source_ip = None
            if self.request.network_interface:
                if is_valid_ip(self.request.network_interface):
                    source_ip = self.request.network_interface
                else:
                    raise ValueError(
                        "Unrecognized IPv4 or IPv6 address for network_interface, got %r"
                        % (self.request.network_interface,)
                    )

            timeout = min(self.request.connect_timeout, self.request.request_timeout)
            if timeout:
                self._timeout = self.io_loop.add_timeout(
                    self.start_time + timeout,
                    functools.partial(self._on_timeout, "while connecting"),
                )
                stream = await self.tcp_client.connect(
                    host,
                    port,
                    af=af,
                    ssl_options=ssl_options,
                    max_buffer_size=self.max_buffer_size,
                    source_ip=source_ip,
                )

                if self.final_callback is None:
                    # final_callback is cleared if we've hit our timeout.
                    stream.close()
                    return
                self.stream = stream
                self.stream.set_close_callback(self.on_connection_close)
                self._remove_timeout()
                if self.final_callback is None:
                    return
                if self.request.request_timeout:
                    self._timeout = self.io_loop.add_timeout(
                        self.start_time + self.request.request_timeout,
                        functools.partial(self._on_timeout, "during request"),
                    )
                if (
                    self.request.method not in self._SUPPORTED_METHODS
                    and not self.request.allow_nonstandard_methods
                ):
                    raise KeyError("unknown method %s" % self.request.method)
                for key in (
                    "proxy_host",
                    "proxy_port",
                    "proxy_username",
                    "proxy_password",
                    "proxy_auth_mode",
                ):
                    if getattr(self.request, key, None):
                        raise NotImplementedError("%s not supported" % key)
                if "Connection" not in self.request.headers:
                    self.request.headers["Connection"] = "close"
                if "Host" not in self.request.headers:
                    if "@" in self.parsed.netloc:
                        self.request.headers["Host"] = self.parsed.netloc.rpartition(
                            "@"
                        )[-1]
                    else:
                        self.request.headers["Host"] = self.parsed.netloc
                username, password = None, None
                if self.parsed.username is not None:
                    username, password = self.parsed.username, self.parsed.password
                elif self.request.auth_username is not None:
                    username = self.request.auth_username
                    password = self.request.auth_password or ""
                if username is not None:
                    assert password is not None
                    if self.request.auth_mode not in (None, "basic"):
                        raise ValueError(
                            "unsupported auth_mode %s", self.request.auth_mode
                        )
                    self.request.headers["Authorization"] = "Basic " + _unicode(
                        base64.b64encode(
                            httputil.encode_username_password(username, password)
                        )
                    )
                if self.request.user_agent:
                    self.request.headers["User-Agent"] = self.request.user_agent
                if not self.request.allow_nonstandard_methods:
                    # Some HTTP methods nearly always have bodies while others
                    # almost never do. Fail in this case unless the user has
                    # opted out of sanity checks with allow_nonstandard_methods.
                    body_expected = self.request.method in ("POST", "PATCH", "PUT")
                    body_present = (
                        self.request.body is not None
                        or self.request.body_producer is not None
                    )
                    if (body_expected and not body_present) or (
                        body_present and not body_expected
                    ):
                        raise ValueError(
                            "Body must %sbe None for method %s (unless "
                            "allow_nonstandard_methods is true)"
                            % ("not " if body_expected else "", self.request.method)
                        )
                if self.request.expect_100_continue:
                    self.request.headers["Expect"] = "100-continue"
                if self.request.body is not None:
                    # When body_producer is used the caller is responsible for
                    # setting Content-Length (or else chunked encoding will be used).
                    self.request.headers["Content-Length"] = str(len(self.request.body))
                if (
                    self.request.method == "POST"
                    and "Content-Type" not in self.request.headers
                ):
                    self.request.headers[
                        "Content-Type"
                    ] = "application/x-www-form-urlencoded"
                if self.request.decompress_response:
                    self.request.headers["Accept-Encoding"] = "gzip"
                req_path = (self.parsed.path or "/") + (
                    ("?" + self.parsed.query) if self.parsed.query else ""
                )
                self.connection = self._create_connection(stream)
                start_line = httputil.RequestStartLine(
                    self.request.method, req_path, ""
                )
                self.connection.write_headers(start_line, self.request.headers)
                if self.request.expect_100_continue:
                    await self.connection.read_response(self)
                else:
                    await self._write_body(True)
        except Exception:
            if not self._handle_exception(*sys.exc_info()):
                raise

    def _get_ssl_options(
        self, scheme: str
    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:
        if scheme == "https":
            if self.request.ssl_options is not None:
                return self.request.ssl_options
            # If we are using the defaults, don't construct a
            # new SSLContext.
            if (
                self.request.validate_cert
                and self.request.ca_certs is None
                and self.request.client_cert is None
                and self.request.client_key is None
            ):
                return _client_ssl_defaults
            ssl_ctx = ssl.create_default_context(
                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs
            )
            if not self.request.validate_cert:
                ssl_ctx.check_hostname = False
                ssl_ctx.verify_mode = ssl.CERT_NONE
            if self.request.client_cert is not None:
                ssl_ctx.load_cert_chain(
                    self.request.client_cert, self.request.client_key
                )
            if hasattr(ssl, "OP_NO_COMPRESSION"):
                # See netutil.ssl_options_to_context
                ssl_ctx.options |= ssl.OP_NO_COMPRESSION
            return ssl_ctx
        return None

    def _on_timeout(self, info: str = None) -> None:
        """Timeout callback of _HTTPConnection instance.

        Raise a `HTTPTimeoutError` when a timeout occurs.

        :info string key: More detailed timeout information.
        """
        self._timeout = None
        error_message = "Timeout {0}".format(info) if info else "Timeout"
        if self.final_callback is not None:
            self._handle_exception(
                HTTPTimeoutError, HTTPTimeoutError(error_message), None
            )

    def _remove_timeout(self) -> None:
        if self._timeout is not None:
            self.io_loop.remove_timeout(self._timeout)
            self._timeout = None

    def _create_connection(self, stream: IOStream) -> HTTP1Connection:
        stream.set_nodelay(True)
        connection = HTTP1Connection(
            stream,
            True,
            HTTP1ConnectionParameters(
                no_keep_alive=True,
                max_header_size=self.max_header_size,
                max_body_size=self.max_body_size,
                decompress=bool(self.request.decompress_response),
            ),
            self._sockaddr,
        )
        return connection

    async def _write_body(self, start_read: bool) -> None:
        if self.request.body is not None:
            self.connection.write(self.request.body)
        elif self.request.body_producer is not None:
            fut = self.request.body_producer(self.connection.write)
            if fut is not None:
                await fut
        self.connection.finish()
        if start_read:
            try:
                await self.connection.read_response(self)
            except StreamClosedError:
                if not self._handle_exception(*sys.exc_info()):
                    raise

    def _release(self) -> None:
        if self.release_callback is not None:
            release_callback = self.release_callback
            self.release_callback = None  # type: ignore
            release_callback()

    def _run_callback(self, response: HTTPResponse) -> None:
        self._release()
        if self.final_callback is not None:
            final_callback = self.final_callback
            self.final_callback = None  # type: ignore
            self.io_loop.add_callback(final_callback, response)

    def _handle_exception(
        self,
        typ: Optional[Type[BaseException]],
        value: Optional[BaseException],
        tb: Optional[TracebackType],
    ) -> bool:
        if self.final_callback:
            self._remove_timeout()
            if isinstance(value, StreamClosedError):
                if value.real_error is None:
                    value = HTTPStreamClosedError("Stream closed")
                else:
                    value = value.real_error
            self._run_callback(
                HTTPResponse(
                    self.request,
                    599,
                    error=value,
                    request_time=self.io_loop.time() - self.start_time,
                    start_time=self.start_wall_time,
                )
            )

            if hasattr(self, "stream"):
                # TODO: this may cause a StreamClosedError to be raised
                # by the connection's Future.  Should we cancel the
                # connection more gracefully?
                self.stream.close()
            return True
        else:
            # If our callback has already been called, we are probably
            # catching an exception that is not caused by us but rather
            # some child of our callback. Rather than drop it on the floor,
            # pass it along, unless it's just the stream being closed.
            return isinstance(value, StreamClosedError)

    def on_connection_close(self) -> None:
        if self.final_callback is not None:
            message = "Connection closed"
            if self.stream.error:
                raise self.stream.error
            try:
                raise HTTPStreamClosedError(message)
            except HTTPStreamClosedError:
                self._handle_exception(*sys.exc_info())

    async def headers_received(
        self,
        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],
        headers: httputil.HTTPHeaders,
    ) -> None:
        assert isinstance(first_line, httputil.ResponseStartLine)
        if self.request.expect_100_continue and first_line.code == 100:
            await self._write_body(False)
            return
        self.code = first_line.code
        self.reason = first_line.reason
        self.headers = headers

        if self._should_follow_redirect():
            return

        if self.request.header_callback is not None:
            # Reassemble the start line.
            self.request.header_callback("%s %s %s\r\n" % first_line)
            for k, v in self.headers.get_all():
                self.request.header_callback("%s: %s\r\n" % (k, v))
            self.request.header_callback("\r\n")

    def _should_follow_redirect(self) -> bool:
        if self.request.follow_redirects:
            assert self.request.max_redirects is not None
            return (
                self.code in (301, 302, 303, 307, 308)
                and self.request.max_redirects > 0
                and self.headers is not None
                and self.headers.get("Location") is not None
            )
        return False

    def finish(self) -> None:
        assert self.code is not None
        data = b"".join(self.chunks)
        self._remove_timeout()
        original_request = getattr(self.request, "original_request", self.request)
        if self._should_follow_redirect():
            assert isinstance(self.request, _RequestProxy)
            new_request = copy.copy(self.request.request)
            new_request.url = urllib.parse.urljoin(
                self.request.url, self.headers["Location"]
            )
            new_request.max_redirects = self.request.max_redirects - 1
            del new_request.headers["Host"]
            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4
            # Client SHOULD make a GET request after a 303.
            # According to the spec, 302 should be followed by the same
            # method as the original request, but in practice browsers
            # treat 302 the same as 303, and many servers use 302 for
            # compatibility with pre-HTTP/1.1 user agents which don't
            # understand the 303 status.
            if self.code in (302, 303):
                new_request.method = "GET"
                new_request.body = None
                for h in [
                    "Content-Length",
                    "Content-Type",
                    "Content-Encoding",
                    "Transfer-Encoding",
                ]:
                    try:
                        del self.request.headers[h]
                    except KeyError:
                        pass
            new_request.original_request = original_request
            final_callback = self.final_callback
            self.final_callback = None
            self._release()
            fut = self.client.fetch(new_request, raise_error=False)
            fut.add_done_callback(lambda f: final_callback(f.result()))
            self._on_end_request()
            return
        if self.request.streaming_callback:
            buffer = BytesIO()
        else:
            buffer = BytesIO(data)  # TODO: don't require one big string?
        response = HTTPResponse(
            original_request,
            self.code,
            reason=getattr(self, "reason", None),
            headers=self.headers,
            request_time=self.io_loop.time() - self.start_time,
            start_time=self.start_wall_time,
            buffer=buffer,
            effective_url=self.request.url,
        )
        self._run_callback(response)
        self._on_end_request()

    def _on_end_request(self) -> None:
        self.stream.close()

    def data_received(self, chunk: bytes) -> None:
        if self._should_follow_redirect():
            # We're going to follow a redirect so just discard the body.
            return
        if self.request.streaming_callback is not None:
            self.request.streaming_callback(chunk)
        else:
            self.chunks.append(chunk)


if __name__ == "__main__":
    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)
    main()

```
--- END FILE ---

Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits to fix the issue.

Every *SEARCH/REPLACE* edit must use this format:
1. The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
4. The dividing line: =======
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE

Here is an example:

```python
### mathweb/flask/app.py
<<<<<<< SEARCH
from flask import Flask
=======
import math
from flask import Flask
>>>>>>> REPLACE
```
</solution>

Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If you would like to add the line '        print(x)', you must fully write that out, with all those spaces before the code!
Wrap each *SEARCH/REPLACE* edit in a code block as shown in the example above. If you have multiple *SEARCH/REPLACE* edits, use a separate code block for each one.
user