{
    "file_diffs": [
        {
            "old_file_content": "\"\"\"Blocking and non-blocking HTTP client interfaces.\n\nThis module defines a common interface shared by two implementations,\n``simple_httpclient`` and ``curl_httpclient``.  Applications may either\ninstantiate their chosen implementation class directly or use the\n`AsyncHTTPClient` class from this module, which selects an implementation\nthat can be overridden with the `AsyncHTTPClient.configure` method.\n\nThe default implementation is ``simple_httpclient``, and this is expected\nto be suitable for most users' needs.  However, some applications may wish\nto switch to ``curl_httpclient`` for reasons such as the following:\n\n* ``curl_httpclient`` has some features not found in ``simple_httpclient``,\n  including support for HTTP proxies and the ability to use a specified\n  network interface.\n\n* ``curl_httpclient`` is more likely to be compatible with sites that are\n  not-quite-compliant with the HTTP spec, or sites that use little-exercised\n  features of HTTP.\n\n* ``curl_httpclient`` is faster.\n\n* ``curl_httpclient`` was the default prior to Tornado 2.0.\n\nNote that if you are using ``curl_httpclient``, it is highly\nrecommended that you use a recent version of ``libcurl`` and\n``pycurl``.  Currently the minimum supported version of libcurl is\n7.22.0, and the minimum version of pycurl is 7.18.2.  It is highly\nrecommended that your ``libcurl`` installation is built with\nasynchronous DNS resolver (threaded or c-ares), otherwise you may\nencounter various problems with request timeouts (for more\ninformation, see\nhttp://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTCONNECTTIMEOUTMS\nand comments in curl_httpclient.py).\n\nTo select ``curl_httpclient``, call `AsyncHTTPClient.configure` at startup::\n\n    AsyncHTTPClient.configure(\"tornado.curl_httpclient.CurlAsyncHTTPClient\")\n\"\"\"\n\nimport datetime\nimport functools\nfrom io import BytesIO\nimport ssl\nimport time\nimport weakref\n\nfrom tornado.concurrent import Future, future_set_result_unless_cancelled\nfrom tornado.escape import utf8, native_str\nfrom tornado import gen, httputil\nfrom tornado.ioloop import IOLoop\nfrom tornado.util import Configurable\n\nfrom typing import Type, Any, Union, Dict, Callable, Optional, cast\n\n\nclass HTTPClient(object):\n    \"\"\"A blocking HTTP client.\n\n    This interface is provided to make it easier to share code between\n    synchronous and asynchronous applications. Applications that are\n    running an `.IOLoop` must use `AsyncHTTPClient` instead.\n\n    Typical usage looks like this::\n\n        http_client = httpclient.HTTPClient()\n        try:\n            response = http_client.fetch(\"http://www.google.com/\")\n            print(response.body)\n        except httpclient.HTTPError as e:\n            # HTTPError is raised for non-200 responses; the response\n            # can be found in e.response.\n            print(\"Error: \" + str(e))\n        except Exception as e:\n            # Other errors are possible, such as IOError.\n            print(\"Error: \" + str(e))\n        http_client.close()\n\n    .. versionchanged:: 5.0\n\n       Due to limitations in `asyncio`, it is no longer possible to\n       use the synchronous ``HTTPClient`` while an `.IOLoop` is running.\n       Use `AsyncHTTPClient` instead.\n\n    \"\"\"\n\n    def __init__(\n        self, async_client_class: Type[\"AsyncHTTPClient\"] = None, **kwargs: Any\n    ) -> None:\n        # Initialize self._closed at the beginning of the constructor\n        # so that an exception raised here doesn't lead to confusing\n        # failures in __del__.\n        self._closed = True\n        self._io_loop = IOLoop(make_current=False)\n        if async_client_class is None:\n            async_client_class = AsyncHTTPClient\n\n        # Create the client while our IOLoop is \"current\", without\n        # clobbering the thread's real current IOLoop (if any).\n        async def make_client() -> \"AsyncHTTPClient\":\n            await gen.sleep(0)\n            assert async_client_class is not None\n            return async_client_class(**kwargs)\n\n        self._async_client = self._io_loop.run_sync(make_client)\n        self._closed = False\n\n    def __del__(self) -> None:\n        self.close()\n\n    def close(self) -> None:\n        \"\"\"Closes the HTTPClient, freeing any resources used.\"\"\"\n        if not self._closed:\n            self._async_client.close()\n            self._io_loop.close()\n            self._closed = True\n\n    def fetch(\n        self, request: Union[\"HTTPRequest\", str], **kwargs: Any\n    ) -> \"HTTPResponse\":\n        \"\"\"Executes a request, returning an `HTTPResponse`.\n\n        The request may be either a string URL or an `HTTPRequest` object.\n        If it is a string, we construct an `HTTPRequest` using any additional\n        kwargs: ``HTTPRequest(request, **kwargs)``\n\n        If an error occurs during the fetch, we raise an `HTTPError` unless\n        the ``raise_error`` keyword argument is set to False.\n        \"\"\"\n        response = self._io_loop.run_sync(\n            functools.partial(self._async_client.fetch, request, **kwargs)\n        )\n        return response\n\n\nclass AsyncHTTPClient(Configurable):\n    \"\"\"An non-blocking HTTP client.\n\n    Example usage::\n\n        async def f():\n            http_client = AsyncHTTPClient()\n            try:\n                response = await http_client.fetch(\"http://www.google.com\")\n            except Exception as e:\n                print(\"Error: %s\" % e)\n            else:\n                print(response.body)\n\n    The constructor for this class is magic in several respects: It\n    actually creates an instance of an implementation-specific\n    subclass, and instances are reused as a kind of pseudo-singleton\n    (one per `.IOLoop`). The keyword argument ``force_instance=True``\n    can be used to suppress this singleton behavior. Unless\n    ``force_instance=True`` is used, no arguments should be passed to\n    the `AsyncHTTPClient` constructor. The implementation subclass as\n    well as arguments to its constructor can be set with the static\n    method `configure()`\n\n    All `AsyncHTTPClient` implementations support a ``defaults``\n    keyword argument, which can be used to set default values for\n    `HTTPRequest` attributes.  For example::\n\n        AsyncHTTPClient.configure(\n            None, defaults=dict(user_agent=\"MyUserAgent\"))\n        # or with force_instance:\n        client = AsyncHTTPClient(force_instance=True,\n            defaults=dict(user_agent=\"MyUserAgent\"))\n\n    .. versionchanged:: 5.0\n       The ``io_loop`` argument (deprecated since version 4.1) has been removed.\n\n    \"\"\"\n\n    _instance_cache = None  # type: Dict[IOLoop, AsyncHTTPClient]\n\n    @classmethod\n    def configurable_base(cls) -> Type[Configurable]:\n        return AsyncHTTPClient\n\n    @classmethod\n    def configurable_default(cls) -> Type[Configurable]:\n        from tornado.simple_httpclient import SimpleAsyncHTTPClient\n\n        return SimpleAsyncHTTPClient\n\n    @classmethod\n    def _async_clients(cls) -> Dict[IOLoop, \"AsyncHTTPClient\"]:\n        attr_name = \"_async_client_dict_\" + cls.__name__\n        if not hasattr(cls, attr_name):\n            setattr(cls, attr_name, weakref.WeakKeyDictionary())\n        return getattr(cls, attr_name)\n\n    def __new__(cls, force_instance: bool = False, **kwargs: Any) -> \"AsyncHTTPClient\":\n        io_loop = IOLoop.current()\n        if force_instance:\n            instance_cache = None\n        else:\n            instance_cache = cls._async_clients()\n        if instance_cache is not None and io_loop in instance_cache:\n            return instance_cache[io_loop]\n        instance = super(AsyncHTTPClient, cls).__new__(cls, **kwargs)  # type: ignore\n        # Make sure the instance knows which cache to remove itself from.\n        # It can't simply call _async_clients() because we may be in\n        # __new__(AsyncHTTPClient) but instance.__class__ may be\n        # SimpleAsyncHTTPClient.\n        instance._instance_cache = instance_cache\n        if instance_cache is not None:\n            instance_cache[instance.io_loop] = instance\n        return instance\n\n    def initialize(self, defaults: Dict[str, Any] = None) -> None:\n        self.io_loop = IOLoop.current()\n        self.defaults = dict(HTTPRequest._DEFAULTS)\n        if defaults is not None:\n            self.defaults.update(defaults)\n        self._closed = False\n\n    def close(self) -> None:\n        \"\"\"Destroys this HTTP client, freeing any file descriptors used.\n\n        This method is **not needed in normal use** due to the way\n        that `AsyncHTTPClient` objects are transparently reused.\n        ``close()`` is generally only necessary when either the\n        `.IOLoop` is also being closed, or the ``force_instance=True``\n        argument was used when creating the `AsyncHTTPClient`.\n\n        No other methods may be called on the `AsyncHTTPClient` after\n        ``close()``.\n\n        \"\"\"\n        if self._closed:\n            return\n        self._closed = True\n        if self._instance_cache is not None:\n            if self._instance_cache.get(self.io_loop) is not self:\n                raise RuntimeError(\"inconsistent AsyncHTTPClient cache\")\n            del self._instance_cache[self.io_loop]\n\n    def fetch(\n        self,\n        request: Union[str, \"HTTPRequest\"],\n        raise_error: bool = True,\n        **kwargs: Any\n    ) -> \"Future[HTTPResponse]\":\n        \"\"\"Executes a request, asynchronously returning an `HTTPResponse`.\n\n        The request may be either a string URL or an `HTTPRequest` object.\n        If it is a string, we construct an `HTTPRequest` using any additional\n        kwargs: ``HTTPRequest(request, **kwargs)``\n\n        This method returns a `.Future` whose result is an\n        `HTTPResponse`. By default, the ``Future`` will raise an\n        `HTTPError` if the request returned a non-200 response code\n        (other errors may also be raised if the server could not be\n        contacted). Instead, if ``raise_error`` is set to False, the\n        response will always be returned regardless of the response\n        code.\n\n        If a ``callback`` is given, it will be invoked with the `HTTPResponse`.\n        In the callback interface, `HTTPError` is not automatically raised.\n        Instead, you must check the response's ``error`` attribute or\n        call its `~HTTPResponse.rethrow` method.\n\n        .. versionchanged:: 6.0\n\n           The ``callback`` argument was removed. Use the returned\n           `.Future` instead.\n\n           The ``raise_error=False`` argument only affects the\n           `HTTPError` raised when a non-200 response code is used,\n           instead of suppressing all errors.\n        \"\"\"\n        if self._closed:\n            raise RuntimeError(\"fetch() called on closed AsyncHTTPClient\")\n        if not isinstance(request, HTTPRequest):\n            request = HTTPRequest(url=request, **kwargs)\n        else:\n            if kwargs:\n                raise ValueError(\n                    \"kwargs can't be used if request is an HTTPRequest object\"\n                )\n        # We may modify this (to add Host, Accept-Encoding, etc),\n        # so make sure we don't modify the caller's object.  This is also\n        # where normal dicts get converted to HTTPHeaders objects.\n        request.headers = httputil.HTTPHeaders(request.headers)\n        request_proxy = _RequestProxy(request, self.defaults)\n        future = Future()  # type: Future[HTTPResponse]\n\n        def handle_response(response: \"HTTPResponse\") -> None:\n            if response.error:\n                if raise_error or not response._error_is_response_code:\n                    future.set_exception(response.error)\n                    return\n            future_set_result_unless_cancelled(future, response)\n\n        self.fetch_impl(cast(HTTPRequest, request_proxy), handle_response)\n        return future\n\n    def fetch_impl(\n        self, request: \"HTTPRequest\", callback: Callable[[\"HTTPResponse\"], None]\n    ) -> None:\n        raise NotImplementedError()\n\n    @classmethod\n    def configure(\n        cls, impl: Union[None, str, Type[Configurable]], **kwargs: Any\n    ) -> None:\n        \"\"\"Configures the `AsyncHTTPClient` subclass to use.\n\n        ``AsyncHTTPClient()`` actually creates an instance of a subclass.\n        This method may be called with either a class object or the\n        fully-qualified name of such a class (or ``None`` to use the default,\n        ``SimpleAsyncHTTPClient``)\n\n        If additional keyword arguments are given, they will be passed\n        to the constructor of each subclass instance created.  The\n        keyword argument ``max_clients`` determines the maximum number\n        of simultaneous `~AsyncHTTPClient.fetch()` operations that can\n        execute in parallel on each `.IOLoop`.  Additional arguments\n        may be supported depending on the implementation class in use.\n\n        Example::\n\n           AsyncHTTPClient.configure(\"tornado.curl_httpclient.CurlAsyncHTTPClient\")\n        \"\"\"\n        super(AsyncHTTPClient, cls).configure(impl, **kwargs)\n\n\nclass HTTPRequest(object):\n    \"\"\"HTTP client request object.\"\"\"\n\n    _headers = None  # type: Union[Dict[str, str], httputil.HTTPHeaders]\n\n    # Default values for HTTPRequest parameters.\n    # Merged with the values on the request object by AsyncHTTPClient\n    # implementations.\n    _DEFAULTS = dict(\n        connect_timeout=20.0,\n        request_timeout=20.0,\n        follow_redirects=True,\n        max_redirects=5,\n        decompress_response=True,\n        proxy_password=\"\",\n        allow_nonstandard_methods=False,\n        validate_cert=True,\n    )\n\n    def __init__(\n        self,\n        url: str,\n        method: str = \"GET\",\n        headers: Union[Dict[str, str], httputil.HTTPHeaders] = None,\n        body: Union[bytes, str] = None,\n        auth_username: str = None,\n        auth_password: str = None,\n        auth_mode: str = None,\n        connect_timeout: float = None,\n        request_timeout: float = None,\n        if_modified_since: Union[float, datetime.datetime] = None,\n        follow_redirects: bool = None,\n        max_redirects: int = None,\n        user_agent: str = None,\n        use_gzip: bool = None,\n        network_interface: str = None,\n        streaming_callback: Callable[[bytes], None] = None,\n        header_callback: Callable[[str], None] = None,\n        prepare_curl_callback: Callable[[Any], None] = None,\n        proxy_host: str = None,\n        proxy_port: int = None,\n        proxy_username: str = None,\n        proxy_password: str = None,\n        proxy_auth_mode: str = None,\n        allow_nonstandard_methods: bool = None,\n        validate_cert: bool = None,\n        ca_certs: str = None,\n        allow_ipv6: bool = None,\n        client_key: str = None,\n        client_cert: str = None,\n        body_producer: Callable[[Callable[[bytes], None]], \"Future[None]\"] = None,\n        expect_100_continue: bool = False,\n        decompress_response: bool = None,\n        ssl_options: Union[Dict[str, Any], ssl.SSLContext] = None,\n    ) -> None:\n        r\"\"\"All parameters except ``url`` are optional.\n\n        :arg str url: URL to fetch\n        :arg str method: HTTP method, e.g. \"GET\" or \"POST\"\n        :arg headers: Additional HTTP headers to pass on the request\n        :type headers: `~tornado.httputil.HTTPHeaders` or `dict`\n        :arg body: HTTP request body as a string (byte or unicode; if unicode\n           the utf-8 encoding will be used)\n        :arg body_producer: Callable used for lazy/asynchronous request bodies.\n           It is called with one argument, a ``write`` function, and should\n           return a `.Future`.  It should call the write function with new\n           data as it becomes available.  The write function returns a\n           `.Future` which can be used for flow control.\n           Only one of ``body`` and ``body_producer`` may\n           be specified.  ``body_producer`` is not supported on\n           ``curl_httpclient``.  When using ``body_producer`` it is recommended\n           to pass a ``Content-Length`` in the headers as otherwise chunked\n           encoding will be used, and many servers do not support chunked\n           encoding on requests.  New in Tornado 4.0\n        :arg str auth_username: Username for HTTP authentication\n        :arg str auth_password: Password for HTTP authentication\n        :arg str auth_mode: Authentication mode; default is \"basic\".\n           Allowed values are implementation-defined; ``curl_httpclient``\n           supports \"basic\" and \"digest\"; ``simple_httpclient`` only supports\n           \"basic\"\n        :arg float connect_timeout: Timeout for initial connection in seconds,\n           default 20 seconds\n        :arg float request_timeout: Timeout for entire request in seconds,\n           default 20 seconds\n        :arg if_modified_since: Timestamp for ``If-Modified-Since`` header\n        :type if_modified_since: `datetime` or `float`\n        :arg bool follow_redirects: Should redirects be followed automatically\n           or return the 3xx response? Default True.\n        :arg int max_redirects: Limit for ``follow_redirects``, default 5.\n        :arg str user_agent: String to send as ``User-Agent`` header\n        :arg bool decompress_response: Request a compressed response from\n           the server and decompress it after downloading.  Default is True.\n           New in Tornado 4.0.\n        :arg bool use_gzip: Deprecated alias for ``decompress_response``\n           since Tornado 4.0.\n        :arg str network_interface: Network interface to use for request.\n           ``curl_httpclient`` only; see note below.\n        :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will\n           be run with each chunk of data as it is received, and\n           ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in\n           the final response.\n        :arg collections.abc.Callable header_callback: If set, ``header_callback`` will\n           be run with each header line as it is received (including the\n           first line, e.g. ``HTTP/1.0 200 OK\\r\\n``, and a final line\n           containing only ``\\r\\n``.  All lines include the trailing newline\n           characters).  ``HTTPResponse.headers`` will be empty in the final\n           response.  This is most useful in conjunction with\n           ``streaming_callback``, because it's the only way to get access to\n           header data while the request is in progress.\n        :arg collections.abc.Callable prepare_curl_callback: If set, will be called with\n           a ``pycurl.Curl`` object to allow the application to make additional\n           ``setopt`` calls.\n        :arg str proxy_host: HTTP proxy hostname.  To use proxies,\n           ``proxy_host`` and ``proxy_port`` must be set; ``proxy_username``,\n           ``proxy_pass`` and ``proxy_auth_mode`` are optional.  Proxies are\n           currently only supported with ``curl_httpclient``.\n        :arg int proxy_port: HTTP proxy port\n        :arg str proxy_username: HTTP proxy username\n        :arg str proxy_password: HTTP proxy password\n        :arg str proxy_auth_mode: HTTP proxy Authentication mode;\n           default is \"basic\". supports \"basic\" and \"digest\"\n        :arg bool allow_nonstandard_methods: Allow unknown values for ``method``\n           argument? Default is False.\n        :arg bool validate_cert: For HTTPS requests, validate the server's\n           certificate? Default is True.\n        :arg str ca_certs: filename of CA certificates in PEM format,\n           or None to use defaults.  See note below when used with\n           ``curl_httpclient``.\n        :arg str client_key: Filename for client SSL key, if any.  See\n           note below when used with ``curl_httpclient``.\n        :arg str client_cert: Filename for client SSL certificate, if any.\n           See note below when used with ``curl_httpclient``.\n        :arg ssl.SSLContext ssl_options: `ssl.SSLContext` object for use in\n           ``simple_httpclient`` (unsupported by ``curl_httpclient``).\n           Overrides ``validate_cert``, ``ca_certs``, ``client_key``,\n           and ``client_cert``.\n        :arg bool allow_ipv6: Use IPv6 when available?  Default is true.\n        :arg bool expect_100_continue: If true, send the\n           ``Expect: 100-continue`` header and wait for a continue response\n           before sending the request body.  Only supported with\n           simple_httpclient.\n\n        .. note::\n\n            When using ``curl_httpclient`` certain options may be\n            inherited by subsequent fetches because ``pycurl`` does\n            not allow them to be cleanly reset.  This applies to the\n            ``ca_certs``, ``client_key``, ``client_cert``, and\n            ``network_interface`` arguments.  If you use these\n            options, you should pass them on every request (you don't\n            have to always use the same values, but it's not possible\n            to mix requests that specify these options with ones that\n            use the defaults).\n\n        .. versionadded:: 3.1\n           The ``auth_mode`` argument.\n\n        .. versionadded:: 4.0\n           The ``body_producer`` and ``expect_100_continue`` arguments.\n\n        .. versionadded:: 4.2\n           The ``ssl_options`` argument.\n\n        .. versionadded:: 4.5\n           The ``proxy_auth_mode`` argument.\n        \"\"\"\n        # Note that some of these attributes go through property setters\n        # defined below.\n        self.headers = headers\n        if if_modified_since:\n            self.headers[\"If-Modified-Since\"] = httputil.format_timestamp(\n                if_modified_since\n            )\n        self.proxy_host = proxy_host\n        self.proxy_port = proxy_port\n        self.proxy_username = proxy_username\n        self.proxy_password = proxy_password\n        self.proxy_auth_mode = proxy_auth_mode\n        self.url = url\n        self.method = method\n        self.body = body\n        self.body_producer = body_producer\n        self.auth_username = auth_username\n        self.auth_password = auth_password\n        self.auth_mode = auth_mode\n        self.connect_timeout = connect_timeout\n        self.request_timeout = request_timeout\n        self.follow_redirects = follow_redirects\n        self.max_redirects = max_redirects\n        self.user_agent = user_agent\n        if decompress_response is not None:\n            self.decompress_response = decompress_response  # type: Optional[bool]\n        else:\n            self.decompress_response = use_gzip\n        self.network_interface = network_interface\n        self.streaming_callback = streaming_callback\n        self.header_callback = header_callback\n        self.prepare_curl_callback = prepare_curl_callback\n        self.allow_nonstandard_methods = allow_nonstandard_methods\n        self.validate_cert = validate_cert\n        self.ca_certs = ca_certs\n        self.allow_ipv6 = allow_ipv6\n        self.client_key = client_key\n        self.client_cert = client_cert\n        self.ssl_options = ssl_options\n        self.expect_100_continue = expect_100_continue\n        self.start_time = time.time()\n\n    @property\n    def headers(self) -> httputil.HTTPHeaders:\n        # TODO: headers may actually be a plain dict until fairly late in\n        # the process (AsyncHTTPClient.fetch), but practically speaking,\n        # whenever the property is used they're already HTTPHeaders.\n        return self._headers  # type: ignore\n\n    @headers.setter\n    def headers(self, value: Union[Dict[str, str], httputil.HTTPHeaders]) -> None:\n        if value is None:\n            self._headers = httputil.HTTPHeaders()\n        else:\n            self._headers = value  # type: ignore\n\n    @property\n    def body(self) -> bytes:\n        return self._body\n\n    @body.setter\n    def body(self, value: Union[bytes, str]) -> None:\n        self._body = utf8(value)\n\n\nclass HTTPResponse(object):\n    \"\"\"HTTP Response object.\n\n    Attributes:\n\n    * request: HTTPRequest object\n\n    * code: numeric HTTP status code, e.g. 200 or 404\n\n    * reason: human-readable reason phrase describing the status code\n\n    * headers: `tornado.httputil.HTTPHeaders` object\n\n    * effective_url: final location of the resource after following any\n      redirects\n\n    * buffer: ``cStringIO`` object for response body\n\n    * body: response body as bytes (created on demand from ``self.buffer``)\n\n    * error: Exception object, if any\n\n    * request_time: seconds from request start to finish. Includes all network\n      operations from DNS resolution to receiving the last byte of data.\n      Does not include time spent in the queue (due to the ``max_clients`` option).\n      If redirects were followed, only includes the final request.\n\n    * start_time: Time at which the HTTP operation started, based on `time.time`\n      (not the monotonic clock used by `.IOLoop.time`). May be ``None`` if the request\n      timed out while in the queue.\n\n    * time_info: dictionary of diagnostic timing information from the request.\n      Available data are subject to change, but currently uses timings\n      available from http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html,\n      plus ``queue``, which is the delay (if any) introduced by waiting for\n      a slot under `AsyncHTTPClient`'s ``max_clients`` setting.\n\n    .. versionadded:: 5.1\n\n       Added the ``start_time`` attribute.\n\n    .. versionchanged:: 5.1\n\n       The ``request_time`` attribute previously included time spent in the queue\n       for ``simple_httpclient``, but not in ``curl_httpclient``. Now queueing time\n       is excluded in both implementations. ``request_time`` is now more accurate for\n       ``curl_httpclient`` because it uses a monotonic clock when available.\n    \"\"\"\n\n    # I'm not sure why these don't get type-inferred from the references in __init__.\n    error = None  # type: Optional[BaseException]\n    _error_is_response_code = False\n    request = None  # type: HTTPRequest\n\n    def __init__(\n        self,\n        request: HTTPRequest,\n        code: int,\n        headers: httputil.HTTPHeaders = None,\n        buffer: BytesIO = None,\n        effective_url: str = None,\n        error: BaseException = None,\n        request_time: float = None,\n        time_info: Dict[str, float] = None,\n        reason: str = None,\n        start_time: float = None,\n    ) -> None:\n        if isinstance(request, _RequestProxy):\n            self.request = request.request\n        else:\n            self.request = request\n        self.code = code\n        self.reason = reason or httputil.responses.get(code, \"Unknown\")\n        if headers is not None:\n            self.headers = headers\n        else:\n            self.headers = httputil.HTTPHeaders()\n        self.buffer = buffer\n        self._body = None  # type: Optional[bytes]\n        if effective_url is None:\n            self.effective_url = request.url\n        else:\n            self.effective_url = effective_url\n        self._error_is_response_code = False\n        if error is None:\n            if self.code < 200 or self.code >= 300:\n                self._error_is_response_code = True\n                self.error = HTTPError(self.code, message=self.reason, response=self)\n            else:\n                self.error = None\n        else:\n            self.error = error\n        self.start_time = start_time\n        self.request_time = request_time\n        self.time_info = time_info or {}\n\n    @property\n    def body(self) -> bytes:\n        if self.buffer is None:\n            raise ValueError(\"body not set\")\n        elif self._body is None:\n            self._body = self.buffer.getvalue()\n\n        return self._body\n\n    def rethrow(self) -> None:\n        \"\"\"If there was an error on the request, raise an `HTTPError`.\"\"\"\n        if self.error:\n            raise self.error\n\n    def __repr__(self) -> str:\n        args = \",\".join(\"%s=%r\" % i for i in sorted(self.__dict__.items()))\n        return \"%s(%s)\" % (self.__class__.__name__, args)\n\n\nclass HTTPClientError(Exception):\n    \"\"\"Exception thrown for an unsuccessful HTTP request.\n\n    Attributes:\n\n    * ``code`` - HTTP error integer error code, e.g. 404.  Error code 599 is\n      used when no HTTP response was received, e.g. for a timeout.\n\n    * ``response`` - `HTTPResponse` object, if any.\n\n    Note that if ``follow_redirects`` is False, redirects become HTTPErrors,\n    and you can look at ``error.response.headers['Location']`` to see the\n    destination of the redirect.\n\n    .. versionchanged:: 5.1\n\n       Renamed from ``HTTPError`` to ``HTTPClientError`` to avoid collisions with\n       `tornado.web.HTTPError`. The name ``tornado.httpclient.HTTPError`` remains\n       as an alias.\n    \"\"\"\n\n    def __init__(\n        self, code: int, message: str = None, response: HTTPResponse = None\n    ) -> None:\n        self.code = code\n        self.message = message or httputil.responses.get(code, \"Unknown\")\n        self.response = response\n        super(HTTPClientError, self).__init__(code, message, response)\n\n    def __str__(self) -> str:\n        return \"HTTP %d: %s\" % (self.code, self.message)\n\n    # There is a cyclic reference between self and self.response,\n    # which breaks the default __repr__ implementation.\n    # (especially on pypy, which doesn't have the same recursion\n    # detection as cpython).\n    __repr__ = __str__\n\n\nHTTPError = HTTPClientError\n\n\nclass _RequestProxy(object):\n    \"\"\"Combines an object with a dictionary of defaults.\n\n    Used internally by AsyncHTTPClient implementations.\n    \"\"\"\n\n    def __init__(\n        self, request: HTTPRequest, defaults: Optional[Dict[str, Any]]\n    ) -> None:\n        self.request = request\n        self.defaults = defaults\n\n    def __getattr__(self, name: str) -> Any:\n        request_attr = getattr(self.request, name)\n        if request_attr is not None:\n            return request_attr\n        elif self.defaults is not None:\n            return self.defaults.get(name, None)\n        else:\n            return None\n\n\ndef main() -> None:\n    from tornado.options import define, options, parse_command_line\n\n    define(\"print_headers\", type=bool, default=False)\n    define(\"print_body\", type=bool, default=True)\n    define(\"follow_redirects\", type=bool, default=True)\n    define(\"validate_cert\", type=bool, default=True)\n    define(\"proxy_host\", type=str)\n    define(\"proxy_port\", type=int)\n    args = parse_command_line()\n    client = HTTPClient()\n    for arg in args:\n        try:\n            response = client.fetch(\n                arg,\n                follow_redirects=options.follow_redirects,\n                validate_cert=options.validate_cert,\n                proxy_host=options.proxy_host,\n                proxy_port=options.proxy_port,\n            )\n        except HTTPError as e:\n            if e.response is not None:\n                response = e.response\n            else:\n                raise\n        if options.print_headers:\n            print(response.headers)\n        if options.print_body:\n            print(native_str(response.body))\n    client.close()\n\n\nif __name__ == \"__main__\":\n    main()\n",
            "new_file_content": "\"\"\"Blocking and non-blocking HTTP client interfaces.\n\nThis module defines a common interface shared by two implementations,\n``simple_httpclient`` and ``curl_httpclient``.  Applications may either\ninstantiate their chosen implementation class directly or use the\n`AsyncHTTPClient` class from this module, which selects an implementation\nthat can be overridden with the `AsyncHTTPClient.configure` method.\n\nThe default implementation is ``simple_httpclient``, and this is expected\nto be suitable for most users' needs.  However, some applications may wish\nto switch to ``curl_httpclient`` for reasons such as the following:\n\n* ``curl_httpclient`` has some features not found in ``simple_httpclient``,\n  including support for HTTP proxies and the ability to use a specified\n  network interface.\n\n* ``curl_httpclient`` is more likely to be compatible with sites that are\n  not-quite-compliant with the HTTP spec, or sites that use little-exercised\n  features of HTTP.\n\n* ``curl_httpclient`` is faster.\n\n* ``curl_httpclient`` was the default prior to Tornado 2.0.\n\nNote that if you are using ``curl_httpclient``, it is highly\nrecommended that you use a recent version of ``libcurl`` and\n``pycurl``.  Currently the minimum supported version of libcurl is\n7.22.0, and the minimum version of pycurl is 7.18.2.  It is highly\nrecommended that your ``libcurl`` installation is built with\nasynchronous DNS resolver (threaded or c-ares), otherwise you may\nencounter various problems with request timeouts (for more\ninformation, see\nhttp://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTCONNECTTIMEOUTMS\nand comments in curl_httpclient.py).\n\nTo select ``curl_httpclient``, call `AsyncHTTPClient.configure` at startup::\n\n    AsyncHTTPClient.configure(\"tornado.curl_httpclient.CurlAsyncHTTPClient\")\n\"\"\"\n\nimport datetime\nimport functools\nfrom io import BytesIO\nimport ssl\nimport time\nimport weakref\n\nfrom tornado.concurrent import Future, future_set_result_unless_cancelled\nfrom tornado.escape import utf8, native_str\nfrom tornado import gen, httputil\nfrom tornado.ioloop import IOLoop\nfrom tornado.util import Configurable\n\nfrom typing import Type, Any, Union, Dict, Callable, Optional, cast\n\n\nclass HTTPClient(object):\n    \"\"\"A blocking HTTP client.\n\n    This interface is provided to make it easier to share code between\n    synchronous and asynchronous applications. Applications that are\n    running an `.IOLoop` must use `AsyncHTTPClient` instead.\n\n    Typical usage looks like this::\n\n        http_client = httpclient.HTTPClient()\n        try:\n            response = http_client.fetch(\"http://www.google.com/\")\n            print(response.body)\n        except httpclient.HTTPError as e:\n            # HTTPError is raised for non-200 responses; the response\n            # can be found in e.response.\n            print(\"Error: \" + str(e))\n        except Exception as e:\n            # Other errors are possible, such as IOError.\n            print(\"Error: \" + str(e))\n        http_client.close()\n\n    .. versionchanged:: 5.0\n\n       Due to limitations in `asyncio`, it is no longer possible to\n       use the synchronous ``HTTPClient`` while an `.IOLoop` is running.\n       Use `AsyncHTTPClient` instead.\n\n    \"\"\"\n\n    def __init__(\n        self, async_client_class: Type[\"AsyncHTTPClient\"] = None, **kwargs: Any\n    ) -> None:\n        # Initialize self._closed at the beginning of the constructor\n        # so that an exception raised here doesn't lead to confusing\n        # failures in __del__.\n        self._closed = True\n        self._io_loop = IOLoop(make_current=False)\n        if async_client_class is None:\n            async_client_class = AsyncHTTPClient\n\n        # Create the client while our IOLoop is \"current\", without\n        # clobbering the thread's real current IOLoop (if any).\n        async def make_client() -> \"AsyncHTTPClient\":\n            await gen.sleep(0)\n            assert async_client_class is not None\n            return async_client_class(**kwargs)\n\n        self._async_client = self._io_loop.run_sync(make_client)\n        self._closed = False\n\n    def __del__(self) -> None:\n        self.close()\n\n    def close(self) -> None:\n        \"\"\"Closes the HTTPClient, freeing any resources used.\"\"\"\n        if not self._closed:\n            self._async_client.close()\n            self._io_loop.close()\n            self._closed = True\n\n    def fetch(\n        self, request: Union[\"HTTPRequest\", str], **kwargs: Any\n    ) -> \"HTTPResponse\":\n        \"\"\"Executes a request, returning an `HTTPResponse`.\n\n        The request may be either a string URL or an `HTTPRequest` object.\n        If it is a string, we construct an `HTTPRequest` using any additional\n        kwargs: ``HTTPRequest(request, **kwargs)``\n\n        If an error occurs during the fetch, we raise an `HTTPError` unless\n        the ``raise_error`` keyword argument is set to False.\n        \"\"\"\n        response = self._io_loop.run_sync(\n            functools.partial(self._async_client.fetch, request, **kwargs)\n        )\n        return response\n\n\nclass AsyncHTTPClient(Configurable):\n    \"\"\"An non-blocking HTTP client.\n\n    Example usage::\n\n        async def f():\n            http_client = AsyncHTTPClient()\n            try:\n                response = await http_client.fetch(\"http://www.google.com\")\n            except Exception as e:\n                print(\"Error: %s\" % e)\n            else:\n                print(response.body)\n\n    The constructor for this class is magic in several respects: It\n    actually creates an instance of an implementation-specific\n    subclass, and instances are reused as a kind of pseudo-singleton\n    (one per `.IOLoop`). The keyword argument ``force_instance=True``\n    can be used to suppress this singleton behavior. Unless\n    ``force_instance=True`` is used, no arguments should be passed to\n    the `AsyncHTTPClient` constructor. The implementation subclass as\n    well as arguments to its constructor can be set with the static\n    method `configure()`\n\n    All `AsyncHTTPClient` implementations support a ``defaults``\n    keyword argument, which can be used to set default values for\n    `HTTPRequest` attributes.  For example::\n\n        AsyncHTTPClient.configure(\n            None, defaults=dict(user_agent=\"MyUserAgent\"))\n        # or with force_instance:\n        client = AsyncHTTPClient(force_instance=True,\n            defaults=dict(user_agent=\"MyUserAgent\"))\n\n    .. versionchanged:: 5.0\n       The ``io_loop`` argument (deprecated since version 4.1) has been removed.\n\n    \"\"\"\n\n    _instance_cache = None  # type: Dict[IOLoop, AsyncHTTPClient]\n\n    @classmethod\n    def configurable_base(cls) -> Type[Configurable]:\n        return AsyncHTTPClient\n\n    @classmethod\n    def configurable_default(cls) -> Type[Configurable]:\n        from tornado.simple_httpclient import SimpleAsyncHTTPClient\n\n        return SimpleAsyncHTTPClient\n\n    @classmethod\n    def _async_clients(cls) -> Dict[IOLoop, \"AsyncHTTPClient\"]:\n        attr_name = \"_async_client_dict_\" + cls.__name__\n        if not hasattr(cls, attr_name):\n            setattr(cls, attr_name, weakref.WeakKeyDictionary())\n        return getattr(cls, attr_name)\n\n    def __new__(cls, force_instance: bool = False, **kwargs: Any) -> \"AsyncHTTPClient\":\n        io_loop = IOLoop.current()\n        if force_instance:\n            instance_cache = None\n        else:\n            instance_cache = cls._async_clients()\n        if instance_cache is not None and io_loop in instance_cache:\n            return instance_cache[io_loop]\n        instance = super(AsyncHTTPClient, cls).__new__(cls, **kwargs)  # type: ignore\n        # Make sure the instance knows which cache to remove itself from.\n        # It can't simply call _async_clients() because we may be in\n        # __new__(AsyncHTTPClient) but instance.__class__ may be\n        # SimpleAsyncHTTPClient.\n        instance._instance_cache = instance_cache\n        if instance_cache is not None:\n            instance_cache[instance.io_loop] = instance\n        return instance\n\n    def initialize(self, defaults: Dict[str, Any] = None) -> None:\n        self.io_loop = IOLoop.current()\n        self.defaults = dict(HTTPRequest._DEFAULTS)\n        if defaults is not None:\n            self.defaults.update(defaults)\n        self._closed = False\n\n    def close(self) -> None:\n        \"\"\"Destroys this HTTP client, freeing any file descriptors used.\n\n        This method is **not needed in normal use** due to the way\n        that `AsyncHTTPClient` objects are transparently reused.\n        ``close()`` is generally only necessary when either the\n        `.IOLoop` is also being closed, or the ``force_instance=True``\n        argument was used when creating the `AsyncHTTPClient`.\n\n        No other methods may be called on the `AsyncHTTPClient` after\n        ``close()``.\n\n        \"\"\"\n        if self._closed:\n            return\n        self._closed = True\n        if self._instance_cache is not None:\n            if self._instance_cache.get(self.io_loop) is not self:\n                raise RuntimeError(\"inconsistent AsyncHTTPClient cache\")\n            del self._instance_cache[self.io_loop]\n\n    def fetch(\n        self,\n        request: Union[str, \"HTTPRequest\"],\n        raise_error: bool = True,\n        **kwargs: Any\n    ) -> \"Future[HTTPResponse]\":\n        \"\"\"Executes a request, asynchronously returning an `HTTPResponse`.\n\n        The request may be either a string URL or an `HTTPRequest` object.\n        If it is a string, we construct an `HTTPRequest` using any additional\n        kwargs: ``HTTPRequest(request, **kwargs)``\n\n        This method returns a `.Future` whose result is an\n        `HTTPResponse`. By default, the ``Future`` will raise an\n        `HTTPError` if the request returned a non-200 response code\n        (other errors may also be raised if the server could not be\n        contacted). Instead, if ``raise_error`` is set to False, the\n        response will always be returned regardless of the response\n        code.\n\n        If a ``callback`` is given, it will be invoked with the `HTTPResponse`.\n        In the callback interface, `HTTPError` is not automatically raised.\n        Instead, you must check the response's ``error`` attribute or\n        call its `~HTTPResponse.rethrow` method.\n\n        .. versionchanged:: 6.0\n\n           The ``callback`` argument was removed. Use the returned\n           `.Future` instead.\n\n           The ``raise_error=False`` argument only affects the\n           `HTTPError` raised when a non-200 response code is used,\n           instead of suppressing all errors.\n        \"\"\"\n        if self._closed:\n            raise RuntimeError(\"fetch() called on closed AsyncHTTPClient\")\n        if not isinstance(request, HTTPRequest):\n            request = HTTPRequest(url=request, **kwargs)\n        else:\n            if kwargs:\n                raise ValueError(\n                    \"kwargs can't be used if request is an HTTPRequest object\"\n                )\n        # We may modify this (to add Host, Accept-Encoding, etc),\n        # so make sure we don't modify the caller's object.  This is also\n        # where normal dicts get converted to HTTPHeaders objects.\n        request.headers = httputil.HTTPHeaders(request.headers)\n        request_proxy = _RequestProxy(request, self.defaults)\n        future = Future()  # type: Future[HTTPResponse]\n\n        def handle_response(response: \"HTTPResponse\") -> None:\n            if response.error:\n                if raise_error or not response._error_is_response_code:\n                    future.set_exception(response.error)\n                    return\n            future_set_result_unless_cancelled(future, response)\n\n        self.fetch_impl(cast(HTTPRequest, request_proxy), handle_response)\n        return future\n\n    def fetch_impl(\n        self, request: \"HTTPRequest\", callback: Callable[[\"HTTPResponse\"], None]\n    ) -> None:\n        raise NotImplementedError()\n\n    @classmethod\n    def configure(\n        cls, impl: Union[None, str, Type[Configurable]], **kwargs: Any\n    ) -> None:\n        \"\"\"Configures the `AsyncHTTPClient` subclass to use.\n\n        ``AsyncHTTPClient()`` actually creates an instance of a subclass.\n        This method may be called with either a class object or the\n        fully-qualified name of such a class (or ``None`` to use the default,\n        ``SimpleAsyncHTTPClient``)\n\n        If additional keyword arguments are given, they will be passed\n        to the constructor of each subclass instance created.  The\n        keyword argument ``max_clients`` determines the maximum number\n        of simultaneous `~AsyncHTTPClient.fetch()` operations that can\n        execute in parallel on each `.IOLoop`.  Additional arguments\n        may be supported depending on the implementation class in use.\n\n        Example::\n\n           AsyncHTTPClient.configure(\"tornado.curl_httpclient.CurlAsyncHTTPClient\")\n        \"\"\"\n        super(AsyncHTTPClient, cls).configure(impl, **kwargs)\n\n\nclass HTTPRequest(object):\n    \"\"\"HTTP client request object.\"\"\"\n\n    _headers = None  # type: Union[Dict[str, str], httputil.HTTPHeaders]\n\n    # Default values for HTTPRequest parameters.\n    # Merged with the values on the request object by AsyncHTTPClient\n    # implementations.\n    _DEFAULTS = dict(\n        connect_timeout=20.0,\n        request_timeout=20.0,\n        follow_redirects=True,\n        max_redirects=5,\n        decompress_response=True,\n        proxy_password=\"\",\n        allow_nonstandard_methods=False,\n        validate_cert=True,\n    )\n\n    def __init__(\n        self,\n        url: str,\n        method: str = \"GET\",\n        headers: Union[Dict[str, str], httputil.HTTPHeaders] = None,\n        body: Union[bytes, str] = None,\n        auth_username: str = None,\n        auth_password: str = None,\n        auth_mode: str = None,\n        connect_timeout: float = None,\n        request_timeout: float = None,\n        if_modified_since: Union[float, datetime.datetime] = None,\n        follow_redirects: bool = None,\n        max_redirects: int = None,\n        user_agent: str = None,\n        use_gzip: bool = None,\n        network_interface: str = None,\n        streaming_callback: Callable[[bytes], None] = None,\n        header_callback: Callable[[str], None] = None,\n        prepare_curl_callback: Callable[[Any], None] = None,\n        proxy_host: str = None,\n        proxy_port: int = None,\n        proxy_username: str = None,\n        proxy_password: str = None,\n        proxy_auth_mode: str = None,\n        allow_nonstandard_methods: bool = None,\n        validate_cert: bool = None,\n        ca_certs: str = None,\n        allow_ipv6: bool = None,\n        client_key: str = None,\n        client_cert: str = None,\n        body_producer: Callable[[Callable[[bytes], None]], \"Future[None]\"] = None,\n        expect_100_continue: bool = False,\n        decompress_response: bool = None,\n        ssl_options: Union[Dict[str, Any], ssl.SSLContext] = None,\n    ) -> None:\n        r\"\"\"All parameters except ``url`` are optional.\n\n        :arg str url: URL to fetch\n        :arg str method: HTTP method, e.g. \"GET\" or \"POST\"\n        :arg headers: Additional HTTP headers to pass on the request\n        :type headers: `~tornado.httputil.HTTPHeaders` or `dict`\n        :arg body: HTTP request body as a string (byte or unicode; if unicode\n           the utf-8 encoding will be used)\n        :arg body_producer: Callable used for lazy/asynchronous request bodies.\n           It is called with one argument, a ``write`` function, and should\n           return a `.Future`.  It should call the write function with new\n           data as it becomes available.  The write function returns a\n           `.Future` which can be used for flow control.\n           Only one of ``body`` and ``body_producer`` may\n           be specified.  ``body_producer`` is not supported on\n           ``curl_httpclient``.  When using ``body_producer`` it is recommended\n           to pass a ``Content-Length`` in the headers as otherwise chunked\n           encoding will be used, and many servers do not support chunked\n           encoding on requests.  New in Tornado 4.0\n        :arg str auth_username: Username for HTTP authentication\n        :arg str auth_password: Password for HTTP authentication\n        :arg str auth_mode: Authentication mode; default is \"basic\".\n           Allowed values are implementation-defined; ``curl_httpclient``\n           supports \"basic\" and \"digest\"; ``simple_httpclient`` only supports\n           \"basic\"\n        :arg float connect_timeout: Timeout for initial connection in seconds,\n           default 20 seconds\n        :arg float request_timeout: Timeout for entire request in seconds,\n           default 20 seconds\n        :arg if_modified_since: Timestamp for ``If-Modified-Since`` header\n        :type if_modified_since: `datetime` or `float`\n        :arg bool follow_redirects: Should redirects be followed automatically\n           or return the 3xx response? Default True.\n        :arg int max_redirects: Limit for ``follow_redirects``, default 5.\n        :arg str user_agent: String to send as ``User-Agent`` header\n        :arg bool decompress_response: Request a compressed response from\n           the server and decompress it after downloading.  Default is True.\n           New in Tornado 4.0.\n        :arg bool use_gzip: Deprecated alias for ``decompress_response``\n           since Tornado 4.0.\n        :arg str network_interface: Network interface or source IP to use for request.\n           See ``curl_httpclient`` note below.\n        :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will\n           be run with each chunk of data as it is received, and\n           ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in\n           the final response.\n        :arg collections.abc.Callable header_callback: If set, ``header_callback`` will\n           be run with each header line as it is received (including the\n           first line, e.g. ``HTTP/1.0 200 OK\\r\\n``, and a final line\n           containing only ``\\r\\n``.  All lines include the trailing newline\n           characters).  ``HTTPResponse.headers`` will be empty in the final\n           response.  This is most useful in conjunction with\n           ``streaming_callback``, because it's the only way to get access to\n           header data while the request is in progress.\n        :arg collections.abc.Callable prepare_curl_callback: If set, will be called with\n           a ``pycurl.Curl`` object to allow the application to make additional\n           ``setopt`` calls.\n        :arg str proxy_host: HTTP proxy hostname.  To use proxies,\n           ``proxy_host`` and ``proxy_port`` must be set; ``proxy_username``,\n           ``proxy_pass`` and ``proxy_auth_mode`` are optional.  Proxies are\n           currently only supported with ``curl_httpclient``.\n        :arg int proxy_port: HTTP proxy port\n        :arg str proxy_username: HTTP proxy username\n        :arg str proxy_password: HTTP proxy password\n        :arg str proxy_auth_mode: HTTP proxy Authentication mode;\n           default is \"basic\". supports \"basic\" and \"digest\"\n        :arg bool allow_nonstandard_methods: Allow unknown values for ``method``\n           argument? Default is False.\n        :arg bool validate_cert: For HTTPS requests, validate the server's\n           certificate? Default is True.\n        :arg str ca_certs: filename of CA certificates in PEM format,\n           or None to use defaults.  See note below when used with\n           ``curl_httpclient``.\n        :arg str client_key: Filename for client SSL key, if any.  See\n           note below when used with ``curl_httpclient``.\n        :arg str client_cert: Filename for client SSL certificate, if any.\n           See note below when used with ``curl_httpclient``.\n        :arg ssl.SSLContext ssl_options: `ssl.SSLContext` object for use in\n           ``simple_httpclient`` (unsupported by ``curl_httpclient``).\n           Overrides ``validate_cert``, ``ca_certs``, ``client_key``,\n           and ``client_cert``.\n        :arg bool allow_ipv6: Use IPv6 when available?  Default is true.\n        :arg bool expect_100_continue: If true, send the\n           ``Expect: 100-continue`` header and wait for a continue response\n           before sending the request body.  Only supported with\n           simple_httpclient.\n\n        .. note::\n\n            When using ``curl_httpclient`` certain options may be\n            inherited by subsequent fetches because ``pycurl`` does\n            not allow them to be cleanly reset.  This applies to the\n            ``ca_certs``, ``client_key``, ``client_cert``, and\n            ``network_interface`` arguments.  If you use these\n            options, you should pass them on every request (you don't\n            have to always use the same values, but it's not possible\n            to mix requests that specify these options with ones that\n            use the defaults).\n\n        .. versionadded:: 3.1\n           The ``auth_mode`` argument.\n\n        .. versionadded:: 4.0\n           The ``body_producer`` and ``expect_100_continue`` arguments.\n\n        .. versionadded:: 4.2\n           The ``ssl_options`` argument.\n\n        .. versionadded:: 4.5\n           The ``proxy_auth_mode`` argument.\n        \"\"\"\n        # Note that some of these attributes go through property setters\n        # defined below.\n        self.headers = headers\n        if if_modified_since:\n            self.headers[\"If-Modified-Since\"] = httputil.format_timestamp(\n                if_modified_since\n            )\n        self.proxy_host = proxy_host\n        self.proxy_port = proxy_port\n        self.proxy_username = proxy_username\n        self.proxy_password = proxy_password\n        self.proxy_auth_mode = proxy_auth_mode\n        self.url = url\n        self.method = method\n        self.body = body\n        self.body_producer = body_producer\n        self.auth_username = auth_username\n        self.auth_password = auth_password\n        self.auth_mode = auth_mode\n        self.connect_timeout = connect_timeout\n        self.request_timeout = request_timeout\n        self.follow_redirects = follow_redirects\n        self.max_redirects = max_redirects\n        self.user_agent = user_agent\n        if decompress_response is not None:\n            self.decompress_response = decompress_response  # type: Optional[bool]\n        else:\n            self.decompress_response = use_gzip\n        self.network_interface = network_interface\n        self.streaming_callback = streaming_callback\n        self.header_callback = header_callback\n        self.prepare_curl_callback = prepare_curl_callback\n        self.allow_nonstandard_methods = allow_nonstandard_methods\n        self.validate_cert = validate_cert\n        self.ca_certs = ca_certs\n        self.allow_ipv6 = allow_ipv6\n        self.client_key = client_key\n        self.client_cert = client_cert\n        self.ssl_options = ssl_options\n        self.expect_100_continue = expect_100_continue\n        self.start_time = time.time()\n\n    @property\n    def headers(self) -> httputil.HTTPHeaders:\n        # TODO: headers may actually be a plain dict until fairly late in\n        # the process (AsyncHTTPClient.fetch), but practically speaking,\n        # whenever the property is used they're already HTTPHeaders.\n        return self._headers  # type: ignore\n\n    @headers.setter\n    def headers(self, value: Union[Dict[str, str], httputil.HTTPHeaders]) -> None:\n        if value is None:\n            self._headers = httputil.HTTPHeaders()\n        else:\n            self._headers = value  # type: ignore\n\n    @property\n    def body(self) -> bytes:\n        return self._body\n\n    @body.setter\n    def body(self, value: Union[bytes, str]) -> None:\n        self._body = utf8(value)\n\n\nclass HTTPResponse(object):\n    \"\"\"HTTP Response object.\n\n    Attributes:\n\n    * request: HTTPRequest object\n\n    * code: numeric HTTP status code, e.g. 200 or 404\n\n    * reason: human-readable reason phrase describing the status code\n\n    * headers: `tornado.httputil.HTTPHeaders` object\n\n    * effective_url: final location of the resource after following any\n      redirects\n\n    * buffer: ``cStringIO`` object for response body\n\n    * body: response body as bytes (created on demand from ``self.buffer``)\n\n    * error: Exception object, if any\n\n    * request_time: seconds from request start to finish. Includes all network\n      operations from DNS resolution to receiving the last byte of data.\n      Does not include time spent in the queue (due to the ``max_clients`` option).\n      If redirects were followed, only includes the final request.\n\n    * start_time: Time at which the HTTP operation started, based on `time.time`\n      (not the monotonic clock used by `.IOLoop.time`). May be ``None`` if the request\n      timed out while in the queue.\n\n    * time_info: dictionary of diagnostic timing information from the request.\n      Available data are subject to change, but currently uses timings\n      available from http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html,\n      plus ``queue``, which is the delay (if any) introduced by waiting for\n      a slot under `AsyncHTTPClient`'s ``max_clients`` setting.\n\n    .. versionadded:: 5.1\n\n       Added the ``start_time`` attribute.\n\n    .. versionchanged:: 5.1\n\n       The ``request_time`` attribute previously included time spent in the queue\n       for ``simple_httpclient``, but not in ``curl_httpclient``. Now queueing time\n       is excluded in both implementations. ``request_time`` is now more accurate for\n       ``curl_httpclient`` because it uses a monotonic clock when available.\n    \"\"\"\n\n    # I'm not sure why these don't get type-inferred from the references in __init__.\n    error = None  # type: Optional[BaseException]\n    _error_is_response_code = False\n    request = None  # type: HTTPRequest\n\n    def __init__(\n        self,\n        request: HTTPRequest,\n        code: int,\n        headers: httputil.HTTPHeaders = None,\n        buffer: BytesIO = None,\n        effective_url: str = None,\n        error: BaseException = None,\n        request_time: float = None,\n        time_info: Dict[str, float] = None,\n        reason: str = None,\n        start_time: float = None,\n    ) -> None:\n        if isinstance(request, _RequestProxy):\n            self.request = request.request\n        else:\n            self.request = request\n        self.code = code\n        self.reason = reason or httputil.responses.get(code, \"Unknown\")\n        if headers is not None:\n            self.headers = headers\n        else:\n            self.headers = httputil.HTTPHeaders()\n        self.buffer = buffer\n        self._body = None  # type: Optional[bytes]\n        if effective_url is None:\n            self.effective_url = request.url\n        else:\n            self.effective_url = effective_url\n        self._error_is_response_code = False\n        if error is None:\n            if self.code < 200 or self.code >= 300:\n                self._error_is_response_code = True\n                self.error = HTTPError(self.code, message=self.reason, response=self)\n            else:\n                self.error = None\n        else:\n            self.error = error\n        self.start_time = start_time\n        self.request_time = request_time\n        self.time_info = time_info or {}\n\n    @property\n    def body(self) -> bytes:\n        if self.buffer is None:\n            raise ValueError(\"body not set\")\n        elif self._body is None:\n            self._body = self.buffer.getvalue()\n\n        return self._body\n\n    def rethrow(self) -> None:\n        \"\"\"If there was an error on the request, raise an `HTTPError`.\"\"\"\n        if self.error:\n            raise self.error\n\n    def __repr__(self) -> str:\n        args = \",\".join(\"%s=%r\" % i for i in sorted(self.__dict__.items()))\n        return \"%s(%s)\" % (self.__class__.__name__, args)\n\n\nclass HTTPClientError(Exception):\n    \"\"\"Exception thrown for an unsuccessful HTTP request.\n\n    Attributes:\n\n    * ``code`` - HTTP error integer error code, e.g. 404.  Error code 599 is\n      used when no HTTP response was received, e.g. for a timeout.\n\n    * ``response`` - `HTTPResponse` object, if any.\n\n    Note that if ``follow_redirects`` is False, redirects become HTTPErrors,\n    and you can look at ``error.response.headers['Location']`` to see the\n    destination of the redirect.\n\n    .. versionchanged:: 5.1\n\n       Renamed from ``HTTPError`` to ``HTTPClientError`` to avoid collisions with\n       `tornado.web.HTTPError`. The name ``tornado.httpclient.HTTPError`` remains\n       as an alias.\n    \"\"\"\n\n    def __init__(\n        self, code: int, message: str = None, response: HTTPResponse = None\n    ) -> None:\n        self.code = code\n        self.message = message or httputil.responses.get(code, \"Unknown\")\n        self.response = response\n        super(HTTPClientError, self).__init__(code, message, response)\n\n    def __str__(self) -> str:\n        return \"HTTP %d: %s\" % (self.code, self.message)\n\n    # There is a cyclic reference between self and self.response,\n    # which breaks the default __repr__ implementation.\n    # (especially on pypy, which doesn't have the same recursion\n    # detection as cpython).\n    __repr__ = __str__\n\n\nHTTPError = HTTPClientError\n\n\nclass _RequestProxy(object):\n    \"\"\"Combines an object with a dictionary of defaults.\n\n    Used internally by AsyncHTTPClient implementations.\n    \"\"\"\n\n    def __init__(\n        self, request: HTTPRequest, defaults: Optional[Dict[str, Any]]\n    ) -> None:\n        self.request = request\n        self.defaults = defaults\n\n    def __getattr__(self, name: str) -> Any:\n        request_attr = getattr(self.request, name)\n        if request_attr is not None:\n            return request_attr\n        elif self.defaults is not None:\n            return self.defaults.get(name, None)\n        else:\n            return None\n\n\ndef main() -> None:\n    from tornado.options import define, options, parse_command_line\n\n    define(\"print_headers\", type=bool, default=False)\n    define(\"print_body\", type=bool, default=True)\n    define(\"follow_redirects\", type=bool, default=True)\n    define(\"validate_cert\", type=bool, default=True)\n    define(\"proxy_host\", type=str)\n    define(\"proxy_port\", type=int)\n    args = parse_command_line()\n    client = HTTPClient()\n    for arg in args:\n        try:\n            response = client.fetch(\n                arg,\n                follow_redirects=options.follow_redirects,\n                validate_cert=options.validate_cert,\n                proxy_host=options.proxy_host,\n                proxy_port=options.proxy_port,\n            )\n        except HTTPError as e:\n            if e.response is not None:\n                response = e.response\n            else:\n                raise\n        if options.print_headers:\n            print(response.headers)\n        if options.print_body:\n            print(native_str(response.body))\n    client.close()\n\n\nif __name__ == \"__main__\":\n    main()\n",
            "header": {
                "file": {
                    "path": "tornado/httpclient.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "e2623d53",
                "new_commit_hash": "d1c92a49",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/tornado/httpclient.py"
            },
            "plus_file": {
                "path": "b/tornado/httpclient.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 422,
                            "length": 8
                        },
                        "new_range": {
                            "start": 422,
                            "length": 8
                        },
                        "section": "class HTTPRequest(object):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "           New in Tornado 4.0.",
                                "type": "context"
                            },
                            {
                                "content": "        :arg bool use_gzip: Deprecated alias for ``decompress_response``",
                                "type": "context"
                            },
                            {
                                "content": "           since Tornado 4.0.",
                                "type": "context"
                            },
                            {
                                "content": "        :arg str network_interface: Network interface to use for request.",
                                "type": "deleted"
                            },
                            {
                                "content": "           ``curl_httpclient`` only; see note below.",
                                "type": "deleted"
                            },
                            {
                                "content": "        :arg str network_interface: Network interface or source IP to use for request.",
                                "type": "added"
                            },
                            {
                                "content": "           See ``curl_httpclient`` note below.",
                                "type": "added"
                            },
                            {
                                "content": "        :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will",
                                "type": "context"
                            },
                            {
                                "content": "           be run with each chunk of data as it is received, and",
                                "type": "context"
                            },
                            {
                                "content": "           ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/httpclient.py",
                            "type": "class",
                            "name": "HTTPRequest",
                            "content": "class HTTPRequest(object):\n    \"\"\"HTTP client request object.\"\"\"\n\n    _headers = None  # type: Union[Dict[str, str], httputil.HTTPHeaders]\n\n    # Default values for HTTPRequest parameters.\n    # Merged with the values on the request object by AsyncHTTPClient\n    # implementations.\n    _DEFAULTS = dict(\n        connect_timeout=20.0,\n        request_timeout=20.0,\n        follow_redirects=True,\n        max_redirects=5,\n        decompress_response=True,\n        proxy_password=\"\",\n        allow_nonstandard_methods=False,\n        validate_cert=True,\n    )\n\n    def __init__(\n        self,\n        url: str,\n        method: str = \"GET\",\n        headers: Union[Dict[str, str], httputil.HTTPHeaders] = None,\n        body: Union[bytes, str] = None,\n        auth_username: str = None,\n        auth_password: str = None,\n        auth_mode: str = None,\n        connect_timeout: float = None,\n        request_timeout: float = None,\n        if_modified_since: Union[float, datetime.datetime] = None,\n        follow_redirects: bool = None,\n        max_redirects: int = None,\n        user_agent: str = None,\n        use_gzip: bool = None,\n        network_interface: str = None,\n        streaming_callback: Callable[[bytes], None] = None,\n        header_callback: Callable[[str], None] = None,\n        prepare_curl_callback: Callable[[Any], None] = None,\n        proxy_host: str = None,\n        proxy_port: int = None,\n        proxy_username: str = None,\n        proxy_password: str = None,\n        proxy_auth_mode: str = None,\n        allow_nonstandard_methods: bool = None,\n        validate_cert: bool = None,\n        ca_certs: str = None,\n        allow_ipv6: bool = None,\n        client_key: str = None,\n        client_cert: str = None,\n        body_producer: Callable[[Callable[[bytes], None]], \"Future[None]\"] = None,\n        expect_100_continue: bool = False,\n        decompress_response: bool = None,\n        ssl_options: Union[Dict[str, Any], ssl.SSLContext] = None,\n    ) -> None:\n        r\"\"\"All parameters except ``url`` are optional.\n\n        :arg str url: URL to fetch\n        :arg str method: HTTP method, e.g. \"GET\" or \"POST\"\n        :arg headers: Additional HTTP headers to pass on the request\n        :type headers: `~tornado.httputil.HTTPHeaders` or `dict`\n        :arg body: HTTP request body as a string (byte or unicode; if unicode\n           the utf-8 encoding will be used)\n        :arg body_producer: Callable used for lazy/asynchronous request bodies.\n           It is called with one argument, a ``write`` function, and should\n           return a `.Future`.  It should call the write function with new\n           data as it becomes available.  The write function returns a\n           `.Future` which can be used for flow control.\n           Only one of ``body`` and ``body_producer`` may\n           be specified.  ``body_producer`` is not supported on\n           ``curl_httpclient``.  When using ``body_producer`` it is recommended\n           to pass a ``Content-Length`` in the headers as otherwise chunked\n           encoding will be used, and many servers do not support chunked\n           encoding on requests.  New in Tornado 4.0\n        :arg str auth_username: Username for HTTP authentication\n        :arg str auth_password: Password for HTTP authentication\n        :arg str auth_mode: Authentication mode; default is \"basic\".\n           Allowed values are implementation-defined; ``curl_httpclient``\n           supports \"basic\" and \"digest\"; ``simple_httpclient`` only supports\n           \"basic\"\n        :arg float connect_timeout: Timeout for initial connection in seconds,\n           default 20 seconds\n        :arg float request_timeout: Timeout for entire request in seconds,\n           default 20 seconds\n        :arg if_modified_since: Timestamp for ``If-Modified-Since`` header\n        :type if_modified_since: `datetime` or `float`\n        :arg bool follow_redirects: Should redirects be followed automatically\n           or return the 3xx response? Default True.\n        :arg int max_redirects: Limit for ``follow_redirects``, default 5.\n        :arg str user_agent: String to send as ``User-Agent`` header\n        :arg bool decompress_response: Request a compressed response from\n           the server and decompress it after downloading.  Default is True.\n           New in Tornado 4.0.\n        :arg bool use_gzip: Deprecated alias for ``decompress_response``\n           since Tornado 4.0.\n        :arg str network_interface: Network interface to use for request.\n           ``curl_httpclient`` only; see note below.\n        :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will\n           be run with each chunk of data as it is received, and\n           ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in\n           the final response.\n        :arg collections.abc.Callable header_callback: If set, ``header_callback`` will\n           be run with each header line as it is received (including the\n           first line, e.g. ``HTTP/1.0 200 OK\\r\\n``, and a final line\n           containing only ``\\r\\n``.  All lines include the trailing newline\n           characters).  ``HTTPResponse.headers`` will be empty in the final\n           response.  This is most useful in conjunction with\n           ``streaming_callback``, because it's the only way to get access to\n           header data while the request is in progress.\n        :arg collections.abc.Callable prepare_curl_callback: If set, will be called with\n           a ``pycurl.Curl`` object to allow the application to make additional\n           ``setopt`` calls.\n        :arg str proxy_host: HTTP proxy hostname.  To use proxies,\n           ``proxy_host`` and ``proxy_port`` must be set; ``proxy_username``,\n           ``proxy_pass`` and ``proxy_auth_mode`` are optional.  Proxies are\n           currently only supported with ``curl_httpclient``.\n        :arg int proxy_port: HTTP proxy port\n        :arg str proxy_username: HTTP proxy username\n        :arg str proxy_password: HTTP proxy password\n        :arg str proxy_auth_mode: HTTP proxy Authentication mode;\n           default is \"basic\". supports \"basic\" and \"digest\"\n        :arg bool allow_nonstandard_methods: Allow unknown values for ``method``\n           argument? Default is False.\n        :arg bool validate_cert: For HTTPS requests, validate the server's\n           certificate? Default is True.\n        :arg str ca_certs: filename of CA certificates in PEM format,\n           or None to use defaults.  See note below when used with\n           ``curl_httpclient``.\n        :arg str client_key: Filename for client SSL key, if any.  See\n           note below when used with ``curl_httpclient``.\n        :arg str client_cert: Filename for client SSL certificate, if any.\n           See note below when used with ``curl_httpclient``.\n        :arg ssl.SSLContext ssl_options: `ssl.SSLContext` object for use in\n           ``simple_httpclient`` (unsupported by ``curl_httpclient``).\n           Overrides ``validate_cert``, ``ca_certs``, ``client_key``,\n           and ``client_cert``.\n        :arg bool allow_ipv6: Use IPv6 when available?  Default is true.\n        :arg bool expect_100_continue: If true, send the\n           ``Expect: 100-continue`` header and wait for a continue response\n           before sending the request body.  Only supported with\n           simple_httpclient.\n\n        .. note::\n\n            When using ``curl_httpclient`` certain options may be\n            inherited by subsequent fetches because ``pycurl`` does\n            not allow them to be cleanly reset.  This applies to the\n            ``ca_certs``, ``client_key``, ``client_cert``, and\n            ``network_interface`` arguments.  If you use these\n            options, you should pass them on every request (you don't\n            have to always use the same values, but it's not possible\n            to mix requests that specify these options with ones that\n            use the defaults).\n\n        .. versionadded:: 3.1\n           The ``auth_mode`` argument.\n\n        .. versionadded:: 4.0\n           The ``body_producer`` and ``expect_100_continue`` arguments.\n\n        .. versionadded:: 4.2\n           The ``ssl_options`` argument.\n\n        .. versionadded:: 4.5\n           The ``proxy_auth_mode`` argument.\n        \"\"\"\n        # Note that some of these attributes go through property setters\n        # defined below.\n        self.headers = headers\n        if if_modified_since:\n            self.headers[\"If-Modified-Since\"] = httputil.format_timestamp(\n                if_modified_since\n            )\n        self.proxy_host = proxy_host\n        self.proxy_port = proxy_port\n        self.proxy_username = proxy_username\n        self.proxy_password = proxy_password\n        self.proxy_auth_mode = proxy_auth_mode\n        self.url = url\n        self.method = method\n        self.body = body\n        self.body_producer = body_producer\n        self.auth_username = auth_username\n        self.auth_password = auth_password\n        self.auth_mode = auth_mode\n        self.connect_timeout = connect_timeout\n        self.request_timeout = request_timeout\n        self.follow_redirects = follow_redirects\n        self.max_redirects = max_redirects\n        self.user_agent = user_agent\n        if decompress_response is not None:\n            self.decompress_response = decompress_response  # type: Optional[bool]\n        else:\n            self.decompress_response = use_gzip\n        self.network_interface = network_interface\n        self.streaming_callback = streaming_callback\n        self.header_callback = header_callback\n        self.prepare_curl_callback = prepare_curl_callback\n        self.allow_nonstandard_methods = allow_nonstandard_methods\n        self.validate_cert = validate_cert\n        self.ca_certs = ca_certs\n        self.allow_ipv6 = allow_ipv6\n        self.client_key = client_key\n        self.client_cert = client_cert\n        self.ssl_options = ssl_options\n        self.expect_100_continue = expect_100_continue\n        self.start_time = time.time()\n\n    @property\n    def headers(self) -> httputil.HTTPHeaders:\n        # TODO: headers may actually be a plain dict until fairly late in\n        # the process (AsyncHTTPClient.fetch), but practically speaking,\n        # whenever the property is used they're already HTTPHeaders.\n        return self._headers  # type: ignore\n\n    @headers.setter\n    def headers(self, value: Union[Dict[str, str], httputil.HTTPHeaders]) -> None:\n        if value is None:\n            self._headers = httputil.HTTPHeaders()\n        else:\n            self._headers = value  # type: ignore\n\n    @property\n    def body(self) -> bytes:\n        return self._body\n\n    @body.setter\n    def body(self, value: Union[bytes, str]) -> None:\n        self._body = utf8(value)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 330,
                            "end_lineno": 558,
                            "parent": null
                        },
                        {
                            "file_name": "tornado/httpclient.py",
                            "type": "method",
                            "name": "HTTPRequest.__init__",
                            "content": "class HTTPRequest(object):\n    \"\"\"HTTP client request object.\"\"\"\n\n    _headers = None  # type: Union[Dict[str, str], httputil.HTTPHeaders]\n\n    # Default values for HTTPRequest parameters.\n    # Merged with the values on the request object by AsyncHTTPClient\n    # implementations.\n    _DEFAULTS = dict(\n        connect_timeout=20.0,\n        request_timeout=20.0,\n        follow_redirects=True,\n        max_redirects=5,\n        decompress_response=True,\n        proxy_password=\"\",\n        allow_nonstandard_methods=False,\n        validate_cert=True,\n    )\n\n    def __init__(\n        self,\n        url: str,\n        method: str = \"GET\",\n        headers: Union[Dict[str, str], httputil.HTTPHeaders] = None,\n        body: Union[bytes, str] = None,\n        auth_username: str = None,\n        auth_password: str = None,\n        auth_mode: str = None,\n        connect_timeout: float = None,\n        request_timeout: float = None,\n        if_modified_since: Union[float, datetime.datetime] = None,\n        follow_redirects: bool = None,\n        max_redirects: int = None,\n        user_agent: str = None,\n        use_gzip: bool = None,\n        network_interface: str = None,\n        streaming_callback: Callable[[bytes], None] = None,\n        header_callback: Callable[[str], None] = None,\n        prepare_curl_callback: Callable[[Any], None] = None,\n        proxy_host: str = None,\n        proxy_port: int = None,\n        proxy_username: str = None,\n        proxy_password: str = None,\n        proxy_auth_mode: str = None,\n        allow_nonstandard_methods: bool = None,\n        validate_cert: bool = None,\n        ca_certs: str = None,\n        allow_ipv6: bool = None,\n        client_key: str = None,\n        client_cert: str = None,\n        body_producer: Callable[[Callable[[bytes], None]], \"Future[None]\"] = None,\n        expect_100_continue: bool = False,\n        decompress_response: bool = None,\n        ssl_options: Union[Dict[str, Any], ssl.SSLContext] = None,\n    ) -> None:\n        r\"\"\"All parameters except ``url`` are optional.\n\n        :arg str url: URL to fetch\n        :arg str method: HTTP method, e.g. \"GET\" or \"POST\"\n        :arg headers: Additional HTTP headers to pass on the request\n        :type headers: `~tornado.httputil.HTTPHeaders` or `dict`\n        :arg body: HTTP request body as a string (byte or unicode; if unicode\n           the utf-8 encoding will be used)\n        :arg body_producer: Callable used for lazy/asynchronous request bodies.\n           It is called with one argument, a ``write`` function, and should\n           return a `.Future`.  It should call the write function with new\n           data as it becomes available.  The write function returns a\n           `.Future` which can be used for flow control.\n           Only one of ``body`` and ``body_producer`` may\n           be specified.  ``body_producer`` is not supported on\n           ``curl_httpclient``.  When using ``body_producer`` it is recommended\n           to pass a ``Content-Length`` in the headers as otherwise chunked\n           encoding will be used, and many servers do not support chunked\n           encoding on requests.  New in Tornado 4.0\n        :arg str auth_username: Username for HTTP authentication\n        :arg str auth_password: Password for HTTP authentication\n        :arg str auth_mode: Authentication mode; default is \"basic\".\n           Allowed values are implementation-defined; ``curl_httpclient``\n           supports \"basic\" and \"digest\"; ``simple_httpclient`` only supports\n           \"basic\"\n        :arg float connect_timeout: Timeout for initial connection in seconds,\n           default 20 seconds\n        :arg float request_timeout: Timeout for entire request in seconds,\n           default 20 seconds\n        :arg if_modified_since: Timestamp for ``If-Modified-Since`` header\n        :type if_modified_since: `datetime` or `float`\n        :arg bool follow_redirects: Should redirects be followed automatically\n           or return the 3xx response? Default True.\n        :arg int max_redirects: Limit for ``follow_redirects``, default 5.\n        :arg str user_agent: String to send as ``User-Agent`` header\n        :arg bool decompress_response: Request a compressed response from\n           the server and decompress it after downloading.  Default is True.\n           New in Tornado 4.0.\n        :arg bool use_gzip: Deprecated alias for ``decompress_response``\n           since Tornado 4.0.\n        :arg str network_interface: Network interface to use for request.\n           ``curl_httpclient`` only; see note below.\n        :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will\n           be run with each chunk of data as it is received, and\n           ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in\n           the final response.\n        :arg collections.abc.Callable header_callback: If set, ``header_callback`` will\n           be run with each header line as it is received (including the\n           first line, e.g. ``HTTP/1.0 200 OK\\r\\n``, and a final line\n           containing only ``\\r\\n``.  All lines include the trailing newline\n           characters).  ``HTTPResponse.headers`` will be empty in the final\n           response.  This is most useful in conjunction with\n           ``streaming_callback``, because it's the only way to get access to\n           header data while the request is in progress.\n        :arg collections.abc.Callable prepare_curl_callback: If set, will be called with\n           a ``pycurl.Curl`` object to allow the application to make additional\n           ``setopt`` calls.\n        :arg str proxy_host: HTTP proxy hostname.  To use proxies,\n           ``proxy_host`` and ``proxy_port`` must be set; ``proxy_username``,\n           ``proxy_pass`` and ``proxy_auth_mode`` are optional.  Proxies are\n           currently only supported with ``curl_httpclient``.\n        :arg int proxy_port: HTTP proxy port\n        :arg str proxy_username: HTTP proxy username\n        :arg str proxy_password: HTTP proxy password\n        :arg str proxy_auth_mode: HTTP proxy Authentication mode;\n           default is \"basic\". supports \"basic\" and \"digest\"\n        :arg bool allow_nonstandard_methods: Allow unknown values for ``method``\n           argument? Default is False.\n        :arg bool validate_cert: For HTTPS requests, validate the server's\n           certificate? Default is True.\n        :arg str ca_certs: filename of CA certificates in PEM format,\n           or None to use defaults.  See note below when used with\n           ``curl_httpclient``.\n        :arg str client_key: Filename for client SSL key, if any.  See\n           note below when used with ``curl_httpclient``.\n        :arg str client_cert: Filename for client SSL certificate, if any.\n           See note below when used with ``curl_httpclient``.\n        :arg ssl.SSLContext ssl_options: `ssl.SSLContext` object for use in\n           ``simple_httpclient`` (unsupported by ``curl_httpclient``).\n           Overrides ``validate_cert``, ``ca_certs``, ``client_key``,\n           and ``client_cert``.\n        :arg bool allow_ipv6: Use IPv6 when available?  Default is true.\n        :arg bool expect_100_continue: If true, send the\n           ``Expect: 100-continue`` header and wait for a continue response\n           before sending the request body.  Only supported with\n           simple_httpclient.\n\n        .. note::\n\n            When using ``curl_httpclient`` certain options may be\n            inherited by subsequent fetches because ``pycurl`` does\n            not allow them to be cleanly reset.  This applies to the\n            ``ca_certs``, ``client_key``, ``client_cert``, and\n            ``network_interface`` arguments.  If you use these\n            options, you should pass them on every request (you don't\n            have to always use the same values, but it's not possible\n            to mix requests that specify these options with ones that\n            use the defaults).\n\n        .. versionadded:: 3.1\n           The ``auth_mode`` argument.\n\n        .. versionadded:: 4.0\n           The ``body_producer`` and ``expect_100_continue`` arguments.\n\n        .. versionadded:: 4.2\n           The ``ssl_options`` argument.\n\n        .. versionadded:: 4.5\n           The ``proxy_auth_mode`` argument.\n        \"\"\"\n        # Note that some of these attributes go through property setters\n        # defined below.\n        self.headers = headers\n        if if_modified_since:\n            self.headers[\"If-Modified-Since\"] = httputil.format_timestamp(\n                if_modified_since\n            )\n        self.proxy_host = proxy_host\n        self.proxy_port = proxy_port\n        self.proxy_username = proxy_username\n        self.proxy_password = proxy_password\n        self.proxy_auth_mode = proxy_auth_mode\n        self.url = url\n        self.method = method\n        self.body = body\n        self.body_producer = body_producer\n        self.auth_username = auth_username\n        self.auth_password = auth_password\n        self.auth_mode = auth_mode\n        self.connect_timeout = connect_timeout\n        self.request_timeout = request_timeout\n        self.follow_redirects = follow_redirects\n        self.max_redirects = max_redirects\n        self.user_agent = user_agent\n        if decompress_response is not None:\n            self.decompress_response = decompress_response  # type: Optional[bool]\n        else:\n            self.decompress_response = use_gzip\n        self.network_interface = network_interface\n        self.streaming_callback = streaming_callback\n        self.header_callback = header_callback\n        self.prepare_curl_callback = prepare_curl_callback\n        self.allow_nonstandard_methods = allow_nonstandard_methods\n        self.validate_cert = validate_cert\n        self.ca_certs = ca_certs\n        self.allow_ipv6 = allow_ipv6\n        self.client_key = client_key\n        self.client_cert = client_cert\n        self.ssl_options = ssl_options\n        self.expect_100_continue = expect_100_continue\n        self.start_time = time.time()\n\n    @property\n    def headers(self) -> httputil.HTTPHeaders:\n        # TODO: headers may actually be a plain dict until fairly late in\n        # the process (AsyncHTTPClient.fetch), but practically speaking,\n        # whenever the property is used they're already HTTPHeaders.\n        return self._headers  # type: ignore\n\n    @headers.setter\n    def headers(self, value: Union[Dict[str, str], httputil.HTTPHeaders]) -> None:\n        if value is None:\n            self._headers = httputil.HTTPHeaders()\n        else:\n            self._headers = value  # type: ignore\n\n    @property\n    def body(self) -> bytes:\n        return self._body\n\n    @body.setter\n    def body(self, value: Union[bytes, str]) -> None:\n        self._body = utf8(value)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 349,
                            "end_lineno": 536,
                            "parent": {
                                "file_name": "tornado/httpclient.py",
                                "type": "class",
                                "name": "HTTPRequest",
                                "content": "class HTTPRequest(object):\n    \"\"\"HTTP client request object.\"\"\"\n\n    _headers = None  # type: Union[Dict[str, str], httputil.HTTPHeaders]\n\n    # Default values for HTTPRequest parameters.\n    # Merged with the values on the request object by AsyncHTTPClient\n    # implementations.\n    _DEFAULTS = dict(\n        connect_timeout=20.0,\n        request_timeout=20.0,\n        follow_redirects=True,\n        max_redirects=5,\n        decompress_response=True,\n        proxy_password=\"\",\n        allow_nonstandard_methods=False,\n        validate_cert=True,\n    )\n\n    def __init__(\n        self,\n        url: str,\n        method: str = \"GET\",\n        headers: Union[Dict[str, str], httputil.HTTPHeaders] = None,\n        body: Union[bytes, str] = None,\n        auth_username: str = None,\n        auth_password: str = None,\n        auth_mode: str = None,\n        connect_timeout: float = None,\n        request_timeout: float = None,\n        if_modified_since: Union[float, datetime.datetime] = None,\n        follow_redirects: bool = None,\n        max_redirects: int = None,\n        user_agent: str = None,\n        use_gzip: bool = None,\n        network_interface: str = None,\n        streaming_callback: Callable[[bytes], None] = None,\n        header_callback: Callable[[str], None] = None,\n        prepare_curl_callback: Callable[[Any], None] = None,\n        proxy_host: str = None,\n        proxy_port: int = None,\n        proxy_username: str = None,\n        proxy_password: str = None,\n        proxy_auth_mode: str = None,\n        allow_nonstandard_methods: bool = None,\n        validate_cert: bool = None,\n        ca_certs: str = None,\n        allow_ipv6: bool = None,\n        client_key: str = None,\n        client_cert: str = None,\n        body_producer: Callable[[Callable[[bytes], None]], \"Future[None]\"] = None,\n        expect_100_continue: bool = False,\n        decompress_response: bool = None,\n        ssl_options: Union[Dict[str, Any], ssl.SSLContext] = None,\n    ) -> None:\n        r\"\"\"All parameters except ``url`` are optional.\n\n        :arg str url: URL to fetch\n        :arg str method: HTTP method, e.g. \"GET\" or \"POST\"\n        :arg headers: Additional HTTP headers to pass on the request\n        :type headers: `~tornado.httputil.HTTPHeaders` or `dict`\n        :arg body: HTTP request body as a string (byte or unicode; if unicode\n           the utf-8 encoding will be used)\n        :arg body_producer: Callable used for lazy/asynchronous request bodies.\n           It is called with one argument, a ``write`` function, and should\n           return a `.Future`.  It should call the write function with new\n           data as it becomes available.  The write function returns a\n           `.Future` which can be used for flow control.\n           Only one of ``body`` and ``body_producer`` may\n           be specified.  ``body_producer`` is not supported on\n           ``curl_httpclient``.  When using ``body_producer`` it is recommended\n           to pass a ``Content-Length`` in the headers as otherwise chunked\n           encoding will be used, and many servers do not support chunked\n           encoding on requests.  New in Tornado 4.0\n        :arg str auth_username: Username for HTTP authentication\n        :arg str auth_password: Password for HTTP authentication\n        :arg str auth_mode: Authentication mode; default is \"basic\".\n           Allowed values are implementation-defined; ``curl_httpclient``\n           supports \"basic\" and \"digest\"; ``simple_httpclient`` only supports\n           \"basic\"\n        :arg float connect_timeout: Timeout for initial connection in seconds,\n           default 20 seconds\n        :arg float request_timeout: Timeout for entire request in seconds,\n           default 20 seconds\n        :arg if_modified_since: Timestamp for ``If-Modified-Since`` header\n        :type if_modified_since: `datetime` or `float`\n        :arg bool follow_redirects: Should redirects be followed automatically\n           or return the 3xx response? Default True.\n        :arg int max_redirects: Limit for ``follow_redirects``, default 5.\n        :arg str user_agent: String to send as ``User-Agent`` header\n        :arg bool decompress_response: Request a compressed response from\n           the server and decompress it after downloading.  Default is True.\n           New in Tornado 4.0.\n        :arg bool use_gzip: Deprecated alias for ``decompress_response``\n           since Tornado 4.0.\n        :arg str network_interface: Network interface to use for request.\n           ``curl_httpclient`` only; see note below.\n        :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will\n           be run with each chunk of data as it is received, and\n           ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in\n           the final response.\n        :arg collections.abc.Callable header_callback: If set, ``header_callback`` will\n           be run with each header line as it is received (including the\n           first line, e.g. ``HTTP/1.0 200 OK\\r\\n``, and a final line\n           containing only ``\\r\\n``.  All lines include the trailing newline\n           characters).  ``HTTPResponse.headers`` will be empty in the final\n           response.  This is most useful in conjunction with\n           ``streaming_callback``, because it's the only way to get access to\n           header data while the request is in progress.\n        :arg collections.abc.Callable prepare_curl_callback: If set, will be called with\n           a ``pycurl.Curl`` object to allow the application to make additional\n           ``setopt`` calls.\n        :arg str proxy_host: HTTP proxy hostname.  To use proxies,\n           ``proxy_host`` and ``proxy_port`` must be set; ``proxy_username``,\n           ``proxy_pass`` and ``proxy_auth_mode`` are optional.  Proxies are\n           currently only supported with ``curl_httpclient``.\n        :arg int proxy_port: HTTP proxy port\n        :arg str proxy_username: HTTP proxy username\n        :arg str proxy_password: HTTP proxy password\n        :arg str proxy_auth_mode: HTTP proxy Authentication mode;\n           default is \"basic\". supports \"basic\" and \"digest\"\n        :arg bool allow_nonstandard_methods: Allow unknown values for ``method``\n           argument? Default is False.\n        :arg bool validate_cert: For HTTPS requests, validate the server's\n           certificate? Default is True.\n        :arg str ca_certs: filename of CA certificates in PEM format,\n           or None to use defaults.  See note below when used with\n           ``curl_httpclient``.\n        :arg str client_key: Filename for client SSL key, if any.  See\n           note below when used with ``curl_httpclient``.\n        :arg str client_cert: Filename for client SSL certificate, if any.\n           See note below when used with ``curl_httpclient``.\n        :arg ssl.SSLContext ssl_options: `ssl.SSLContext` object for use in\n           ``simple_httpclient`` (unsupported by ``curl_httpclient``).\n           Overrides ``validate_cert``, ``ca_certs``, ``client_key``,\n           and ``client_cert``.\n        :arg bool allow_ipv6: Use IPv6 when available?  Default is true.\n        :arg bool expect_100_continue: If true, send the\n           ``Expect: 100-continue`` header and wait for a continue response\n           before sending the request body.  Only supported with\n           simple_httpclient.\n\n        .. note::\n\n            When using ``curl_httpclient`` certain options may be\n            inherited by subsequent fetches because ``pycurl`` does\n            not allow them to be cleanly reset.  This applies to the\n            ``ca_certs``, ``client_key``, ``client_cert``, and\n            ``network_interface`` arguments.  If you use these\n            options, you should pass them on every request (you don't\n            have to always use the same values, but it's not possible\n            to mix requests that specify these options with ones that\n            use the defaults).\n\n        .. versionadded:: 3.1\n           The ``auth_mode`` argument.\n\n        .. versionadded:: 4.0\n           The ``body_producer`` and ``expect_100_continue`` arguments.\n\n        .. versionadded:: 4.2\n           The ``ssl_options`` argument.\n\n        .. versionadded:: 4.5\n           The ``proxy_auth_mode`` argument.\n        \"\"\"\n        # Note that some of these attributes go through property setters\n        # defined below.\n        self.headers = headers\n        if if_modified_since:\n            self.headers[\"If-Modified-Since\"] = httputil.format_timestamp(\n                if_modified_since\n            )\n        self.proxy_host = proxy_host\n        self.proxy_port = proxy_port\n        self.proxy_username = proxy_username\n        self.proxy_password = proxy_password\n        self.proxy_auth_mode = proxy_auth_mode\n        self.url = url\n        self.method = method\n        self.body = body\n        self.body_producer = body_producer\n        self.auth_username = auth_username\n        self.auth_password = auth_password\n        self.auth_mode = auth_mode\n        self.connect_timeout = connect_timeout\n        self.request_timeout = request_timeout\n        self.follow_redirects = follow_redirects\n        self.max_redirects = max_redirects\n        self.user_agent = user_agent\n        if decompress_response is not None:\n            self.decompress_response = decompress_response  # type: Optional[bool]\n        else:\n            self.decompress_response = use_gzip\n        self.network_interface = network_interface\n        self.streaming_callback = streaming_callback\n        self.header_callback = header_callback\n        self.prepare_curl_callback = prepare_curl_callback\n        self.allow_nonstandard_methods = allow_nonstandard_methods\n        self.validate_cert = validate_cert\n        self.ca_certs = ca_certs\n        self.allow_ipv6 = allow_ipv6\n        self.client_key = client_key\n        self.client_cert = client_cert\n        self.ssl_options = ssl_options\n        self.expect_100_continue = expect_100_continue\n        self.start_time = time.time()\n\n    @property\n    def headers(self) -> httputil.HTTPHeaders:\n        # TODO: headers may actually be a plain dict until fairly late in\n        # the process (AsyncHTTPClient.fetch), but practically speaking,\n        # whenever the property is used they're already HTTPHeaders.\n        return self._headers  # type: ignore\n\n    @headers.setter\n    def headers(self, value: Union[Dict[str, str], httputil.HTTPHeaders]) -> None:\n        if value is None:\n            self._headers = httputil.HTTPHeaders()\n        else:\n            self._headers = value  # type: ignore\n\n    @property\n    def body(self) -> bytes:\n        return self._body\n\n    @body.setter\n    def body(self, value: Union[bytes, str]) -> None:\n        self._body = utf8(value)",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 330,
                                "end_lineno": 558,
                                "parent": null
                            }
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": "from tornado.escape import _unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPResponse,\n    HTTPError,\n    AsyncHTTPClient,\n    main,\n    _RequestProxy,\n    HTTPRequest,\n)\nfrom tornado import httputil\nfrom tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import StreamClosedError, IOStream\nfrom tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults\nfrom tornado.log import gen_log\nfrom tornado.tcpclient import TCPClient\n\nimport base64\nimport collections\nimport copy\nimport functools\nimport re\nimport socket\nimport ssl\nimport sys\nimport time\nfrom io import BytesIO\nimport urllib.parse\n\nfrom typing import Dict, Any, Callable, Optional, Type, Union\nfrom types import TracebackType\nimport typing\n\nif typing.TYPE_CHECKING:\n    from typing import Deque, Tuple, List  # noqa: F401\n\n\nclass HTTPTimeoutError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient on timeout.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPTimeoutError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Timeout\"\n\n\nclass HTTPStreamClosedError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.\n\n    When a more specific exception is available (such as `ConnectionResetError`),\n    it may be raised instead of this one.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPStreamClosedError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Stream closed\"\n\n\nclass SimpleAsyncHTTPClient(AsyncHTTPClient):\n    \"\"\"Non-blocking HTTP client with no external dependencies.\n\n    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.\n    Some features found in the curl-based AsyncHTTPClient are not yet\n    supported.  In particular, proxies are not supported, connections\n    are not reused, and callers cannot select the network interface to be\n    used.\n    \"\"\"\n\n    def initialize(  # type: ignore\n        self,\n        max_clients: int = 10,\n        hostname_mapping: Dict[str, str] = None,\n        max_buffer_size: int = 104857600,\n        resolver: Resolver = None,\n        defaults: Dict[str, Any] = None,\n        max_header_size: int = None,\n        max_body_size: int = None,\n    ) -> None:\n        \"\"\"Creates a AsyncHTTPClient.\n\n        Only a single AsyncHTTPClient instance exists per IOLoop\n        in order to provide limitations on the number of pending connections.\n        ``force_instance=True`` may be used to suppress this behavior.\n\n        Note that because of this implicit reuse, unless ``force_instance``\n        is used, only the first call to the constructor actually uses\n        its arguments. It is recommended to use the ``configure`` method\n        instead of the constructor to ensure that arguments take effect.\n\n        ``max_clients`` is the number of concurrent requests that can be\n        in progress; when this limit is reached additional requests will be\n        queued. Note that time spent waiting in this queue still counts\n        against the ``request_timeout``.\n\n        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.\n        It can be used to make local DNS changes when modifying system-wide\n        settings like ``/etc/hosts`` is not possible or desirable (e.g. in\n        unittests).\n\n        ``max_buffer_size`` (default 100MB) is the number of bytes\n        that can be read into memory at once. ``max_body_size``\n        (defaults to ``max_buffer_size``) is the largest response body\n        that the client will accept.  Without a\n        ``streaming_callback``, the smaller of these two limits\n        applies; with a ``streaming_callback`` only ``max_body_size``\n        does.\n\n        .. versionchanged:: 4.2\n           Added the ``max_body_size`` argument.\n        \"\"\"\n        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)\n        self.max_clients = max_clients\n        self.queue = (\n            collections.deque()\n        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.active = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.waiting = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]\n        self.max_buffer_size = max_buffer_size\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        # TCPClient could create a Resolver for us, but we have to do it\n        # ourselves to support hostname_mapping.\n        if resolver:\n            self.resolver = resolver\n            self.own_resolver = False\n        else:\n            self.resolver = Resolver()\n            self.own_resolver = True\n        if hostname_mapping is not None:\n            self.resolver = OverrideResolver(\n                resolver=self.resolver, mapping=hostname_mapping\n            )\n        self.tcp_client = TCPClient(resolver=self.resolver)\n\n    def close(self) -> None:\n        super(SimpleAsyncHTTPClient, self).close()\n        if self.own_resolver:\n            self.resolver.close()\n        self.tcp_client.close()\n\n    def fetch_impl(\n        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]\n    ) -> None:\n        key = object()\n        self.queue.append((key, request, callback))\n        if not len(self.active) < self.max_clients:\n            assert request.connect_timeout is not None\n            assert request.request_timeout is not None\n            timeout_handle = self.io_loop.add_timeout(\n                self.io_loop.time()\n                + min(request.connect_timeout, request.request_timeout),\n                functools.partial(self._on_timeout, key, \"in request queue\"),\n            )\n        else:\n            timeout_handle = None\n        self.waiting[key] = (request, callback, timeout_handle)\n        self._process_queue()\n        if self.queue:\n            gen_log.debug(\n                \"max_clients limit reached, request queued. \"\n                \"%d active, %d queued requests.\" % (len(self.active), len(self.queue))\n            )\n\n    def _process_queue(self) -> None:\n        while self.queue and len(self.active) < self.max_clients:\n            key, request, callback = self.queue.popleft()\n            if key not in self.waiting:\n                continue\n            self._remove_timeout(key)\n            self.active[key] = (request, callback)\n            release_callback = functools.partial(self._release_fetch, key)\n            self._handle_request(request, release_callback, callback)\n\n    def _connection_class(self) -> type:\n        return _HTTPConnection\n\n    def _handle_request(\n        self,\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n    ) -> None:\n        self._connection_class()(\n            self,\n            request,\n            release_callback,\n            final_callback,\n            self.max_buffer_size,\n            self.tcp_client,\n            self.max_header_size,\n            self.max_body_size,\n        )\n\n    def _release_fetch(self, key: object) -> None:\n        del self.active[key]\n        self._process_queue()\n\n    def _remove_timeout(self, key: object) -> None:\n        if key in self.waiting:\n            request, callback, timeout_handle = self.waiting[key]\n            if timeout_handle is not None:\n                self.io_loop.remove_timeout(timeout_handle)\n            del self.waiting[key]\n\n    def _on_timeout(self, key: object, info: str = None) -> None:\n        \"\"\"Timeout callback of request.\n\n        Construct a timeout HTTPResponse when a timeout occurs.\n\n        :arg object key: A simple object to mark the request.\n        :info string key: More detailed timeout information.\n        \"\"\"\n        request, callback, timeout_handle = self.waiting[key]\n        self.queue.remove((key, request, callback))\n\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        timeout_response = HTTPResponse(\n            request,\n            599,\n            error=HTTPTimeoutError(error_message),\n            request_time=self.io_loop.time() - request.start_time,\n        )\n        self.io_loop.add_callback(callback, timeout_response)\n        del self.waiting[key]\n\n\nclass _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"network_interface\",\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)\n\n\nif __name__ == \"__main__\":\n    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)\n    main()\n",
            "new_file_content": "from tornado.escape import _unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPResponse,\n    HTTPError,\n    AsyncHTTPClient,\n    main,\n    _RequestProxy,\n    HTTPRequest,\n)\nfrom tornado import httputil\nfrom tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import StreamClosedError, IOStream\nfrom tornado.netutil import (\n    Resolver,\n    OverrideResolver,\n    _client_ssl_defaults,\n    is_valid_ip,\n)\nfrom tornado.log import gen_log\nfrom tornado.tcpclient import TCPClient\n\nimport base64\nimport collections\nimport copy\nimport functools\nimport re\nimport socket\nimport ssl\nimport sys\nimport time\nfrom io import BytesIO\nimport urllib.parse\n\nfrom typing import Dict, Any, Callable, Optional, Type, Union\nfrom types import TracebackType\nimport typing\n\nif typing.TYPE_CHECKING:\n    from typing import Deque, Tuple, List  # noqa: F401\n\n\nclass HTTPTimeoutError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient on timeout.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPTimeoutError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Timeout\"\n\n\nclass HTTPStreamClosedError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.\n\n    When a more specific exception is available (such as `ConnectionResetError`),\n    it may be raised instead of this one.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPStreamClosedError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Stream closed\"\n\n\nclass SimpleAsyncHTTPClient(AsyncHTTPClient):\n    \"\"\"Non-blocking HTTP client with no external dependencies.\n\n    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.\n    Some features found in the curl-based AsyncHTTPClient are not yet\n    supported.  In particular, proxies are not supported, connections\n    are not reused, and callers cannot select the network interface to be\n    used.\n    \"\"\"\n\n    def initialize(  # type: ignore\n        self,\n        max_clients: int = 10,\n        hostname_mapping: Dict[str, str] = None,\n        max_buffer_size: int = 104857600,\n        resolver: Resolver = None,\n        defaults: Dict[str, Any] = None,\n        max_header_size: int = None,\n        max_body_size: int = None,\n    ) -> None:\n        \"\"\"Creates a AsyncHTTPClient.\n\n        Only a single AsyncHTTPClient instance exists per IOLoop\n        in order to provide limitations on the number of pending connections.\n        ``force_instance=True`` may be used to suppress this behavior.\n\n        Note that because of this implicit reuse, unless ``force_instance``\n        is used, only the first call to the constructor actually uses\n        its arguments. It is recommended to use the ``configure`` method\n        instead of the constructor to ensure that arguments take effect.\n\n        ``max_clients`` is the number of concurrent requests that can be\n        in progress; when this limit is reached additional requests will be\n        queued. Note that time spent waiting in this queue still counts\n        against the ``request_timeout``.\n\n        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.\n        It can be used to make local DNS changes when modifying system-wide\n        settings like ``/etc/hosts`` is not possible or desirable (e.g. in\n        unittests).\n\n        ``max_buffer_size`` (default 100MB) is the number of bytes\n        that can be read into memory at once. ``max_body_size``\n        (defaults to ``max_buffer_size``) is the largest response body\n        that the client will accept.  Without a\n        ``streaming_callback``, the smaller of these two limits\n        applies; with a ``streaming_callback`` only ``max_body_size``\n        does.\n\n        .. versionchanged:: 4.2\n           Added the ``max_body_size`` argument.\n        \"\"\"\n        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)\n        self.max_clients = max_clients\n        self.queue = (\n            collections.deque()\n        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.active = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.waiting = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]\n        self.max_buffer_size = max_buffer_size\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        # TCPClient could create a Resolver for us, but we have to do it\n        # ourselves to support hostname_mapping.\n        if resolver:\n            self.resolver = resolver\n            self.own_resolver = False\n        else:\n            self.resolver = Resolver()\n            self.own_resolver = True\n        if hostname_mapping is not None:\n            self.resolver = OverrideResolver(\n                resolver=self.resolver, mapping=hostname_mapping\n            )\n        self.tcp_client = TCPClient(resolver=self.resolver)\n\n    def close(self) -> None:\n        super(SimpleAsyncHTTPClient, self).close()\n        if self.own_resolver:\n            self.resolver.close()\n        self.tcp_client.close()\n\n    def fetch_impl(\n        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]\n    ) -> None:\n        key = object()\n        self.queue.append((key, request, callback))\n        if not len(self.active) < self.max_clients:\n            assert request.connect_timeout is not None\n            assert request.request_timeout is not None\n            timeout_handle = self.io_loop.add_timeout(\n                self.io_loop.time()\n                + min(request.connect_timeout, request.request_timeout),\n                functools.partial(self._on_timeout, key, \"in request queue\"),\n            )\n        else:\n            timeout_handle = None\n        self.waiting[key] = (request, callback, timeout_handle)\n        self._process_queue()\n        if self.queue:\n            gen_log.debug(\n                \"max_clients limit reached, request queued. \"\n                \"%d active, %d queued requests.\" % (len(self.active), len(self.queue))\n            )\n\n    def _process_queue(self) -> None:\n        while self.queue and len(self.active) < self.max_clients:\n            key, request, callback = self.queue.popleft()\n            if key not in self.waiting:\n                continue\n            self._remove_timeout(key)\n            self.active[key] = (request, callback)\n            release_callback = functools.partial(self._release_fetch, key)\n            self._handle_request(request, release_callback, callback)\n\n    def _connection_class(self) -> type:\n        return _HTTPConnection\n\n    def _handle_request(\n        self,\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n    ) -> None:\n        self._connection_class()(\n            self,\n            request,\n            release_callback,\n            final_callback,\n            self.max_buffer_size,\n            self.tcp_client,\n            self.max_header_size,\n            self.max_body_size,\n        )\n\n    def _release_fetch(self, key: object) -> None:\n        del self.active[key]\n        self._process_queue()\n\n    def _remove_timeout(self, key: object) -> None:\n        if key in self.waiting:\n            request, callback, timeout_handle = self.waiting[key]\n            if timeout_handle is not None:\n                self.io_loop.remove_timeout(timeout_handle)\n            del self.waiting[key]\n\n    def _on_timeout(self, key: object, info: str = None) -> None:\n        \"\"\"Timeout callback of request.\n\n        Construct a timeout HTTPResponse when a timeout occurs.\n\n        :arg object key: A simple object to mark the request.\n        :info string key: More detailed timeout information.\n        \"\"\"\n        request, callback, timeout_handle = self.waiting[key]\n        self.queue.remove((key, request, callback))\n\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        timeout_response = HTTPResponse(\n            request,\n            599,\n            error=HTTPTimeoutError(error_message),\n            request_time=self.io_loop.time() - request.start_time,\n        )\n        self.io_loop.add_callback(callback, timeout_response)\n        del self.waiting[key]\n\n\nclass _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)\n\n\nif __name__ == \"__main__\":\n    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)\n    main()\n",
            "header": {
                "file": {
                    "path": "tornado/simple_httpclient.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "d6ac20f9",
                "new_commit_hash": "81a0fd24",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/tornado/simple_httpclient.py"
            },
            "plus_file": {
                "path": "b/tornado/simple_httpclient.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 12,
                            "length": 7
                        },
                        "new_range": {
                            "start": 12,
                            "length": 12
                        },
                        "section": "from tornado import httputil"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "from tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters",
                                "type": "context"
                            },
                            {
                                "content": "from tornado.ioloop import IOLoop",
                                "type": "context"
                            },
                            {
                                "content": "from tornado.iostream import StreamClosedError, IOStream",
                                "type": "context"
                            },
                            {
                                "content": "from tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults",
                                "type": "deleted"
                            },
                            {
                                "content": "from tornado.netutil import (",
                                "type": "added"
                            },
                            {
                                "content": "    Resolver,",
                                "type": "added"
                            },
                            {
                                "content": "    OverrideResolver,",
                                "type": "added"
                            },
                            {
                                "content": "    _client_ssl_defaults,",
                                "type": "added"
                            },
                            {
                                "content": "    is_valid_ip,",
                                "type": "added"
                            },
                            {
                                "content": ")",
                                "type": "added"
                            },
                            {
                                "content": "from tornado.log import gen_log",
                                "type": "context"
                            },
                            {
                                "content": "from tornado.tcpclient import TCPClient",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [],
                    "added_entities": [
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "statement",
                            "name": "",
                            "content": "from tornado.netutil import (\n    Resolver,\n    OverrideResolver,\n    _client_ssl_defaults,\n    is_valid_ip,\n)",
                            "ast_type_str": "ImportFrom",
                            "start_lineno": 15,
                            "end_lineno": 20,
                            "parent": null
                        }
                    ],
                    "deleted_entities": [
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "statement",
                            "name": "",
                            "content": "from tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults",
                            "ast_type_str": "ImportFrom",
                            "start_lineno": 15,
                            "end_lineno": 15,
                            "parent": null
                        }
                    ]
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 306,
                            "length": 6
                        },
                        "new_range": {
                            "start": 311,
                            "length": 16
                        },
                        "section": "class _HTTPConnection(httputil.HTTPMessageDelegate):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "            ssl_options = self._get_ssl_options(self.parsed.scheme)",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "            source_ip = None",
                                "type": "added"
                            },
                            {
                                "content": "            if self.request.network_interface:",
                                "type": "added"
                            },
                            {
                                "content": "                if is_valid_ip(self.request.network_interface):",
                                "type": "added"
                            },
                            {
                                "content": "                    source_ip = self.request.network_interface",
                                "type": "added"
                            },
                            {
                                "content": "                else:",
                                "type": "added"
                            },
                            {
                                "content": "                    raise ValueError(",
                                "type": "added"
                            },
                            {
                                "content": "                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"",
                                "type": "added"
                            },
                            {
                                "content": "                        % (self.request.network_interface,)",
                                "type": "added"
                            },
                            {
                                "content": "                    )",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "            timeout = min(self.request.connect_timeout, self.request.request_timeout)",
                                "type": "context"
                            },
                            {
                                "content": "            if timeout:",
                                "type": "context"
                            },
                            {
                                "content": "                self._timeout = self.io_loop.add_timeout(",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "class",
                            "name": "_HTTPConnection",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 251,
                            "end_lineno": 682,
                            "parent": null
                        },
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "method",
                            "name": "_HTTPConnection.run",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 289,
                            "end_lineno": 441,
                            "parent": {
                                "file_name": "tornado/simple_httpclient.py",
                                "type": "class",
                                "name": "_HTTPConnection",
                                "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 251,
                                "end_lineno": 682,
                                "parent": null
                            }
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 318,
                            "length": 6
                        },
                        "new_range": {
                            "start": 333,
                            "length": 7
                        },
                        "section": "class _HTTPConnection(httputil.HTTPMessageDelegate):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "                    af=af,",
                                "type": "context"
                            },
                            {
                                "content": "                    ssl_options=ssl_options,",
                                "type": "context"
                            },
                            {
                                "content": "                    max_buffer_size=self.max_buffer_size,",
                                "type": "context"
                            },
                            {
                                "content": "                    source_ip=source_ip,",
                                "type": "added"
                            },
                            {
                                "content": "                )",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "                if self.final_callback is None:",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "class",
                            "name": "_HTTPConnection",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 251,
                            "end_lineno": 682,
                            "parent": null
                        },
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "method",
                            "name": "_HTTPConnection.run",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 289,
                            "end_lineno": 441,
                            "parent": {
                                "file_name": "tornado/simple_httpclient.py",
                                "type": "class",
                                "name": "_HTTPConnection",
                                "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            source_ip = None\n            if self.request.network_interface:\n                if is_valid_ip(self.request.network_interface):\n                    source_ip = self.request.network_interface\n                else:\n                    raise ValueError(\n                        \"Unrecognized IPv4 or IPv6 address for network_interface, got %r\"\n                        % (self.request.network_interface,)\n                    )\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                    source_ip=source_ip,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 251,
                                "end_lineno": 682,
                                "parent": null
                            }
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 340,
                            "length": 7
                        },
                        "new_range": {
                            "start": 356,
                            "length": 6
                        },
                        "section": "class _HTTPConnection(httputil.HTTPMessageDelegate):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "                ):",
                                "type": "context"
                            },
                            {
                                "content": "                    raise KeyError(\"unknown method %s\" % self.request.method)",
                                "type": "context"
                            },
                            {
                                "content": "                for key in (",
                                "type": "context"
                            },
                            {
                                "content": "                    \"network_interface\",",
                                "type": "deleted"
                            },
                            {
                                "content": "                    \"proxy_host\",",
                                "type": "context"
                            },
                            {
                                "content": "                    \"proxy_port\",",
                                "type": "context"
                            },
                            {
                                "content": "                    \"proxy_username\",",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "class",
                            "name": "_HTTPConnection",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"network_interface\",\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 246,
                            "end_lineno": 667,
                            "parent": null
                        },
                        {
                            "file_name": "tornado/simple_httpclient.py",
                            "type": "method",
                            "name": "_HTTPConnection.run",
                            "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"network_interface\",\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 284,
                            "end_lineno": 426,
                            "parent": {
                                "file_name": "tornado/simple_httpclient.py",
                                "type": "class",
                                "name": "_HTTPConnection",
                                "content": "class _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"network_interface\",\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 246,
                                "end_lineno": 667,
                                "parent": null
                            }
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": "# -*- coding: utf-8 -*-\nimport base64\nimport binascii\nfrom contextlib import closing\nimport copy\nimport threading\nimport datetime\nfrom io import BytesIO\nimport time\nimport typing  # noqa: F401\nimport unicodedata\nimport unittest\n\nfrom tornado.escape import utf8, native_str\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPRequest,\n    HTTPResponse,\n    _RequestProxy,\n    HTTPError,\n    HTTPClient,\n)\nfrom tornado.httpserver import HTTPServer\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import IOStream\nfrom tornado.log import gen_log\nfrom tornado import netutil\nfrom tornado.testing import AsyncHTTPTestCase, bind_unused_port, gen_test, ExpectLog\nfrom tornado.test.util import skipOnTravis\nfrom tornado.web import Application, RequestHandler, url\nfrom tornado.httputil import format_timestamp, HTTPHeaders\n\n\nclass HelloWorldHandler(RequestHandler):\n    def get(self):\n        name = self.get_argument(\"name\", \"world\")\n        self.set_header(\"Content-Type\", \"text/plain\")\n        self.finish(\"Hello %s!\" % name)\n\n\nclass PostHandler(RequestHandler):\n    def post(self):\n        self.finish(\n            \"Post arg1: %s, arg2: %s\"\n            % (self.get_argument(\"arg1\"), self.get_argument(\"arg2\"))\n        )\n\n\nclass PutHandler(RequestHandler):\n    def put(self):\n        self.write(\"Put body: \")\n        self.write(self.request.body)\n\n\nclass RedirectHandler(RequestHandler):\n    def prepare(self):\n        self.write(\"redirects can have bodies too\")\n        self.redirect(\n            self.get_argument(\"url\"), status=int(self.get_argument(\"status\", \"302\"))\n        )\n\n\nclass ChunkHandler(RequestHandler):\n    @gen.coroutine\n    def get(self):\n        self.write(\"asdf\")\n        self.flush()\n        # Wait a bit to ensure the chunks are sent and received separately.\n        yield gen.sleep(0.01)\n        self.write(\"qwer\")\n\n\nclass AuthHandler(RequestHandler):\n    def get(self):\n        self.finish(self.request.headers[\"Authorization\"])\n\n\nclass CountdownHandler(RequestHandler):\n    def get(self, count):\n        count = int(count)\n        if count > 0:\n            self.redirect(self.reverse_url(\"countdown\", count - 1))\n        else:\n            self.write(\"Zero\")\n\n\nclass EchoPostHandler(RequestHandler):\n    def post(self):\n        self.write(self.request.body)\n\n\nclass UserAgentHandler(RequestHandler):\n    def get(self):\n        self.write(self.request.headers.get(\"User-Agent\", \"User agent not set\"))\n\n\nclass ContentLength304Handler(RequestHandler):\n    def get(self):\n        self.set_status(304)\n        self.set_header(\"Content-Length\", 42)\n\n    def _clear_headers_for_304(self):\n        # Tornado strips content-length from 304 responses, but here we\n        # want to simulate servers that include the headers anyway.\n        pass\n\n\nclass PatchHandler(RequestHandler):\n    def patch(self):\n        \"Return the request payload - so we can check it is being kept\"\n        self.write(self.request.body)\n\n\nclass AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = post = put = delete = options = patch = other = method  # type: ignore\n\n\nclass SetHeaderHandler(RequestHandler):\n    def get(self):\n        # Use get_arguments for keys to get strings, but\n        # request.arguments for values to get bytes.\n        for k, v in zip(self.get_arguments(\"k\"), self.request.arguments[\"v\"]):\n            self.set_header(k, v)\n\n\n# These tests end up getting run redundantly: once here with the default\n# HTTPClient implementation, and then again in each implementation's own\n# test suite.\n\n\nclass HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n\nclass RequestProxyTest(unittest.TestCase):\n    def test_request_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", user_agent=\"foo\"), dict()\n        )\n        self.assertEqual(proxy.user_agent, \"foo\")\n\n    def test_default_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\"), dict(network_interface=\"foo\")\n        )\n        self.assertEqual(proxy.network_interface, \"foo\")\n\n    def test_both_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", proxy_host=\"foo\"), dict(proxy_host=\"bar\")\n        )\n        self.assertEqual(proxy.proxy_host, \"foo\")\n\n    def test_neither_set(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        self.assertIs(proxy.auth_username, None)\n\n    def test_bad_attribute(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        with self.assertRaises(AttributeError):\n            proxy.foo\n\n    def test_defaults_none(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), None)\n        self.assertIs(proxy.auth_username, None)\n\n\nclass HTTPResponseTestCase(unittest.TestCase):\n    def test_str(self):\n        response = HTTPResponse(  # type: ignore\n            HTTPRequest(\"http://example.com\"), 200, headers={}, buffer=BytesIO()\n        )\n        s = str(response)\n        self.assertTrue(s.startswith(\"HTTPResponse(\"))\n        self.assertIn(\"code=200\", s)\n\n\nclass SyncHTTPClientTest(unittest.TestCase):\n    def setUp(self):\n        self.server_ioloop = IOLoop()\n        event = threading.Event()\n\n        @gen.coroutine\n        def init_server():\n            sock, self.port = bind_unused_port()\n            app = Application([(\"/\", HelloWorldHandler)])\n            self.server = HTTPServer(app)\n            self.server.add_socket(sock)\n            event.set()\n\n        def start():\n            self.server_ioloop.run_sync(init_server)\n            self.server_ioloop.start()\n\n        self.server_thread = threading.Thread(target=start)\n        self.server_thread.start()\n        event.wait()\n\n        self.http_client = HTTPClient()\n\n    def tearDown(self):\n        def stop_server():\n            self.server.stop()\n            # Delay the shutdown of the IOLoop by several iterations because\n            # the server may still have some cleanup work left when\n            # the client finishes with the response (this is noticeable\n            # with http/2, which leaves a Future with an unexamined\n            # StreamClosedError on the loop).\n\n            @gen.coroutine\n            def slow_stop():\n                # The number of iterations is difficult to predict. Typically,\n                # one is sufficient, although sometimes it needs more.\n                for i in range(5):\n                    yield\n                self.server_ioloop.stop()\n\n            self.server_ioloop.add_callback(slow_stop)\n\n        self.server_ioloop.add_callback(stop_server)\n        self.server_thread.join()\n        self.http_client.close()\n        self.server_ioloop.close(all_fds=True)\n\n    def get_url(self, path):\n        return \"http://127.0.0.1:%d%s\" % (self.port, path)\n\n    def test_sync_client(self):\n        response = self.http_client.fetch(self.get_url(\"/\"))\n        self.assertEqual(b\"Hello world!\", response.body)\n\n    def test_sync_client_error(self):\n        # Synchronous HTTPClient raises errors directly; no need for\n        # response.rethrow()\n        with self.assertRaises(HTTPError) as assertion:\n            self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(assertion.exception.code, 404)\n\n\nclass HTTPRequestTestCase(unittest.TestCase):\n    def test_headers(self):\n        request = HTTPRequest(\"http://example.com\", headers={\"foo\": \"bar\"})\n        self.assertEqual(request.headers, {\"foo\": \"bar\"})\n\n    def test_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = {\"bar\": \"baz\"}  # type: ignore\n        self.assertEqual(request.headers, {\"bar\": \"baz\"})\n\n    def test_null_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = None  # type: ignore\n        self.assertEqual(request.headers, {})\n\n    def test_body(self):\n        request = HTTPRequest(\"http://example.com\", body=\"foo\")\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_body_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.body = \"foo\"  # type: ignore\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_if_modified_since(self):\n        http_date = datetime.datetime.utcnow()\n        request = HTTPRequest(\"http://example.com\", if_modified_since=http_date)\n        self.assertEqual(\n            request.headers, {\"If-Modified-Since\": format_timestamp(http_date)}\n        )\n\n\nclass HTTPErrorTestCase(unittest.TestCase):\n    def test_copy(self):\n        e = HTTPError(403)\n        e2 = copy.copy(e)\n        self.assertIsNot(e, e2)\n        self.assertEqual(e.code, e2.code)\n\n    def test_plain_error(self):\n        e = HTTPError(403)\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n\n    def test_error_with_response(self):\n        resp = HTTPResponse(HTTPRequest(\"http://example.com/\"), 403)\n        with self.assertRaises(HTTPError) as cm:\n            resp.rethrow()\n        e = cm.exception\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n",
            "new_file_content": "# -*- coding: utf-8 -*-\nimport base64\nimport binascii\nfrom contextlib import closing\nimport copy\nimport threading\nimport datetime\nfrom io import BytesIO\nimport time\nimport typing  # noqa: F401\nimport unicodedata\nimport unittest\n\nfrom tornado.escape import utf8, native_str\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPRequest,\n    HTTPResponse,\n    _RequestProxy,\n    HTTPError,\n    HTTPClient,\n)\nfrom tornado.httpserver import HTTPServer\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import IOStream\nfrom tornado.log import gen_log\nfrom tornado import netutil\nfrom tornado.testing import AsyncHTTPTestCase, bind_unused_port, gen_test, ExpectLog\nfrom tornado.test.util import skipOnTravis\nfrom tornado.web import Application, RequestHandler, url\nfrom tornado.httputil import format_timestamp, HTTPHeaders\n\n\nclass HelloWorldHandler(RequestHandler):\n    def get(self):\n        name = self.get_argument(\"name\", \"world\")\n        self.set_header(\"Content-Type\", \"text/plain\")\n        self.finish(\"Hello %s!\" % name)\n\n\nclass PostHandler(RequestHandler):\n    def post(self):\n        self.finish(\n            \"Post arg1: %s, arg2: %s\"\n            % (self.get_argument(\"arg1\"), self.get_argument(\"arg2\"))\n        )\n\n\nclass PutHandler(RequestHandler):\n    def put(self):\n        self.write(\"Put body: \")\n        self.write(self.request.body)\n\n\nclass RedirectHandler(RequestHandler):\n    def prepare(self):\n        self.write(\"redirects can have bodies too\")\n        self.redirect(\n            self.get_argument(\"url\"), status=int(self.get_argument(\"status\", \"302\"))\n        )\n\n\nclass ChunkHandler(RequestHandler):\n    @gen.coroutine\n    def get(self):\n        self.write(\"asdf\")\n        self.flush()\n        # Wait a bit to ensure the chunks are sent and received separately.\n        yield gen.sleep(0.01)\n        self.write(\"qwer\")\n\n\nclass AuthHandler(RequestHandler):\n    def get(self):\n        self.finish(self.request.headers[\"Authorization\"])\n\n\nclass CountdownHandler(RequestHandler):\n    def get(self, count):\n        count = int(count)\n        if count > 0:\n            self.redirect(self.reverse_url(\"countdown\", count - 1))\n        else:\n            self.write(\"Zero\")\n\n\nclass EchoPostHandler(RequestHandler):\n    def post(self):\n        self.write(self.request.body)\n\n\nclass UserAgentHandler(RequestHandler):\n    def get(self):\n        self.write(self.request.headers.get(\"User-Agent\", \"User agent not set\"))\n\n\nclass ContentLength304Handler(RequestHandler):\n    def get(self):\n        self.set_status(304)\n        self.set_header(\"Content-Length\", 42)\n\n    def _clear_headers_for_304(self):\n        # Tornado strips content-length from 304 responses, but here we\n        # want to simulate servers that include the headers anyway.\n        pass\n\n\nclass PatchHandler(RequestHandler):\n    def patch(self):\n        \"Return the request payload - so we can check it is being kept\"\n        self.write(self.request.body)\n\n\nclass AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = post = put = delete = options = patch = other = method  # type: ignore\n\n\nclass SetHeaderHandler(RequestHandler):\n    def get(self):\n        # Use get_arguments for keys to get strings, but\n        # request.arguments for values to get bytes.\n        for k, v in zip(self.get_arguments(\"k\"), self.request.arguments[\"v\"]):\n            self.set_header(k, v)\n\n\n# These tests end up getting run redundantly: once here with the default\n# HTTPClient implementation, and then again in each implementation's own\n# test suite.\n\n\nclass HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n\nclass RequestProxyTest(unittest.TestCase):\n    def test_request_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", user_agent=\"foo\"), dict()\n        )\n        self.assertEqual(proxy.user_agent, \"foo\")\n\n    def test_default_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\"), dict(network_interface=\"foo\")\n        )\n        self.assertEqual(proxy.network_interface, \"foo\")\n\n    def test_both_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", proxy_host=\"foo\"), dict(proxy_host=\"bar\")\n        )\n        self.assertEqual(proxy.proxy_host, \"foo\")\n\n    def test_neither_set(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        self.assertIs(proxy.auth_username, None)\n\n    def test_bad_attribute(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        with self.assertRaises(AttributeError):\n            proxy.foo\n\n    def test_defaults_none(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), None)\n        self.assertIs(proxy.auth_username, None)\n\n\nclass HTTPResponseTestCase(unittest.TestCase):\n    def test_str(self):\n        response = HTTPResponse(  # type: ignore\n            HTTPRequest(\"http://example.com\"), 200, headers={}, buffer=BytesIO()\n        )\n        s = str(response)\n        self.assertTrue(s.startswith(\"HTTPResponse(\"))\n        self.assertIn(\"code=200\", s)\n\n\nclass SyncHTTPClientTest(unittest.TestCase):\n    def setUp(self):\n        self.server_ioloop = IOLoop()\n        event = threading.Event()\n\n        @gen.coroutine\n        def init_server():\n            sock, self.port = bind_unused_port()\n            app = Application([(\"/\", HelloWorldHandler)])\n            self.server = HTTPServer(app)\n            self.server.add_socket(sock)\n            event.set()\n\n        def start():\n            self.server_ioloop.run_sync(init_server)\n            self.server_ioloop.start()\n\n        self.server_thread = threading.Thread(target=start)\n        self.server_thread.start()\n        event.wait()\n\n        self.http_client = HTTPClient()\n\n    def tearDown(self):\n        def stop_server():\n            self.server.stop()\n            # Delay the shutdown of the IOLoop by several iterations because\n            # the server may still have some cleanup work left when\n            # the client finishes with the response (this is noticeable\n            # with http/2, which leaves a Future with an unexamined\n            # StreamClosedError on the loop).\n\n            @gen.coroutine\n            def slow_stop():\n                # The number of iterations is difficult to predict. Typically,\n                # one is sufficient, although sometimes it needs more.\n                for i in range(5):\n                    yield\n                self.server_ioloop.stop()\n\n            self.server_ioloop.add_callback(slow_stop)\n\n        self.server_ioloop.add_callback(stop_server)\n        self.server_thread.join()\n        self.http_client.close()\n        self.server_ioloop.close(all_fds=True)\n\n    def get_url(self, path):\n        return \"http://127.0.0.1:%d%s\" % (self.port, path)\n\n    def test_sync_client(self):\n        response = self.http_client.fetch(self.get_url(\"/\"))\n        self.assertEqual(b\"Hello world!\", response.body)\n\n    def test_sync_client_error(self):\n        # Synchronous HTTPClient raises errors directly; no need for\n        # response.rethrow()\n        with self.assertRaises(HTTPError) as assertion:\n            self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(assertion.exception.code, 404)\n\n\nclass HTTPRequestTestCase(unittest.TestCase):\n    def test_headers(self):\n        request = HTTPRequest(\"http://example.com\", headers={\"foo\": \"bar\"})\n        self.assertEqual(request.headers, {\"foo\": \"bar\"})\n\n    def test_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = {\"bar\": \"baz\"}  # type: ignore\n        self.assertEqual(request.headers, {\"bar\": \"baz\"})\n\n    def test_null_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = None  # type: ignore\n        self.assertEqual(request.headers, {})\n\n    def test_body(self):\n        request = HTTPRequest(\"http://example.com\", body=\"foo\")\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_body_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.body = \"foo\"  # type: ignore\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_if_modified_since(self):\n        http_date = datetime.datetime.utcnow()\n        request = HTTPRequest(\"http://example.com\", if_modified_since=http_date)\n        self.assertEqual(\n            request.headers, {\"If-Modified-Since\": format_timestamp(http_date)}\n        )\n\n\nclass HTTPErrorTestCase(unittest.TestCase):\n    def test_copy(self):\n        e = HTTPError(403)\n        e2 = copy.copy(e)\n        self.assertIsNot(e, e2)\n        self.assertEqual(e.code, e2.code)\n\n    def test_plain_error(self):\n        e = HTTPError(403)\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n\n    def test_error_with_response(self):\n        resp = HTTPResponse(HTTPRequest(\"http://example.com/\"), 403)\n        with self.assertRaises(HTTPError) as cm:\n            resp.rethrow()\n        e = cm.exception\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n",
            "header": {
                "file": {
                    "path": "tornado/test/httpclient_test.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "c30c1b56",
                "new_commit_hash": "0a732ddc",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/tornado/test/httpclient_test.py"
            },
            "plus_file": {
                "path": "b/tornado/test/httpclient_test.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 467,
                            "length": 6
                        },
                        "new_range": {
                            "start": 467,
                            "length": 18
                        },
                        "section": "X-XSS-Protection: 1;"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "        response2 = yield self.http_client.fetch(response.request)",
                                "type": "context"
                            },
                            {
                                "content": "        self.assertEqual(response2.body, b\"Hello world!\")",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "    @gen_test",
                                "type": "added"
                            },
                            {
                                "content": "    def test_bind_source_ip(self):",
                                "type": "added"
                            },
                            {
                                "content": "        url = self.get_url(\"/hello\")",
                                "type": "added"
                            },
                            {
                                "content": "        request = HTTPRequest(url, network_interface=\"127.0.0.1\")",
                                "type": "added"
                            },
                            {
                                "content": "        response = yield self.http_client.fetch(request)",
                                "type": "added"
                            },
                            {
                                "content": "        self.assertEqual(response.code, 200)",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "        with self.assertRaises((ValueError, HTTPError)) as context:",
                                "type": "added"
                            },
                            {
                                "content": "            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")",
                                "type": "added"
                            },
                            {
                                "content": "            yield self.http_client.fetch(request)",
                                "type": "added"
                            },
                            {
                                "content": "        self.assertIn(\"not-interface-or-ip\", str(context.exception))",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "    def test_all_methods(self):",
                                "type": "context"
                            },
                            {
                                "content": "        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:",
                                "type": "context"
                            },
                            {
                                "content": "            response = self.fetch(\"/all_methods\", method=method)",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "tornado/test/httpclient_test.py",
                            "type": "class",
                            "name": "HTTPClientCommonTestCase",
                            "content": "class HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 136,
                            "end_lineno": 571,
                            "parent": null
                        }
                    ],
                    "added_entities": [
                        {
                            "file_name": "tornado/test/httpclient_test.py",
                            "type": "method",
                            "name": "HTTPClientCommonTestCase.test_bind_source_ip",
                            "content": "class HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))",
                            "ast_type_str": "ClassDef",
                            "start_lineno": 471,
                            "end_lineno": 480,
                            "parent": {
                                "file_name": "tornado/test/httpclient_test.py",
                                "type": "class",
                                "name": "HTTPClientCommonTestCase",
                                "content": "class HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))",
                                "ast_type_str": "ClassDef",
                                "start_lineno": 136,
                                "end_lineno": 571,
                                "parent": null
                            }
                        }
                    ],
                    "deleted_entities": []
                }
            ]
        }
    ],
    "old_commit_hash": "9ba61c636899ed0f7472fb2f4975a53720d1b0ff^",
    "new_commit_hash": "9ba61c636899ed0f7472fb2f4975a53720d1b0ff",
    "commit_message": "Add support for source IP address to SimpleAsyncHTTPClient.\n\nExtend support for HTTPRequest's network_interface parameter to\nSimpleAsyncHTTPClient. This enables binding to specific local IP and\nenables connecting to WebSockets via a specific IP address.\n\nTo use it you create a HTTPRequest with network_interface parameter set\nto local IP address to be used as source IP. websocket_connect then\npasses the request to SimpleAsyncHTTPClient.\n\nNote that even though the parameter name is `network_interface`, the\nCurlAsyncHTTPClient may support (OS dependant) passing in interface\nname, resolvable hostname or IP address, while this patch only adds\nIP address support to SimpleAsyncHTTPClient.",
    "commit_date": "2018-11-02T13:38:22+01:00",
    "metadata": {}
}
{
    "repo_name": "tornado",
    "new_commit_hash": "9ba61c636899ed0f7472fb2f4975a53720d1b0ff",
    "test_file_codes": [
        "# -*- coding: utf-8 -*-\nimport base64\nimport binascii\nfrom contextlib import closing\nimport copy\nimport threading\nimport datetime\nfrom io import BytesIO\nimport time\nimport typing  # noqa: F401\nimport unicodedata\nimport unittest\n\nfrom tornado.escape import utf8, native_str\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPRequest,\n    HTTPResponse,\n    _RequestProxy,\n    HTTPError,\n    HTTPClient,\n)\nfrom tornado.httpserver import HTTPServer\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import IOStream\nfrom tornado.log import gen_log\nfrom tornado import netutil\nfrom tornado.testing import AsyncHTTPTestCase, bind_unused_port, gen_test, ExpectLog\nfrom tornado.test.util import skipOnTravis\nfrom tornado.web import Application, RequestHandler, url\nfrom tornado.httputil import format_timestamp, HTTPHeaders\n\n\nclass HelloWorldHandler(RequestHandler):\n    def get(self):\n        name = self.get_argument(\"name\", \"world\")\n        self.set_header(\"Content-Type\", \"text/plain\")\n        self.finish(\"Hello %s!\" % name)\n\n\nclass PostHandler(RequestHandler):\n    def post(self):\n        self.finish(\n            \"Post arg1: %s, arg2: %s\"\n            % (self.get_argument(\"arg1\"), self.get_argument(\"arg2\"))\n        )\n\n\nclass PutHandler(RequestHandler):\n    def put(self):\n        self.write(\"Put body: \")\n        self.write(self.request.body)\n\n\nclass RedirectHandler(RequestHandler):\n    def prepare(self):\n        self.write(\"redirects can have bodies too\")\n        self.redirect(\n            self.get_argument(\"url\"), status=int(self.get_argument(\"status\", \"302\"))\n        )\n\n\nclass ChunkHandler(RequestHandler):\n    @gen.coroutine\n    def get(self):\n        self.write(\"asdf\")\n        self.flush()\n        # Wait a bit to ensure the chunks are sent and received separately.\n        yield gen.sleep(0.01)\n        self.write(\"qwer\")\n\n\nclass AuthHandler(RequestHandler):\n    def get(self):\n        self.finish(self.request.headers[\"Authorization\"])\n\n\nclass CountdownHandler(RequestHandler):\n    def get(self, count):\n        count = int(count)\n        if count > 0:\n            self.redirect(self.reverse_url(\"countdown\", count - 1))\n        else:\n            self.write(\"Zero\")\n\n\nclass EchoPostHandler(RequestHandler):\n    def post(self):\n        self.write(self.request.body)\n\n\nclass UserAgentHandler(RequestHandler):\n    def get(self):\n        self.write(self.request.headers.get(\"User-Agent\", \"User agent not set\"))\n\n\nclass ContentLength304Handler(RequestHandler):\n    def get(self):\n        self.set_status(304)\n        self.set_header(\"Content-Length\", 42)\n\n    def _clear_headers_for_304(self):\n        # Tornado strips content-length from 304 responses, but here we\n        # want to simulate servers that include the headers anyway.\n        pass\n\n\nclass PatchHandler(RequestHandler):\n    def patch(self):\n        \"Return the request payload - so we can check it is being kept\"\n        self.write(self.request.body)\n\n\nclass AllMethodsHandler(RequestHandler):\n    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + (\"OTHER\",)  # type: ignore\n\n    def method(self):\n        self.write(self.request.method)\n\n    get = post = put = delete = options = patch = other = method  # type: ignore\n\n\nclass SetHeaderHandler(RequestHandler):\n    def get(self):\n        # Use get_arguments for keys to get strings, but\n        # request.arguments for values to get bytes.\n        for k, v in zip(self.get_arguments(\"k\"), self.request.arguments[\"v\"]):\n            self.set_header(k, v)\n\n\n# These tests end up getting run redundantly: once here with the default\n# HTTPClient implementation, and then again in each implementation's own\n# test suite.\n\n\nclass HTTPClientCommonTestCase(AsyncHTTPTestCase):\n    def get_app(self):\n        return Application(\n            [\n                url(\"/hello\", HelloWorldHandler),\n                url(\"/post\", PostHandler),\n                url(\"/put\", PutHandler),\n                url(\"/redirect\", RedirectHandler),\n                url(\"/chunk\", ChunkHandler),\n                url(\"/auth\", AuthHandler),\n                url(\"/countdown/([0-9]+)\", CountdownHandler, name=\"countdown\"),\n                url(\"/echopost\", EchoPostHandler),\n                url(\"/user_agent\", UserAgentHandler),\n                url(\"/304_with_content_length\", ContentLength304Handler),\n                url(\"/all_methods\", AllMethodsHandler),\n                url(\"/patch\", PatchHandler),\n                url(\"/set_header\", SetHeaderHandler),\n            ],\n            gzip=True,\n        )\n\n    def test_patch_receives_payload(self):\n        body = b\"some patch data\"\n        response = self.fetch(\"/patch\", method=\"PATCH\", body=body)\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, body)\n\n    @skipOnTravis\n    def test_hello_world(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.headers[\"Content-Type\"], \"text/plain\")\n        self.assertEqual(response.body, b\"Hello world!\")\n        self.assertEqual(int(response.request_time), 0)\n\n        response = self.fetch(\"/hello?name=Ben\")\n        self.assertEqual(response.body, b\"Hello Ben!\")\n\n    def test_streaming_callback(self):\n        # streaming_callback is also tested in test_chunked\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/hello\", streaming_callback=chunks.append)\n        # with streaming_callback, data goes to the callback and not response.body\n        self.assertEqual(chunks, [b\"Hello world!\"])\n        self.assertFalse(response.body)\n\n    def test_post(self):\n        response = self.fetch(\"/post\", method=\"POST\", body=\"arg1=foo&arg2=bar\")\n        self.assertEqual(response.code, 200)\n        self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_chunked(self):\n        response = self.fetch(\"/chunk\")\n        self.assertEqual(response.body, b\"asdfqwer\")\n\n        chunks = []  # type: typing.List[bytes]\n        response = self.fetch(\"/chunk\", streaming_callback=chunks.append)\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n        self.assertFalse(response.body)\n\n    def test_chunked_close(self):\n        # test case in which chunks spread read-callback processing\n        # over several ioloop iterations, but the connection is already closed.\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                # fake an HTTP server using chunked encoding where the final chunks\n                # and connection close all happen at once\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nTransfer-Encoding: chunked\n\n1\n1\n1\n2\n0\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.body, b\"12\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_basic_auth(self):\n        # This test data appears in section 2 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\n                \"/auth\", auth_username=\"Aladdin\", auth_password=\"open sesame\"\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_explicit_mode(self):\n        self.assertEqual(\n            self.fetch(\n                \"/auth\",\n                auth_username=\"Aladdin\",\n                auth_password=\"open sesame\",\n                auth_mode=\"basic\",\n            ).body,\n            b\"Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==\",\n        )\n\n    def test_basic_auth_unicode(self):\n        # This test data appears in section 2.1 of RFC 7617.\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=\"test\", auth_password=\"123£\").body,\n            b\"Basic dGVzdDoxMjPCow==\",\n        )\n\n        # The standard mandates NFC. Give it a decomposed username\n        # and ensure it is normalized to composed form.\n        username = unicodedata.normalize(\"NFD\", u\"josé\")\n        self.assertEqual(\n            self.fetch(\"/auth\", auth_username=username, auth_password=\"səcrət\").body,\n            b\"Basic am9zw6k6c8mZY3LJmXQ=\",\n        )\n\n    def test_unsupported_auth_mode(self):\n        # curl and simple clients handle errors a bit differently; the\n        # important thing is that they don't fall back to basic auth\n        # on an unknown mode.\n        with ExpectLog(gen_log, \"uncaught exception\", required=False):\n            with self.assertRaises((ValueError, HTTPError)):\n                self.fetch(\n                    \"/auth\",\n                    auth_username=\"Aladdin\",\n                    auth_password=\"open sesame\",\n                    auth_mode=\"asdf\",\n                    raise_error=True,\n                )\n\n    def test_follow_redirect(self):\n        response = self.fetch(\"/countdown/2\", follow_redirects=False)\n        self.assertEqual(302, response.code)\n        self.assertTrue(response.headers[\"Location\"].endswith(\"/countdown/1\"))\n\n        response = self.fetch(\"/countdown/2\")\n        self.assertEqual(200, response.code)\n        self.assertTrue(response.effective_url.endswith(\"/countdown/0\"))\n        self.assertEqual(b\"Zero\", response.body)\n\n    def test_credentials_in_url(self):\n        url = self.get_url(\"/auth\").replace(\"http://\", \"http://me:secret@\")\n        response = self.fetch(url)\n        self.assertEqual(b\"Basic \" + base64.b64encode(b\"me:secret\"), response.body)\n\n    def test_body_encoding(self):\n        unicode_body = u\"\\xe9\"\n        byte_body = binascii.a2b_hex(b\"e9\")\n\n        # unicode string in body gets converted to utf8\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=unicode_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"2\")\n        self.assertEqual(response.body, utf8(unicode_body))\n\n        # byte strings pass through directly\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n        # Mixing unicode in headers and byte string bodies shouldn't\n        # break anything\n        response = self.fetch(\n            \"/echopost\",\n            method=\"POST\",\n            body=byte_body,\n            headers={\"Content-Type\": \"application/blah\"},\n            user_agent=u\"foo\",\n        )\n        self.assertEqual(response.headers[\"Content-Length\"], \"1\")\n        self.assertEqual(response.body, byte_body)\n\n    def test_types(self):\n        response = self.fetch(\"/hello\")\n        self.assertEqual(type(response.body), bytes)\n        self.assertEqual(type(response.headers[\"Content-Type\"]), str)\n        self.assertEqual(type(response.code), int)\n        self.assertEqual(type(response.effective_url), str)\n\n    def test_header_callback(self):\n        first_line = []\n        headers = {}\n        chunks = []\n\n        def header_callback(header_line):\n            if header_line.startswith(\"HTTP/1.1 101\"):\n                # Upgrading to HTTP/2\n                pass\n            elif header_line.startswith(\"HTTP/\"):\n                first_line.append(header_line)\n            elif header_line != \"\\r\\n\":\n                k, v = header_line.split(\":\", 1)\n                headers[k.lower()] = v.strip()\n\n        def streaming_callback(chunk):\n            # All header callbacks are run before any streaming callbacks,\n            # so the header data is available to process the data as it\n            # comes in.\n            self.assertEqual(headers[\"content-type\"], \"text/html; charset=UTF-8\")\n            chunks.append(chunk)\n\n        self.fetch(\n            \"/chunk\",\n            header_callback=header_callback,\n            streaming_callback=streaming_callback,\n        )\n        self.assertEqual(len(first_line), 1, first_line)\n        self.assertRegexpMatches(first_line[0], \"HTTP/[0-9]\\\\.[0-9] 200.*\\r\\n\")\n        self.assertEqual(chunks, [b\"asdf\", b\"qwer\"])\n\n    @gen_test\n    def test_configure_defaults(self):\n        defaults = dict(user_agent=\"TestDefaultUserAgent\", allow_ipv6=False)\n        # Construct a new instance of the configured client class\n        client = self.http_client.__class__(force_instance=True, defaults=defaults)\n        try:\n            response = yield client.fetch(self.get_url(\"/user_agent\"))\n            self.assertEqual(response.body, b\"TestDefaultUserAgent\")\n        finally:\n            client.close()\n\n    def test_header_types(self):\n        # Header values may be passed as character or utf8 byte strings,\n        # in a plain dictionary or an HTTPHeaders object.\n        # Keys must always be the native str type.\n        # All combinations should have the same results on the wire.\n        for value in [u\"MyUserAgent\", b\"MyUserAgent\"]:\n            for container in [dict, HTTPHeaders]:\n                headers = container()\n                headers[\"User-Agent\"] = value\n                resp = self.fetch(\"/user_agent\", headers=headers)\n                self.assertEqual(\n                    resp.body,\n                    b\"MyUserAgent\",\n                    \"response=%r, value=%r, container=%r\"\n                    % (resp.body, value, container),\n                )\n\n    def test_multi_line_headers(self):\n        # Multi-line http headers are rare but rfc-allowed\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2\n        sock, port = bind_unused_port()\n        with closing(sock):\n\n            @gen.coroutine\n            def accept_callback(conn, address):\n                stream = IOStream(conn)\n                request_data = yield stream.read_until(b\"\\r\\n\\r\\n\")\n                if b\"HTTP/1.\" not in request_data:\n                    self.skipTest(\"requires HTTP/1.x\")\n                yield stream.write(\n                    b\"\"\"\\\nHTTP/1.1 200 OK\nX-XSS-Protection: 1;\n\\tmode=block\n\n\"\"\".replace(\n                        b\"\\n\", b\"\\r\\n\"\n                    )\n                )\n                stream.close()\n\n            netutil.add_accept_handler(sock, accept_callback)  # type: ignore\n            resp = self.fetch(\"http://127.0.0.1:%d/\" % port)\n            resp.rethrow()\n            self.assertEqual(resp.headers[\"X-XSS-Protection\"], \"1; mode=block\")\n            self.io_loop.remove_handler(sock.fileno())\n\n    def test_304_with_content_length(self):\n        # According to the spec 304 responses SHOULD NOT include\n        # Content-Length or other entity headers, but some servers do it\n        # anyway.\n        # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5\n        response = self.fetch(\"/304_with_content_length\")\n        self.assertEqual(response.code, 304)\n        self.assertEqual(response.headers[\"Content-Length\"], \"42\")\n\n    @gen_test\n    def test_future_interface(self):\n        response = yield self.http_client.fetch(self.get_url(\"/hello\"))\n        self.assertEqual(response.body, b\"Hello world!\")\n\n    @gen_test\n    def test_future_http_error(self):\n        with self.assertRaises(HTTPError) as context:\n            yield self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(context.exception.code, 404)\n        self.assertEqual(context.exception.response.code, 404)\n\n    @gen_test\n    def test_future_http_error_no_raise(self):\n        response = yield self.http_client.fetch(\n            self.get_url(\"/notfound\"), raise_error=False\n        )\n        self.assertEqual(response.code, 404)\n\n    @gen_test\n    def test_reuse_request_from_response(self):\n        # The response.request attribute should be an HTTPRequest, not\n        # a _RequestProxy.\n        # This test uses self.http_client.fetch because self.fetch calls\n        # self.get_url on the input unconditionally.\n        url = self.get_url(\"/hello\")\n        response = yield self.http_client.fetch(url)\n        self.assertEqual(response.request.url, url)\n        self.assertTrue(isinstance(response.request, HTTPRequest))\n        response2 = yield self.http_client.fetch(response.request)\n        self.assertEqual(response2.body, b\"Hello world!\")\n\n    @gen_test\n    def test_bind_source_ip(self):\n        url = self.get_url(\"/hello\")\n        request = HTTPRequest(url, network_interface=\"127.0.0.1\")\n        response = yield self.http_client.fetch(request)\n        self.assertEqual(response.code, 200)\n\n        with self.assertRaises((ValueError, HTTPError)) as context:\n            request = HTTPRequest(url, network_interface=\"not-interface-or-ip\")\n            yield self.http_client.fetch(request)\n        self.assertIn(\"not-interface-or-ip\", str(context.exception))\n\n    def test_all_methods(self):\n        for method in [\"GET\", \"DELETE\", \"OPTIONS\"]:\n            response = self.fetch(\"/all_methods\", method=method)\n            self.assertEqual(response.body, utf8(method))\n        for method in [\"POST\", \"PUT\", \"PATCH\"]:\n            response = self.fetch(\"/all_methods\", method=method, body=b\"\")\n            self.assertEqual(response.body, utf8(method))\n        response = self.fetch(\"/all_methods\", method=\"HEAD\")\n        self.assertEqual(response.body, b\"\")\n        response = self.fetch(\n            \"/all_methods\", method=\"OTHER\", allow_nonstandard_methods=True\n        )\n        self.assertEqual(response.body, b\"OTHER\")\n\n    def test_body_sanity_checks(self):\n        # These methods require a body.\n        for method in (\"POST\", \"PUT\", \"PATCH\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\"/all_methods\", method=method, raise_error=True)\n            self.assertIn(\"must not be None\", str(context.exception))\n\n            resp = self.fetch(\n                \"/all_methods\", method=method, allow_nonstandard_methods=True\n            )\n            self.assertEqual(resp.code, 200)\n\n        # These methods don't allow a body.\n        for method in (\"GET\", \"DELETE\", \"OPTIONS\"):\n            with self.assertRaises(ValueError) as context:\n                self.fetch(\n                    \"/all_methods\", method=method, body=b\"asdf\", raise_error=True\n                )\n            self.assertIn(\"must be None\", str(context.exception))\n\n            # In most cases this can be overridden, but curl_httpclient\n            # does not allow body with a GET at all.\n            if method != \"GET\":\n                self.fetch(\n                    \"/all_methods\",\n                    method=method,\n                    body=b\"asdf\",\n                    allow_nonstandard_methods=True,\n                    raise_error=True,\n                )\n                self.assertEqual(resp.code, 200)\n\n    # This test causes odd failures with the combination of\n    # curl_httpclient (at least with the version of libcurl available\n    # on ubuntu 12.04), TwistedIOLoop, and epoll.  For POST (but not PUT),\n    # curl decides the response came back too soon and closes the connection\n    # to start again.  It does this *before* telling the socket callback to\n    # unregister the FD.  Some IOLoop implementations have special kernel\n    # integration to discover this immediately.  Tornado's IOLoops\n    # ignore errors on remove_handler to accommodate this behavior, but\n    # Twisted's reactor does not.  The removeReader call fails and so\n    # do all future removeAll calls (which our tests do at cleanup).\n    #\n    # def test_post_307(self):\n    #    response = self.fetch(\"/redirect?status=307&url=/post\",\n    #                          method=\"POST\", body=b\"arg1=foo&arg2=bar\")\n    #    self.assertEqual(response.body, b\"Post arg1: foo, arg2: bar\")\n\n    def test_put_307(self):\n        response = self.fetch(\n            \"/redirect?status=307&url=/put\", method=\"PUT\", body=b\"hello\"\n        )\n        response.rethrow()\n        self.assertEqual(response.body, b\"Put body: hello\")\n\n    def test_non_ascii_header(self):\n        # Non-ascii headers are sent as latin1.\n        response = self.fetch(\"/set_header?k=foo&v=%E9\")\n        response.rethrow()\n        self.assertEqual(response.headers[\"Foo\"], native_str(u\"\\u00e9\"))\n\n    def test_response_times(self):\n        # A few simple sanity checks of the response time fields to\n        # make sure they're using the right basis (between the\n        # wall-time and monotonic clocks).\n        start_time = time.time()\n        response = self.fetch(\"/hello\")\n        response.rethrow()\n        self.assertGreaterEqual(response.request_time, 0)\n        self.assertLess(response.request_time, 1.0)\n        # A very crude check to make sure that start_time is based on\n        # wall time and not the monotonic clock.\n        self.assertLess(abs(response.start_time - start_time), 1.0)\n\n        for k, v in response.time_info.items():\n            self.assertTrue(0 <= v < 1.0, \"time_info[%s] out of bounds: %s\" % (k, v))\n\n\nclass RequestProxyTest(unittest.TestCase):\n    def test_request_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", user_agent=\"foo\"), dict()\n        )\n        self.assertEqual(proxy.user_agent, \"foo\")\n\n    def test_default_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\"), dict(network_interface=\"foo\")\n        )\n        self.assertEqual(proxy.network_interface, \"foo\")\n\n    def test_both_set(self):\n        proxy = _RequestProxy(\n            HTTPRequest(\"http://example.com/\", proxy_host=\"foo\"), dict(proxy_host=\"bar\")\n        )\n        self.assertEqual(proxy.proxy_host, \"foo\")\n\n    def test_neither_set(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        self.assertIs(proxy.auth_username, None)\n\n    def test_bad_attribute(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), dict())\n        with self.assertRaises(AttributeError):\n            proxy.foo\n\n    def test_defaults_none(self):\n        proxy = _RequestProxy(HTTPRequest(\"http://example.com/\"), None)\n        self.assertIs(proxy.auth_username, None)\n\n\nclass HTTPResponseTestCase(unittest.TestCase):\n    def test_str(self):\n        response = HTTPResponse(  # type: ignore\n            HTTPRequest(\"http://example.com\"), 200, headers={}, buffer=BytesIO()\n        )\n        s = str(response)\n        self.assertTrue(s.startswith(\"HTTPResponse(\"))\n        self.assertIn(\"code=200\", s)\n\n\nclass SyncHTTPClientTest(unittest.TestCase):\n    def setUp(self):\n        self.server_ioloop = IOLoop()\n        event = threading.Event()\n\n        @gen.coroutine\n        def init_server():\n            sock, self.port = bind_unused_port()\n            app = Application([(\"/\", HelloWorldHandler)])\n            self.server = HTTPServer(app)\n            self.server.add_socket(sock)\n            event.set()\n\n        def start():\n            self.server_ioloop.run_sync(init_server)\n            self.server_ioloop.start()\n\n        self.server_thread = threading.Thread(target=start)\n        self.server_thread.start()\n        event.wait()\n\n        self.http_client = HTTPClient()\n\n    def tearDown(self):\n        def stop_server():\n            self.server.stop()\n            # Delay the shutdown of the IOLoop by several iterations because\n            # the server may still have some cleanup work left when\n            # the client finishes with the response (this is noticeable\n            # with http/2, which leaves a Future with an unexamined\n            # StreamClosedError on the loop).\n\n            @gen.coroutine\n            def slow_stop():\n                # The number of iterations is difficult to predict. Typically,\n                # one is sufficient, although sometimes it needs more.\n                for i in range(5):\n                    yield\n                self.server_ioloop.stop()\n\n            self.server_ioloop.add_callback(slow_stop)\n\n        self.server_ioloop.add_callback(stop_server)\n        self.server_thread.join()\n        self.http_client.close()\n        self.server_ioloop.close(all_fds=True)\n\n    def get_url(self, path):\n        return \"http://127.0.0.1:%d%s\" % (self.port, path)\n\n    def test_sync_client(self):\n        response = self.http_client.fetch(self.get_url(\"/\"))\n        self.assertEqual(b\"Hello world!\", response.body)\n\n    def test_sync_client_error(self):\n        # Synchronous HTTPClient raises errors directly; no need for\n        # response.rethrow()\n        with self.assertRaises(HTTPError) as assertion:\n            self.http_client.fetch(self.get_url(\"/notfound\"))\n        self.assertEqual(assertion.exception.code, 404)\n\n\nclass HTTPRequestTestCase(unittest.TestCase):\n    def test_headers(self):\n        request = HTTPRequest(\"http://example.com\", headers={\"foo\": \"bar\"})\n        self.assertEqual(request.headers, {\"foo\": \"bar\"})\n\n    def test_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = {\"bar\": \"baz\"}  # type: ignore\n        self.assertEqual(request.headers, {\"bar\": \"baz\"})\n\n    def test_null_headers_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.headers = None  # type: ignore\n        self.assertEqual(request.headers, {})\n\n    def test_body(self):\n        request = HTTPRequest(\"http://example.com\", body=\"foo\")\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_body_setter(self):\n        request = HTTPRequest(\"http://example.com\")\n        request.body = \"foo\"  # type: ignore\n        self.assertEqual(request.body, utf8(\"foo\"))\n\n    def test_if_modified_since(self):\n        http_date = datetime.datetime.utcnow()\n        request = HTTPRequest(\"http://example.com\", if_modified_since=http_date)\n        self.assertEqual(\n            request.headers, {\"If-Modified-Since\": format_timestamp(http_date)}\n        )\n\n\nclass HTTPErrorTestCase(unittest.TestCase):\n    def test_copy(self):\n        e = HTTPError(403)\n        e2 = copy.copy(e)\n        self.assertIsNot(e, e2)\n        self.assertEqual(e.code, e2.code)\n\n    def test_plain_error(self):\n        e = HTTPError(403)\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n\n    def test_error_with_response(self):\n        resp = HTTPResponse(HTTPRequest(\"http://example.com/\"), 403)\n        with self.assertRaises(HTTPError) as cm:\n            resp.rethrow()\n        e = cm.exception\n        self.assertEqual(str(e), \"HTTP 403: Forbidden\")\n        self.assertEqual(repr(e), \"HTTP 403: Forbidden\")\n",
        "#!/usr/bin/env python\nimport unittest\nimport sys\nimport time\nimport traceback\nimport platform\nimport os\n\n\nclass PytestLikeResult(unittest.TestResult):\n    def __init__(self, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n        self.start_time = None\n        # Store (test, outcome, info) for each test:\n        #   outcome ∈ { \"passed\", \"failed\", \"error\", \"skipped\" }\n        #   info can hold the exception or skip reason, etc.\n        self.results = []\n\n    def startTestRun(self):\n        \"\"\"Called once before all tests.\"\"\"\n        self.start_time = time.time()\n        # Print a header similar to pytest\n        print(\"=\" * 29 + \" test session starts \" + \"=\" * 29)\n        # You can customize the platform/versions/paths as you like:\n        print(\n            f\"platform {platform.system().lower()} -- Python {platform.python_version()}\"\n        )\n        print(f\"rootdir: {os.path.abspath(os.curdir)}\")\n\n    def stopTestRun(self):\n        \"\"\"Called once after all tests are done.\"\"\"\n        elapsed = time.time() - self.start_time\n        print()  # blank line after final dot\n\n        # Summarize\n        ntests = len(self.results)\n        print(f\"collected {ntests} items\")\n\n        # Show details of failures/errors\n        failures = [r for r in self.results if r[1] == \"failed\"]\n        errors = [r for r in self.results if r[1] == \"error\"]\n\n        if failures:\n            print(\"\\n\" + \"=\" * 35 + \" FAILURES \" + \"=\" * 35)\n            for test, outcome, err_info in failures:\n                test = test.split(\"::\")\n                if len(test) == 3:\n                    test = f\"{test[1]}.{test[2]}\"\n                else:\n                    test = f\"{test[1]}\"\n                print(f\"__________ {test} __________\")\n                self._print_traceback(err_info)\n        if errors:\n            print(\"\\n\" + \"=\" * 35 + \" ERRORS \" + \"=\" * 37)\n            for test, outcome, err_info in errors:\n                test = test.split(\"::\")\n                if len(test) == 3:\n                    test = f\"{test[1]}.{test[2]}\"\n                else:\n                    test = f\"{test[1]}\"\n                print(f\"__________ {test} __________\")\n                self._print_traceback(err_info)\n\n        # Optional short summary lines for passes/fails/skips\n        print(\"\\n==================== short test summary info ====================\")\n        for test, outcome, info in self.results:\n            if outcome in (\"passed\", \"failed\", \"error\", \"skipped\"):\n                print(f\"{outcome.upper()} {test}\")\n\n        # Final one-line summary: “1 failed, 1 errors, 10 passed in 2.04s”\n        passed_count = sum(1 for r in self.results if r[1] == \"passed\")\n        failed_count = len(failures)\n        error_count = len(errors)\n        skipped_count = sum(1 for r in self.results if r[1] == \"skipped\")\n\n        summary_bits = []\n        if failed_count:\n            summary_bits.append(f\"{failed_count} failed\")\n        if error_count:\n            summary_bits.append(f\"{error_count} error{'s' if error_count != 1 else ''}\")\n        if passed_count:\n            summary_bits.append(f\"{passed_count} passed\")\n        if skipped_count:\n            summary_bits.append(f\"{skipped_count} skipped\")\n\n        summary_str = \", \".join(summary_bits) if summary_bits else \"no tests run\"\n        print(\n            f\"=================== {summary_str} in {elapsed:.2f}s ===================\"\n        )\n\n    def startTest(self, test):\n        \"\"\"Called right before each test method.\"\"\"\n        super().startTest(test)\n        # Print a dot or some indicator\n        print(\".\", end=\"\", flush=True)\n\n    def addSuccess(self, test):\n        super().addSuccess(test)\n        self.results.append((self._test_id(test), \"passed\", None))\n\n    def addFailure(self, test, err):\n        super().addFailure(test, err)\n        self.results.append((self._test_id(test), \"failed\", err))\n\n    def addError(self, test, err):\n        super().addError(test, err)\n        self.results.append((self._test_id(test), \"error\", err))\n\n    def addSkip(self, test, reason):\n        super().addSkip(test, reason)\n        self.results.append((self._test_id(test), \"skipped\", reason))\n\n    def _print_traceback(self, err_info):\n        \"\"\"Helper to print the traceback like pytest does.\"\"\"\n        if isinstance(err_info, tuple) and len(err_info) == 3:\n            exc_type, exc_value, tb = err_info\n            traceback.print_exception(exc_type, exc_value, tb, file=sys.stdout)\n        else:\n            print(str(err_info))\n\n    def _test_id(self, test):\n        try:\n            return f\"{test.__class__.__module__}::{test.__class__.__name__}::{test._testMethodName}\"\n        except AttributeError:\n            # Fallback for _ErrorHolder objects\n            return f\"{test.__class__.__module__}::{test.__class__.__name__}\"\n\n\nclass PytestLikeRunner(unittest.TextTestRunner):\n    \"\"\"A custom TextTestRunner that uses the PytestLikeResult.\"\"\"\n\n    resultclass = PytestLikeResult\n\n    def run(self, test):\n        result = self._makeResult()\n        # Overriding run to ensure we call startTestRun/stopTestRun\n        result.startTestRun()\n        test(result)\n        result.stopTestRun()\n        return result\n\n    def tearDown(self):\n        # Reset signal handlers\n        from twisted.internet import reactor\n\n        reactor.removeAll()\n        reactor.stop()\n        super().tearDown()\n\n\ndef main():\n    loader = unittest.TestLoader()\n    suite = loader.discover(\".\", pattern=\"test_*.py\")  # discover tests in r2e_tests/\n    runner = PytestLikeRunner(verbosity=0)\n    result = runner.run(suite)\n    # Exit with code 0 if all tests passed/skipped, 1 if there were fails/errors\n    sys.exit(not result.wasSuccessful())\n\n\nif __name__ == \"__main__\":\n    main()\n"
    ],
    "test_file_names": [
        "test_1.py",
        "tornado_unittest_runner.py"
    ],
    "setup_res_code": 0,
    "setup_res_stdout": "",
    "setup_res_stderr": "Using CPython 3.9.21\nCreating virtual environment at: .venv\nActivate with: source .venv/bin/activate\nResolved 1 package in 680ms\nPrepared 1 package in 1.14s\nInstalled 1 package in 3ms\n + tornado==6.0.dev1 (from file:///home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff)\nResolved 29 packages in 17ms\nInstalled 29 packages in 241ms\n + attrs==24.3.0\n + automat==24.8.1\n + cachetools==5.5.0\n + cffi==1.17.1\n + chardet==5.2.0\n + colorama==0.4.6\n + constantly==23.10.4\n + distlib==0.3.9\n + exceptiongroup==1.2.2\n + filelock==3.16.1\n + hyperlink==21.0.0\n + idna==3.10\n + incremental==24.7.2\n + iniconfig==2.0.0\n + packaging==24.2\n + platformdirs==4.3.6\n + pluggy==1.5.0\n + pycares==4.5.0\n + pycparser==2.22\n + pycurl==7.45.4\n + pyproject-api==1.8.0\n + pytest==8.3.4\n + setuptools==75.6.0\n + tomli==2.2.1\n + tox==4.23.2\n + twisted==24.11.0\n + typing-extensions==4.12.2\n + virtualenv==20.28.0\n + zope-interface==7.2\n",
    "new_commit_res_code": 0,
    "new_commit_res_stdout": "============================= test session starts =============================\nplatform linux -- Python 3.9.21\nrootdir: /home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff\n...............................................\ncollected 47 items\n\n==================== short test summary info ====================\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_304_with_content_length\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_all_methods\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_explicit_mode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_unicode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_bind_source_ip\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_encoding\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_sanity_checks\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked_close\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_configure_defaults\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_credentials_in_url\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_follow_redirect\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error_no_raise\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_interface\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_hello_world\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_multi_line_headers\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_non_ascii_header\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_patch_receives_payload\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_post\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_put_307\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_response_times\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_reuse_request_from_response\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_streaming_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_unsupported_auth_mode\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_copy\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_error_with_response\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_plain_error\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_if_modified_since\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_null_headers_setter\nPASSED r2e_tests.test_1::HTTPResponseTestCase::test_str\nPASSED r2e_tests.test_1::RequestProxyTest::test_bad_attribute\nPASSED r2e_tests.test_1::RequestProxyTest::test_both_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_default_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_defaults_none\nPASSED r2e_tests.test_1::RequestProxyTest::test_neither_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_request_set\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client_error\n=================== 47 passed in 0.14s ===================\n",
    "new_commit_res_stderr": "WARNING:tornado.access:405 HEAD /all_methods (127.0.0.1) 0.23ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.28ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.25ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.53ms\n",
    "old_commit_res_code": 1,
    "old_commit_res_stdout": "============================= test session starts =============================\nplatform linux -- Python 3.9.21\nrootdir: /home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff\n...............................................\ncollected 47 items\n\n=================================== ERRORS =====================================\n__________ HTTPClientCommonTestCase.test_bind_source_ip __________\nTraceback (most recent call last):\n  File \"/home/gcpuser/.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/unittest/case.py\", line 59, in testPartExecutor\n    yield\n  File \"/home/gcpuser/.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/unittest/case.py\", line 592, in run\n    self._callTestMethod(testMethod)\n  File \"/home/gcpuser/.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/unittest/case.py\", line 550, in _callTestMethod\n    method()\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/tornado/testing.py\", line 93, in __call__\n    result = self.orig_method(*args, **kwargs)\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/tornado/testing.py\", line 557, in post_coroutine\n    return self.io_loop.run_sync(\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/tornado/ioloop.py\", line 535, in run_sync\n    return future_cell[0].result()\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/tornado/gen.py\", line 743, in run\n    yielded = self.gen.throw(*exc_info)  # type: ignore\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/r2e_tests/test_1.py\", line 474, in test_bind_source_ip\n    response = yield self.http_client.fetch(request)\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/tornado/gen.py\", line 736, in run\n    value = future.result()\n  File \"/home/gcpuser/buckets/local_repoeval_bucket/repos/tornado_9ba61c636899ed0f7472fb2f4975a53720d1b0ff/tornado/simple_httpclient.py\", line 351, in run\n    raise NotImplementedError(\"%s not supported\" % key)\nNotImplementedError: network_interface not supported\n\n==================== short test summary info ====================\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_304_with_content_length\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_all_methods\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_explicit_mode\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_basic_auth_unicode\nERROR r2e_tests.test_1::HTTPClientCommonTestCase::test_bind_source_ip\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_encoding\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_body_sanity_checks\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_chunked_close\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_configure_defaults\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_credentials_in_url\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_follow_redirect\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_http_error_no_raise\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_future_interface\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_header_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_hello_world\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_multi_line_headers\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_non_ascii_header\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_patch_receives_payload\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_post\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_put_307\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_response_times\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_reuse_request_from_response\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_streaming_callback\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_types\nPASSED r2e_tests.test_1::HTTPClientCommonTestCase::test_unsupported_auth_mode\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_copy\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_error_with_response\nPASSED r2e_tests.test_1::HTTPErrorTestCase::test_plain_error\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_body_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_headers_setter\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_if_modified_since\nPASSED r2e_tests.test_1::HTTPRequestTestCase::test_null_headers_setter\nPASSED r2e_tests.test_1::HTTPResponseTestCase::test_str\nPASSED r2e_tests.test_1::RequestProxyTest::test_bad_attribute\nPASSED r2e_tests.test_1::RequestProxyTest::test_both_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_default_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_defaults_none\nPASSED r2e_tests.test_1::RequestProxyTest::test_neither_set\nPASSED r2e_tests.test_1::RequestProxyTest::test_request_set\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client\nPASSED r2e_tests.test_1::SyncHTTPClientTest::test_sync_client_error\n=================== 1 error, 46 passed in 0.14s ===================\n",
    "old_commit_res_stderr": "WARNING:tornado.access:405 HEAD /all_methods (127.0.0.1) 0.27ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.24ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.22ms\nWARNING:tornado.access:404 GET /notfound (127.0.0.1) 0.59ms\n"
}
tornado/httpclient.py tornado/simple_httpclient.py tornado/test/httpclient_test.py
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[ISSUE]
**Title:** `network_interface` Parameter Not Supported by `SimpleAsyncHTTPClient`

**Description:**
When attempting to use the `network_interface` parameter with `SimpleAsyncHTTPClient`, it raises a `NotImplementedError`. This prevents binding HTTP requests to a specific local IP address, which is necessary for certain networking scenarios.

**Example Code:**
```python
from tornado.httpclient import AsyncHTTPClient, HTTPRequest

async def fetch_with_source_ip(url, source_ip):
    client = AsyncHTTPClient()
    request = HTTPRequest(url, network_interface=source_ip)
    response = await client.fetch(request)
    return response

# Attempting to bind to a specific IP address
await fetch_with_source_ip("http://example.com", "127.0.0.1")
```

**Expected Behavior:**
The `SimpleAsyncHTTPClient` should support the `network_interface` parameter, allowing the user to bind the HTTP request to the specified local IP address. This enables more precise control over network traffic routing.

**Actual Behavior:**
Using the `network_interface` parameter with `SimpleAsyncHTTPClient` results in the following error:
```
NotImplementedError: network_interface not supported
```
This indicates that the client does not recognize or handle the `network_interface` parameter, leading to failed HTTP requests when attempting to specify a source IP.

[/ISSUE]
--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.

--- BEGIN FILE ---
```
### tornado/simple_httpclient.py
from tornado.escape import _unicode
from tornado import gen
from tornado.httpclient import (
    HTTPResponse,
    HTTPError,
    AsyncHTTPClient,
    main,
    _RequestProxy,
    HTTPRequest,
)
from tornado import httputil
from tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters
from tornado.ioloop import IOLoop
from tornado.iostream import StreamClosedError, IOStream
from tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults
from tornado.log import gen_log
from tornado.tcpclient import TCPClient

import base64
import collections
import copy
import functools
import re
import socket
import ssl
import sys
import time
from io import BytesIO
import urllib.parse

from typing import Dict, Any, Callable, Optional, Type, Union
from types import TracebackType
import typing

if typing.TYPE_CHECKING:
    from typing import Deque, Tuple, List  # noqa: F401


class HTTPTimeoutError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient on timeout.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPTimeoutError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Timeout"


class HTTPStreamClosedError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.

    When a more specific exception is available (such as `ConnectionResetError`),
    it may be raised instead of this one.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPStreamClosedError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Stream closed"


class SimpleAsyncHTTPClient(AsyncHTTPClient):
    """Non-blocking HTTP client with no external dependencies.

    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.
    Some features found in the curl-based AsyncHTTPClient are not yet
    supported.  In particular, proxies are not supported, connections
    are not reused, and callers cannot select the network interface to be
    used.
    """

    def initialize(  # type: ignore
        self,
        max_clients: int = 10,
        hostname_mapping: Dict[str, str] = None,
        max_buffer_size: int = 104857600,
        resolver: Resolver = None,
        defaults: Dict[str, Any] = None,
        max_header_size: int = None,
        max_body_size: int = None,
    ) -> None:
        """Creates a AsyncHTTPClient.

        Only a single AsyncHTTPClient instance exists per IOLoop
        in order to provide limitations on the number of pending connections.
        ``force_instance=True`` may be used to suppress this behavior.

        Note that because of this implicit reuse, unless ``force_instance``
        is used, only the first call to the constructor actually uses
        its arguments. It is recommended to use the ``configure`` method
        instead of the constructor to ensure that arguments take effect.

        ``max_clients`` is the number of concurrent requests that can be
        in progress; when this limit is reached additional requests will be
        queued. Note that time spent waiting in this queue still counts
        against the ``request_timeout``.

        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.
        It can be used to make local DNS changes when modifying system-wide
        settings like ``/etc/hosts`` is not possible or desirable (e.g. in
        unittests).

        ``max_buffer_size`` (default 100MB) is the number of bytes
        that can be read into memory at once. ``max_body_size``
        (defaults to ``max_buffer_size``) is the largest response body
        that the client will accept.  Without a
        ``streaming_callback``, the smaller of these two limits
        applies; with a ``streaming_callback`` only ``max_body_size``
        does.

        .. versionchanged:: 4.2
           Added the ``max_body_size`` argument.
        """
        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)
        self.max_clients = max_clients
        self.queue = (
            collections.deque()
        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]
        self.active = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]
        self.waiting = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]
        self.max_buffer_size = max_buffer_size
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        # TCPClient could create a Resolver for us, but we have to do it
        # ourselves to support hostname_mapping.
        if resolver:
            self.resolver = resolver
            self.own_resolver = False
        else:
            self.resolver = Resolver()
            self.own_resolver = True
        if hostname_mapping is not None:
            self.resolver = OverrideResolver(
                resolver=self.resolver, mapping=hostname_mapping
            )
        self.tcp_client = TCPClient(resolver=self.resolver)

    def close(self) -> None:
        super(SimpleAsyncHTTPClient, self).close()
        if self.own_resolver:
            self.resolver.close()
        self.tcp_client.close()

    def fetch_impl(
        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]
    ) -> None:
        key = object()
        self.queue.append((key, request, callback))
        if not len(self.active) < self.max_clients:
            assert request.connect_timeout is not None
            assert request.request_timeout is not None
            timeout_handle = self.io_loop.add_timeout(
                self.io_loop.time()
                + min(request.connect_timeout, request.request_timeout),
                functools.partial(self._on_timeout, key, "in request queue"),
            )
        else:
            timeout_handle = None
        self.waiting[key] = (request, callback, timeout_handle)
        self._process_queue()
        if self.queue:
            gen_log.debug(
                "max_clients limit reached, request queued. "
                "%d active, %d queued requests." % (len(self.active), len(self.queue))
            )

    def _process_queue(self) -> None:
        while self.queue and len(self.active) < self.max_clients:
            key, request, callback = self.queue.popleft()
            if key not in self.waiting:
                continue
            self._remove_timeout(key)
            self.active[key] = (request, callback)
            release_callback = functools.partial(self._release_fetch, key)
            self._handle_request(request, release_callback, callback)

    def _connection_class(self) -> type:
        return _HTTPConnection

    def _handle_request(
        self,
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
    ) -> None:
        self._connection_class()(
            self,
            request,
            release_callback,
            final_callback,
            self.max_buffer_size,
            self.tcp_client,
            self.max_header_size,
            self.max_body_size,
        )

    def _release_fetch(self, key: object) -> None:
        del self.active[key]
        self._process_queue()

    def _remove_timeout(self, key: object) -> None:
        if key in self.waiting:
            request, callback, timeout_handle = self.waiting[key]
            if timeout_handle is not None:
                self.io_loop.remove_timeout(timeout_handle)
            del self.waiting[key]

    def _on_timeout(self, key: object, info: str = None) -> None:
        """Timeout callback of request.

        Construct a timeout HTTPResponse when a timeout occurs.

        :arg object key: A simple object to mark the request.
        :info string key: More detailed timeout information.
        """
        request, callback, timeout_handle = self.waiting[key]
        self.queue.remove((key, request, callback))

        error_message = "Timeout {0}".format(info) if info else "Timeout"
        timeout_response = HTTPResponse(
            request,
            599,
            error=HTTPTimeoutError(error_message),
            request_time=self.io_loop.time() - request.start_time,
        )
        self.io_loop.add_callback(callback, timeout_response)
        del self.waiting[key]


class _HTTPConnection(httputil.HTTPMessageDelegate):
    _SUPPORTED_METHODS = set(
        ["GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"]
    )

    def __init__(
        self,
        client: Optional[SimpleAsyncHTTPClient],
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
        max_buffer_size: int,
        tcp_client: TCPClient,
        max_header_size: int,
        max_body_size: int,
    ) -> None:
        self.io_loop = IOLoop.current()
        self.start_time = self.io_loop.time()
        self.start_wall_time = time.time()
        self.client = client
        self.request = request
        self.release_callback = release_callback
        self.final_callback = final_callback
        self.max_buffer_size = max_buffer_size
        self.tcp_client = tcp_client
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        self.code = None  # type: Optional[int]
        self.headers = None  # type: Optional[httputil.HTTPHeaders]
        self.chunks = []  # type: List[bytes]
        self._decompressor = None
        # Timeout handle returned by IOLoop.add_timeout
        self._timeout = None  # type: object
        self._sockaddr = None
        IOLoop.current().add_future(
            gen.convert_yielded(self.run()), lambda f: f.result()
        )

    async def run(self) -> None:
        try:
            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))
            if self.parsed.scheme not in ("http", "https"):
                raise ValueError("Unsupported url scheme: %s" % self.request.url)
            # urlsplit results have hostname and port results, but they
            # didn't support ipv6 literals until python 2.7.
            netloc = self.parsed.netloc
            if "@" in netloc:
                userpass, _, netloc = netloc.rpartition("@")
            host, port = httputil.split_host_and_port(netloc)
            if port is None:
                port = 443 if self.parsed.scheme == "https" else 80
            if re.match(r"^\[.*\]$", host):
                # raw ipv6 addresses in urls are enclosed in brackets
                host = host[1:-1]
            self.parsed_hostname = host  # save final host for _on_connect

            if self.request.allow_ipv6 is False:
                af = socket.AF_INET
            else:
                af = socket.AF_UNSPEC

            ssl_options = self._get_ssl_options(self.parsed.scheme)

            timeout = min(self.request.connect_timeout, self.request.request_timeout)
            if timeout:
                self._timeout = self.io_loop.add_timeout(
                    self.start_time + timeout,
                    functools.partial(self._on_timeout, "while connecting"),
                )
                stream = await self.tcp_client.connect(
                    host,
                    port,
                    af=af,
                    ssl_options=ssl_options,
                    max_buffer_size=self.max_buffer_size,
                )

                if self.final_callback is None:
                    # final_callback is cleared if we've hit our timeout.
                    stream.close()
                    return
                self.stream = stream
                self.stream.set_close_callback(self.on_connection_close)
                self._remove_timeout()
                if self.final_callback is None:
                    return
                if self.request.request_timeout:
                    self._timeout = self.io_loop.add_timeout(
                        self.start_time + self.request.request_timeout,
                        functools.partial(self._on_timeout, "during request"),
                    )
                if (
                    self.request.method not in self._SUPPORTED_METHODS
                    and not self.request.allow_nonstandard_methods
                ):
                    raise KeyError("unknown method %s" % self.request.method)
                for key in (
                    "network_interface",
                    "proxy_host",
                    "proxy_port",
                    "proxy_username",
                    "proxy_password",
                    "proxy_auth_mode",
                ):
                    if getattr(self.request, key, None):
                        raise NotImplementedError("%s not supported" % key)
                if "Connection" not in self.request.headers:
                    self.request.headers["Connection"] = "close"
                if "Host" not in self.request.headers:
                    if "@" in self.parsed.netloc:
                        self.request.headers["Host"] = self.parsed.netloc.rpartition(
                            "@"
                        )[-1]
                    else:
                        self.request.headers["Host"] = self.parsed.netloc
                username, password = None, None
                if self.parsed.username is not None:
                    username, password = self.parsed.username, self.parsed.password
                elif self.request.auth_username is not None:
                    username = self.request.auth_username
                    password = self.request.auth_password or ""
                if username is not None:
                    assert password is not None
                    if self.request.auth_mode not in (None, "basic"):
                        raise ValueError(
                            "unsupported auth_mode %s", self.request.auth_mode
                        )
                    self.request.headers["Authorization"] = "Basic " + _unicode(
                        base64.b64encode(
                            httputil.encode_username_password(username, password)
                        )
                    )
                if self.request.user_agent:
                    self.request.headers["User-Agent"] = self.request.user_agent
                if not self.request.allow_nonstandard_methods:
                    # Some HTTP methods nearly always have bodies while others
                    # almost never do. Fail in this case unless the user has
                    # opted out of sanity checks with allow_nonstandard_methods.
                    body_expected = self.request.method in ("POST", "PATCH", "PUT")
                    body_present = (
                        self.request.body is not None
                        or self.request.body_producer is not None
                    )
                    if (body_expected and not body_present) or (
                        body_present and not body_expected
                    ):
                        raise ValueError(
                            "Body must %sbe None for method %s (unless "
                            "allow_nonstandard_methods is true)"
                            % ("not " if body_expected else "", self.request.method)
                        )
                if self.request.expect_100_continue:
                    self.request.headers["Expect"] = "100-continue"
                if self.request.body is not None:
                    # When body_producer is used the caller is responsible for
                    # setting Content-Length (or else chunked encoding will be used).
                    self.request.headers["Content-Length"] = str(len(self.request.body))
                if (
                    self.request.method == "POST"
                    and "Content-Type" not in self.request.headers
                ):
                    self.request.headers[
                        "Content-Type"
                    ] = "application/x-www-form-urlencoded"
                if self.request.decompress_response:
                    self.request.headers["Accept-Encoding"] = "gzip"
                req_path = (self.parsed.path or "/") + (
                    ("?" + self.parsed.query) if self.parsed.query else ""
                )
                self.connection = self._create_connection(stream)
                start_line = httputil.RequestStartLine(
                    self.request.method, req_path, ""
                )
                self.connection.write_headers(start_line, self.request.headers)
                if self.request.expect_100_continue:
                    await self.connection.read_response(self)
                else:
                    await self._write_body(True)
        except Exception:
            if not self._handle_exception(*sys.exc_info()):
                raise

    def _get_ssl_options(
        self, scheme: str
    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:
        if scheme == "https":
            if self.request.ssl_options is not None:
                return self.request.ssl_options
            # If we are using the defaults, don't construct a
            # new SSLContext.
            if (
                self.request.validate_cert
                and self.request.ca_certs is None
                and self.request.client_cert is None
                and self.request.client_key is None
            ):
                return _client_ssl_defaults
            ssl_ctx = ssl.create_default_context(
                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs
            )
            if not self.request.validate_cert:
                ssl_ctx.check_hostname = False
                ssl_ctx.verify_mode = ssl.CERT_NONE
            if self.request.client_cert is not None:
                ssl_ctx.load_cert_chain(
                    self.request.client_cert, self.request.client_key
                )
            if hasattr(ssl, "OP_NO_COMPRESSION"):
                # See netutil.ssl_options_to_context
                ssl_ctx.options |= ssl.OP_NO_COMPRESSION
            return ssl_ctx
        return None

    def _on_timeout(self, info: str = None) -> None:
        """Timeout callback of _HTTPConnection instance.

        Raise a `HTTPTimeoutError` when a timeout occurs.

        :info string key: More detailed timeout information.
        """
        self._timeout = None
        error_message = "Timeout {0}".format(info) if info else "Timeout"
        if self.final_callback is not None:
            self._handle_exception(
                HTTPTimeoutError, HTTPTimeoutError(error_message), None
            )

    def _remove_timeout(self) -> None:
        if self._timeout is not None:
            self.io_loop.remove_timeout(self._timeout)
            self._timeout = None

    def _create_connection(self, stream: IOStream) -> HTTP1Connection:
        stream.set_nodelay(True)
        connection = HTTP1Connection(
            stream,
            True,
            HTTP1ConnectionParameters(
                no_keep_alive=True,
                max_header_size=self.max_header_size,
                max_body_size=self.max_body_size,
                decompress=bool(self.request.decompress_response),
            ),
            self._sockaddr,
        )
        return connection

    async def _write_body(self, start_read: bool) -> None:
        if self.request.body is not None:
            self.connection.write(self.request.body)
        elif self.request.body_producer is not None:
            fut = self.request.body_producer(self.connection.write)
            if fut is not None:
                await fut
        self.connection.finish()
        if start_read:
            try:
                await self.connection.read_response(self)
            except StreamClosedError:
                if not self._handle_exception(*sys.exc_info()):
                    raise

    def _release(self) -> None:
        if self.release_callback is not None:
            release_callback = self.release_callback
            self.release_callback = None  # type: ignore
            release_callback()

    def _run_callback(self, response: HTTPResponse) -> None:
        self._release()
        if self.final_callback is not None:
            final_callback = self.final_callback
            self.final_callback = None  # type: ignore
            self.io_loop.add_callback(final_callback, response)

    def _handle_exception(
        self,
        typ: Optional[Type[BaseException]],
        value: Optional[BaseException],
        tb: Optional[TracebackType],
    ) -> bool:
        if self.final_callback:
            self._remove_timeout()
            if isinstance(value, StreamClosedError):
                if value.real_error is None:
                    value = HTTPStreamClosedError("Stream closed")
                else:
                    value = value.real_error
            self._run_callback(
                HTTPResponse(
                    self.request,
                    599,
                    error=value,
                    request_time=self.io_loop.time() - self.start_time,
                    start_time=self.start_wall_time,
                )
            )

            if hasattr(self, "stream"):
                # TODO: this may cause a StreamClosedError to be raised
                # by the connection's Future.  Should we cancel the
                # connection more gracefully?
                self.stream.close()
            return True
        else:
            # If our callback has already been called, we are probably
            # catching an exception that is not caused by us but rather
            # some child of our callback. Rather than drop it on the floor,
            # pass it along, unless it's just the stream being closed.
            return isinstance(value, StreamClosedError)

    def on_connection_close(self) -> None:
        if self.final_callback is not None:
            message = "Connection closed"
            if self.stream.error:
                raise self.stream.error
            try:
                raise HTTPStreamClosedError(message)
            except HTTPStreamClosedError:
                self._handle_exception(*sys.exc_info())

    async def headers_received(
        self,
        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],
        headers: httputil.HTTPHeaders,
    ) -> None:
        assert isinstance(first_line, httputil.ResponseStartLine)
        if self.request.expect_100_continue and first_line.code == 100:
            await self._write_body(False)
            return
        self.code = first_line.code
        self.reason = first_line.reason
        self.headers = headers

        if self._should_follow_redirect():
            return

        if self.request.header_callback is not None:
            # Reassemble the start line.
            self.request.header_callback("%s %s %s\r\n" % first_line)
            for k, v in self.headers.get_all():
                self.request.header_callback("%s: %s\r\n" % (k, v))
            self.request.header_callback("\r\n")

    def _should_follow_redirect(self) -> bool:
        if self.request.follow_redirects:
            assert self.request.max_redirects is not None
            return (
                self.code in (301, 302, 303, 307, 308)
                and self.request.max_redirects > 0
            )
        return False

    def finish(self) -> None:
        assert self.code is not None
        data = b"".join(self.chunks)
        self._remove_timeout()
        original_request = getattr(self.request, "original_request", self.request)
        if self._should_follow_redirect():
            assert isinstance(self.request, _RequestProxy)
            new_request = copy.copy(self.request.request)
            new_request.url = urllib.parse.urljoin(
                self.request.url, self.headers["Location"]
            )
            new_request.max_redirects = self.request.max_redirects - 1
            del new_request.headers["Host"]
            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4
            # Client SHOULD make a GET request after a 303.
            # According to the spec, 302 should be followed by the same
            # method as the original request, but in practice browsers
            # treat 302 the same as 303, and many servers use 302 for
            # compatibility with pre-HTTP/1.1 user agents which don't
            # understand the 303 status.
            if self.code in (302, 303):
                new_request.method = "GET"
                new_request.body = None
                for h in [
                    "Content-Length",
                    "Content-Type",
                    "Content-Encoding",
                    "Transfer-Encoding",
                ]:
                    try:
                        del self.request.headers[h]
                    except KeyError:
                        pass
            new_request.original_request = original_request
            final_callback = self.final_callback
            self.final_callback = None
            self._release()
            fut = self.client.fetch(new_request, raise_error=False)
            fut.add_done_callback(lambda f: final_callback(f.result()))
            self._on_end_request()
            return
        if self.request.streaming_callback:
            buffer = BytesIO()
        else:
            buffer = BytesIO(data)  # TODO: don't require one big string?
        response = HTTPResponse(
            original_request,
            self.code,
            reason=getattr(self, "reason", None),
            headers=self.headers,
            request_time=self.io_loop.time() - self.start_time,
            start_time=self.start_wall_time,
            buffer=buffer,
            effective_url=self.request.url,
        )
        self._run_callback(response)
        self._on_end_request()

    def _on_end_request(self) -> None:
        self.stream.close()

    def data_received(self, chunk: bytes) -> None:
        if self._should_follow_redirect():
            # We're going to follow a redirect so just discard the body.
            return
        if self.request.streaming_callback is not None:
            self.request.streaming_callback(chunk)
        else:
            self.chunks.append(chunk)


if __name__ == "__main__":
    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)
    main()

```
--- END FILE ---

Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits to fix the issue.

Every *SEARCH/REPLACE* edit must use this format:
1. The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
4. The dividing line: =======
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE

Here is an example:

```python
### mathweb/flask/app.py
<<<<<<< SEARCH
from flask import Flask
=======
import math
from flask import Flask
>>>>>>> REPLACE
```
</solution>

Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If you would like to add the line '        print(x)', you must fully write that out, with all those spaces before the code!
Wrap each *SEARCH/REPLACE* edit in a code block as shown in the example above. If you have multiple *SEARCH/REPLACE* edits, use a separate code block for each one.
[ISSUE]
**Title:** `network_interface` Parameter Not Supported by `SimpleAsyncHTTPClient`

**Description:**
When attempting to use the `network_interface` parameter with `SimpleAsyncHTTPClient`, it raises a `NotImplementedError`. This prevents binding HTTP requests to a specific local IP address, which is necessary for certain networking scenarios.

**Example Code:**
```python
from tornado.httpclient import AsyncHTTPClient, HTTPRequest

async def fetch_with_source_ip(url, source_ip):
    client = AsyncHTTPClient()
    request = HTTPRequest(url, network_interface=source_ip)
    response = await client.fetch(request)
    return response

# Attempting to bind to a specific IP address
await fetch_with_source_ip("http://example.com", "127.0.0.1")
```

**Expected Behavior:**
The `SimpleAsyncHTTPClient` should support the `network_interface` parameter, allowing the user to bind the HTTP request to the specified local IP address. This enables more precise control over network traffic routing.

**Actual Behavior:**
Using the `network_interface` parameter with `SimpleAsyncHTTPClient` results in the following error:
```
NotImplementedError: network_interface not supported
```
This indicates that the client does not recognize or handle the `network_interface` parameter, leading to failed HTTP requests when attempting to specify a source IP.

[/ISSUE]
{
    "HTTPClientCommonTestCase.test_304_with_content_length": "PASSED",
    "HTTPClientCommonTestCase.test_all_methods": "PASSED",
    "HTTPClientCommonTestCase.test_basic_auth": "PASSED",
    "HTTPClientCommonTestCase.test_basic_auth_explicit_mode": "PASSED",
    "HTTPClientCommonTestCase.test_basic_auth_unicode": "PASSED",
    "HTTPClientCommonTestCase.test_bind_source_ip": "PASSED",
    "HTTPClientCommonTestCase.test_body_encoding": "PASSED",
    "HTTPClientCommonTestCase.test_body_sanity_checks": "PASSED",
    "HTTPClientCommonTestCase.test_chunked": "PASSED",
    "HTTPClientCommonTestCase.test_chunked_close": "PASSED",
    "HTTPClientCommonTestCase.test_configure_defaults": "PASSED",
    "HTTPClientCommonTestCase.test_credentials_in_url": "PASSED",
    "HTTPClientCommonTestCase.test_follow_redirect": "PASSED",
    "HTTPClientCommonTestCase.test_future_http_error": "PASSED",
    "HTTPClientCommonTestCase.test_future_http_error_no_raise": "PASSED",
    "HTTPClientCommonTestCase.test_future_interface": "PASSED",
    "HTTPClientCommonTestCase.test_header_callback": "PASSED",
    "HTTPClientCommonTestCase.test_header_types": "PASSED",
    "HTTPClientCommonTestCase.test_hello_world": "PASSED",
    "HTTPClientCommonTestCase.test_multi_line_headers": "PASSED",
    "HTTPClientCommonTestCase.test_non_ascii_header": "PASSED",
    "HTTPClientCommonTestCase.test_patch_receives_payload": "PASSED",
    "HTTPClientCommonTestCase.test_post": "PASSED",
    "HTTPClientCommonTestCase.test_put_307": "PASSED",
    "HTTPClientCommonTestCase.test_response_times": "PASSED",
    "HTTPClientCommonTestCase.test_reuse_request_from_response": "PASSED",
    "HTTPClientCommonTestCase.test_streaming_callback": "PASSED",
    "HTTPClientCommonTestCase.test_types": "PASSED",
    "HTTPClientCommonTestCase.test_unsupported_auth_mode": "PASSED",
    "HTTPErrorTestCase.test_copy": "PASSED",
    "HTTPErrorTestCase.test_error_with_response": "PASSED",
    "HTTPErrorTestCase.test_plain_error": "PASSED",
    "HTTPRequestTestCase.test_body": "PASSED",
    "HTTPRequestTestCase.test_body_setter": "PASSED",
    "HTTPRequestTestCase.test_headers": "PASSED",
    "HTTPRequestTestCase.test_headers_setter": "PASSED",
    "HTTPRequestTestCase.test_if_modified_since": "PASSED",
    "HTTPRequestTestCase.test_null_headers_setter": "PASSED",
    "HTTPResponseTestCase.test_str": "PASSED",
    "RequestProxyTest.test_bad_attribute": "PASSED",
    "RequestProxyTest.test_both_set": "PASSED",
    "RequestProxyTest.test_default_set": "PASSED",
    "RequestProxyTest.test_defaults_none": "PASSED",
    "RequestProxyTest.test_neither_set": "PASSED",
    "RequestProxyTest.test_request_set": "PASSED",
    "SyncHTTPClientTest.test_sync_client": "PASSED",
    "SyncHTTPClientTest.test_sync_client_error": "PASSED"
}
diff --git a/tornado/httpclient.py b/tornado/httpclient.py
index e2623d53..d1c92a49 100644
--- a/tornado/httpclient.py
+++ b/tornado/httpclient.py
@@ -422,8 +422,8 @@ class HTTPRequest(object):
            New in Tornado 4.0.
         :arg bool use_gzip: Deprecated alias for ``decompress_response``
            since Tornado 4.0.
-        :arg str network_interface: Network interface to use for request.
-           ``curl_httpclient`` only; see note below.
+        :arg str network_interface: Network interface or source IP to use for request.
+           See ``curl_httpclient`` note below.
         :arg collections.abc.Callable streaming_callback: If set, ``streaming_callback`` will
            be run with each chunk of data as it is received, and
            ``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in
diff --git a/tornado/simple_httpclient.py b/tornado/simple_httpclient.py
index d6ac20f9..81a0fd24 100644
--- a/tornado/simple_httpclient.py
+++ b/tornado/simple_httpclient.py
@@ -12,7 +12,12 @@ from tornado import httputil
 from tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters
 from tornado.ioloop import IOLoop
 from tornado.iostream import StreamClosedError, IOStream
-from tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults
+from tornado.netutil import (
+    Resolver,
+    OverrideResolver,
+    _client_ssl_defaults,
+    is_valid_ip,
+)
 from tornado.log import gen_log
 from tornado.tcpclient import TCPClient
 
@@ -306,6 +311,16 @@ class _HTTPConnection(httputil.HTTPMessageDelegate):
 
             ssl_options = self._get_ssl_options(self.parsed.scheme)
 
+            source_ip = None
+            if self.request.network_interface:
+                if is_valid_ip(self.request.network_interface):
+                    source_ip = self.request.network_interface
+                else:
+                    raise ValueError(
+                        "Unrecognized IPv4 or IPv6 address for network_interface, got %r"
+                        % (self.request.network_interface,)
+                    )
+
             timeout = min(self.request.connect_timeout, self.request.request_timeout)
             if timeout:
                 self._timeout = self.io_loop.add_timeout(
@@ -318,6 +333,7 @@ class _HTTPConnection(httputil.HTTPMessageDelegate):
                     af=af,
                     ssl_options=ssl_options,
                     max_buffer_size=self.max_buffer_size,
+                    source_ip=source_ip,
                 )
 
                 if self.final_callback is None:
@@ -340,7 +356,6 @@ class _HTTPConnection(httputil.HTTPMessageDelegate):
                 ):
                     raise KeyError("unknown method %s" % self.request.method)
                 for key in (
-                    "network_interface",
                     "proxy_host",
                     "proxy_port",
                     "proxy_username",

{"tornado/simple_httpclient.py": "from tornado.escape import _unicode\nfrom tornado import gen\nfrom tornado.httpclient import (\n    HTTPResponse,\n    HTTPError,\n    AsyncHTTPClient,\n    main,\n    _RequestProxy,\n    HTTPRequest,\n)\nfrom tornado import httputil\nfrom tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters\nfrom tornado.ioloop import IOLoop\nfrom tornado.iostream import StreamClosedError, IOStream\nfrom tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults\nfrom tornado.log import gen_log\nfrom tornado.tcpclient import TCPClient\n\nimport base64\nimport collections\nimport copy\nimport functools\nimport re\nimport socket\nimport ssl\nimport sys\nimport time\nfrom io import BytesIO\nimport urllib.parse\n\nfrom typing import Dict, Any, Callable, Optional, Type, Union\nfrom types import TracebackType\nimport typing\n\nif typing.TYPE_CHECKING:\n    from typing import Deque, Tuple, List  # noqa: F401\n\n\nclass HTTPTimeoutError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient on timeout.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPTimeoutError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Timeout\"\n\n\nclass HTTPStreamClosedError(HTTPError):\n    \"\"\"Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.\n\n    When a more specific exception is available (such as `ConnectionResetError`),\n    it may be raised instead of this one.\n\n    For historical reasons, this is a subclass of `.HTTPClientError`\n    which simulates a response code of 599.\n\n    .. versionadded:: 5.1\n    \"\"\"\n\n    def __init__(self, message: str) -> None:\n        super(HTTPStreamClosedError, self).__init__(599, message=message)\n\n    def __str__(self) -> str:\n        return self.message or \"Stream closed\"\n\n\nclass SimpleAsyncHTTPClient(AsyncHTTPClient):\n    \"\"\"Non-blocking HTTP client with no external dependencies.\n\n    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.\n    Some features found in the curl-based AsyncHTTPClient are not yet\n    supported.  In particular, proxies are not supported, connections\n    are not reused, and callers cannot select the network interface to be\n    used.\n    \"\"\"\n\n    def initialize(  # type: ignore\n        self,\n        max_clients: int = 10,\n        hostname_mapping: Dict[str, str] = None,\n        max_buffer_size: int = 104857600,\n        resolver: Resolver = None,\n        defaults: Dict[str, Any] = None,\n        max_header_size: int = None,\n        max_body_size: int = None,\n    ) -> None:\n        \"\"\"Creates a AsyncHTTPClient.\n\n        Only a single AsyncHTTPClient instance exists per IOLoop\n        in order to provide limitations on the number of pending connections.\n        ``force_instance=True`` may be used to suppress this behavior.\n\n        Note that because of this implicit reuse, unless ``force_instance``\n        is used, only the first call to the constructor actually uses\n        its arguments. It is recommended to use the ``configure`` method\n        instead of the constructor to ensure that arguments take effect.\n\n        ``max_clients`` is the number of concurrent requests that can be\n        in progress; when this limit is reached additional requests will be\n        queued. Note that time spent waiting in this queue still counts\n        against the ``request_timeout``.\n\n        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.\n        It can be used to make local DNS changes when modifying system-wide\n        settings like ``/etc/hosts`` is not possible or desirable (e.g. in\n        unittests).\n\n        ``max_buffer_size`` (default 100MB) is the number of bytes\n        that can be read into memory at once. ``max_body_size``\n        (defaults to ``max_buffer_size``) is the largest response body\n        that the client will accept.  Without a\n        ``streaming_callback``, the smaller of these two limits\n        applies; with a ``streaming_callback`` only ``max_body_size``\n        does.\n\n        .. versionchanged:: 4.2\n           Added the ``max_body_size`` argument.\n        \"\"\"\n        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)\n        self.max_clients = max_clients\n        self.queue = (\n            collections.deque()\n        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.active = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]\n        self.waiting = (\n            {}\n        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]\n        self.max_buffer_size = max_buffer_size\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        # TCPClient could create a Resolver for us, but we have to do it\n        # ourselves to support hostname_mapping.\n        if resolver:\n            self.resolver = resolver\n            self.own_resolver = False\n        else:\n            self.resolver = Resolver()\n            self.own_resolver = True\n        if hostname_mapping is not None:\n            self.resolver = OverrideResolver(\n                resolver=self.resolver, mapping=hostname_mapping\n            )\n        self.tcp_client = TCPClient(resolver=self.resolver)\n\n    def close(self) -> None:\n        super(SimpleAsyncHTTPClient, self).close()\n        if self.own_resolver:\n            self.resolver.close()\n        self.tcp_client.close()\n\n    def fetch_impl(\n        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]\n    ) -> None:\n        key = object()\n        self.queue.append((key, request, callback))\n        if not len(self.active) < self.max_clients:\n            assert request.connect_timeout is not None\n            assert request.request_timeout is not None\n            timeout_handle = self.io_loop.add_timeout(\n                self.io_loop.time()\n                + min(request.connect_timeout, request.request_timeout),\n                functools.partial(self._on_timeout, key, \"in request queue\"),\n            )\n        else:\n            timeout_handle = None\n        self.waiting[key] = (request, callback, timeout_handle)\n        self._process_queue()\n        if self.queue:\n            gen_log.debug(\n                \"max_clients limit reached, request queued. \"\n                \"%d active, %d queued requests.\" % (len(self.active), len(self.queue))\n            )\n\n    def _process_queue(self) -> None:\n        while self.queue and len(self.active) < self.max_clients:\n            key, request, callback = self.queue.popleft()\n            if key not in self.waiting:\n                continue\n            self._remove_timeout(key)\n            self.active[key] = (request, callback)\n            release_callback = functools.partial(self._release_fetch, key)\n            self._handle_request(request, release_callback, callback)\n\n    def _connection_class(self) -> type:\n        return _HTTPConnection\n\n    def _handle_request(\n        self,\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n    ) -> None:\n        self._connection_class()(\n            self,\n            request,\n            release_callback,\n            final_callback,\n            self.max_buffer_size,\n            self.tcp_client,\n            self.max_header_size,\n            self.max_body_size,\n        )\n\n    def _release_fetch(self, key: object) -> None:\n        del self.active[key]\n        self._process_queue()\n\n    def _remove_timeout(self, key: object) -> None:\n        if key in self.waiting:\n            request, callback, timeout_handle = self.waiting[key]\n            if timeout_handle is not None:\n                self.io_loop.remove_timeout(timeout_handle)\n            del self.waiting[key]\n\n    def _on_timeout(self, key: object, info: str = None) -> None:\n        \"\"\"Timeout callback of request.\n\n        Construct a timeout HTTPResponse when a timeout occurs.\n\n        :arg object key: A simple object to mark the request.\n        :info string key: More detailed timeout information.\n        \"\"\"\n        request, callback, timeout_handle = self.waiting[key]\n        self.queue.remove((key, request, callback))\n\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        timeout_response = HTTPResponse(\n            request,\n            599,\n            error=HTTPTimeoutError(error_message),\n            request_time=self.io_loop.time() - request.start_time,\n        )\n        self.io_loop.add_callback(callback, timeout_response)\n        del self.waiting[key]\n\n\nclass _HTTPConnection(httputil.HTTPMessageDelegate):\n    _SUPPORTED_METHODS = set(\n        [\"GET\", \"HEAD\", \"POST\", \"PUT\", \"DELETE\", \"PATCH\", \"OPTIONS\"]\n    )\n\n    def __init__(\n        self,\n        client: Optional[SimpleAsyncHTTPClient],\n        request: HTTPRequest,\n        release_callback: Callable[[], None],\n        final_callback: Callable[[HTTPResponse], None],\n        max_buffer_size: int,\n        tcp_client: TCPClient,\n        max_header_size: int,\n        max_body_size: int,\n    ) -> None:\n        self.io_loop = IOLoop.current()\n        self.start_time = self.io_loop.time()\n        self.start_wall_time = time.time()\n        self.client = client\n        self.request = request\n        self.release_callback = release_callback\n        self.final_callback = final_callback\n        self.max_buffer_size = max_buffer_size\n        self.tcp_client = tcp_client\n        self.max_header_size = max_header_size\n        self.max_body_size = max_body_size\n        self.code = None  # type: Optional[int]\n        self.headers = None  # type: Optional[httputil.HTTPHeaders]\n        self.chunks = []  # type: List[bytes]\n        self._decompressor = None\n        # Timeout handle returned by IOLoop.add_timeout\n        self._timeout = None  # type: object\n        self._sockaddr = None\n        IOLoop.current().add_future(\n            gen.convert_yielded(self.run()), lambda f: f.result()\n        )\n\n    async def run(self) -> None:\n        try:\n            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))\n            if self.parsed.scheme not in (\"http\", \"https\"):\n                raise ValueError(\"Unsupported url scheme: %s\" % self.request.url)\n            # urlsplit results have hostname and port results, but they\n            # didn't support ipv6 literals until python 2.7.\n            netloc = self.parsed.netloc\n            if \"@\" in netloc:\n                userpass, _, netloc = netloc.rpartition(\"@\")\n            host, port = httputil.split_host_and_port(netloc)\n            if port is None:\n                port = 443 if self.parsed.scheme == \"https\" else 80\n            if re.match(r\"^\\[.*\\]$\", host):\n                # raw ipv6 addresses in urls are enclosed in brackets\n                host = host[1:-1]\n            self.parsed_hostname = host  # save final host for _on_connect\n\n            if self.request.allow_ipv6 is False:\n                af = socket.AF_INET\n            else:\n                af = socket.AF_UNSPEC\n\n            ssl_options = self._get_ssl_options(self.parsed.scheme)\n\n            timeout = min(self.request.connect_timeout, self.request.request_timeout)\n            if timeout:\n                self._timeout = self.io_loop.add_timeout(\n                    self.start_time + timeout,\n                    functools.partial(self._on_timeout, \"while connecting\"),\n                )\n                stream = await self.tcp_client.connect(\n                    host,\n                    port,\n                    af=af,\n                    ssl_options=ssl_options,\n                    max_buffer_size=self.max_buffer_size,\n                )\n\n                if self.final_callback is None:\n                    # final_callback is cleared if we've hit our timeout.\n                    stream.close()\n                    return\n                self.stream = stream\n                self.stream.set_close_callback(self.on_connection_close)\n                self._remove_timeout()\n                if self.final_callback is None:\n                    return\n                if self.request.request_timeout:\n                    self._timeout = self.io_loop.add_timeout(\n                        self.start_time + self.request.request_timeout,\n                        functools.partial(self._on_timeout, \"during request\"),\n                    )\n                if (\n                    self.request.method not in self._SUPPORTED_METHODS\n                    and not self.request.allow_nonstandard_methods\n                ):\n                    raise KeyError(\"unknown method %s\" % self.request.method)\n                for key in (\n                    \"network_interface\",\n                    \"proxy_host\",\n                    \"proxy_port\",\n                    \"proxy_username\",\n                    \"proxy_password\",\n                    \"proxy_auth_mode\",\n                ):\n                    if getattr(self.request, key, None):\n                        raise NotImplementedError(\"%s not supported\" % key)\n                if \"Connection\" not in self.request.headers:\n                    self.request.headers[\"Connection\"] = \"close\"\n                if \"Host\" not in self.request.headers:\n                    if \"@\" in self.parsed.netloc:\n                        self.request.headers[\"Host\"] = self.parsed.netloc.rpartition(\n                            \"@\"\n                        )[-1]\n                    else:\n                        self.request.headers[\"Host\"] = self.parsed.netloc\n                username, password = None, None\n                if self.parsed.username is not None:\n                    username, password = self.parsed.username, self.parsed.password\n                elif self.request.auth_username is not None:\n                    username = self.request.auth_username\n                    password = self.request.auth_password or \"\"\n                if username is not None:\n                    assert password is not None\n                    if self.request.auth_mode not in (None, \"basic\"):\n                        raise ValueError(\n                            \"unsupported auth_mode %s\", self.request.auth_mode\n                        )\n                    self.request.headers[\"Authorization\"] = \"Basic \" + _unicode(\n                        base64.b64encode(\n                            httputil.encode_username_password(username, password)\n                        )\n                    )\n                if self.request.user_agent:\n                    self.request.headers[\"User-Agent\"] = self.request.user_agent\n                if not self.request.allow_nonstandard_methods:\n                    # Some HTTP methods nearly always have bodies while others\n                    # almost never do. Fail in this case unless the user has\n                    # opted out of sanity checks with allow_nonstandard_methods.\n                    body_expected = self.request.method in (\"POST\", \"PATCH\", \"PUT\")\n                    body_present = (\n                        self.request.body is not None\n                        or self.request.body_producer is not None\n                    )\n                    if (body_expected and not body_present) or (\n                        body_present and not body_expected\n                    ):\n                        raise ValueError(\n                            \"Body must %sbe None for method %s (unless \"\n                            \"allow_nonstandard_methods is true)\"\n                            % (\"not \" if body_expected else \"\", self.request.method)\n                        )\n                if self.request.expect_100_continue:\n                    self.request.headers[\"Expect\"] = \"100-continue\"\n                if self.request.body is not None:\n                    # When body_producer is used the caller is responsible for\n                    # setting Content-Length (or else chunked encoding will be used).\n                    self.request.headers[\"Content-Length\"] = str(len(self.request.body))\n                if (\n                    self.request.method == \"POST\"\n                    and \"Content-Type\" not in self.request.headers\n                ):\n                    self.request.headers[\n                        \"Content-Type\"\n                    ] = \"application/x-www-form-urlencoded\"\n                if self.request.decompress_response:\n                    self.request.headers[\"Accept-Encoding\"] = \"gzip\"\n                req_path = (self.parsed.path or \"/\") + (\n                    (\"?\" + self.parsed.query) if self.parsed.query else \"\"\n                )\n                self.connection = self._create_connection(stream)\n                start_line = httputil.RequestStartLine(\n                    self.request.method, req_path, \"\"\n                )\n                self.connection.write_headers(start_line, self.request.headers)\n                if self.request.expect_100_continue:\n                    await self.connection.read_response(self)\n                else:\n                    await self._write_body(True)\n        except Exception:\n            if not self._handle_exception(*sys.exc_info()):\n                raise\n\n    def _get_ssl_options(\n        self, scheme: str\n    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:\n        if scheme == \"https\":\n            if self.request.ssl_options is not None:\n                return self.request.ssl_options\n            # If we are using the defaults, don't construct a\n            # new SSLContext.\n            if (\n                self.request.validate_cert\n                and self.request.ca_certs is None\n                and self.request.client_cert is None\n                and self.request.client_key is None\n            ):\n                return _client_ssl_defaults\n            ssl_ctx = ssl.create_default_context(\n                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs\n            )\n            if not self.request.validate_cert:\n                ssl_ctx.check_hostname = False\n                ssl_ctx.verify_mode = ssl.CERT_NONE\n            if self.request.client_cert is not None:\n                ssl_ctx.load_cert_chain(\n                    self.request.client_cert, self.request.client_key\n                )\n            if hasattr(ssl, \"OP_NO_COMPRESSION\"):\n                # See netutil.ssl_options_to_context\n                ssl_ctx.options |= ssl.OP_NO_COMPRESSION\n            return ssl_ctx\n        return None\n\n    def _on_timeout(self, info: str = None) -> None:\n        \"\"\"Timeout callback of _HTTPConnection instance.\n\n        Raise a `HTTPTimeoutError` when a timeout occurs.\n\n        :info string key: More detailed timeout information.\n        \"\"\"\n        self._timeout = None\n        error_message = \"Timeout {0}\".format(info) if info else \"Timeout\"\n        if self.final_callback is not None:\n            self._handle_exception(\n                HTTPTimeoutError, HTTPTimeoutError(error_message), None\n            )\n\n    def _remove_timeout(self) -> None:\n        if self._timeout is not None:\n            self.io_loop.remove_timeout(self._timeout)\n            self._timeout = None\n\n    def _create_connection(self, stream: IOStream) -> HTTP1Connection:\n        stream.set_nodelay(True)\n        connection = HTTP1Connection(\n            stream,\n            True,\n            HTTP1ConnectionParameters(\n                no_keep_alive=True,\n                max_header_size=self.max_header_size,\n                max_body_size=self.max_body_size,\n                decompress=bool(self.request.decompress_response),\n            ),\n            self._sockaddr,\n        )\n        return connection\n\n    async def _write_body(self, start_read: bool) -> None:\n        if self.request.body is not None:\n            self.connection.write(self.request.body)\n        elif self.request.body_producer is not None:\n            fut = self.request.body_producer(self.connection.write)\n            if fut is not None:\n                await fut\n        self.connection.finish()\n        if start_read:\n            try:\n                await self.connection.read_response(self)\n            except StreamClosedError:\n                if not self._handle_exception(*sys.exc_info()):\n                    raise\n\n    def _release(self) -> None:\n        if self.release_callback is not None:\n            release_callback = self.release_callback\n            self.release_callback = None  # type: ignore\n            release_callback()\n\n    def _run_callback(self, response: HTTPResponse) -> None:\n        self._release()\n        if self.final_callback is not None:\n            final_callback = self.final_callback\n            self.final_callback = None  # type: ignore\n            self.io_loop.add_callback(final_callback, response)\n\n    def _handle_exception(\n        self,\n        typ: Optional[Type[BaseException]],\n        value: Optional[BaseException],\n        tb: Optional[TracebackType],\n    ) -> bool:\n        if self.final_callback:\n            self._remove_timeout()\n            if isinstance(value, StreamClosedError):\n                if value.real_error is None:\n                    value = HTTPStreamClosedError(\"Stream closed\")\n                else:\n                    value = value.real_error\n            self._run_callback(\n                HTTPResponse(\n                    self.request,\n                    599,\n                    error=value,\n                    request_time=self.io_loop.time() - self.start_time,\n                    start_time=self.start_wall_time,\n                )\n            )\n\n            if hasattr(self, \"stream\"):\n                # TODO: this may cause a StreamClosedError to be raised\n                # by the connection's Future.  Should we cancel the\n                # connection more gracefully?\n                self.stream.close()\n            return True\n        else:\n            # If our callback has already been called, we are probably\n            # catching an exception that is not caused by us but rather\n            # some child of our callback. Rather than drop it on the floor,\n            # pass it along, unless it's just the stream being closed.\n            return isinstance(value, StreamClosedError)\n\n    def on_connection_close(self) -> None:\n        if self.final_callback is not None:\n            message = \"Connection closed\"\n            if self.stream.error:\n                raise self.stream.error\n            try:\n                raise HTTPStreamClosedError(message)\n            except HTTPStreamClosedError:\n                self._handle_exception(*sys.exc_info())\n\n    async def headers_received(\n        self,\n        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],\n        headers: httputil.HTTPHeaders,\n    ) -> None:\n        assert isinstance(first_line, httputil.ResponseStartLine)\n        if self.request.expect_100_continue and first_line.code == 100:\n            await self._write_body(False)\n            return\n        self.code = first_line.code\n        self.reason = first_line.reason\n        self.headers = headers\n\n        if self._should_follow_redirect():\n            return\n\n        if self.request.header_callback is not None:\n            # Reassemble the start line.\n            self.request.header_callback(\"%s %s %s\\r\\n\" % first_line)\n            for k, v in self.headers.get_all():\n                self.request.header_callback(\"%s: %s\\r\\n\" % (k, v))\n            self.request.header_callback(\"\\r\\n\")\n\n    def _should_follow_redirect(self) -> bool:\n        if self.request.follow_redirects:\n            assert self.request.max_redirects is not None\n            return (\n                self.code in (301, 302, 303, 307, 308)\n                and self.request.max_redirects > 0\n            )\n        return False\n\n    def finish(self) -> None:\n        assert self.code is not None\n        data = b\"\".join(self.chunks)\n        self._remove_timeout()\n        original_request = getattr(self.request, \"original_request\", self.request)\n        if self._should_follow_redirect():\n            assert isinstance(self.request, _RequestProxy)\n            new_request = copy.copy(self.request.request)\n            new_request.url = urllib.parse.urljoin(\n                self.request.url, self.headers[\"Location\"]\n            )\n            new_request.max_redirects = self.request.max_redirects - 1\n            del new_request.headers[\"Host\"]\n            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4\n            # Client SHOULD make a GET request after a 303.\n            # According to the spec, 302 should be followed by the same\n            # method as the original request, but in practice browsers\n            # treat 302 the same as 303, and many servers use 302 for\n            # compatibility with pre-HTTP/1.1 user agents which don't\n            # understand the 303 status.\n            if self.code in (302, 303):\n                new_request.method = \"GET\"\n                new_request.body = None\n                for h in [\n                    \"Content-Length\",\n                    \"Content-Type\",\n                    \"Content-Encoding\",\n                    \"Transfer-Encoding\",\n                ]:\n                    try:\n                        del self.request.headers[h]\n                    except KeyError:\n                        pass\n            new_request.original_request = original_request\n            final_callback = self.final_callback\n            self.final_callback = None\n            self._release()\n            fut = self.client.fetch(new_request, raise_error=False)\n            fut.add_done_callback(lambda f: final_callback(f.result()))\n            self._on_end_request()\n            return\n        if self.request.streaming_callback:\n            buffer = BytesIO()\n        else:\n            buffer = BytesIO(data)  # TODO: don't require one big string?\n        response = HTTPResponse(\n            original_request,\n            self.code,\n            reason=getattr(self, \"reason\", None),\n            headers=self.headers,\n            request_time=self.io_loop.time() - self.start_time,\n            start_time=self.start_wall_time,\n            buffer=buffer,\n            effective_url=self.request.url,\n        )\n        self._run_callback(response)\n        self._on_end_request()\n\n    def _on_end_request(self) -> None:\n        self.stream.close()\n\n    def data_received(self, chunk: bytes) -> None:\n        if self._should_follow_redirect():\n            # We're going to follow a redirect so just discard the body.\n            return\n        if self.request.streaming_callback is not None:\n            self.request.streaming_callback(chunk)\n        else:\n            self.chunks.append(chunk)\n\n\nif __name__ == \"__main__\":\n    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)\n    main()\n"}
A user will ask you to solve a task. You should first draft your thinking process (inner monologue). Then, generate the solution.

Your response format must follow the template below:
<think>
Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct solution.
</think>
<solution>
Final solution presented to the user.
</solution>
system
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[ISSUE]
**Title:** `network_interface` Parameter Not Supported by `SimpleAsyncHTTPClient`

**Description:**
When attempting to use the `network_interface` parameter with `SimpleAsyncHTTPClient`, it raises a `NotImplementedError`. This prevents binding HTTP requests to a specific local IP address, which is necessary for certain networking scenarios.

**Example Code:**
```python
from tornado.httpclient import AsyncHTTPClient, HTTPRequest

async def fetch_with_source_ip(url, source_ip):
    client = AsyncHTTPClient()
    request = HTTPRequest(url, network_interface=source_ip)
    response = await client.fetch(request)
    return response

# Attempting to bind to a specific IP address
await fetch_with_source_ip("http://example.com", "127.0.0.1")
```

**Expected Behavior:**
The `SimpleAsyncHTTPClient` should support the `network_interface` parameter, allowing the user to bind the HTTP request to the specified local IP address. This enables more precise control over network traffic routing.

**Actual Behavior:**
Using the `network_interface` parameter with `SimpleAsyncHTTPClient` results in the following error:
```
NotImplementedError: network_interface not supported
```
This indicates that the client does not recognize or handle the `network_interface` parameter, leading to failed HTTP requests when attempting to specify a source IP.

[/ISSUE]
--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.

--- BEGIN FILE ---
```
### tornado/simple_httpclient.py
from tornado.escape import _unicode
from tornado import gen
from tornado.httpclient import (
    HTTPResponse,
    HTTPError,
    AsyncHTTPClient,
    main,
    _RequestProxy,
    HTTPRequest,
)
from tornado import httputil
from tornado.http1connection import HTTP1Connection, HTTP1ConnectionParameters
from tornado.ioloop import IOLoop
from tornado.iostream import StreamClosedError, IOStream
from tornado.netutil import Resolver, OverrideResolver, _client_ssl_defaults
from tornado.log import gen_log
from tornado.tcpclient import TCPClient

import base64
import collections
import copy
import functools
import re
import socket
import ssl
import sys
import time
from io import BytesIO
import urllib.parse

from typing import Dict, Any, Callable, Optional, Type, Union
from types import TracebackType
import typing

if typing.TYPE_CHECKING:
    from typing import Deque, Tuple, List  # noqa: F401


class HTTPTimeoutError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient on timeout.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPTimeoutError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Timeout"


class HTTPStreamClosedError(HTTPError):
    """Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.

    When a more specific exception is available (such as `ConnectionResetError`),
    it may be raised instead of this one.

    For historical reasons, this is a subclass of `.HTTPClientError`
    which simulates a response code of 599.

    .. versionadded:: 5.1
    """

    def __init__(self, message: str) -> None:
        super(HTTPStreamClosedError, self).__init__(599, message=message)

    def __str__(self) -> str:
        return self.message or "Stream closed"


class SimpleAsyncHTTPClient(AsyncHTTPClient):
    """Non-blocking HTTP client with no external dependencies.

    This class implements an HTTP 1.1 client on top of Tornado's IOStreams.
    Some features found in the curl-based AsyncHTTPClient are not yet
    supported.  In particular, proxies are not supported, connections
    are not reused, and callers cannot select the network interface to be
    used.
    """

    def initialize(  # type: ignore
        self,
        max_clients: int = 10,
        hostname_mapping: Dict[str, str] = None,
        max_buffer_size: int = 104857600,
        resolver: Resolver = None,
        defaults: Dict[str, Any] = None,
        max_header_size: int = None,
        max_body_size: int = None,
    ) -> None:
        """Creates a AsyncHTTPClient.

        Only a single AsyncHTTPClient instance exists per IOLoop
        in order to provide limitations on the number of pending connections.
        ``force_instance=True`` may be used to suppress this behavior.

        Note that because of this implicit reuse, unless ``force_instance``
        is used, only the first call to the constructor actually uses
        its arguments. It is recommended to use the ``configure`` method
        instead of the constructor to ensure that arguments take effect.

        ``max_clients`` is the number of concurrent requests that can be
        in progress; when this limit is reached additional requests will be
        queued. Note that time spent waiting in this queue still counts
        against the ``request_timeout``.

        ``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.
        It can be used to make local DNS changes when modifying system-wide
        settings like ``/etc/hosts`` is not possible or desirable (e.g. in
        unittests).

        ``max_buffer_size`` (default 100MB) is the number of bytes
        that can be read into memory at once. ``max_body_size``
        (defaults to ``max_buffer_size``) is the largest response body
        that the client will accept.  Without a
        ``streaming_callback``, the smaller of these two limits
        applies; with a ``streaming_callback`` only ``max_body_size``
        does.

        .. versionchanged:: 4.2
           Added the ``max_body_size`` argument.
        """
        super(SimpleAsyncHTTPClient, self).initialize(defaults=defaults)
        self.max_clients = max_clients
        self.queue = (
            collections.deque()
        )  # type: Deque[Tuple[object, HTTPRequest, Callable[[HTTPResponse], None]]]
        self.active = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None]]]
        self.waiting = (
            {}
        )  # type: Dict[object, Tuple[HTTPRequest, Callable[[HTTPResponse], None], object]]
        self.max_buffer_size = max_buffer_size
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        # TCPClient could create a Resolver for us, but we have to do it
        # ourselves to support hostname_mapping.
        if resolver:
            self.resolver = resolver
            self.own_resolver = False
        else:
            self.resolver = Resolver()
            self.own_resolver = True
        if hostname_mapping is not None:
            self.resolver = OverrideResolver(
                resolver=self.resolver, mapping=hostname_mapping
            )
        self.tcp_client = TCPClient(resolver=self.resolver)

    def close(self) -> None:
        super(SimpleAsyncHTTPClient, self).close()
        if self.own_resolver:
            self.resolver.close()
        self.tcp_client.close()

    def fetch_impl(
        self, request: HTTPRequest, callback: Callable[[HTTPResponse], None]
    ) -> None:
        key = object()
        self.queue.append((key, request, callback))
        if not len(self.active) < self.max_clients:
            assert request.connect_timeout is not None
            assert request.request_timeout is not None
            timeout_handle = self.io_loop.add_timeout(
                self.io_loop.time()
                + min(request.connect_timeout, request.request_timeout),
                functools.partial(self._on_timeout, key, "in request queue"),
            )
        else:
            timeout_handle = None
        self.waiting[key] = (request, callback, timeout_handle)
        self._process_queue()
        if self.queue:
            gen_log.debug(
                "max_clients limit reached, request queued. "
                "%d active, %d queued requests." % (len(self.active), len(self.queue))
            )

    def _process_queue(self) -> None:
        while self.queue and len(self.active) < self.max_clients:
            key, request, callback = self.queue.popleft()
            if key not in self.waiting:
                continue
            self._remove_timeout(key)
            self.active[key] = (request, callback)
            release_callback = functools.partial(self._release_fetch, key)
            self._handle_request(request, release_callback, callback)

    def _connection_class(self) -> type:
        return _HTTPConnection

    def _handle_request(
        self,
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
    ) -> None:
        self._connection_class()(
            self,
            request,
            release_callback,
            final_callback,
            self.max_buffer_size,
            self.tcp_client,
            self.max_header_size,
            self.max_body_size,
        )

    def _release_fetch(self, key: object) -> None:
        del self.active[key]
        self._process_queue()

    def _remove_timeout(self, key: object) -> None:
        if key in self.waiting:
            request, callback, timeout_handle = self.waiting[key]
            if timeout_handle is not None:
                self.io_loop.remove_timeout(timeout_handle)
            del self.waiting[key]

    def _on_timeout(self, key: object, info: str = None) -> None:
        """Timeout callback of request.

        Construct a timeout HTTPResponse when a timeout occurs.

        :arg object key: A simple object to mark the request.
        :info string key: More detailed timeout information.
        """
        request, callback, timeout_handle = self.waiting[key]
        self.queue.remove((key, request, callback))

        error_message = "Timeout {0}".format(info) if info else "Timeout"
        timeout_response = HTTPResponse(
            request,
            599,
            error=HTTPTimeoutError(error_message),
            request_time=self.io_loop.time() - request.start_time,
        )
        self.io_loop.add_callback(callback, timeout_response)
        del self.waiting[key]


class _HTTPConnection(httputil.HTTPMessageDelegate):
    _SUPPORTED_METHODS = set(
        ["GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"]
    )

    def __init__(
        self,
        client: Optional[SimpleAsyncHTTPClient],
        request: HTTPRequest,
        release_callback: Callable[[], None],
        final_callback: Callable[[HTTPResponse], None],
        max_buffer_size: int,
        tcp_client: TCPClient,
        max_header_size: int,
        max_body_size: int,
    ) -> None:
        self.io_loop = IOLoop.current()
        self.start_time = self.io_loop.time()
        self.start_wall_time = time.time()
        self.client = client
        self.request = request
        self.release_callback = release_callback
        self.final_callback = final_callback
        self.max_buffer_size = max_buffer_size
        self.tcp_client = tcp_client
        self.max_header_size = max_header_size
        self.max_body_size = max_body_size
        self.code = None  # type: Optional[int]
        self.headers = None  # type: Optional[httputil.HTTPHeaders]
        self.chunks = []  # type: List[bytes]
        self._decompressor = None
        # Timeout handle returned by IOLoop.add_timeout
        self._timeout = None  # type: object
        self._sockaddr = None
        IOLoop.current().add_future(
            gen.convert_yielded(self.run()), lambda f: f.result()
        )

    async def run(self) -> None:
        try:
            self.parsed = urllib.parse.urlsplit(_unicode(self.request.url))
            if self.parsed.scheme not in ("http", "https"):
                raise ValueError("Unsupported url scheme: %s" % self.request.url)
            # urlsplit results have hostname and port results, but they
            # didn't support ipv6 literals until python 2.7.
            netloc = self.parsed.netloc
            if "@" in netloc:
                userpass, _, netloc = netloc.rpartition("@")
            host, port = httputil.split_host_and_port(netloc)
            if port is None:
                port = 443 if self.parsed.scheme == "https" else 80
            if re.match(r"^\[.*\]$", host):
                # raw ipv6 addresses in urls are enclosed in brackets
                host = host[1:-1]
            self.parsed_hostname = host  # save final host for _on_connect

            if self.request.allow_ipv6 is False:
                af = socket.AF_INET
            else:
                af = socket.AF_UNSPEC

            ssl_options = self._get_ssl_options(self.parsed.scheme)

            timeout = min(self.request.connect_timeout, self.request.request_timeout)
            if timeout:
                self._timeout = self.io_loop.add_timeout(
                    self.start_time + timeout,
                    functools.partial(self._on_timeout, "while connecting"),
                )
                stream = await self.tcp_client.connect(
                    host,
                    port,
                    af=af,
                    ssl_options=ssl_options,
                    max_buffer_size=self.max_buffer_size,
                )

                if self.final_callback is None:
                    # final_callback is cleared if we've hit our timeout.
                    stream.close()
                    return
                self.stream = stream
                self.stream.set_close_callback(self.on_connection_close)
                self._remove_timeout()
                if self.final_callback is None:
                    return
                if self.request.request_timeout:
                    self._timeout = self.io_loop.add_timeout(
                        self.start_time + self.request.request_timeout,
                        functools.partial(self._on_timeout, "during request"),
                    )
                if (
                    self.request.method not in self._SUPPORTED_METHODS
                    and not self.request.allow_nonstandard_methods
                ):
                    raise KeyError("unknown method %s" % self.request.method)
                for key in (
                    "network_interface",
                    "proxy_host",
                    "proxy_port",
                    "proxy_username",
                    "proxy_password",
                    "proxy_auth_mode",
                ):
                    if getattr(self.request, key, None):
                        raise NotImplementedError("%s not supported" % key)
                if "Connection" not in self.request.headers:
                    self.request.headers["Connection"] = "close"
                if "Host" not in self.request.headers:
                    if "@" in self.parsed.netloc:
                        self.request.headers["Host"] = self.parsed.netloc.rpartition(
                            "@"
                        )[-1]
                    else:
                        self.request.headers["Host"] = self.parsed.netloc
                username, password = None, None
                if self.parsed.username is not None:
                    username, password = self.parsed.username, self.parsed.password
                elif self.request.auth_username is not None:
                    username = self.request.auth_username
                    password = self.request.auth_password or ""
                if username is not None:
                    assert password is not None
                    if self.request.auth_mode not in (None, "basic"):
                        raise ValueError(
                            "unsupported auth_mode %s", self.request.auth_mode
                        )
                    self.request.headers["Authorization"] = "Basic " + _unicode(
                        base64.b64encode(
                            httputil.encode_username_password(username, password)
                        )
                    )
                if self.request.user_agent:
                    self.request.headers["User-Agent"] = self.request.user_agent
                if not self.request.allow_nonstandard_methods:
                    # Some HTTP methods nearly always have bodies while others
                    # almost never do. Fail in this case unless the user has
                    # opted out of sanity checks with allow_nonstandard_methods.
                    body_expected = self.request.method in ("POST", "PATCH", "PUT")
                    body_present = (
                        self.request.body is not None
                        or self.request.body_producer is not None
                    )
                    if (body_expected and not body_present) or (
                        body_present and not body_expected
                    ):
                        raise ValueError(
                            "Body must %sbe None for method %s (unless "
                            "allow_nonstandard_methods is true)"
                            % ("not " if body_expected else "", self.request.method)
                        )
                if self.request.expect_100_continue:
                    self.request.headers["Expect"] = "100-continue"
                if self.request.body is not None:
                    # When body_producer is used the caller is responsible for
                    # setting Content-Length (or else chunked encoding will be used).
                    self.request.headers["Content-Length"] = str(len(self.request.body))
                if (
                    self.request.method == "POST"
                    and "Content-Type" not in self.request.headers
                ):
                    self.request.headers[
                        "Content-Type"
                    ] = "application/x-www-form-urlencoded"
                if self.request.decompress_response:
                    self.request.headers["Accept-Encoding"] = "gzip"
                req_path = (self.parsed.path or "/") + (
                    ("?" + self.parsed.query) if self.parsed.query else ""
                )
                self.connection = self._create_connection(stream)
                start_line = httputil.RequestStartLine(
                    self.request.method, req_path, ""
                )
                self.connection.write_headers(start_line, self.request.headers)
                if self.request.expect_100_continue:
                    await self.connection.read_response(self)
                else:
                    await self._write_body(True)
        except Exception:
            if not self._handle_exception(*sys.exc_info()):
                raise

    def _get_ssl_options(
        self, scheme: str
    ) -> Union[None, Dict[str, Any], ssl.SSLContext]:
        if scheme == "https":
            if self.request.ssl_options is not None:
                return self.request.ssl_options
            # If we are using the defaults, don't construct a
            # new SSLContext.
            if (
                self.request.validate_cert
                and self.request.ca_certs is None
                and self.request.client_cert is None
                and self.request.client_key is None
            ):
                return _client_ssl_defaults
            ssl_ctx = ssl.create_default_context(
                ssl.Purpose.SERVER_AUTH, cafile=self.request.ca_certs
            )
            if not self.request.validate_cert:
                ssl_ctx.check_hostname = False
                ssl_ctx.verify_mode = ssl.CERT_NONE
            if self.request.client_cert is not None:
                ssl_ctx.load_cert_chain(
                    self.request.client_cert, self.request.client_key
                )
            if hasattr(ssl, "OP_NO_COMPRESSION"):
                # See netutil.ssl_options_to_context
                ssl_ctx.options |= ssl.OP_NO_COMPRESSION
            return ssl_ctx
        return None

    def _on_timeout(self, info: str = None) -> None:
        """Timeout callback of _HTTPConnection instance.

        Raise a `HTTPTimeoutError` when a timeout occurs.

        :info string key: More detailed timeout information.
        """
        self._timeout = None
        error_message = "Timeout {0}".format(info) if info else "Timeout"
        if self.final_callback is not None:
            self._handle_exception(
                HTTPTimeoutError, HTTPTimeoutError(error_message), None
            )

    def _remove_timeout(self) -> None:
        if self._timeout is not None:
            self.io_loop.remove_timeout(self._timeout)
            self._timeout = None

    def _create_connection(self, stream: IOStream) -> HTTP1Connection:
        stream.set_nodelay(True)
        connection = HTTP1Connection(
            stream,
            True,
            HTTP1ConnectionParameters(
                no_keep_alive=True,
                max_header_size=self.max_header_size,
                max_body_size=self.max_body_size,
                decompress=bool(self.request.decompress_response),
            ),
            self._sockaddr,
        )
        return connection

    async def _write_body(self, start_read: bool) -> None:
        if self.request.body is not None:
            self.connection.write(self.request.body)
        elif self.request.body_producer is not None:
            fut = self.request.body_producer(self.connection.write)
            if fut is not None:
                await fut
        self.connection.finish()
        if start_read:
            try:
                await self.connection.read_response(self)
            except StreamClosedError:
                if not self._handle_exception(*sys.exc_info()):
                    raise

    def _release(self) -> None:
        if self.release_callback is not None:
            release_callback = self.release_callback
            self.release_callback = None  # type: ignore
            release_callback()

    def _run_callback(self, response: HTTPResponse) -> None:
        self._release()
        if self.final_callback is not None:
            final_callback = self.final_callback
            self.final_callback = None  # type: ignore
            self.io_loop.add_callback(final_callback, response)

    def _handle_exception(
        self,
        typ: Optional[Type[BaseException]],
        value: Optional[BaseException],
        tb: Optional[TracebackType],
    ) -> bool:
        if self.final_callback:
            self._remove_timeout()
            if isinstance(value, StreamClosedError):
                if value.real_error is None:
                    value = HTTPStreamClosedError("Stream closed")
                else:
                    value = value.real_error
            self._run_callback(
                HTTPResponse(
                    self.request,
                    599,
                    error=value,
                    request_time=self.io_loop.time() - self.start_time,
                    start_time=self.start_wall_time,
                )
            )

            if hasattr(self, "stream"):
                # TODO: this may cause a StreamClosedError to be raised
                # by the connection's Future.  Should we cancel the
                # connection more gracefully?
                self.stream.close()
            return True
        else:
            # If our callback has already been called, we are probably
            # catching an exception that is not caused by us but rather
            # some child of our callback. Rather than drop it on the floor,
            # pass it along, unless it's just the stream being closed.
            return isinstance(value, StreamClosedError)

    def on_connection_close(self) -> None:
        if self.final_callback is not None:
            message = "Connection closed"
            if self.stream.error:
                raise self.stream.error
            try:
                raise HTTPStreamClosedError(message)
            except HTTPStreamClosedError:
                self._handle_exception(*sys.exc_info())

    async def headers_received(
        self,
        first_line: Union[httputil.ResponseStartLine, httputil.RequestStartLine],
        headers: httputil.HTTPHeaders,
    ) -> None:
        assert isinstance(first_line, httputil.ResponseStartLine)
        if self.request.expect_100_continue and first_line.code == 100:
            await self._write_body(False)
            return
        self.code = first_line.code
        self.reason = first_line.reason
        self.headers = headers

        if self._should_follow_redirect():
            return

        if self.request.header_callback is not None:
            # Reassemble the start line.
            self.request.header_callback("%s %s %s\r\n" % first_line)
            for k, v in self.headers.get_all():
                self.request.header_callback("%s: %s\r\n" % (k, v))
            self.request.header_callback("\r\n")

    def _should_follow_redirect(self) -> bool:
        if self.request.follow_redirects:
            assert self.request.max_redirects is not None
            return (
                self.code in (301, 302, 303, 307, 308)
                and self.request.max_redirects > 0
            )
        return False

    def finish(self) -> None:
        assert self.code is not None
        data = b"".join(self.chunks)
        self._remove_timeout()
        original_request = getattr(self.request, "original_request", self.request)
        if self._should_follow_redirect():
            assert isinstance(self.request, _RequestProxy)
            new_request = copy.copy(self.request.request)
            new_request.url = urllib.parse.urljoin(
                self.request.url, self.headers["Location"]
            )
            new_request.max_redirects = self.request.max_redirects - 1
            del new_request.headers["Host"]
            # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4
            # Client SHOULD make a GET request after a 303.
            # According to the spec, 302 should be followed by the same
            # method as the original request, but in practice browsers
            # treat 302 the same as 303, and many servers use 302 for
            # compatibility with pre-HTTP/1.1 user agents which don't
            # understand the 303 status.
            if self.code in (302, 303):
                new_request.method = "GET"
                new_request.body = None
                for h in [
                    "Content-Length",
                    "Content-Type",
                    "Content-Encoding",
                    "Transfer-Encoding",
                ]:
                    try:
                        del self.request.headers[h]
                    except KeyError:
                        pass
            new_request.original_request = original_request
            final_callback = self.final_callback
            self.final_callback = None
            self._release()
            fut = self.client.fetch(new_request, raise_error=False)
            fut.add_done_callback(lambda f: final_callback(f.result()))
            self._on_end_request()
            return
        if self.request.streaming_callback:
            buffer = BytesIO()
        else:
            buffer = BytesIO(data)  # TODO: don't require one big string?
        response = HTTPResponse(
            original_request,
            self.code,
            reason=getattr(self, "reason", None),
            headers=self.headers,
            request_time=self.io_loop.time() - self.start_time,
            start_time=self.start_wall_time,
            buffer=buffer,
            effective_url=self.request.url,
        )
        self._run_callback(response)
        self._on_end_request()

    def _on_end_request(self) -> None:
        self.stream.close()

    def data_received(self, chunk: bytes) -> None:
        if self._should_follow_redirect():
            # We're going to follow a redirect so just discard the body.
            return
        if self.request.streaming_callback is not None:
            self.request.streaming_callback(chunk)
        else:
            self.chunks.append(chunk)


if __name__ == "__main__":
    AsyncHTTPClient.configure(SimpleAsyncHTTPClient)
    main()

```
--- END FILE ---

Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits to fix the issue.

Every *SEARCH/REPLACE* edit must use this format:
1. The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
4. The dividing line: =======
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE

Here is an example:

```python
### mathweb/flask/app.py
<<<<<<< SEARCH
from flask import Flask
=======
import math
from flask import Flask
>>>>>>> REPLACE
```
</solution>

Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If you would like to add the line '        print(x)', you must fully write that out, with all those spaces before the code!
Wrap each *SEARCH/REPLACE* edit in a code block as shown in the example above. If you have multiple *SEARCH/REPLACE* edits, use a separate code block for each one.
user