We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Usage of HttpCompressionMiddleware needs to be reflected in Scrapy stats
## Summary
Usage of `HttpCompressionMiddleware` needs to be relfected in Scrapy stats.
## Motivation
In order to estimate scrapy memory usage efficiency and prevent.. memory leaks like [this](https://stackoverflow.com/q/63936759/10884791).
I will need to know:
1. number of request/response objects that can be active (can be achieved by using [`trackref`](https://docs.scrapy.org/en/latest/topics/leaks.html#debugging-memory-leaks-with-trackref) )
2. size of memory required to store that number of request/response objects. 

A lot of websites use compression to reduce traffic. In this case I would like to calculate average size of **decompressed** responses to estimate p.2.

Decompression process means that at some point application will require to allocate memory to store both compressed and decompressed response body and I will need to know this sizes to have more complete vision of scrapy memory usage.

Also size of  decompressed body will be several times more than size of compressed response and it will affect scrapy memory usage.

## Describe alternatives you've considered
The easiest one - is to change priority of `DownloaderStats` middleware and check difference in `downloader/response_bytes` stats parameter.
```
    custom_settings = {"DOWNLOAD_DELAY":1,
                       "DOWNLOADER_MIDDLEWARES":{
                           'scrapy.downloadermiddlewares.stats.DownloaderStats':50
                       }
```
Stats from quotes.toscrape.com spider (it uses `gzip` compression) with default settings:

```
{'downloader/request_bytes': 2642,
 'downloader/request_count': 10,
 'downloader/request_method_count/GET': 10,
 'downloader/response_bytes': 24534,
```
 
And with changed priority of `DownloaderStats` middleware:

```
{'downloader/request_bytes': 912, # size reduced as it didn't count size of request headers populated by downloader middlewares
 'downloader/request_count': 10,
 'downloader/request_method_count/GET': 10,
 'downloader/response_bytes': 110191,   # it counted size of decompressed data 
```

Average size of compressed response (by default) - 2453 bytes.
Average size of decompressed response  - 11019 bytes (~4.5 times more).

## Additional context
Potential solution is to add something like this:
` self.stats.inc_value('decompressed_bytes', spider=spider)`
into `process_response` method of `HttpCompressionMiddleware`

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `scrapy/downloadermiddlewares/httpcompression.py`
Content:
```
1 import io
2 import zlib
3 
4 from scrapy.utils.gz import gunzip
5 from scrapy.http import Response, TextResponse
6 from scrapy.responsetypes import responsetypes
7 from scrapy.exceptions import NotConfigured
8 
9 
10 ACCEPTED_ENCODINGS = [b'gzip', b'deflate']
11 
12 try:
13     import brotli
14     ACCEPTED_ENCODINGS.append(b'br')
15 except ImportError:
16     pass
17 
18 try:
19     import zstandard
20     ACCEPTED_ENCODINGS.append(b'zstd')
21 except ImportError:
22     pass
23 
24 
25 class HttpCompressionMiddleware:
26     """This middleware allows compressed (gzip, deflate) traffic to be
27     sent/received from web sites"""
28     @classmethod
29     def from_crawler(cls, crawler):
30         if not crawler.settings.getbool('COMPRESSION_ENABLED'):
31             raise NotConfigured
32         return cls()
33 
34     def process_request(self, request, spider):
35         request.headers.setdefault('Accept-Encoding',
36                                    b", ".join(ACCEPTED_ENCODINGS))
37 
38     def process_response(self, request, response, spider):
39 
40         if request.method == 'HEAD':
41             return response
42         if isinstance(response, Response):
43             content_encoding = response.headers.getlist('Content-Encoding')
44             if content_encoding:
45                 encoding = content_encoding.pop()
46                 decoded_body = self._decode(response.body, encoding.lower())
47                 respcls = responsetypes.from_args(
48                     headers=response.headers, url=response.url, body=decoded_body
49                 )
50                 kwargs = dict(cls=respcls, body=decoded_body)
51                 if issubclass(respcls, TextResponse):
52                     # force recalculating the encoding until we make sure the
53                     # responsetypes guessing is reliable
54                     kwargs['encoding'] = None
55                 response = response.replace(**kwargs)
56                 if not content_encoding:
57                     del response.headers['Content-Encoding']
58 
59         return response
60 
61     def _decode(self, body, encoding):
62         if encoding == b'gzip' or encoding == b'x-gzip':
63             body = gunzip(body)
64 
65         if encoding == b'deflate':
66             try:
67                 body = zlib.decompress(body)
68             except zlib.error:
69                 # ugly hack to work with raw deflate content that may
70                 # be sent by microsoft servers. For more information, see:
71                 # http://carsten.codimi.de/gzip.yaws/
72                 # http://www.port80software.com/200ok/archive/2005/10/31/868.aspx
73                 # http://www.gzip.org/zlib/zlib_faq.html#faq38
74                 body = zlib.decompress(body, -15)
75         if encoding == b'br' and b'br' in ACCEPTED_ENCODINGS:
76             body = brotli.decompress(body)
77         if encoding == b'zstd' and b'zstd' in ACCEPTED_ENCODINGS:
78             # Using its streaming API since its simple API could handle only cases
79             # where there is content size data embedded in the frame
80             reader = zstandard.ZstdDecompressor().stream_reader(io.BytesIO(body))
81             body = reader.read()
82         return body
83
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/scrapy/downloadermiddlewares/httpcompression.py b/scrapy/downloadermiddlewares/httpcompression.py
--- a/scrapy/downloadermiddlewares/httpcompression.py
+++ b/scrapy/downloadermiddlewares/httpcompression.py
@@ -1,10 +1,12 @@
 import io
+import warnings
 import zlib
 
-from scrapy.utils.gz import gunzip
+from scrapy.exceptions import NotConfigured
 from scrapy.http import Response, TextResponse
 from scrapy.responsetypes import responsetypes
-from scrapy.exceptions import NotConfigured
+from scrapy.utils.deprecate import ScrapyDeprecationWarning
+from scrapy.utils.gz import gunzip
 
 
 ACCEPTED_ENCODINGS = [b'gzip', b'deflate']
@@ -25,11 +27,25 @@
 class HttpCompressionMiddleware:
     """This middleware allows compressed (gzip, deflate) traffic to be
     sent/received from web sites"""
+    def __init__(self, stats=None):
+        self.stats = stats
+
     @classmethod
     def from_crawler(cls, crawler):
         if not crawler.settings.getbool('COMPRESSION_ENABLED'):
             raise NotConfigured
-        return cls()
+        try:
+            return cls(stats=crawler.stats)
+        except TypeError:
+            warnings.warn(
+                "HttpCompressionMiddleware subclasses must either modify "
+                "their '__init__' method to support a 'stats' parameter or "
+                "reimplement the 'from_crawler' method.",
+                ScrapyDeprecationWarning,
+            )
+            result = cls()
+            result.stats = crawler.stats
+            return result
 
     def process_request(self, request, spider):
         request.headers.setdefault('Accept-Encoding',
@@ -44,6 +60,9 @@
             if content_encoding:
                 encoding = content_encoding.pop()
                 decoded_body = self._decode(response.body, encoding.lower())
+                if self.stats:
+                    self.stats.inc_value('httpcompression/response_bytes', len(decoded_body), spider=spider)
+                    self.stats.inc_value('httpcompression/response_count', spider=spider)
                 respcls = responsetypes.from_args(
                     headers=response.headers, url=response.url, body=decoded_body
                 )

{"golden_diff": "diff --git a/scrapy/downloadermiddlewares/httpcompression.py b/scrapy/downloadermiddlewares/httpcompression.py\n--- a/scrapy/downloadermiddlewares/httpcompression.py\n+++ b/scrapy/downloadermiddlewares/httpcompression.py\n@@ -1,10 +1,12 @@\n import io\n+import warnings\n import zlib\n \n-from scrapy.utils.gz import gunzip\n+from scrapy.exceptions import NotConfigured\n from scrapy.http import Response, TextResponse\n from scrapy.responsetypes import responsetypes\n-from scrapy.exceptions import NotConfigured\n+from scrapy.utils.deprecate import ScrapyDeprecationWarning\n+from scrapy.utils.gz import gunzip\n \n \n ACCEPTED_ENCODINGS = [b'gzip', b'deflate']\n@@ -25,11 +27,25 @@\n class HttpCompressionMiddleware:\n     \"\"\"This middleware allows compressed (gzip, deflate) traffic to be\n     sent/received from web sites\"\"\"\n+    def __init__(self, stats=None):\n+        self.stats = stats\n+\n     @classmethod\n     def from_crawler(cls, crawler):\n         if not crawler.settings.getbool('COMPRESSION_ENABLED'):\n             raise NotConfigured\n-        return cls()\n+        try:\n+            return cls(stats=crawler.stats)\n+        except TypeError:\n+            warnings.warn(\n+                \"HttpCompressionMiddleware subclasses must either modify \"\n+                \"their '__init__' method to support a 'stats' parameter or \"\n+                \"reimplement the 'from_crawler' method.\",\n+                ScrapyDeprecationWarning,\n+            )\n+            result = cls()\n+            result.stats = crawler.stats\n+            return result\n \n     def process_request(self, request, spider):\n         request.headers.setdefault('Accept-Encoding',\n@@ -44,6 +60,9 @@\n             if content_encoding:\n                 encoding = content_encoding.pop()\n                 decoded_body = self._decode(response.body, encoding.lower())\n+                if self.stats:\n+                    self.stats.inc_value('httpcompression/response_bytes', len(decoded_body), spider=spider)\n+                    self.stats.inc_value('httpcompression/response_count', spider=spider)\n                 respcls = responsetypes.from_args(\n                     headers=response.headers, url=response.url, body=decoded_body\n                 )\n", "issue": "Usage of HttpCompressionMiddleware needs to be reflected in Scrapy stats\n## Summary\r\nUsage of `HttpCompressionMiddleware` needs to be relfected in Scrapy stats.\r\n## Motivation\r\nIn order to estimate scrapy memory usage efficiency and prevent.. memory leaks like [this](https://stackoverflow.com/q/63936759/10884791).\r\nI will need to know:\r\n1. number of request/response objects that can be active (can be achieved by using [`trackref`](https://docs.scrapy.org/en/latest/topics/leaks.html#debugging-memory-leaks-with-trackref) )\r\n2. size of memory required to store that number of request/response objects. \r\n\r\nA lot of websites use compression to reduce traffic. In this case I would like to calculate average size of **decompressed** responses to estimate p.2.\r\n\r\nDecompression process means that at some point application will require to allocate memory to store both compressed and decompressed response body and I will need to know this sizes to have more complete vision of scrapy memory usage.\r\n\r\nAlso size of  decompressed body will be several times more than size of compressed response and it will affect scrapy memory usage.\r\n\r\n## Describe alternatives you've considered\r\nThe easiest one - is to change priority of `DownloaderStats` middleware and check difference in `downloader/response_bytes` stats parameter.\r\n```\r\n    custom_settings = {\"DOWNLOAD_DELAY\":1,\r\n                       \"DOWNLOADER_MIDDLEWARES\":{\r\n                           'scrapy.downloadermiddlewares.stats.DownloaderStats':50\r\n                       }\r\n```\r\nStats from quotes.toscrape.com spider (it uses `gzip` compression) with default settings:\r\n\r\n```\r\n{'downloader/request_bytes': 2642,\r\n 'downloader/request_count': 10,\r\n 'downloader/request_method_count/GET': 10,\r\n 'downloader/response_bytes': 24534,\r\n```\r\n \r\nAnd with changed priority of `DownloaderStats` middleware:\r\n\r\n```\r\n{'downloader/request_bytes': 912, # size reduced as it didn't count size of request headers populated by downloader middlewares\r\n 'downloader/request_count': 10,\r\n 'downloader/request_method_count/GET': 10,\r\n 'downloader/response_bytes': 110191,   # it counted size of decompressed data \r\n```\r\n\r\nAverage size of compressed response (by default) - 2453 bytes.\r\nAverage size of decompressed response  - 11019 bytes (~4.5 times more).\r\n\r\n## Additional context\r\nPotential solution is to add something like this:\r\n` self.stats.inc_value('decompressed_bytes', spider=spider)`\r\ninto `process_response` method of `HttpCompressionMiddleware`\n", "before_files": [{"content": "import io\nimport zlib\n\nfrom scrapy.utils.gz import gunzip\nfrom scrapy.http import Response, TextResponse\nfrom scrapy.responsetypes import responsetypes\nfrom scrapy.exceptions import NotConfigured\n\n\nACCEPTED_ENCODINGS = [b'gzip', b'deflate']\n\ntry:\n    import brotli\n    ACCEPTED_ENCODINGS.append(b'br')\nexcept ImportError:\n    pass\n\ntry:\n    import zstandard\n    ACCEPTED_ENCODINGS.append(b'zstd')\nexcept ImportError:\n    pass\n\n\nclass HttpCompressionMiddleware:\n    \"\"\"This middleware allows compressed (gzip, deflate) traffic to be\n    sent/received from web sites\"\"\"\n    @classmethod\n    def from_crawler(cls, crawler):\n        if not crawler.settings.getbool('COMPRESSION_ENABLED'):\n            raise NotConfigured\n        return cls()\n\n    def process_request(self, request, spider):\n        request.headers.setdefault('Accept-Encoding',\n                                   b\", \".join(ACCEPTED_ENCODINGS))\n\n    def process_response(self, request, response, spider):\n\n        if request.method == 'HEAD':\n            return response\n        if isinstance(response, Response):\n            content_encoding = response.headers.getlist('Content-Encoding')\n            if content_encoding:\n                encoding = content_encoding.pop()\n                decoded_body = self._decode(response.body, encoding.lower())\n                respcls = responsetypes.from_args(\n                    headers=response.headers, url=response.url, body=decoded_body\n                )\n                kwargs = dict(cls=respcls, body=decoded_body)\n                if issubclass(respcls, TextResponse):\n                    # force recalculating the encoding until we make sure the\n                    # responsetypes guessing is reliable\n                    kwargs['encoding'] = None\n                response = response.replace(**kwargs)\n                if not content_encoding:\n                    del response.headers['Content-Encoding']\n\n        return response\n\n    def _decode(self, body, encoding):\n        if encoding == b'gzip' or encoding == b'x-gzip':\n            body = gunzip(body)\n\n        if encoding == b'deflate':\n            try:\n                body = zlib.decompress(body)\n            except zlib.error:\n                # ugly hack to work with raw deflate content that may\n                # be sent by microsoft servers. For more information, see:\n                # http://carsten.codimi.de/gzip.yaws/\n                # http://www.port80software.com/200ok/archive/2005/10/31/868.aspx\n                # http://www.gzip.org/zlib/zlib_faq.html#faq38\n                body = zlib.decompress(body, -15)\n        if encoding == b'br' and b'br' in ACCEPTED_ENCODINGS:\n            body = brotli.decompress(body)\n        if encoding == b'zstd' and b'zstd' in ACCEPTED_ENCODINGS:\n            # Using its streaming API since its simple API could handle only cases\n            # where there is content size data embedded in the frame\n            reader = zstandard.ZstdDecompressor().stream_reader(io.BytesIO(body))\n            body = reader.read()\n        return body\n", "path": "scrapy/downloadermiddlewares/httpcompression.py"}], "after_files": [{"content": "import io\nimport warnings\nimport zlib\n\nfrom scrapy.exceptions import NotConfigured\nfrom scrapy.http import Response, TextResponse\nfrom scrapy.responsetypes import responsetypes\nfrom scrapy.utils.deprecate import ScrapyDeprecationWarning\nfrom scrapy.utils.gz import gunzip\n\n\nACCEPTED_ENCODINGS = [b'gzip', b'deflate']\n\ntry:\n    import brotli\n    ACCEPTED_ENCODINGS.append(b'br')\nexcept ImportError:\n    pass\n\ntry:\n    import zstandard\n    ACCEPTED_ENCODINGS.append(b'zstd')\nexcept ImportError:\n    pass\n\n\nclass HttpCompressionMiddleware:\n    \"\"\"This middleware allows compressed (gzip, deflate) traffic to be\n    sent/received from web sites\"\"\"\n    def __init__(self, stats=None):\n        self.stats = stats\n\n    @classmethod\n    def from_crawler(cls, crawler):\n        if not crawler.settings.getbool('COMPRESSION_ENABLED'):\n            raise NotConfigured\n        try:\n            return cls(stats=crawler.stats)\n        except TypeError:\n            warnings.warn(\n                \"HttpCompressionMiddleware subclasses must either modify \"\n                \"their '__init__' method to support a 'stats' parameter or \"\n                \"reimplement the 'from_crawler' method.\",\n                ScrapyDeprecationWarning,\n            )\n            result = cls()\n            result.stats = crawler.stats\n            return result\n\n    def process_request(self, request, spider):\n        request.headers.setdefault('Accept-Encoding',\n                                   b\", \".join(ACCEPTED_ENCODINGS))\n\n    def process_response(self, request, response, spider):\n\n        if request.method == 'HEAD':\n            return response\n        if isinstance(response, Response):\n            content_encoding = response.headers.getlist('Content-Encoding')\n            if content_encoding:\n                encoding = content_encoding.pop()\n                decoded_body = self._decode(response.body, encoding.lower())\n                if self.stats:\n                    self.stats.inc_value('httpcompression/response_bytes', len(decoded_body), spider=spider)\n                    self.stats.inc_value('httpcompression/response_count', spider=spider)\n                respcls = responsetypes.from_args(\n                    headers=response.headers, url=response.url, body=decoded_body\n                )\n                kwargs = dict(cls=respcls, body=decoded_body)\n                if issubclass(respcls, TextResponse):\n                    # force recalculating the encoding until we make sure the\n                    # responsetypes guessing is reliable\n                    kwargs['encoding'] = None\n                response = response.replace(**kwargs)\n                if not content_encoding:\n                    del response.headers['Content-Encoding']\n\n        return response\n\n    def _decode(self, body, encoding):\n        if encoding == b'gzip' or encoding == b'x-gzip':\n            body = gunzip(body)\n\n        if encoding == b'deflate':\n            try:\n                body = zlib.decompress(body)\n            except zlib.error:\n                # ugly hack to work with raw deflate content that may\n                # be sent by microsoft servers. For more information, see:\n                # http://carsten.codimi.de/gzip.yaws/\n                # http://www.port80software.com/200ok/archive/2005/10/31/868.aspx\n                # http://www.gzip.org/zlib/zlib_faq.html#faq38\n                body = zlib.decompress(body, -15)\n        if encoding == b'br' and b'br' in ACCEPTED_ENCODINGS:\n            body = brotli.decompress(body)\n        if encoding == b'zstd' and b'zstd' in ACCEPTED_ENCODINGS:\n            # Using its streaming API since its simple API could handle only cases\n            # where there is content size data embedded in the frame\n            reader = zstandard.ZstdDecompressor().stream_reader(io.BytesIO(body))\n            body = reader.read()\n        return body\n", "path": "scrapy/downloadermiddlewares/httpcompression.py"}]}