We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
`StaticFiles` causes Internal Server Error when user accesses existing files as directory
### Checklist

<!-- Please make sure you check all these items before submitting your bug report. -->

- [x] The bug is reproducible against the latest release and/or `master`.
- [x] There are no similar issues or pull requests to fix it yet.

### Describe the bug

<!-- A clear and concise description of what the bug is. -->
`StaticFiles` causes Internal Server Error when user accesses existing files as directory (e.g. `/static/somefile.txt/foobar`)

### To reproduce

<!-- Provide a *minimal* example with steps to reproduce the bug locally.

NOTE: try to keep any external dependencies *at an absolute minimum*
(middleware, servers, proxies, certificates...).
In other words, remove anything that doesn't make the bug go away.
-->

1. create virtual env and activate it with `python -m venv venv && source venv/bin/activate`
2. install dependencies with `pip install starlette uvicorn aiofiles`
3. setup application as follows:

directory structure:
```
.
├── poc.py
├── static
│   └── sample.txt
└── venv
```

poc.py
```py
# code from https://www.starlette.io/staticfiles/
from starlette.applications import Starlette
from starlette.routing import Mount
from starlette.staticfiles import StaticFiles


routes = [
    Mount("/static", app=StaticFiles(directory="static"), name="static"),
]

app = Starlette(routes=routes)
```

4. run application with `uvicorn poc:app`
5. access `http://127.0.0.1:8000/static/sample.txt/foo`

### Expected behavior

<!-- A clear and concise description of what you expected to happen. -->
`StaticFiles` returns "404 Not Found" HTTP response (as Apache HTTP Server and Nginx does).

### Actual behavior

<!-- A clear and concise description of what actually happens. -->
`StaticFiles` returns "500 Internal Server Error" HTTP response.

### Debugging material

<!-- Any tracebacks, screenshots, etc. that can help understanding the problem.

NOTE:
- Please list tracebacks in full (don't truncate them).
- Consider using `<details>` to make tracebacks/logs collapsible if they're very large (see https://gist.github.com/ericclemmons/b146fe5da72ca1f706b2ef72a20ac39d).
-->
console log with tracebacks:
```
INFO:     Started server process [13052]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:64288 - "GET /static/sample.txt/foo HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 394, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/routing.py", line 582, in __call__
    await route.handle(scope, receive, send)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/routing.py", line 392, in handle
    await self.app(scope, receive, send)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/staticfiles.py", line 97, in __call__
    response = await self.get_response(path, scope)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/staticfiles.py", line 114, in get_response
    full_path, stat_result = await self.lookup_path(path)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/staticfiles.py", line 154, in lookup_path
    stat_result = await aio_stat(full_path)
  File "/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/aiofiles/os.py", line 13, in run
    return await loop.run_in_executor(executor, pfunc)
  File "/usr/local/Cellar/python@3.9/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
NotADirectoryError: [Errno 20] Not a directory: '/Users/xkhorasan/programs/test_starlette/static/sample.txt/foo'
```

### Environment

- OS: macOS
- Python version: 3.9.1
- Starlette version:  0.14.1

### Additional context

<!-- Any additional information that can help understanding the problem.

Eg. linked issues, or a description of what you were trying to achieve. -->

Apache HTTP Server and Nginx treat this case (access existing file as directory) as "404 Not Found".
Samples:
* Apache HTTP Server: https://httpd.apache.org/images/httpd_logo_wide_new.png/foo
* Nginx: https://nginx.org/download/nginx-1.18.0.tar.gz/foo
Return 404 Not Found for accessing file as directory pattern
fixes #1123

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `starlette/staticfiles.py`
Content:
```
1 import importlib.util
2 import os
3 import stat
4 import typing
5 from email.utils import parsedate
6 
7 import anyio
8 
9 from starlette.datastructures import URL, Headers
10 from starlette.responses import (
11     FileResponse,
12     PlainTextResponse,
13     RedirectResponse,
14     Response,
15 )
16 from starlette.types import Receive, Scope, Send
17 
18 PathLike = typing.Union[str, "os.PathLike[str]"]
19 
20 
21 class NotModifiedResponse(Response):
22     NOT_MODIFIED_HEADERS = (
23         "cache-control",
24         "content-location",
25         "date",
26         "etag",
27         "expires",
28         "vary",
29     )
30 
31     def __init__(self, headers: Headers):
32         super().__init__(
33             status_code=304,
34             headers={
35                 name: value
36                 for name, value in headers.items()
37                 if name in self.NOT_MODIFIED_HEADERS
38             },
39         )
40 
41 
42 class StaticFiles:
43     def __init__(
44         self,
45         *,
46         directory: PathLike = None,
47         packages: typing.List[str] = None,
48         html: bool = False,
49         check_dir: bool = True,
50     ) -> None:
51         self.directory = directory
52         self.packages = packages
53         self.all_directories = self.get_directories(directory, packages)
54         self.html = html
55         self.config_checked = False
56         if check_dir and directory is not None and not os.path.isdir(directory):
57             raise RuntimeError(f"Directory '{directory}' does not exist")
58 
59     def get_directories(
60         self, directory: PathLike = None, packages: typing.List[str] = None
61     ) -> typing.List[PathLike]:
62         """
63         Given `directory` and `packages` arguments, return a list of all the
64         directories that should be used for serving static files from.
65         """
66         directories = []
67         if directory is not None:
68             directories.append(directory)
69 
70         for package in packages or []:
71             spec = importlib.util.find_spec(package)
72             assert spec is not None, f"Package {package!r} could not be found."
73             assert (
74                 spec.origin is not None
75             ), f"Directory 'statics' in package {package!r} could not be found."
76             package_directory = os.path.normpath(
77                 os.path.join(spec.origin, "..", "statics")
78             )
79             assert os.path.isdir(
80                 package_directory
81             ), f"Directory 'statics' in package {package!r} could not be found."
82             directories.append(package_directory)
83 
84         return directories
85 
86     async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
87         """
88         The ASGI entry point.
89         """
90         assert scope["type"] == "http"
91 
92         if not self.config_checked:
93             await self.check_config()
94             self.config_checked = True
95 
96         path = self.get_path(scope)
97         response = await self.get_response(path, scope)
98         await response(scope, receive, send)
99 
100     def get_path(self, scope: Scope) -> str:
101         """
102         Given the ASGI scope, return the `path` string to serve up,
103         with OS specific path seperators, and any '..', '.' components removed.
104         """
105         return os.path.normpath(os.path.join(*scope["path"].split("/")))
106 
107     async def get_response(self, path: str, scope: Scope) -> Response:
108         """
109         Returns an HTTP response, given the incoming path, method and request headers.
110         """
111         if scope["method"] not in ("GET", "HEAD"):
112             return PlainTextResponse("Method Not Allowed", status_code=405)
113 
114         full_path, stat_result = await self.lookup_path(path)
115 
116         if stat_result and stat.S_ISREG(stat_result.st_mode):
117             # We have a static file to serve.
118             return self.file_response(full_path, stat_result, scope)
119 
120         elif stat_result and stat.S_ISDIR(stat_result.st_mode) and self.html:
121             # We're in HTML mode, and have got a directory URL.
122             # Check if we have 'index.html' file to serve.
123             index_path = os.path.join(path, "index.html")
124             full_path, stat_result = await self.lookup_path(index_path)
125             if stat_result is not None and stat.S_ISREG(stat_result.st_mode):
126                 if not scope["path"].endswith("/"):
127                     # Directory URLs should redirect to always end in "/".
128                     url = URL(scope=scope)
129                     url = url.replace(path=url.path + "/")
130                     return RedirectResponse(url=url)
131                 return self.file_response(full_path, stat_result, scope)
132 
133         if self.html:
134             # Check for '404.html' if we're in HTML mode.
135             full_path, stat_result = await self.lookup_path("404.html")
136             if stat_result is not None and stat.S_ISREG(stat_result.st_mode):
137                 return FileResponse(
138                     full_path,
139                     stat_result=stat_result,
140                     method=scope["method"],
141                     status_code=404,
142                 )
143 
144         return PlainTextResponse("Not Found", status_code=404)
145 
146     async def lookup_path(
147         self, path: str
148     ) -> typing.Tuple[str, typing.Optional[os.stat_result]]:
149         for directory in self.all_directories:
150             full_path = os.path.realpath(os.path.join(directory, path))
151             directory = os.path.realpath(directory)
152             if os.path.commonprefix([full_path, directory]) != directory:
153                 # Don't allow misbehaving clients to break out of the static files
154                 # directory.
155                 continue
156             try:
157                 stat_result = await anyio.to_thread.run_sync(os.stat, full_path)
158                 return full_path, stat_result
159             except FileNotFoundError:
160                 pass
161         return "", None
162 
163     def file_response(
164         self,
165         full_path: PathLike,
166         stat_result: os.stat_result,
167         scope: Scope,
168         status_code: int = 200,
169     ) -> Response:
170         method = scope["method"]
171         request_headers = Headers(scope=scope)
172 
173         response = FileResponse(
174             full_path, status_code=status_code, stat_result=stat_result, method=method
175         )
176         if self.is_not_modified(response.headers, request_headers):
177             return NotModifiedResponse(response.headers)
178         return response
179 
180     async def check_config(self) -> None:
181         """
182         Perform a one-off configuration check that StaticFiles is actually
183         pointed at a directory, so that we can raise loud errors rather than
184         just returning 404 responses.
185         """
186         if self.directory is None:
187             return
188 
189         try:
190             stat_result = await anyio.to_thread.run_sync(os.stat, self.directory)
191         except FileNotFoundError:
192             raise RuntimeError(
193                 f"StaticFiles directory '{self.directory}' does not exist."
194             )
195         if not (stat.S_ISDIR(stat_result.st_mode) or stat.S_ISLNK(stat_result.st_mode)):
196             raise RuntimeError(
197                 f"StaticFiles path '{self.directory}' is not a directory."
198             )
199 
200     def is_not_modified(
201         self, response_headers: Headers, request_headers: Headers
202     ) -> bool:
203         """
204         Given the request and response headers, return `True` if an HTTP
205         "Not Modified" response could be returned instead.
206         """
207         try:
208             if_none_match = request_headers["if-none-match"]
209             etag = response_headers["etag"]
210             if if_none_match == etag:
211                 return True
212         except KeyError:
213             pass
214 
215         try:
216             if_modified_since = parsedate(request_headers["if-modified-since"])
217             last_modified = parsedate(response_headers["last-modified"])
218             if (
219                 if_modified_since is not None
220                 and last_modified is not None
221                 and if_modified_since >= last_modified
222             ):
223                 return True
224         except KeyError:
225             pass
226 
227         return False
228
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/starlette/staticfiles.py b/starlette/staticfiles.py
--- a/starlette/staticfiles.py
+++ b/starlette/staticfiles.py
@@ -7,12 +7,8 @@
 import anyio
 
 from starlette.datastructures import URL, Headers
-from starlette.responses import (
-    FileResponse,
-    PlainTextResponse,
-    RedirectResponse,
-    Response,
-)
+from starlette.exceptions import HTTPException
+from starlette.responses import FileResponse, RedirectResponse, Response
 from starlette.types import Receive, Scope, Send
 
 PathLike = typing.Union[str, "os.PathLike[str]"]
@@ -109,9 +105,30 @@
         Returns an HTTP response, given the incoming path, method and request headers.
         """
         if scope["method"] not in ("GET", "HEAD"):
-            return PlainTextResponse("Method Not Allowed", status_code=405)
+            raise HTTPException(status_code=405)
 
-        full_path, stat_result = await self.lookup_path(path)
+        try:
+            full_path, stat_result = await anyio.to_thread.run_sync(
+                self.lookup_path, path
+            )
+        except (FileNotFoundError, NotADirectoryError):
+            if self.html:
+                # Check for '404.html' if we're in HTML mode.
+                full_path, stat_result = await anyio.to_thread.run_sync(
+                    self.lookup_path, "404.html"
+                )
+                if stat_result and stat.S_ISREG(stat_result.st_mode):
+                    return FileResponse(
+                        full_path,
+                        stat_result=stat_result,
+                        method=scope["method"],
+                        status_code=404,
+                    )
+            raise HTTPException(status_code=404)
+        except PermissionError:
+            raise HTTPException(status_code=401)
+        except OSError:
+            raise
 
         if stat_result and stat.S_ISREG(stat_result.st_mode):
             # We have a static file to serve.
@@ -121,7 +138,9 @@
             # We're in HTML mode, and have got a directory URL.
             # Check if we have 'index.html' file to serve.
             index_path = os.path.join(path, "index.html")
-            full_path, stat_result = await self.lookup_path(index_path)
+            full_path, stat_result = await anyio.to_thread.run_sync(
+                self.lookup_path, index_path
+            )
             if stat_result is not None and stat.S_ISREG(stat_result.st_mode):
                 if not scope["path"].endswith("/"):
                     # Directory URLs should redirect to always end in "/".
@@ -130,20 +149,9 @@
                     return RedirectResponse(url=url)
                 return self.file_response(full_path, stat_result, scope)
 
-        if self.html:
-            # Check for '404.html' if we're in HTML mode.
-            full_path, stat_result = await self.lookup_path("404.html")
-            if stat_result is not None and stat.S_ISREG(stat_result.st_mode):
-                return FileResponse(
-                    full_path,
-                    stat_result=stat_result,
-                    method=scope["method"],
-                    status_code=404,
-                )
-
-        return PlainTextResponse("Not Found", status_code=404)
+        raise HTTPException(status_code=404)
 
-    async def lookup_path(
+    def lookup_path(
         self, path: str
     ) -> typing.Tuple[str, typing.Optional[os.stat_result]]:
         for directory in self.all_directories:
@@ -153,11 +161,7 @@
                 # Don't allow misbehaving clients to break out of the static files
                 # directory.
                 continue
-            try:
-                stat_result = await anyio.to_thread.run_sync(os.stat, full_path)
-                return full_path, stat_result
-            except FileNotFoundError:
-                pass
+            return full_path, os.stat(full_path)
         return "", None
 
     def file_response(

{"golden_diff": "diff --git a/starlette/staticfiles.py b/starlette/staticfiles.py\n--- a/starlette/staticfiles.py\n+++ b/starlette/staticfiles.py\n@@ -7,12 +7,8 @@\n import anyio\n \n from starlette.datastructures import URL, Headers\n-from starlette.responses import (\n-    FileResponse,\n-    PlainTextResponse,\n-    RedirectResponse,\n-    Response,\n-)\n+from starlette.exceptions import HTTPException\n+from starlette.responses import FileResponse, RedirectResponse, Response\n from starlette.types import Receive, Scope, Send\n \n PathLike = typing.Union[str, \"os.PathLike[str]\"]\n@@ -109,9 +105,30 @@\n         Returns an HTTP response, given the incoming path, method and request headers.\n         \"\"\"\n         if scope[\"method\"] not in (\"GET\", \"HEAD\"):\n-            return PlainTextResponse(\"Method Not Allowed\", status_code=405)\n+            raise HTTPException(status_code=405)\n \n-        full_path, stat_result = await self.lookup_path(path)\n+        try:\n+            full_path, stat_result = await anyio.to_thread.run_sync(\n+                self.lookup_path, path\n+            )\n+        except (FileNotFoundError, NotADirectoryError):\n+            if self.html:\n+                # Check for '404.html' if we're in HTML mode.\n+                full_path, stat_result = await anyio.to_thread.run_sync(\n+                    self.lookup_path, \"404.html\"\n+                )\n+                if stat_result and stat.S_ISREG(stat_result.st_mode):\n+                    return FileResponse(\n+                        full_path,\n+                        stat_result=stat_result,\n+                        method=scope[\"method\"],\n+                        status_code=404,\n+                    )\n+            raise HTTPException(status_code=404)\n+        except PermissionError:\n+            raise HTTPException(status_code=401)\n+        except OSError:\n+            raise\n \n         if stat_result and stat.S_ISREG(stat_result.st_mode):\n             # We have a static file to serve.\n@@ -121,7 +138,9 @@\n             # We're in HTML mode, and have got a directory URL.\n             # Check if we have 'index.html' file to serve.\n             index_path = os.path.join(path, \"index.html\")\n-            full_path, stat_result = await self.lookup_path(index_path)\n+            full_path, stat_result = await anyio.to_thread.run_sync(\n+                self.lookup_path, index_path\n+            )\n             if stat_result is not None and stat.S_ISREG(stat_result.st_mode):\n                 if not scope[\"path\"].endswith(\"/\"):\n                     # Directory URLs should redirect to always end in \"/\".\n@@ -130,20 +149,9 @@\n                     return RedirectResponse(url=url)\n                 return self.file_response(full_path, stat_result, scope)\n \n-        if self.html:\n-            # Check for '404.html' if we're in HTML mode.\n-            full_path, stat_result = await self.lookup_path(\"404.html\")\n-            if stat_result is not None and stat.S_ISREG(stat_result.st_mode):\n-                return FileResponse(\n-                    full_path,\n-                    stat_result=stat_result,\n-                    method=scope[\"method\"],\n-                    status_code=404,\n-                )\n-\n-        return PlainTextResponse(\"Not Found\", status_code=404)\n+        raise HTTPException(status_code=404)\n \n-    async def lookup_path(\n+    def lookup_path(\n         self, path: str\n     ) -> typing.Tuple[str, typing.Optional[os.stat_result]]:\n         for directory in self.all_directories:\n@@ -153,11 +161,7 @@\n                 # Don't allow misbehaving clients to break out of the static files\n                 # directory.\n                 continue\n-            try:\n-                stat_result = await anyio.to_thread.run_sync(os.stat, full_path)\n-                return full_path, stat_result\n-            except FileNotFoundError:\n-                pass\n+            return full_path, os.stat(full_path)\n         return \"\", None\n \n     def file_response(\n", "issue": "`StaticFiles` causes Internal Server Error when user accesses existing files as directory\n### Checklist\r\n\r\n<!-- Please make sure you check all these items before submitting your bug report. -->\r\n\r\n- [x] The bug is reproducible against the latest release and/or `master`.\r\n- [x] There are no similar issues or pull requests to fix it yet.\r\n\r\n### Describe the bug\r\n\r\n<!-- A clear and concise description of what the bug is. -->\r\n`StaticFiles` causes Internal Server Error when user accesses existing files as directory (e.g. `/static/somefile.txt/foobar`)\r\n\r\n### To reproduce\r\n\r\n<!-- Provide a *minimal* example with steps to reproduce the bug locally.\r\n\r\nNOTE: try to keep any external dependencies *at an absolute minimum*\r\n(middleware, servers, proxies, certificates...).\r\nIn other words, remove anything that doesn't make the bug go away.\r\n-->\r\n\r\n1. create virtual env and activate it with `python -m venv venv && source venv/bin/activate`\r\n2. install dependencies with `pip install starlette uvicorn aiofiles`\r\n3. setup application as follows:\r\n\r\ndirectory structure:\r\n```\r\n.\r\n\u251c\u2500\u2500 poc.py\r\n\u251c\u2500\u2500 static\r\n\u2502   \u2514\u2500\u2500 sample.txt\r\n\u2514\u2500\u2500 venv\r\n```\r\n\r\npoc.py\r\n```py\r\n# code from https://www.starlette.io/staticfiles/\r\nfrom starlette.applications import Starlette\r\nfrom starlette.routing import Mount\r\nfrom starlette.staticfiles import StaticFiles\r\n\r\n\r\nroutes = [\r\n    Mount(\"/static\", app=StaticFiles(directory=\"static\"), name=\"static\"),\r\n]\r\n\r\napp = Starlette(routes=routes)\r\n```\r\n\r\n4. run application with `uvicorn poc:app`\r\n5. access `http://127.0.0.1:8000/static/sample.txt/foo`\r\n\r\n### Expected behavior\r\n\r\n<!-- A clear and concise description of what you expected to happen. -->\r\n`StaticFiles` returns \"404 Not Found\" HTTP response (as Apache HTTP Server and Nginx does).\r\n\r\n### Actual behavior\r\n\r\n<!-- A clear and concise description of what actually happens. -->\r\n`StaticFiles` returns \"500 Internal Server Error\" HTTP response.\r\n\r\n### Debugging material\r\n\r\n<!-- Any tracebacks, screenshots, etc. that can help understanding the problem.\r\n\r\nNOTE:\r\n- Please list tracebacks in full (don't truncate them).\r\n- Consider using `<details>` to make tracebacks/logs collapsible if they're very large (see https://gist.github.com/ericclemmons/b146fe5da72ca1f706b2ef72a20ac39d).\r\n-->\r\nconsole log with tracebacks:\r\n```\r\nINFO:     Started server process [13052]\r\nINFO:     Waiting for application startup.\r\nINFO:     Application startup complete.\r\nINFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\r\nINFO:     127.0.0.1:64288 - \"GET /static/sample.txt/foo HTTP/1.1\" 500 Internal Server Error\r\nERROR:    Exception in ASGI application\r\nTraceback (most recent call last):\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py\", line 394, in run_asgi\r\n    result = await app(self.scope, self.receive, self.send)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py\", line 45, in __call__\r\n    return await self.app(scope, receive, send)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/applications.py\", line 112, in __call__\r\n    await self.middleware_stack(scope, receive, send)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/middleware/errors.py\", line 181, in __call__\r\n    raise exc from None\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/middleware/errors.py\", line 159, in __call__\r\n    await self.app(scope, receive, _send)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/exceptions.py\", line 82, in __call__\r\n    raise exc from None\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/exceptions.py\", line 71, in __call__\r\n    await self.app(scope, receive, sender)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/routing.py\", line 582, in __call__\r\n    await route.handle(scope, receive, send)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/routing.py\", line 392, in handle\r\n    await self.app(scope, receive, send)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/staticfiles.py\", line 97, in __call__\r\n    response = await self.get_response(path, scope)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/staticfiles.py\", line 114, in get_response\r\n    full_path, stat_result = await self.lookup_path(path)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/starlette/staticfiles.py\", line 154, in lookup_path\r\n    stat_result = await aio_stat(full_path)\r\n  File \"/Users/xkhorasan/programs/test_starlette/venv/lib/python3.9/site-packages/aiofiles/os.py\", line 13, in run\r\n    return await loop.run_in_executor(executor, pfunc)\r\n  File \"/usr/local/Cellar/python@3.9/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py\", line 52, in run\r\n    result = self.fn(*self.args, **self.kwargs)\r\nNotADirectoryError: [Errno 20] Not a directory: '/Users/xkhorasan/programs/test_starlette/static/sample.txt/foo'\r\n```\r\n\r\n### Environment\r\n\r\n- OS: macOS\r\n- Python version: 3.9.1\r\n- Starlette version:  0.14.1\r\n\r\n### Additional context\r\n\r\n<!-- Any additional information that can help understanding the problem.\r\n\r\nEg. linked issues, or a description of what you were trying to achieve. -->\r\n\r\nApache HTTP Server and Nginx treat this case (access existing file as directory) as \"404 Not Found\".\r\nSamples:\r\n* Apache HTTP Server: https://httpd.apache.org/images/httpd_logo_wide_new.png/foo\r\n* Nginx: https://nginx.org/download/nginx-1.18.0.tar.gz/foo\nReturn 404 Not Found for accessing file as directory pattern\nfixes #1123\n", "before_files": [{"content": "import importlib.util\nimport os\nimport stat\nimport typing\nfrom email.utils import parsedate\n\nimport anyio\n\nfrom starlette.datastructures import URL, Headers\nfrom starlette.responses import (\n    FileResponse,\n    PlainTextResponse,\n    RedirectResponse,\n    Response,\n)\nfrom starlette.types import Receive, Scope, Send\n\nPathLike = typing.Union[str, \"os.PathLike[str]\"]\n\n\nclass NotModifiedResponse(Response):\n    NOT_MODIFIED_HEADERS = (\n        \"cache-control\",\n        \"content-location\",\n        \"date\",\n        \"etag\",\n        \"expires\",\n        \"vary\",\n    )\n\n    def __init__(self, headers: Headers):\n        super().__init__(\n            status_code=304,\n            headers={\n                name: value\n                for name, value in headers.items()\n                if name in self.NOT_MODIFIED_HEADERS\n            },\n        )\n\n\nclass StaticFiles:\n    def __init__(\n        self,\n        *,\n        directory: PathLike = None,\n        packages: typing.List[str] = None,\n        html: bool = False,\n        check_dir: bool = True,\n    ) -> None:\n        self.directory = directory\n        self.packages = packages\n        self.all_directories = self.get_directories(directory, packages)\n        self.html = html\n        self.config_checked = False\n        if check_dir and directory is not None and not os.path.isdir(directory):\n            raise RuntimeError(f\"Directory '{directory}' does not exist\")\n\n    def get_directories(\n        self, directory: PathLike = None, packages: typing.List[str] = None\n    ) -> typing.List[PathLike]:\n        \"\"\"\n        Given `directory` and `packages` arguments, return a list of all the\n        directories that should be used for serving static files from.\n        \"\"\"\n        directories = []\n        if directory is not None:\n            directories.append(directory)\n\n        for package in packages or []:\n            spec = importlib.util.find_spec(package)\n            assert spec is not None, f\"Package {package!r} could not be found.\"\n            assert (\n                spec.origin is not None\n            ), f\"Directory 'statics' in package {package!r} could not be found.\"\n            package_directory = os.path.normpath(\n                os.path.join(spec.origin, \"..\", \"statics\")\n            )\n            assert os.path.isdir(\n                package_directory\n            ), f\"Directory 'statics' in package {package!r} could not be found.\"\n            directories.append(package_directory)\n\n        return directories\n\n    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:\n        \"\"\"\n        The ASGI entry point.\n        \"\"\"\n        assert scope[\"type\"] == \"http\"\n\n        if not self.config_checked:\n            await self.check_config()\n            self.config_checked = True\n\n        path = self.get_path(scope)\n        response = await self.get_response(path, scope)\n        await response(scope, receive, send)\n\n    def get_path(self, scope: Scope) -> str:\n        \"\"\"\n        Given the ASGI scope, return the `path` string to serve up,\n        with OS specific path seperators, and any '..', '.' components removed.\n        \"\"\"\n        return os.path.normpath(os.path.join(*scope[\"path\"].split(\"/\")))\n\n    async def get_response(self, path: str, scope: Scope) -> Response:\n        \"\"\"\n        Returns an HTTP response, given the incoming path, method and request headers.\n        \"\"\"\n        if scope[\"method\"] not in (\"GET\", \"HEAD\"):\n            return PlainTextResponse(\"Method Not Allowed\", status_code=405)\n\n        full_path, stat_result = await self.lookup_path(path)\n\n        if stat_result and stat.S_ISREG(stat_result.st_mode):\n            # We have a static file to serve.\n            return self.file_response(full_path, stat_result, scope)\n\n        elif stat_result and stat.S_ISDIR(stat_result.st_mode) and self.html:\n            # We're in HTML mode, and have got a directory URL.\n            # Check if we have 'index.html' file to serve.\n            index_path = os.path.join(path, \"index.html\")\n            full_path, stat_result = await self.lookup_path(index_path)\n            if stat_result is not None and stat.S_ISREG(stat_result.st_mode):\n                if not scope[\"path\"].endswith(\"/\"):\n                    # Directory URLs should redirect to always end in \"/\".\n                    url = URL(scope=scope)\n                    url = url.replace(path=url.path + \"/\")\n                    return RedirectResponse(url=url)\n                return self.file_response(full_path, stat_result, scope)\n\n        if self.html:\n            # Check for '404.html' if we're in HTML mode.\n            full_path, stat_result = await self.lookup_path(\"404.html\")\n            if stat_result is not None and stat.S_ISREG(stat_result.st_mode):\n                return FileResponse(\n                    full_path,\n                    stat_result=stat_result,\n                    method=scope[\"method\"],\n                    status_code=404,\n                )\n\n        return PlainTextResponse(\"Not Found\", status_code=404)\n\n    async def lookup_path(\n        self, path: str\n    ) -> typing.Tuple[str, typing.Optional[os.stat_result]]:\n        for directory in self.all_directories:\n            full_path = os.path.realpath(os.path.join(directory, path))\n            directory = os.path.realpath(directory)\n            if os.path.commonprefix([full_path, directory]) != directory:\n                # Don't allow misbehaving clients to break out of the static files\n                # directory.\n                continue\n            try:\n                stat_result = await anyio.to_thread.run_sync(os.stat, full_path)\n                return full_path, stat_result\n            except FileNotFoundError:\n                pass\n        return \"\", None\n\n    def file_response(\n        self,\n        full_path: PathLike,\n        stat_result: os.stat_result,\n        scope: Scope,\n        status_code: int = 200,\n    ) -> Response:\n        method = scope[\"method\"]\n        request_headers = Headers(scope=scope)\n\n        response = FileResponse(\n            full_path, status_code=status_code, stat_result=stat_result, method=method\n        )\n        if self.is_not_modified(response.headers, request_headers):\n            return NotModifiedResponse(response.headers)\n        return response\n\n    async def check_config(self) -> None:\n        \"\"\"\n        Perform a one-off configuration check that StaticFiles is actually\n        pointed at a directory, so that we can raise loud errors rather than\n        just returning 404 responses.\n        \"\"\"\n        if self.directory is None:\n            return\n\n        try:\n            stat_result = await anyio.to_thread.run_sync(os.stat, self.directory)\n        except FileNotFoundError:\n            raise RuntimeError(\n                f\"StaticFiles directory '{self.directory}' does not exist.\"\n            )\n        if not (stat.S_ISDIR(stat_result.st_mode) or stat.S_ISLNK(stat_result.st_mode)):\n            raise RuntimeError(\n                f\"StaticFiles path '{self.directory}' is not a directory.\"\n            )\n\n    def is_not_modified(\n        self, response_headers: Headers, request_headers: Headers\n    ) -> bool:\n        \"\"\"\n        Given the request and response headers, return `True` if an HTTP\n        \"Not Modified\" response could be returned instead.\n        \"\"\"\n        try:\n            if_none_match = request_headers[\"if-none-match\"]\n            etag = response_headers[\"etag\"]\n            if if_none_match == etag:\n                return True\n        except KeyError:\n            pass\n\n        try:\n            if_modified_since = parsedate(request_headers[\"if-modified-since\"])\n            last_modified = parsedate(response_headers[\"last-modified\"])\n            if (\n                if_modified_since is not None\n                and last_modified is not None\n                and if_modified_since >= last_modified\n            ):\n                return True\n        except KeyError:\n            pass\n\n        return False\n", "path": "starlette/staticfiles.py"}], "after_files": [{"content": "import importlib.util\nimport os\nimport stat\nimport typing\nfrom email.utils import parsedate\n\nimport anyio\n\nfrom starlette.datastructures import URL, Headers\nfrom starlette.exceptions import HTTPException\nfrom starlette.responses import FileResponse, RedirectResponse, Response\nfrom starlette.types import Receive, Scope, Send\n\nPathLike = typing.Union[str, \"os.PathLike[str]\"]\n\n\nclass NotModifiedResponse(Response):\n    NOT_MODIFIED_HEADERS = (\n        \"cache-control\",\n        \"content-location\",\n        \"date\",\n        \"etag\",\n        \"expires\",\n        \"vary\",\n    )\n\n    def __init__(self, headers: Headers):\n        super().__init__(\n            status_code=304,\n            headers={\n                name: value\n                for name, value in headers.items()\n                if name in self.NOT_MODIFIED_HEADERS\n            },\n        )\n\n\nclass StaticFiles:\n    def __init__(\n        self,\n        *,\n        directory: PathLike = None,\n        packages: typing.List[str] = None,\n        html: bool = False,\n        check_dir: bool = True,\n    ) -> None:\n        self.directory = directory\n        self.packages = packages\n        self.all_directories = self.get_directories(directory, packages)\n        self.html = html\n        self.config_checked = False\n        if check_dir and directory is not None and not os.path.isdir(directory):\n            raise RuntimeError(f\"Directory '{directory}' does not exist\")\n\n    def get_directories(\n        self, directory: PathLike = None, packages: typing.List[str] = None\n    ) -> typing.List[PathLike]:\n        \"\"\"\n        Given `directory` and `packages` arguments, return a list of all the\n        directories that should be used for serving static files from.\n        \"\"\"\n        directories = []\n        if directory is not None:\n            directories.append(directory)\n\n        for package in packages or []:\n            spec = importlib.util.find_spec(package)\n            assert spec is not None, f\"Package {package!r} could not be found.\"\n            assert (\n                spec.origin is not None\n            ), f\"Directory 'statics' in package {package!r} could not be found.\"\n            package_directory = os.path.normpath(\n                os.path.join(spec.origin, \"..\", \"statics\")\n            )\n            assert os.path.isdir(\n                package_directory\n            ), f\"Directory 'statics' in package {package!r} could not be found.\"\n            directories.append(package_directory)\n\n        return directories\n\n    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:\n        \"\"\"\n        The ASGI entry point.\n        \"\"\"\n        assert scope[\"type\"] == \"http\"\n\n        if not self.config_checked:\n            await self.check_config()\n            self.config_checked = True\n\n        path = self.get_path(scope)\n        response = await self.get_response(path, scope)\n        await response(scope, receive, send)\n\n    def get_path(self, scope: Scope) -> str:\n        \"\"\"\n        Given the ASGI scope, return the `path` string to serve up,\n        with OS specific path seperators, and any '..', '.' components removed.\n        \"\"\"\n        return os.path.normpath(os.path.join(*scope[\"path\"].split(\"/\")))\n\n    async def get_response(self, path: str, scope: Scope) -> Response:\n        \"\"\"\n        Returns an HTTP response, given the incoming path, method and request headers.\n        \"\"\"\n        if scope[\"method\"] not in (\"GET\", \"HEAD\"):\n            raise HTTPException(status_code=405)\n\n        try:\n            full_path, stat_result = await anyio.to_thread.run_sync(\n                self.lookup_path, path\n            )\n        except (FileNotFoundError, NotADirectoryError):\n            if self.html:\n                # Check for '404.html' if we're in HTML mode.\n                full_path, stat_result = await anyio.to_thread.run_sync(\n                    self.lookup_path, \"404.html\"\n                )\n                if stat_result and stat.S_ISREG(stat_result.st_mode):\n                    return FileResponse(\n                        full_path,\n                        stat_result=stat_result,\n                        method=scope[\"method\"],\n                        status_code=404,\n                    )\n            raise HTTPException(status_code=404)\n        except PermissionError:\n            raise HTTPException(status_code=401)\n        except OSError:\n            raise\n\n        if stat_result and stat.S_ISREG(stat_result.st_mode):\n            # We have a static file to serve.\n            return self.file_response(full_path, stat_result, scope)\n\n        elif stat_result and stat.S_ISDIR(stat_result.st_mode) and self.html:\n            # We're in HTML mode, and have got a directory URL.\n            # Check if we have 'index.html' file to serve.\n            index_path = os.path.join(path, \"index.html\")\n            full_path, stat_result = await anyio.to_thread.run_sync(\n                self.lookup_path, index_path\n            )\n            if stat_result is not None and stat.S_ISREG(stat_result.st_mode):\n                if not scope[\"path\"].endswith(\"/\"):\n                    # Directory URLs should redirect to always end in \"/\".\n                    url = URL(scope=scope)\n                    url = url.replace(path=url.path + \"/\")\n                    return RedirectResponse(url=url)\n                return self.file_response(full_path, stat_result, scope)\n\n        raise HTTPException(status_code=404)\n\n    def lookup_path(\n        self, path: str\n    ) -> typing.Tuple[str, typing.Optional[os.stat_result]]:\n        for directory in self.all_directories:\n            full_path = os.path.realpath(os.path.join(directory, path))\n            directory = os.path.realpath(directory)\n            if os.path.commonprefix([full_path, directory]) != directory:\n                # Don't allow misbehaving clients to break out of the static files\n                # directory.\n                continue\n            return full_path, os.stat(full_path)\n        return \"\", None\n\n    def file_response(\n        self,\n        full_path: PathLike,\n        stat_result: os.stat_result,\n        scope: Scope,\n        status_code: int = 200,\n    ) -> Response:\n        method = scope[\"method\"]\n        request_headers = Headers(scope=scope)\n\n        response = FileResponse(\n            full_path, status_code=status_code, stat_result=stat_result, method=method\n        )\n        if self.is_not_modified(response.headers, request_headers):\n            return NotModifiedResponse(response.headers)\n        return response\n\n    async def check_config(self) -> None:\n        \"\"\"\n        Perform a one-off configuration check that StaticFiles is actually\n        pointed at a directory, so that we can raise loud errors rather than\n        just returning 404 responses.\n        \"\"\"\n        if self.directory is None:\n            return\n\n        try:\n            stat_result = await anyio.to_thread.run_sync(os.stat, self.directory)\n        except FileNotFoundError:\n            raise RuntimeError(\n                f\"StaticFiles directory '{self.directory}' does not exist.\"\n            )\n        if not (stat.S_ISDIR(stat_result.st_mode) or stat.S_ISLNK(stat_result.st_mode)):\n            raise RuntimeError(\n                f\"StaticFiles path '{self.directory}' is not a directory.\"\n            )\n\n    def is_not_modified(\n        self, response_headers: Headers, request_headers: Headers\n    ) -> bool:\n        \"\"\"\n        Given the request and response headers, return `True` if an HTTP\n        \"Not Modified\" response could be returned instead.\n        \"\"\"\n        try:\n            if_none_match = request_headers[\"if-none-match\"]\n            etag = response_headers[\"etag\"]\n            if if_none_match == etag:\n                return True\n        except KeyError:\n            pass\n\n        try:\n            if_modified_since = parsedate(request_headers[\"if-modified-since\"])\n            last_modified = parsedate(response_headers[\"last-modified\"])\n            if (\n                if_modified_since is not None\n                and last_modified is not None\n                and if_modified_since >= last_modified\n            ):\n                return True\n        except KeyError:\n            pass\n\n        return False\n", "path": "starlette/staticfiles.py"}]}