We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
T - Update robots.tx (Warnings have spiked on Search Console)
**Summary**
We are seeing a large spike in warnings on Google Search Console. The spike started on December 22, 2019.

<img width="952" alt="Coverage" src="https://user-images.githubusercontent.com/557852/72343521-5fb44000-36cf-11ea-8f1f-12955b98e34d.png">

**Steps To Reproduce (STR)**
1. Go to Search Console
2. Open Coverage Report
3. open "Indexed, though blocked by robots.txt"


**Actual behavior**
It looks like a bunch of URLs are blocked from indexing by Google that should not be blocked. Examples are:
* https://developer.mozilla.org/it/docs/Web/API/GlobalEventHandlers/onresize$translate
* https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/end%24edit
* https://developer.mozilla.org/en-US/docs/Web/API/RTCIceCandidateInit/sdpMid$translate?tolocale=de

When I test these URLs in Search Console's robots.txt tester it tells me that those URLs are allowed, and they should be. $translate, or $edit are not in robots.txt


**Expected behavior**
Those URLs should not be blocked, so that Google can crawl the pages, discover the no-index meta tag and remove them from its index.


**Additional context**
I assume that something is broken when it comes to handling the $ chracter in the URL. Maybe we just remove everything with the $url from our robots.txt. As far as I can tell, almost all of those URLs now live under wiki.developer.mozilla.org anyway.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `kuma/landing/views.py`
Content:
```
1 from django.conf import settings
2 from django.http import HttpResponse
3 from django.shortcuts import redirect, render
4 from django.views import static
5 from django.views.decorators.cache import never_cache
6 from django.views.generic import RedirectView
7 
8 from kuma.core.decorators import ensure_wiki_domain, shared_cache_control
9 from kuma.core.utils import is_wiki
10 from kuma.feeder.models import Bundle
11 from kuma.feeder.sections import SECTION_HACKS
12 from kuma.search.models import Filter
13 
14 from .utils import favicon_url
15 
16 
17 @shared_cache_control
18 def contribute_json(request):
19     return static.serve(request, "contribute.json", document_root=settings.ROOT)
20 
21 
22 @shared_cache_control
23 def home(request):
24     """Home page."""
25     context = {}
26     # Need for both wiki and react homepage
27     context["updates"] = list(Bundle.objects.recent_entries(SECTION_HACKS.updates)[:5])
28 
29     # The default template name
30     template_name = "landing/react_homepage.html"
31     if is_wiki(request):
32         template_name = "landing/homepage.html"
33         context["default_filters"] = Filter.objects.default_filters()
34     return render(request, template_name, context)
35 
36 
37 @ensure_wiki_domain
38 @never_cache
39 def maintenance_mode(request):
40     if settings.MAINTENANCE_MODE:
41         return render(request, "landing/maintenance-mode.html")
42     else:
43         return redirect("home")
44 
45 
46 @ensure_wiki_domain
47 @shared_cache_control
48 def promote_buttons(request):
49     """Bug 646192: MDN affiliate buttons"""
50     return render(request, "landing/promote_buttons.html")
51 
52 
53 ROBOTS_ALLOWED_TXT = """\
54 User-agent: *
55 Sitemap: https://developer.mozilla.org/sitemap.xml
56 
57 Disallow: /admin/
58 Disallow: /api/
59 Disallow: /*/dashboards/*
60 Disallow: /*docs/feeds
61 Disallow: /*docs/templates
62 Disallow: /*docs*Template:
63 Disallow: /*docs/all
64 Disallow: /*docs/tag*
65 Disallow: /*docs/needs-review*
66 Disallow: /*docs/localization-tag*
67 Disallow: /*docs/with-errors
68 Disallow: /*docs/without-parent
69 Disallow: /*docs/top-level
70 Disallow: /*docs/new
71 Disallow: /*docs/get-documents
72 Disallow: /*docs/submit_akismet_spam
73 Disallow: /*docs/load*
74 Disallow: /*docs/Experiment:*
75 Disallow: /*$api
76 Disallow: /*$compare
77 Disallow: /*$revision
78 Disallow: /*$history
79 Disallow: /*$children
80 Disallow: /*$flag
81 Disallow: /*$locales
82 Disallow: /*$toc
83 Disallow: /*$move
84 Disallow: /*$quick-review
85 Disallow: /*$samples
86 Disallow: /*$revert
87 Disallow: /*$repair_breadcrumbs
88 Disallow: /*$delete
89 Disallow: /*$restore
90 Disallow: /*$purge
91 Disallow: /*$subscribe
92 Disallow: /*$subscribe_to_tree
93 Disallow: /*$vote
94 Disallow: /*docs.json
95 Disallow: /*docs/ckeditor_config.js
96 Disallow: /*/files/
97 Disallow: /media
98 Disallow: /*move-requested
99 Disallow: /*preview-wiki-content
100 Disallow: /*profiles*/edit
101 Disallow: /skins
102 Disallow: /*type=feed
103 Disallow: /*users/
104 """ + "\n".join(
105     "Disallow: /{locale}/search".format(locale=locale)
106     for locale in settings.ENABLED_LOCALES
107 )
108 
109 ROBOTS_GO_AWAY_TXT = """\
110 User-Agent: *
111 Disallow: /
112 """
113 
114 
115 @shared_cache_control
116 def robots_txt(request):
117     """Serve robots.txt that allows or forbids robots."""
118     host = request.get_host()
119     if host in settings.ALLOW_ROBOTS_DOMAINS:
120         robots = ""
121     elif host in settings.ALLOW_ROBOTS_WEB_DOMAINS:
122         robots = ROBOTS_ALLOWED_TXT
123     else:
124         robots = ROBOTS_GO_AWAY_TXT
125     return HttpResponse(robots, content_type="text/plain")
126 
127 
128 class FaviconRedirect(RedirectView):
129     """Redirect to the favicon in the static img folder (bug 1402497)"""
130 
131     def get_redirect_url(self, *args, **kwargs):
132         return favicon_url()
133
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/kuma/landing/views.py b/kuma/landing/views.py
--- a/kuma/landing/views.py
+++ b/kuma/landing/views.py
@@ -50,6 +50,13 @@
     return render(request, "landing/promote_buttons.html")
 
 
+ROBOTS_ALL_ALLOWED_TXT = """\
+User-agent: *
+Sitemap: https://wiki.developer.mozilla.org/sitemap.xml
+
+Disallow:
+"""
+
 ROBOTS_ALLOWED_TXT = """\
 User-agent: *
 Sitemap: https://developer.mozilla.org/sitemap.xml
@@ -119,7 +126,10 @@
     if host in settings.ALLOW_ROBOTS_DOMAINS:
         robots = ""
     elif host in settings.ALLOW_ROBOTS_WEB_DOMAINS:
-        robots = ROBOTS_ALLOWED_TXT
+        if host == settings.WIKI_HOST:
+            robots = ROBOTS_ALL_ALLOWED_TXT
+        else:
+            robots = ROBOTS_ALLOWED_TXT
     else:
         robots = ROBOTS_GO_AWAY_TXT
     return HttpResponse(robots, content_type="text/plain")

{"golden_diff": "diff --git a/kuma/landing/views.py b/kuma/landing/views.py\n--- a/kuma/landing/views.py\n+++ b/kuma/landing/views.py\n@@ -50,6 +50,13 @@\n     return render(request, \"landing/promote_buttons.html\")\n \n \n+ROBOTS_ALL_ALLOWED_TXT = \"\"\"\\\n+User-agent: *\n+Sitemap: https://wiki.developer.mozilla.org/sitemap.xml\n+\n+Disallow:\n+\"\"\"\n+\n ROBOTS_ALLOWED_TXT = \"\"\"\\\n User-agent: *\n Sitemap: https://developer.mozilla.org/sitemap.xml\n@@ -119,7 +126,10 @@\n     if host in settings.ALLOW_ROBOTS_DOMAINS:\n         robots = \"\"\n     elif host in settings.ALLOW_ROBOTS_WEB_DOMAINS:\n-        robots = ROBOTS_ALLOWED_TXT\n+        if host == settings.WIKI_HOST:\n+            robots = ROBOTS_ALL_ALLOWED_TXT\n+        else:\n+            robots = ROBOTS_ALLOWED_TXT\n     else:\n         robots = ROBOTS_GO_AWAY_TXT\n     return HttpResponse(robots, content_type=\"text/plain\")\n", "issue": "T - Update robots.tx (Warnings have spiked on Search Console)\n**Summary**\r\nWe are seeing a large spike in warnings on Google Search Console. The spike started on December 22, 2019.\r\n\r\n<img width=\"952\" alt=\"Coverage\" src=\"https://user-images.githubusercontent.com/557852/72343521-5fb44000-36cf-11ea-8f1f-12955b98e34d.png\">\r\n\r\n**Steps To Reproduce (STR)**\r\n1. Go to Search Console\r\n2. Open Coverage Report\r\n3. open \"Indexed, though blocked by robots.txt\"\r\n\r\n\r\n**Actual behavior**\r\nIt looks like a bunch of URLs are blocked from indexing by Google that should not be blocked. Examples are:\r\n* https://developer.mozilla.org/it/docs/Web/API/GlobalEventHandlers/onresize$translate\r\n* https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/end%24edit\r\n* https://developer.mozilla.org/en-US/docs/Web/API/RTCIceCandidateInit/sdpMid$translate?tolocale=de\r\n\r\nWhen I test these URLs in Search Console's robots.txt tester it tells me that those URLs are allowed, and they should be. $translate, or $edit are not in robots.txt\r\n\r\n\r\n**Expected behavior**\r\nThose URLs should not be blocked, so that Google can crawl the pages, discover the no-index meta tag and remove them from its index.\r\n\r\n\r\n**Additional context**\r\nI assume that something is broken when it comes to handling the $ chracter in the URL. Maybe we just remove everything with the $url from our robots.txt. As far as I can tell, almost all of those URLs now live under wiki.developer.mozilla.org anyway.\n", "before_files": [{"content": "from django.conf import settings\nfrom django.http import HttpResponse\nfrom django.shortcuts import redirect, render\nfrom django.views import static\nfrom django.views.decorators.cache import never_cache\nfrom django.views.generic import RedirectView\n\nfrom kuma.core.decorators import ensure_wiki_domain, shared_cache_control\nfrom kuma.core.utils import is_wiki\nfrom kuma.feeder.models import Bundle\nfrom kuma.feeder.sections import SECTION_HACKS\nfrom kuma.search.models import Filter\n\nfrom .utils import favicon_url\n\n\n@shared_cache_control\ndef contribute_json(request):\n    return static.serve(request, \"contribute.json\", document_root=settings.ROOT)\n\n\n@shared_cache_control\ndef home(request):\n    \"\"\"Home page.\"\"\"\n    context = {}\n    # Need for both wiki and react homepage\n    context[\"updates\"] = list(Bundle.objects.recent_entries(SECTION_HACKS.updates)[:5])\n\n    # The default template name\n    template_name = \"landing/react_homepage.html\"\n    if is_wiki(request):\n        template_name = \"landing/homepage.html\"\n        context[\"default_filters\"] = Filter.objects.default_filters()\n    return render(request, template_name, context)\n\n\n@ensure_wiki_domain\n@never_cache\ndef maintenance_mode(request):\n    if settings.MAINTENANCE_MODE:\n        return render(request, \"landing/maintenance-mode.html\")\n    else:\n        return redirect(\"home\")\n\n\n@ensure_wiki_domain\n@shared_cache_control\ndef promote_buttons(request):\n    \"\"\"Bug 646192: MDN affiliate buttons\"\"\"\n    return render(request, \"landing/promote_buttons.html\")\n\n\nROBOTS_ALLOWED_TXT = \"\"\"\\\nUser-agent: *\nSitemap: https://developer.mozilla.org/sitemap.xml\n\nDisallow: /admin/\nDisallow: /api/\nDisallow: /*/dashboards/*\nDisallow: /*docs/feeds\nDisallow: /*docs/templates\nDisallow: /*docs*Template:\nDisallow: /*docs/all\nDisallow: /*docs/tag*\nDisallow: /*docs/needs-review*\nDisallow: /*docs/localization-tag*\nDisallow: /*docs/with-errors\nDisallow: /*docs/without-parent\nDisallow: /*docs/top-level\nDisallow: /*docs/new\nDisallow: /*docs/get-documents\nDisallow: /*docs/submit_akismet_spam\nDisallow: /*docs/load*\nDisallow: /*docs/Experiment:*\nDisallow: /*$api\nDisallow: /*$compare\nDisallow: /*$revision\nDisallow: /*$history\nDisallow: /*$children\nDisallow: /*$flag\nDisallow: /*$locales\nDisallow: /*$toc\nDisallow: /*$move\nDisallow: /*$quick-review\nDisallow: /*$samples\nDisallow: /*$revert\nDisallow: /*$repair_breadcrumbs\nDisallow: /*$delete\nDisallow: /*$restore\nDisallow: /*$purge\nDisallow: /*$subscribe\nDisallow: /*$subscribe_to_tree\nDisallow: /*$vote\nDisallow: /*docs.json\nDisallow: /*docs/ckeditor_config.js\nDisallow: /*/files/\nDisallow: /media\nDisallow: /*move-requested\nDisallow: /*preview-wiki-content\nDisallow: /*profiles*/edit\nDisallow: /skins\nDisallow: /*type=feed\nDisallow: /*users/\n\"\"\" + \"\\n\".join(\n    \"Disallow: /{locale}/search\".format(locale=locale)\n    for locale in settings.ENABLED_LOCALES\n)\n\nROBOTS_GO_AWAY_TXT = \"\"\"\\\nUser-Agent: *\nDisallow: /\n\"\"\"\n\n\n@shared_cache_control\ndef robots_txt(request):\n    \"\"\"Serve robots.txt that allows or forbids robots.\"\"\"\n    host = request.get_host()\n    if host in settings.ALLOW_ROBOTS_DOMAINS:\n        robots = \"\"\n    elif host in settings.ALLOW_ROBOTS_WEB_DOMAINS:\n        robots = ROBOTS_ALLOWED_TXT\n    else:\n        robots = ROBOTS_GO_AWAY_TXT\n    return HttpResponse(robots, content_type=\"text/plain\")\n\n\nclass FaviconRedirect(RedirectView):\n    \"\"\"Redirect to the favicon in the static img folder (bug 1402497)\"\"\"\n\n    def get_redirect_url(self, *args, **kwargs):\n        return favicon_url()\n", "path": "kuma/landing/views.py"}], "after_files": [{"content": "from django.conf import settings\nfrom django.http import HttpResponse\nfrom django.shortcuts import redirect, render\nfrom django.views import static\nfrom django.views.decorators.cache import never_cache\nfrom django.views.generic import RedirectView\n\nfrom kuma.core.decorators import ensure_wiki_domain, shared_cache_control\nfrom kuma.core.utils import is_wiki\nfrom kuma.feeder.models import Bundle\nfrom kuma.feeder.sections import SECTION_HACKS\nfrom kuma.search.models import Filter\n\nfrom .utils import favicon_url\n\n\n@shared_cache_control\ndef contribute_json(request):\n    return static.serve(request, \"contribute.json\", document_root=settings.ROOT)\n\n\n@shared_cache_control\ndef home(request):\n    \"\"\"Home page.\"\"\"\n    context = {}\n    # Need for both wiki and react homepage\n    context[\"updates\"] = list(Bundle.objects.recent_entries(SECTION_HACKS.updates)[:5])\n\n    # The default template name\n    template_name = \"landing/react_homepage.html\"\n    if is_wiki(request):\n        template_name = \"landing/homepage.html\"\n        context[\"default_filters\"] = Filter.objects.default_filters()\n    return render(request, template_name, context)\n\n\n@ensure_wiki_domain\n@never_cache\ndef maintenance_mode(request):\n    if settings.MAINTENANCE_MODE:\n        return render(request, \"landing/maintenance-mode.html\")\n    else:\n        return redirect(\"home\")\n\n\n@ensure_wiki_domain\n@shared_cache_control\ndef promote_buttons(request):\n    \"\"\"Bug 646192: MDN affiliate buttons\"\"\"\n    return render(request, \"landing/promote_buttons.html\")\n\n\nROBOTS_ALL_ALLOWED_TXT = \"\"\"\\\nUser-agent: *\nSitemap: https://wiki.developer.mozilla.org/sitemap.xml\n\nDisallow:\n\"\"\"\n\nROBOTS_ALLOWED_TXT = \"\"\"\\\nUser-agent: *\nSitemap: https://developer.mozilla.org/sitemap.xml\n\nDisallow: /admin/\nDisallow: /api/\nDisallow: /*/dashboards/*\nDisallow: /*docs/feeds\nDisallow: /*docs/templates\nDisallow: /*docs*Template:\nDisallow: /*docs/all\nDisallow: /*docs/tag*\nDisallow: /*docs/needs-review*\nDisallow: /*docs/localization-tag*\nDisallow: /*docs/with-errors\nDisallow: /*docs/without-parent\nDisallow: /*docs/top-level\nDisallow: /*docs/new\nDisallow: /*docs/get-documents\nDisallow: /*docs/submit_akismet_spam\nDisallow: /*docs/load*\nDisallow: /*docs/Experiment:*\nDisallow: /*$api\nDisallow: /*$compare\nDisallow: /*$revision\nDisallow: /*$history\nDisallow: /*$children\nDisallow: /*$flag\nDisallow: /*$locales\nDisallow: /*$toc\nDisallow: /*$move\nDisallow: /*$quick-review\nDisallow: /*$samples\nDisallow: /*$revert\nDisallow: /*$repair_breadcrumbs\nDisallow: /*$delete\nDisallow: /*$restore\nDisallow: /*$purge\nDisallow: /*$subscribe\nDisallow: /*$subscribe_to_tree\nDisallow: /*$vote\nDisallow: /*docs.json\nDisallow: /*docs/ckeditor_config.js\nDisallow: /*/files/\nDisallow: /media\nDisallow: /*move-requested\nDisallow: /*preview-wiki-content\nDisallow: /*profiles*/edit\nDisallow: /skins\nDisallow: /*type=feed\nDisallow: /*users/\n\"\"\" + \"\\n\".join(\n    \"Disallow: /{locale}/search\".format(locale=locale)\n    for locale in settings.ENABLED_LOCALES\n)\n\nROBOTS_GO_AWAY_TXT = \"\"\"\\\nUser-Agent: *\nDisallow: /\n\"\"\"\n\n\n@shared_cache_control\ndef robots_txt(request):\n    \"\"\"Serve robots.txt that allows or forbids robots.\"\"\"\n    host = request.get_host()\n    if host in settings.ALLOW_ROBOTS_DOMAINS:\n        robots = \"\"\n    elif host in settings.ALLOW_ROBOTS_WEB_DOMAINS:\n        if host == settings.WIKI_HOST:\n            robots = ROBOTS_ALL_ALLOWED_TXT\n        else:\n            robots = ROBOTS_ALLOWED_TXT\n    else:\n        robots = ROBOTS_GO_AWAY_TXT\n    return HttpResponse(robots, content_type=\"text/plain\")\n\n\nclass FaviconRedirect(RedirectView):\n    \"\"\"Redirect to the favicon in the static img folder (bug 1402497)\"\"\"\n\n    def get_redirect_url(self, *args, **kwargs):\n        return favicon_url()\n", "path": "kuma/landing/views.py"}]}