We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Bug: Yahoo results for simplified Chinese search sometimes have the first character cut off
<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SearXNG -->

**Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG**

The latest commit as of this report, 75587d03a6766c87f2f1964c82080393559ed623

**How did you install SearXNG?**

```
make run
```

**What happened?**

As the title says. Current code redirects simplified search to the Hong Kong site, which actually uses traditional Chinese, but marks simplified pages with a `<span class="fz-s fc-obsidian ml-4">簡</span>` after the title text node, at the end:

```html
<h3 style="display:block;margin-top:24px;margin-bottom:2px;" class="title tc"><a style="line-height:26px" class=" d-ib fz-20 lh-26 td-hu tc va-bot mxw-100p" href="https://zh.wikipedia.org/wiki/%E7%B9%81%E4%BD%93%E5%AD%97" referrerpolicy="origin" target="_blank" data-439="6515fd318fbac" aria-label="繁体字 - 维基百科，自由的百科全书"><span class=" s-url d-ib p-abs t-0 l-0 fz-14 lh-20 fc-obsidian wr-bw ls-n pb-4">zh.wikipedia.org<span class=" fc-pewter"> › wiki › <b>繁体字</b></span></span>繁体字 - 维基百科，自由的百科全书<span class="fz-s fc-obsidian ml-4">簡</span></a></h3>
```

This character is extracted by `.xpath('span')` in the first line, which, without knowledge of this situation, actually extracted text from two `span`s, making the second line substring off by one:

https://github.com/searxng/searxng/blob/75587d03a6766c87f2f1964c82080393559ed623/searx/engines/yahoo.py#L139-L140

**How To Reproduce**

Search for some simplified Chinese keyword and look for a Yahoo result with 簡 being the last character of its title.

**Expected behavior**

That mark should be ignored, eliminating the off by one.

**Screenshots & Logs**

<img width="664" alt="problematic search result" src="https://github.com/searxng/searxng/assets/4474501/f8ea1959-0487-4fed-b93f-c3e9a061eaea">

You can see that the last breadcrumb is 繁体字, but the title says 体字, cutting off 繁; while there is an extraneous 簡 at the end.

**Additional context**

```diff
diff --git a/searx/engines/yahoo.py b/searx/engines/yahoo.py
index 0fdeacec2..4858e7e3f 100644
--- a/searx/engines/yahoo.py
+++ b/searx/engines/yahoo.py
@@ -136,8 +136,9 @@ def response(resp):
         title = eval_xpath_getindex(result, './/h3/a', 0, default=None)
         if title is None:
             continue
-        offset = len(extract_text(title.xpath('span')))
-        title = extract_text(title)[offset:]
+        offset = len(extract_text(title.xpath('span[1]')))
+        offset_right = -len(extract_text(title.xpath('span[2]'))) or None
+        title = extract_text(title)[offset:offset_right]

         content = eval_xpath_getindex(result, './/div[contains(@class, "compText")]', 0, default='')
         content = extract_text(content, allow_none=True)
```

This is an apparent fix, but I'm not sure it's worth it.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `searx/engines/yahoo.py`
Content:
```
1 # SPDX-License-Identifier: AGPL-3.0-or-later
2 # lint: pylint
3 """Yahoo Search (Web)
4 
5 Languages are supported by mapping the language to a domain.  If domain is not
6 found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.
7 
8 """
9 
10 from urllib.parse import (
11     unquote,
12     urlencode,
13 )
14 from lxml import html
15 
16 from searx.utils import (
17     eval_xpath_getindex,
18     eval_xpath_list,
19     extract_text,
20 )
21 from searx.enginelib.traits import EngineTraits
22 
23 traits: EngineTraits
24 
25 # about
26 about = {
27     "website": 'https://search.yahoo.com/',
28     "wikidata_id": None,
29     "official_api_documentation": 'https://developer.yahoo.com/api/',
30     "use_official_api": False,
31     "require_api_key": False,
32     "results": 'HTML',
33 }
34 
35 # engine dependent config
36 categories = ['general', 'web']
37 paging = True
38 time_range_support = True
39 # send_accept_language_header = True
40 
41 time_range_dict = {
42     'day': ('1d', 'd'),
43     'week': ('1w', 'w'),
44     'month': ('1m', 'm'),
45 }
46 
47 lang2domain = {
48     'zh_chs': 'hk.search.yahoo.com',
49     'zh_cht': 'tw.search.yahoo.com',
50     'any': 'search.yahoo.com',
51     'en': 'search.yahoo.com',
52     'bg': 'search.yahoo.com',
53     'cs': 'search.yahoo.com',
54     'da': 'search.yahoo.com',
55     'el': 'search.yahoo.com',
56     'et': 'search.yahoo.com',
57     'he': 'search.yahoo.com',
58     'hr': 'search.yahoo.com',
59     'ja': 'search.yahoo.com',
60     'ko': 'search.yahoo.com',
61     'sk': 'search.yahoo.com',
62     'sl': 'search.yahoo.com',
63 }
64 """Map language to domain"""
65 
66 locale_aliases = {
67     'zh': 'zh_Hans',
68     'zh-HK': 'zh_Hans',
69     'zh-CN': 'zh_Hans',  # dead since 2015 / routed to hk.search.yahoo.com
70     'zh-TW': 'zh_Hant',
71 }
72 
73 
74 def request(query, params):
75     """build request"""
76 
77     lang = locale_aliases.get(params['language'], None)
78     if not lang:
79         lang = params['language'].split('-')[0]
80     lang = traits.get_language(lang, traits.all_locale)
81 
82     offset = (params['pageno'] - 1) * 7 + 1
83     age, btf = time_range_dict.get(params['time_range'], ('', ''))
84 
85     args = urlencode(
86         {
87             'p': query,
88             'ei': 'UTF-8',
89             'fl': 1,
90             'vl': 'lang_' + lang,
91             'btf': btf,
92             'fr2': 'time',
93             'age': age,
94             'b': offset,
95             'xargs': 0,
96         }
97     )
98 
99     domain = lang2domain.get(lang, '%s.search.yahoo.com' % lang)
100     params['url'] = 'https://%s/search?%s' % (domain, args)
101     return params
102 
103 
104 def parse_url(url_string):
105     """remove yahoo-specific tracking-url"""
106 
107     endings = ['/RS', '/RK']
108     endpositions = []
109     start = url_string.find('http', url_string.find('/RU=') + 1)
110 
111     for ending in endings:
112         endpos = url_string.rfind(ending)
113         if endpos > -1:
114             endpositions.append(endpos)
115 
116     if start == 0 or len(endpositions) == 0:
117         return url_string
118 
119     end = min(endpositions)
120     return unquote(url_string[start:end])
121 
122 
123 def response(resp):
124     """parse response"""
125 
126     results = []
127     dom = html.fromstring(resp.text)
128 
129     # parse results
130     for result in eval_xpath_list(dom, '//div[contains(@class,"algo-sr")]'):
131         url = eval_xpath_getindex(result, './/h3/a/@href', 0, default=None)
132         if url is None:
133             continue
134         url = parse_url(url)
135 
136         title = eval_xpath_getindex(result, './/h3/a', 0, default=None)
137         if title is None:
138             continue
139         offset = len(extract_text(title.xpath('span')))
140         title = extract_text(title)[offset:]
141 
142         content = eval_xpath_getindex(result, './/div[contains(@class, "compText")]', 0, default='')
143         content = extract_text(content, allow_none=True)
144 
145         # append result
146         results.append({'url': url, 'title': title, 'content': content})
147 
148     for suggestion in eval_xpath_list(dom, '//div[contains(@class, "AlsoTry")]//table//a'):
149         # append suggestion
150         results.append({'suggestion': extract_text(suggestion)})
151 
152     return results
153 
154 
155 def fetch_traits(engine_traits: EngineTraits):
156     """Fetch languages from yahoo"""
157 
158     # pylint: disable=import-outside-toplevel
159     import babel
160     from searx import network
161     from searx.locales import language_tag
162 
163     engine_traits.all_locale = 'any'
164 
165     resp = network.get('https://search.yahoo.com/preferences/languages')
166     if not resp.ok:
167         print("ERROR: response from peertube is not OK.")
168 
169     dom = html.fromstring(resp.text)
170     offset = len('lang_')
171 
172     eng2sxng = {'zh_chs': 'zh_Hans', 'zh_cht': 'zh_Hant'}
173 
174     for val in eval_xpath_list(dom, '//div[contains(@class, "lang-item")]/input/@value'):
175         eng_tag = val[offset:]
176 
177         try:
178             sxng_tag = language_tag(babel.Locale.parse(eng2sxng.get(eng_tag, eng_tag)))
179         except babel.UnknownLocaleError:
180             print('ERROR: unknown language --> %s' % eng_tag)
181             continue
182 
183         conflict = engine_traits.languages.get(sxng_tag)
184         if conflict:
185             if conflict != eng_tag:
186                 print("CONFLICT: babel %s --> %s, %s" % (sxng_tag, conflict, eng_tag))
187             continue
188         engine_traits.languages[sxng_tag] = eng_tag
189
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/searx/engines/yahoo.py b/searx/engines/yahoo.py
--- a/searx/engines/yahoo.py
+++ b/searx/engines/yahoo.py
@@ -133,12 +133,7 @@
             continue
         url = parse_url(url)
 
-        title = eval_xpath_getindex(result, './/h3/a', 0, default=None)
-        if title is None:
-            continue
-        offset = len(extract_text(title.xpath('span')))
-        title = extract_text(title)[offset:]
-
+        title = extract_text(result.xpath('.//h3//a/@aria-label'))
         content = eval_xpath_getindex(result, './/div[contains(@class, "compText")]', 0, default='')
         content = extract_text(content, allow_none=True)
 
@@ -164,7 +159,7 @@
 
     resp = network.get('https://search.yahoo.com/preferences/languages')
     if not resp.ok:
-        print("ERROR: response from peertube is not OK.")
+        print("ERROR: response from yahoo is not OK.")
 
     dom = html.fromstring(resp.text)
     offset = len('lang_')

{"golden_diff": "diff --git a/searx/engines/yahoo.py b/searx/engines/yahoo.py\n--- a/searx/engines/yahoo.py\n+++ b/searx/engines/yahoo.py\n@@ -133,12 +133,7 @@\n             continue\n         url = parse_url(url)\n \n-        title = eval_xpath_getindex(result, './/h3/a', 0, default=None)\n-        if title is None:\n-            continue\n-        offset = len(extract_text(title.xpath('span')))\n-        title = extract_text(title)[offset:]\n-\n+        title = extract_text(result.xpath('.//h3//a/@aria-label'))\n         content = eval_xpath_getindex(result, './/div[contains(@class, \"compText\")]', 0, default='')\n         content = extract_text(content, allow_none=True)\n \n@@ -164,7 +159,7 @@\n \n     resp = network.get('https://search.yahoo.com/preferences/languages')\n     if not resp.ok:\n-        print(\"ERROR: response from peertube is not OK.\")\n+        print(\"ERROR: response from yahoo is not OK.\")\n \n     dom = html.fromstring(resp.text)\n     offset = len('lang_')\n", "issue": "Bug: Yahoo results for simplified Chinese search sometimes have the first character cut off\n<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SearXNG -->\r\n\r\n**Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG**\r\n\r\nThe latest commit as of this report, 75587d03a6766c87f2f1964c82080393559ed623\r\n\r\n**How did you install SearXNG?**\r\n\r\n```\r\nmake run\r\n```\r\n\r\n**What happened?**\r\n\r\nAs the title says. Current code redirects simplified search to the Hong Kong site, which actually uses traditional Chinese, but marks simplified pages with a `<span class=\"fz-s fc-obsidian ml-4\">\u7c21</span>` after the title text node, at the end:\r\n\r\n```html\r\n<h3 style=\"display:block;margin-top:24px;margin-bottom:2px;\" class=\"title tc\"><a style=\"line-height:26px\" class=\" d-ib fz-20 lh-26 td-hu tc va-bot mxw-100p\" href=\"https://zh.wikipedia.org/wiki/%E7%B9%81%E4%BD%93%E5%AD%97\" referrerpolicy=\"origin\" target=\"_blank\" data-439=\"6515fd318fbac\" aria-label=\"\u7e41\u4f53\u5b57 - \u7ef4\u57fa\u767e\u79d1\uff0c\u81ea\u7531\u7684\u767e\u79d1\u5168\u4e66\"><span class=\" s-url d-ib p-abs t-0 l-0 fz-14 lh-20 fc-obsidian wr-bw ls-n pb-4\">zh.wikipedia.org<span class=\" fc-pewter\"> \u203a wiki \u203a <b>\u7e41\u4f53\u5b57</b></span></span>\u7e41\u4f53\u5b57 - \u7ef4\u57fa\u767e\u79d1\uff0c\u81ea\u7531\u7684\u767e\u79d1\u5168\u4e66<span class=\"fz-s fc-obsidian ml-4\">\u7c21</span></a></h3>\r\n```\r\n\r\nThis character is extracted by `.xpath('span')` in the first line, which, without knowledge of this situation, actually extracted text from two `span`s, making the second line substring off by one:\r\n\r\nhttps://github.com/searxng/searxng/blob/75587d03a6766c87f2f1964c82080393559ed623/searx/engines/yahoo.py#L139-L140\r\n\r\n**How To Reproduce**\r\n\r\nSearch for some simplified Chinese keyword and look for a Yahoo result with \u7c21 being the last character of its title.\r\n\r\n**Expected behavior**\r\n\r\nThat mark should be ignored, eliminating the off by one.\r\n\r\n**Screenshots & Logs**\r\n\r\n<img width=\"664\" alt=\"problematic search result\" src=\"https://github.com/searxng/searxng/assets/4474501/f8ea1959-0487-4fed-b93f-c3e9a061eaea\">\r\n\r\nYou can see that the last breadcrumb is \u7e41\u4f53\u5b57, but the title says \u4f53\u5b57, cutting off \u7e41; while there is an extraneous \u7c21 at the end.\r\n\r\n**Additional context**\r\n\r\n```diff\r\ndiff --git a/searx/engines/yahoo.py b/searx/engines/yahoo.py\r\nindex 0fdeacec2..4858e7e3f 100644\r\n--- a/searx/engines/yahoo.py\r\n+++ b/searx/engines/yahoo.py\r\n@@ -136,8 +136,9 @@ def response(resp):\r\n         title = eval_xpath_getindex(result, './/h3/a', 0, default=None)\r\n         if title is None:\r\n             continue\r\n-        offset = len(extract_text(title.xpath('span')))\r\n-        title = extract_text(title)[offset:]\r\n+        offset = len(extract_text(title.xpath('span[1]')))\r\n+        offset_right = -len(extract_text(title.xpath('span[2]'))) or None\r\n+        title = extract_text(title)[offset:offset_right]\r\n\r\n         content = eval_xpath_getindex(result, './/div[contains(@class, \"compText\")]', 0, default='')\r\n         content = extract_text(content, allow_none=True)\r\n```\r\n\r\nThis is an apparent fix, but I'm not sure it's worth it.\n", "before_files": [{"content": "# SPDX-License-Identifier: AGPL-3.0-or-later\n# lint: pylint\n\"\"\"Yahoo Search (Web)\n\nLanguages are supported by mapping the language to a domain.  If domain is not\nfound in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.\n\n\"\"\"\n\nfrom urllib.parse import (\n    unquote,\n    urlencode,\n)\nfrom lxml import html\n\nfrom searx.utils import (\n    eval_xpath_getindex,\n    eval_xpath_list,\n    extract_text,\n)\nfrom searx.enginelib.traits import EngineTraits\n\ntraits: EngineTraits\n\n# about\nabout = {\n    \"website\": 'https://search.yahoo.com/',\n    \"wikidata_id\": None,\n    \"official_api_documentation\": 'https://developer.yahoo.com/api/',\n    \"use_official_api\": False,\n    \"require_api_key\": False,\n    \"results\": 'HTML',\n}\n\n# engine dependent config\ncategories = ['general', 'web']\npaging = True\ntime_range_support = True\n# send_accept_language_header = True\n\ntime_range_dict = {\n    'day': ('1d', 'd'),\n    'week': ('1w', 'w'),\n    'month': ('1m', 'm'),\n}\n\nlang2domain = {\n    'zh_chs': 'hk.search.yahoo.com',\n    'zh_cht': 'tw.search.yahoo.com',\n    'any': 'search.yahoo.com',\n    'en': 'search.yahoo.com',\n    'bg': 'search.yahoo.com',\n    'cs': 'search.yahoo.com',\n    'da': 'search.yahoo.com',\n    'el': 'search.yahoo.com',\n    'et': 'search.yahoo.com',\n    'he': 'search.yahoo.com',\n    'hr': 'search.yahoo.com',\n    'ja': 'search.yahoo.com',\n    'ko': 'search.yahoo.com',\n    'sk': 'search.yahoo.com',\n    'sl': 'search.yahoo.com',\n}\n\"\"\"Map language to domain\"\"\"\n\nlocale_aliases = {\n    'zh': 'zh_Hans',\n    'zh-HK': 'zh_Hans',\n    'zh-CN': 'zh_Hans',  # dead since 2015 / routed to hk.search.yahoo.com\n    'zh-TW': 'zh_Hant',\n}\n\n\ndef request(query, params):\n    \"\"\"build request\"\"\"\n\n    lang = locale_aliases.get(params['language'], None)\n    if not lang:\n        lang = params['language'].split('-')[0]\n    lang = traits.get_language(lang, traits.all_locale)\n\n    offset = (params['pageno'] - 1) * 7 + 1\n    age, btf = time_range_dict.get(params['time_range'], ('', ''))\n\n    args = urlencode(\n        {\n            'p': query,\n            'ei': 'UTF-8',\n            'fl': 1,\n            'vl': 'lang_' + lang,\n            'btf': btf,\n            'fr2': 'time',\n            'age': age,\n            'b': offset,\n            'xargs': 0,\n        }\n    )\n\n    domain = lang2domain.get(lang, '%s.search.yahoo.com' % lang)\n    params['url'] = 'https://%s/search?%s' % (domain, args)\n    return params\n\n\ndef parse_url(url_string):\n    \"\"\"remove yahoo-specific tracking-url\"\"\"\n\n    endings = ['/RS', '/RK']\n    endpositions = []\n    start = url_string.find('http', url_string.find('/RU=') + 1)\n\n    for ending in endings:\n        endpos = url_string.rfind(ending)\n        if endpos > -1:\n            endpositions.append(endpos)\n\n    if start == 0 or len(endpositions) == 0:\n        return url_string\n\n    end = min(endpositions)\n    return unquote(url_string[start:end])\n\n\ndef response(resp):\n    \"\"\"parse response\"\"\"\n\n    results = []\n    dom = html.fromstring(resp.text)\n\n    # parse results\n    for result in eval_xpath_list(dom, '//div[contains(@class,\"algo-sr\")]'):\n        url = eval_xpath_getindex(result, './/h3/a/@href', 0, default=None)\n        if url is None:\n            continue\n        url = parse_url(url)\n\n        title = eval_xpath_getindex(result, './/h3/a', 0, default=None)\n        if title is None:\n            continue\n        offset = len(extract_text(title.xpath('span')))\n        title = extract_text(title)[offset:]\n\n        content = eval_xpath_getindex(result, './/div[contains(@class, \"compText\")]', 0, default='')\n        content = extract_text(content, allow_none=True)\n\n        # append result\n        results.append({'url': url, 'title': title, 'content': content})\n\n    for suggestion in eval_xpath_list(dom, '//div[contains(@class, \"AlsoTry\")]//table//a'):\n        # append suggestion\n        results.append({'suggestion': extract_text(suggestion)})\n\n    return results\n\n\ndef fetch_traits(engine_traits: EngineTraits):\n    \"\"\"Fetch languages from yahoo\"\"\"\n\n    # pylint: disable=import-outside-toplevel\n    import babel\n    from searx import network\n    from searx.locales import language_tag\n\n    engine_traits.all_locale = 'any'\n\n    resp = network.get('https://search.yahoo.com/preferences/languages')\n    if not resp.ok:\n        print(\"ERROR: response from peertube is not OK.\")\n\n    dom = html.fromstring(resp.text)\n    offset = len('lang_')\n\n    eng2sxng = {'zh_chs': 'zh_Hans', 'zh_cht': 'zh_Hant'}\n\n    for val in eval_xpath_list(dom, '//div[contains(@class, \"lang-item\")]/input/@value'):\n        eng_tag = val[offset:]\n\n        try:\n            sxng_tag = language_tag(babel.Locale.parse(eng2sxng.get(eng_tag, eng_tag)))\n        except babel.UnknownLocaleError:\n            print('ERROR: unknown language --> %s' % eng_tag)\n            continue\n\n        conflict = engine_traits.languages.get(sxng_tag)\n        if conflict:\n            if conflict != eng_tag:\n                print(\"CONFLICT: babel %s --> %s, %s\" % (sxng_tag, conflict, eng_tag))\n            continue\n        engine_traits.languages[sxng_tag] = eng_tag\n", "path": "searx/engines/yahoo.py"}], "after_files": [{"content": "# SPDX-License-Identifier: AGPL-3.0-or-later\n# lint: pylint\n\"\"\"Yahoo Search (Web)\n\nLanguages are supported by mapping the language to a domain.  If domain is not\nfound in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.\n\n\"\"\"\n\nfrom urllib.parse import (\n    unquote,\n    urlencode,\n)\nfrom lxml import html\n\nfrom searx.utils import (\n    eval_xpath_getindex,\n    eval_xpath_list,\n    extract_text,\n)\nfrom searx.enginelib.traits import EngineTraits\n\ntraits: EngineTraits\n\n# about\nabout = {\n    \"website\": 'https://search.yahoo.com/',\n    \"wikidata_id\": None,\n    \"official_api_documentation\": 'https://developer.yahoo.com/api/',\n    \"use_official_api\": False,\n    \"require_api_key\": False,\n    \"results\": 'HTML',\n}\n\n# engine dependent config\ncategories = ['general', 'web']\npaging = True\ntime_range_support = True\n# send_accept_language_header = True\n\ntime_range_dict = {\n    'day': ('1d', 'd'),\n    'week': ('1w', 'w'),\n    'month': ('1m', 'm'),\n}\n\nlang2domain = {\n    'zh_chs': 'hk.search.yahoo.com',\n    'zh_cht': 'tw.search.yahoo.com',\n    'any': 'search.yahoo.com',\n    'en': 'search.yahoo.com',\n    'bg': 'search.yahoo.com',\n    'cs': 'search.yahoo.com',\n    'da': 'search.yahoo.com',\n    'el': 'search.yahoo.com',\n    'et': 'search.yahoo.com',\n    'he': 'search.yahoo.com',\n    'hr': 'search.yahoo.com',\n    'ja': 'search.yahoo.com',\n    'ko': 'search.yahoo.com',\n    'sk': 'search.yahoo.com',\n    'sl': 'search.yahoo.com',\n}\n\"\"\"Map language to domain\"\"\"\n\nlocale_aliases = {\n    'zh': 'zh_Hans',\n    'zh-HK': 'zh_Hans',\n    'zh-CN': 'zh_Hans',  # dead since 2015 / routed to hk.search.yahoo.com\n    'zh-TW': 'zh_Hant',\n}\n\n\ndef request(query, params):\n    \"\"\"build request\"\"\"\n\n    lang = locale_aliases.get(params['language'], None)\n    if not lang:\n        lang = params['language'].split('-')[0]\n    lang = traits.get_language(lang, traits.all_locale)\n\n    offset = (params['pageno'] - 1) * 7 + 1\n    age, btf = time_range_dict.get(params['time_range'], ('', ''))\n\n    args = urlencode(\n        {\n            'p': query,\n            'ei': 'UTF-8',\n            'fl': 1,\n            'vl': 'lang_' + lang,\n            'btf': btf,\n            'fr2': 'time',\n            'age': age,\n            'b': offset,\n            'xargs': 0,\n        }\n    )\n\n    domain = lang2domain.get(lang, '%s.search.yahoo.com' % lang)\n    params['url'] = 'https://%s/search?%s' % (domain, args)\n    return params\n\n\ndef parse_url(url_string):\n    \"\"\"remove yahoo-specific tracking-url\"\"\"\n\n    endings = ['/RS', '/RK']\n    endpositions = []\n    start = url_string.find('http', url_string.find('/RU=') + 1)\n\n    for ending in endings:\n        endpos = url_string.rfind(ending)\n        if endpos > -1:\n            endpositions.append(endpos)\n\n    if start == 0 or len(endpositions) == 0:\n        return url_string\n\n    end = min(endpositions)\n    return unquote(url_string[start:end])\n\n\ndef response(resp):\n    \"\"\"parse response\"\"\"\n\n    results = []\n    dom = html.fromstring(resp.text)\n\n    # parse results\n    for result in eval_xpath_list(dom, '//div[contains(@class,\"algo-sr\")]'):\n        url = eval_xpath_getindex(result, './/h3/a/@href', 0, default=None)\n        if url is None:\n            continue\n        url = parse_url(url)\n\n        title = extract_text(result.xpath('.//h3//a/@aria-label'))\n        content = eval_xpath_getindex(result, './/div[contains(@class, \"compText\")]', 0, default='')\n        content = extract_text(content, allow_none=True)\n\n        # append result\n        results.append({'url': url, 'title': title, 'content': content})\n\n    for suggestion in eval_xpath_list(dom, '//div[contains(@class, \"AlsoTry\")]//table//a'):\n        # append suggestion\n        results.append({'suggestion': extract_text(suggestion)})\n\n    return results\n\n\ndef fetch_traits(engine_traits: EngineTraits):\n    \"\"\"Fetch languages from yahoo\"\"\"\n\n    # pylint: disable=import-outside-toplevel\n    import babel\n    from searx import network\n    from searx.locales import language_tag\n\n    engine_traits.all_locale = 'any'\n\n    resp = network.get('https://search.yahoo.com/preferences/languages')\n    if not resp.ok:\n        print(\"ERROR: response from yahoo is not OK.\")\n\n    dom = html.fromstring(resp.text)\n    offset = len('lang_')\n\n    eng2sxng = {'zh_chs': 'zh_Hans', 'zh_cht': 'zh_Hant'}\n\n    for val in eval_xpath_list(dom, '//div[contains(@class, \"lang-item\")]/input/@value'):\n        eng_tag = val[offset:]\n\n        try:\n            sxng_tag = language_tag(babel.Locale.parse(eng2sxng.get(eng_tag, eng_tag)))\n        except babel.UnknownLocaleError:\n            print('ERROR: unknown language --> %s' % eng_tag)\n            continue\n\n        conflict = engine_traits.languages.get(sxng_tag)\n        if conflict:\n            if conflict != eng_tag:\n                print(\"CONFLICT: babel %s --> %s, %s\" % (sxng_tag, conflict, eng_tag))\n            continue\n        engine_traits.languages[sxng_tag] = eng_tag\n", "path": "searx/engines/yahoo.py"}]}