We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
ann_utils 当 output_emb_size = 0 时报错
欢迎您反馈PaddleNLP使用问题，非常感谢您对PaddleNLP的贡献！
在留下您的问题时，辛苦您同步提供如下信息：
- 版本、环境信息
1）PaddleNLP和PaddlePaddle版本：请提供您的PaddleNLP和PaddlePaddle版本号，例如PaddleNLP 2.0.4，PaddlePaddle2.1.1
2）系统环境：请您描述系统类型，例如Linux/Windows/MacOS/，python版本
- 复现信息：如为报错，请给出复现环境、复现步骤


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `applications/neural_search/recall/in_batch_negative/ann_util.py`
Content:
```
1 # Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 # coding=UTF-8
16 
17 import numpy as np
18 import hnswlib
19 from paddlenlp.utils.log import logger
20 
21 
22 def build_index(args, data_loader, model):
23 
24     index = hnswlib.Index(space='ip', dim=args.output_emb_size)
25 
26     # Initializing index
27     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
28     # during insertion of an element.
29     # The capacity can be increased by saving/loading the index, see below.
30     #
31     # ef_construction - controls index search speed/build speed tradeoff
32     #
33     # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)
34     # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction
35     index.init_index(
36         max_elements=args.hnsw_max_elements,
37         ef_construction=args.hnsw_ef,
38         M=args.hnsw_m)
39 
40     # Controlling the recall by setting ef:
41     # higher ef leads to better accuracy, but slower search
42     index.set_ef(args.hnsw_ef)
43 
44     # Set number of threads used during batch search/construction
45     # By default using all available cores
46     index.set_num_threads(16)
47 
48     logger.info("start build index..........")
49 
50     all_embeddings = []
51 
52     for text_embeddings in model.get_semantic_embedding(data_loader):
53         all_embeddings.append(text_embeddings.numpy())
54 
55     all_embeddings = np.concatenate(all_embeddings, axis=0)
56     index.add_items(all_embeddings)
57 
58     logger.info("Total index number:{}".format(index.get_current_count()))
59 
60     return index
61
```

Path: `applications/question_answering/faq_system/ann_util.py`
Content:
```
1 # Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import numpy as np
16 import hnswlib
17 from paddlenlp.utils.log import logger
18 
19 
20 def build_index(args, data_loader, model):
21 
22     index = hnswlib.Index(space='ip', dim=args.output_emb_size)
23 
24     # Initializing index
25     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
26     # during insertion of an element.
27     # The capacity can be increased by saving/loading the index, see below.
28     #
29     # ef_construction - controls index search speed/build speed tradeoff
30     #
31     # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)
32     # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction
33     index.init_index(
34         max_elements=args.hnsw_max_elements,
35         ef_construction=args.hnsw_ef,
36         M=args.hnsw_m)
37 
38     # Controlling the recall by setting ef:
39     # higher ef leads to better accuracy, but slower search
40     index.set_ef(args.hnsw_ef)
41 
42     # Set number of threads used during batch search/construction
43     # By default using all available cores
44     index.set_num_threads(16)
45 
46     logger.info("start build index..........")
47 
48     all_embeddings = []
49 
50     for text_embeddings in model.get_semantic_embedding(data_loader):
51         all_embeddings.append(text_embeddings.numpy())
52 
53     all_embeddings = np.concatenate(all_embeddings, axis=0)
54     index.add_items(all_embeddings)
55 
56     logger.info("Total index number:{}".format(index.get_current_count()))
57 
58     return index
59
```

Path: `applications/question_answering/faq_finance/ann_util.py`
Content:
```
1 # Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import numpy as np
16 import hnswlib
17 from paddlenlp.utils.log import logger
18 
19 
20 def build_index(args, data_loader, model):
21 
22     index = hnswlib.Index(space='ip', dim=args.output_emb_size)
23 
24     # Initializing index
25     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
26     # during insertion of an element.
27     # The capacity can be increased by saving/loading the index, see below.
28     #
29     # ef_construction - controls index search speed/build speed tradeoff
30     #
31     # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)
32     # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction
33     index.init_index(
34         max_elements=args.hnsw_max_elements,
35         ef_construction=args.hnsw_ef,
36         M=args.hnsw_m)
37 
38     # Controlling the recall by setting ef:
39     # higher ef leads to better accuracy, but slower search
40     index.set_ef(args.hnsw_ef)
41 
42     # Set number of threads used during batch search/construction
43     # By default using all available cores
44     index.set_num_threads(16)
45 
46     logger.info("start build index..........")
47 
48     all_embeddings = []
49 
50     for text_embeddings in model.get_semantic_embedding(data_loader):
51         all_embeddings.append(text_embeddings.numpy())
52 
53     all_embeddings = np.concatenate(all_embeddings, axis=0)
54     index.add_items(all_embeddings)
55 
56     logger.info("Total index number:{}".format(index.get_current_count()))
57 
58     return index
59
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/applications/neural_search/recall/in_batch_negative/ann_util.py b/applications/neural_search/recall/in_batch_negative/ann_util.py
--- a/applications/neural_search/recall/in_batch_negative/ann_util.py
+++ b/applications/neural_search/recall/in_batch_negative/ann_util.py
@@ -21,7 +21,9 @@
 
 def build_index(args, data_loader, model):
 
-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)
+    index = hnswlib.Index(
+        space='ip',
+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)
 
     # Initializing index
     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
diff --git a/applications/question_answering/faq_finance/ann_util.py b/applications/question_answering/faq_finance/ann_util.py
--- a/applications/question_answering/faq_finance/ann_util.py
+++ b/applications/question_answering/faq_finance/ann_util.py
@@ -19,7 +19,9 @@
 
 def build_index(args, data_loader, model):
 
-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)
+    index = hnswlib.Index(
+        space='ip',
+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)
 
     # Initializing index
     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
diff --git a/applications/question_answering/faq_system/ann_util.py b/applications/question_answering/faq_system/ann_util.py
--- a/applications/question_answering/faq_system/ann_util.py
+++ b/applications/question_answering/faq_system/ann_util.py
@@ -19,7 +19,9 @@
 
 def build_index(args, data_loader, model):
 
-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)
+    index = hnswlib.Index(
+        space='ip',
+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)
 
     # Initializing index
     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded

{"golden_diff": "diff --git a/applications/neural_search/recall/in_batch_negative/ann_util.py b/applications/neural_search/recall/in_batch_negative/ann_util.py\n--- a/applications/neural_search/recall/in_batch_negative/ann_util.py\n+++ b/applications/neural_search/recall/in_batch_negative/ann_util.py\n@@ -21,7 +21,9 @@\n \n def build_index(args, data_loader, model):\n \n-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)\n+    index = hnswlib.Index(\n+        space='ip',\n+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)\n \n     # Initializing index\n     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\ndiff --git a/applications/question_answering/faq_finance/ann_util.py b/applications/question_answering/faq_finance/ann_util.py\n--- a/applications/question_answering/faq_finance/ann_util.py\n+++ b/applications/question_answering/faq_finance/ann_util.py\n@@ -19,7 +19,9 @@\n \n def build_index(args, data_loader, model):\n \n-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)\n+    index = hnswlib.Index(\n+        space='ip',\n+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)\n \n     # Initializing index\n     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\ndiff --git a/applications/question_answering/faq_system/ann_util.py b/applications/question_answering/faq_system/ann_util.py\n--- a/applications/question_answering/faq_system/ann_util.py\n+++ b/applications/question_answering/faq_system/ann_util.py\n@@ -19,7 +19,9 @@\n \n def build_index(args, data_loader, model):\n \n-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)\n+    index = hnswlib.Index(\n+        space='ip',\n+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)\n \n     # Initializing index\n     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n", "issue": "ann_utils \u5f53 output_emb_size = 0 \u65f6\u62a5\u9519\n\u6b22\u8fce\u60a8\u53cd\u9988PaddleNLP\u4f7f\u7528\u95ee\u9898\uff0c\u975e\u5e38\u611f\u8c22\u60a8\u5bf9PaddleNLP\u7684\u8d21\u732e\uff01\r\n\u5728\u7559\u4e0b\u60a8\u7684\u95ee\u9898\u65f6\uff0c\u8f9b\u82e6\u60a8\u540c\u6b65\u63d0\u4f9b\u5982\u4e0b\u4fe1\u606f\uff1a\r\n- \u7248\u672c\u3001\u73af\u5883\u4fe1\u606f\r\n1\uff09PaddleNLP\u548cPaddlePaddle\u7248\u672c\uff1a\u8bf7\u63d0\u4f9b\u60a8\u7684PaddleNLP\u548cPaddlePaddle\u7248\u672c\u53f7\uff0c\u4f8b\u5982PaddleNLP 2.0.4\uff0cPaddlePaddle2.1.1\r\n2\uff09\u7cfb\u7edf\u73af\u5883\uff1a\u8bf7\u60a8\u63cf\u8ff0\u7cfb\u7edf\u7c7b\u578b\uff0c\u4f8b\u5982Linux/Windows/MacOS/\uff0cpython\u7248\u672c\r\n- \u590d\u73b0\u4fe1\u606f\uff1a\u5982\u4e3a\u62a5\u9519\uff0c\u8bf7\u7ed9\u51fa\u590d\u73b0\u73af\u5883\u3001\u590d\u73b0\u6b65\u9aa4\r\n\n", "before_files": [{"content": "# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# coding=UTF-8\n\nimport numpy as np\nimport hnswlib\nfrom paddlenlp.utils.log import logger\n\n\ndef build_index(args, data_loader, model):\n\n    index = hnswlib.Index(space='ip', dim=args.output_emb_size)\n\n    # Initializing index\n    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n    # during insertion of an element.\n    # The capacity can be increased by saving/loading the index, see below.\n    #\n    # ef_construction - controls index search speed/build speed tradeoff\n    #\n    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)\n    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction\n    index.init_index(\n        max_elements=args.hnsw_max_elements,\n        ef_construction=args.hnsw_ef,\n        M=args.hnsw_m)\n\n    # Controlling the recall by setting ef:\n    # higher ef leads to better accuracy, but slower search\n    index.set_ef(args.hnsw_ef)\n\n    # Set number of threads used during batch search/construction\n    # By default using all available cores\n    index.set_num_threads(16)\n\n    logger.info(\"start build index..........\")\n\n    all_embeddings = []\n\n    for text_embeddings in model.get_semantic_embedding(data_loader):\n        all_embeddings.append(text_embeddings.numpy())\n\n    all_embeddings = np.concatenate(all_embeddings, axis=0)\n    index.add_items(all_embeddings)\n\n    logger.info(\"Total index number:{}\".format(index.get_current_count()))\n\n    return index\n", "path": "applications/neural_search/recall/in_batch_negative/ann_util.py"}, {"content": "# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport hnswlib\nfrom paddlenlp.utils.log import logger\n\n\ndef build_index(args, data_loader, model):\n\n    index = hnswlib.Index(space='ip', dim=args.output_emb_size)\n\n    # Initializing index\n    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n    # during insertion of an element.\n    # The capacity can be increased by saving/loading the index, see below.\n    #\n    # ef_construction - controls index search speed/build speed tradeoff\n    #\n    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)\n    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction\n    index.init_index(\n        max_elements=args.hnsw_max_elements,\n        ef_construction=args.hnsw_ef,\n        M=args.hnsw_m)\n\n    # Controlling the recall by setting ef:\n    # higher ef leads to better accuracy, but slower search\n    index.set_ef(args.hnsw_ef)\n\n    # Set number of threads used during batch search/construction\n    # By default using all available cores\n    index.set_num_threads(16)\n\n    logger.info(\"start build index..........\")\n\n    all_embeddings = []\n\n    for text_embeddings in model.get_semantic_embedding(data_loader):\n        all_embeddings.append(text_embeddings.numpy())\n\n    all_embeddings = np.concatenate(all_embeddings, axis=0)\n    index.add_items(all_embeddings)\n\n    logger.info(\"Total index number:{}\".format(index.get_current_count()))\n\n    return index\n", "path": "applications/question_answering/faq_system/ann_util.py"}, {"content": "# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport hnswlib\nfrom paddlenlp.utils.log import logger\n\n\ndef build_index(args, data_loader, model):\n\n    index = hnswlib.Index(space='ip', dim=args.output_emb_size)\n\n    # Initializing index\n    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n    # during insertion of an element.\n    # The capacity can be increased by saving/loading the index, see below.\n    #\n    # ef_construction - controls index search speed/build speed tradeoff\n    #\n    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)\n    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction\n    index.init_index(\n        max_elements=args.hnsw_max_elements,\n        ef_construction=args.hnsw_ef,\n        M=args.hnsw_m)\n\n    # Controlling the recall by setting ef:\n    # higher ef leads to better accuracy, but slower search\n    index.set_ef(args.hnsw_ef)\n\n    # Set number of threads used during batch search/construction\n    # By default using all available cores\n    index.set_num_threads(16)\n\n    logger.info(\"start build index..........\")\n\n    all_embeddings = []\n\n    for text_embeddings in model.get_semantic_embedding(data_loader):\n        all_embeddings.append(text_embeddings.numpy())\n\n    all_embeddings = np.concatenate(all_embeddings, axis=0)\n    index.add_items(all_embeddings)\n\n    logger.info(\"Total index number:{}\".format(index.get_current_count()))\n\n    return index\n", "path": "applications/question_answering/faq_finance/ann_util.py"}], "after_files": [{"content": "# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# coding=UTF-8\n\nimport numpy as np\nimport hnswlib\nfrom paddlenlp.utils.log import logger\n\n\ndef build_index(args, data_loader, model):\n\n    index = hnswlib.Index(\n        space='ip',\n        dim=args.output_emb_size if args.output_emb_size > 0 else 768)\n\n    # Initializing index\n    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n    # during insertion of an element.\n    # The capacity can be increased by saving/loading the index, see below.\n    #\n    # ef_construction - controls index search speed/build speed tradeoff\n    #\n    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)\n    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction\n    index.init_index(\n        max_elements=args.hnsw_max_elements,\n        ef_construction=args.hnsw_ef,\n        M=args.hnsw_m)\n\n    # Controlling the recall by setting ef:\n    # higher ef leads to better accuracy, but slower search\n    index.set_ef(args.hnsw_ef)\n\n    # Set number of threads used during batch search/construction\n    # By default using all available cores\n    index.set_num_threads(16)\n\n    logger.info(\"start build index..........\")\n\n    all_embeddings = []\n\n    for text_embeddings in model.get_semantic_embedding(data_loader):\n        all_embeddings.append(text_embeddings.numpy())\n\n    all_embeddings = np.concatenate(all_embeddings, axis=0)\n    index.add_items(all_embeddings)\n\n    logger.info(\"Total index number:{}\".format(index.get_current_count()))\n\n    return index\n", "path": "applications/neural_search/recall/in_batch_negative/ann_util.py"}, {"content": "# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport hnswlib\nfrom paddlenlp.utils.log import logger\n\n\ndef build_index(args, data_loader, model):\n\n    index = hnswlib.Index(\n        space='ip',\n        dim=args.output_emb_size if args.output_emb_size > 0 else 768)\n\n    # Initializing index\n    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n    # during insertion of an element.\n    # The capacity can be increased by saving/loading the index, see below.\n    #\n    # ef_construction - controls index search speed/build speed tradeoff\n    #\n    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)\n    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction\n    index.init_index(\n        max_elements=args.hnsw_max_elements,\n        ef_construction=args.hnsw_ef,\n        M=args.hnsw_m)\n\n    # Controlling the recall by setting ef:\n    # higher ef leads to better accuracy, but slower search\n    index.set_ef(args.hnsw_ef)\n\n    # Set number of threads used during batch search/construction\n    # By default using all available cores\n    index.set_num_threads(16)\n\n    logger.info(\"start build index..........\")\n\n    all_embeddings = []\n\n    for text_embeddings in model.get_semantic_embedding(data_loader):\n        all_embeddings.append(text_embeddings.numpy())\n\n    all_embeddings = np.concatenate(all_embeddings, axis=0)\n    index.add_items(all_embeddings)\n\n    logger.info(\"Total index number:{}\".format(index.get_current_count()))\n\n    return index\n", "path": "applications/question_answering/faq_system/ann_util.py"}, {"content": "# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport hnswlib\nfrom paddlenlp.utils.log import logger\n\n\ndef build_index(args, data_loader, model):\n\n    index = hnswlib.Index(\n        space='ip',\n        dim=args.output_emb_size if args.output_emb_size > 0 else 768)\n\n    # Initializing index\n    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded\n    # during insertion of an element.\n    # The capacity can be increased by saving/loading the index, see below.\n    #\n    # ef_construction - controls index search speed/build speed tradeoff\n    #\n    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)\n    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction\n    index.init_index(\n        max_elements=args.hnsw_max_elements,\n        ef_construction=args.hnsw_ef,\n        M=args.hnsw_m)\n\n    # Controlling the recall by setting ef:\n    # higher ef leads to better accuracy, but slower search\n    index.set_ef(args.hnsw_ef)\n\n    # Set number of threads used during batch search/construction\n    # By default using all available cores\n    index.set_num_threads(16)\n\n    logger.info(\"start build index..........\")\n\n    all_embeddings = []\n\n    for text_embeddings in model.get_semantic_embedding(data_loader):\n        all_embeddings.append(text_embeddings.numpy())\n\n    all_embeddings = np.concatenate(all_embeddings, axis=0)\n    index.add_items(all_embeddings)\n\n    logger.info(\"Total index number:{}\".format(index.get_current_count()))\n\n    return index\n", "path": "applications/question_answering/faq_finance/ann_util.py"}]}