We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
XGBoost Model Perfomance Issue
/kind bug
XGboost model server perform poorly after deployed to k8s.

**What steps did you take and what happened:**
I did performance test for xgbserver using provided sample model.

`load.cfg`:
```
POST http://<target>/v1/models/default:predict
@request.json
```

`req.json`:
```
{"instances": [[1,2,3,4]]}
```
Running locally (on Mac OS, 8 core machine) with vegeta
```
cat load.cfg | vegeta attack -duration 1m -rate 1000 | vegeta report -type=text
```

I am able to hit 1000 RPS with good latencies result.

```
Requests      [total, rate, throughput]    60000, 1000.02, 1000.00
Duration      [total, attack, wait]        1m0.000012024s, 59.999069988s, 942.036µs
Latencies     [mean, 50, 90, 95, 99, max]  1.070524ms, 946.443µs, 1.414653ms, 1.520775ms, 2.401196ms, 17.650717ms
Bytes In      [total, mean]                1320000, 22.00
Bytes Out     [total, mean]                1560000, 26.00
Success       [ratio]                      100.00%
Status Codes  [code:count]                 200:60000  
```

But after deployed to K8S (with 4 core as CPU request and limit)

The performance deteriorates at 100 RPS
```
cat load-1.cfg | vegeta attack -rate 100 -duration 60s | vegeta report
```

```
Requests      [total, rate, throughput]    6000, 100.02, 89.47
Duration      [total, attack, wait]        1m7.064629048s, 59.989775627s, 7.074853421s
Latencies     [mean, 50, 90, 95, 99, max]  1.524605151s, 3.683733ms, 5.575147148s, 7.007148666s, 7.789309456s, 8.606190814s
Bytes In      [total, mean]                132000, 22.00
Bytes Out     [total, mean]                192000, 32.00
Success       [ratio]                      100.00%
Status Codes  [code:count]                 200:6000
```

**What did you expect to happen:**

The performance should not be affected too much after deployed to K8S.

**Anything else you would like to add:**
I suspected the issue come from `nthread` in xgbserver are left to default (i.e. 0). 
In: 
https://github.com/kubeflow/kfserving/blob/c46a0da8c0cf168dccfcee3b79f306fea135faf4/python/xgbserver/xgbserver/model.py#L36
and
https://github.com/kubeflow/kfserving/blob/c46a0da8c0cf168dccfcee3b79f306fea135faf4/python/xgbserver/xgbserver/model.py#L42

Which will leave the decision of number of thread to use to xgboost by detecting number of cpu. However, possibly due to this https://bugs.python.org/issue36054, the `nthread` is incorrect and causing the performance issues.

I did a test with setting `nthread=1` :
```
class XGBoostModel(kfserving.KFModel):
    def __init__(self, name: str, model_dir: str, booster: XGBModel = None):
        super().__init__(name)
        self.name = name
        self.model_dir = model_dir
        if not booster is None:
            self._booster = booster
            self.ready = True

    def load(self):
        model_file = os.path.join(
            kfserving.Storage.download(self.model_dir), BOOSTER_FILE)
        self._booster = xgb.Booster(params={"nthread" = 1},model_file=model_file)
        self.ready = True

    def predict(self, request: Dict) -> Dict:
        try:
            # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970
            dmatrix = xgb.DMatrix(request["instances"], nthread=1)
            result: xgb.DMatrix = self._booster.predict(dmatrix)
            return { "predictions": result.tolist() }
        except Exception as e:
            raise Exception("Failed to predict %s" % e)
```

```
cat load.cfg | vegeta attack -rate 1000 -duration 60s | vegeta report
```

and it performed much better:

```
Requests      [total, rate, throughput]    60000, 1000.02, 999.99
Duration      [total, attack, wait]        1m0.00064503s, 59.9988685s, 1.77653ms
Latencies     [mean, 50, 90, 95, 99, max]  1.927362ms, 1.73989ms, 1.941139ms, 2.490536ms, 5.660895ms, 56.707145ms
Bytes In      [total, mean]                1320000, 22.00
Bytes Out     [total, mean]                1920000, 32.00
Success       [ratio]                      100.00%
Status Codes  [code:count]                 200:60000
```

The question here is:
1. Is setting `nthread=1` a good idea for other kind of workload?
2. Should this be configurable?

**Environment:**

- Istio Version: 1.1.7
- Knative Version: 0.8.1
- KFServing Version: 0.2.1
- Kubeflow version: NA
- Minikube version: NA
- Kubernetes version: (use `kubectl version`):
- OS (e.g. from `/etc/os-release`):


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `python/xgbserver/xgbserver/model.py`
Content:
```
1 # Copyright 2019 kubeflow.org.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #    http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import kfserving
16 import xgboost as xgb
17 from xgboost import XGBModel
18 import os
19 import numpy as np
20 from typing import List, Dict
21 
22 BOOSTER_FILE = "model.bst"
23 
24 class XGBoostModel(kfserving.KFModel):
25     def __init__(self, name: str, model_dir: str, booster: XGBModel = None):
26         super().__init__(name)
27         self.name = name
28         self.model_dir = model_dir
29         if not booster is None:
30             self._booster = booster
31             self.ready = True
32 
33     def load(self):
34         model_file = os.path.join(
35             kfserving.Storage.download(self.model_dir), BOOSTER_FILE)
36         self._booster = xgb.Booster(model_file=model_file)
37         self.ready = True
38 
39     def predict(self, request: Dict) -> Dict:
40         try:
41             # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970
42             dmatrix = xgb.DMatrix(request["instances"])
43             result: xgb.DMatrix = self._booster.predict(dmatrix)
44             return { "predictions": result.tolist() }
45         except Exception as e:
46             raise Exception("Failed to predict %s" % e)
47
```

Path: `python/xgbserver/xgbserver/__main__.py`
Content:
```
1 # Copyright 2019 kubeflow.org.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #    http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import kfserving
16 import argparse
17 
18 from xgbserver import XGBoostModel
19 
20 DEFAULT_MODEL_NAME = "default"
21 DEFAULT_LOCAL_MODEL_DIR = "/tmp/model"
22 
23 parser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser]) #pylint:disable=c-extension-no-member
24 parser.add_argument('--model_dir', required=True,
25                     help='A URI pointer to the model directory')
26 parser.add_argument('--model_name', default=DEFAULT_MODEL_NAME,
27                     help='The name that the model is served under.')
28 args, _ = parser.parse_known_args()
29 
30 if __name__ == "__main__":
31     model = XGBoostModel(args.model_name, args.model_dir)
32     model.load()
33     kfserving.KFServer().start([model]) #pylint:disable=c-extension-no-member
34
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/python/xgbserver/xgbserver/__main__.py b/python/xgbserver/xgbserver/__main__.py
--- a/python/xgbserver/xgbserver/__main__.py
+++ b/python/xgbserver/xgbserver/__main__.py
@@ -19,15 +19,18 @@
 
 DEFAULT_MODEL_NAME = "default"
 DEFAULT_LOCAL_MODEL_DIR = "/tmp/model"
+DEFAULT_NTHREAD = 1
 
 parser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser]) #pylint:disable=c-extension-no-member
 parser.add_argument('--model_dir', required=True,
                     help='A URI pointer to the model directory')
 parser.add_argument('--model_name', default=DEFAULT_MODEL_NAME,
                     help='The name that the model is served under.')
+parser.add_argument('--nthread', default=DEFAULT_NTHREAD,
+                    help='Number of threads to use by XGBoost.')
 args, _ = parser.parse_known_args()
 
 if __name__ == "__main__":
-    model = XGBoostModel(args.model_name, args.model_dir)
+    model = XGBoostModel(args.model_name, args.model_dir, args.nthread)
     model.load()
     kfserving.KFServer().start([model]) #pylint:disable=c-extension-no-member
diff --git a/python/xgbserver/xgbserver/model.py b/python/xgbserver/xgbserver/model.py
--- a/python/xgbserver/xgbserver/model.py
+++ b/python/xgbserver/xgbserver/model.py
@@ -22,10 +22,12 @@
 BOOSTER_FILE = "model.bst"
 
 class XGBoostModel(kfserving.KFModel):
-    def __init__(self, name: str, model_dir: str, booster: XGBModel = None):
+    def __init__(self, name: str, model_dir: str, nthread: int, booster: \
+        XGBModel = None):
         super().__init__(name)
         self.name = name
         self.model_dir = model_dir
+        self.nthread = nthread
         if not booster is None:
             self._booster = booster
             self.ready = True
@@ -33,13 +35,14 @@
     def load(self):
         model_file = os.path.join(
             kfserving.Storage.download(self.model_dir), BOOSTER_FILE)
-        self._booster = xgb.Booster(model_file=model_file)
+        self._booster = xgb.Booster(params={"nthread" : self.nthread},
+                                    model_file=model_file)
         self.ready = True
 
     def predict(self, request: Dict) -> Dict:
         try:
             # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970
-            dmatrix = xgb.DMatrix(request["instances"])
+            dmatrix = xgb.DMatrix(request["instances"], nthread=self.nthread)
             result: xgb.DMatrix = self._booster.predict(dmatrix)
             return { "predictions": result.tolist() }
         except Exception as e:

{"golden_diff": "diff --git a/python/xgbserver/xgbserver/__main__.py b/python/xgbserver/xgbserver/__main__.py\n--- a/python/xgbserver/xgbserver/__main__.py\n+++ b/python/xgbserver/xgbserver/__main__.py\n@@ -19,15 +19,18 @@\n \n DEFAULT_MODEL_NAME = \"default\"\n DEFAULT_LOCAL_MODEL_DIR = \"/tmp/model\"\n+DEFAULT_NTHREAD = 1\n \n parser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser]) #pylint:disable=c-extension-no-member\n parser.add_argument('--model_dir', required=True,\n                     help='A URI pointer to the model directory')\n parser.add_argument('--model_name', default=DEFAULT_MODEL_NAME,\n                     help='The name that the model is served under.')\n+parser.add_argument('--nthread', default=DEFAULT_NTHREAD,\n+                    help='Number of threads to use by XGBoost.')\n args, _ = parser.parse_known_args()\n \n if __name__ == \"__main__\":\n-    model = XGBoostModel(args.model_name, args.model_dir)\n+    model = XGBoostModel(args.model_name, args.model_dir, args.nthread)\n     model.load()\n     kfserving.KFServer().start([model]) #pylint:disable=c-extension-no-member\ndiff --git a/python/xgbserver/xgbserver/model.py b/python/xgbserver/xgbserver/model.py\n--- a/python/xgbserver/xgbserver/model.py\n+++ b/python/xgbserver/xgbserver/model.py\n@@ -22,10 +22,12 @@\n BOOSTER_FILE = \"model.bst\"\n \n class XGBoostModel(kfserving.KFModel):\n-    def __init__(self, name: str, model_dir: str, booster: XGBModel = None):\n+    def __init__(self, name: str, model_dir: str, nthread: int, booster: \\\n+        XGBModel = None):\n         super().__init__(name)\n         self.name = name\n         self.model_dir = model_dir\n+        self.nthread = nthread\n         if not booster is None:\n             self._booster = booster\n             self.ready = True\n@@ -33,13 +35,14 @@\n     def load(self):\n         model_file = os.path.join(\n             kfserving.Storage.download(self.model_dir), BOOSTER_FILE)\n-        self._booster = xgb.Booster(model_file=model_file)\n+        self._booster = xgb.Booster(params={\"nthread\" : self.nthread},\n+                                    model_file=model_file)\n         self.ready = True\n \n     def predict(self, request: Dict) -> Dict:\n         try:\n             # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970\n-            dmatrix = xgb.DMatrix(request[\"instances\"])\n+            dmatrix = xgb.DMatrix(request[\"instances\"], nthread=self.nthread)\n             result: xgb.DMatrix = self._booster.predict(dmatrix)\n             return { \"predictions\": result.tolist() }\n         except Exception as e:\n", "issue": "XGBoost Model Perfomance Issue\n/kind bug\r\nXGboost model server perform poorly after deployed to k8s.\r\n\r\n**What steps did you take and what happened:**\r\nI did performance test for xgbserver using provided sample model.\r\n\r\n`load.cfg`:\r\n```\r\nPOST http://<target>/v1/models/default:predict\r\n@request.json\r\n```\r\n\r\n`req.json`:\r\n```\r\n{\"instances\": [[1,2,3,4]]}\r\n```\r\nRunning locally (on Mac OS, 8 core machine) with vegeta\r\n```\r\ncat load.cfg | vegeta attack -duration 1m -rate 1000 | vegeta report -type=text\r\n```\r\n\r\nI am able to hit 1000 RPS with good latencies result.\r\n\r\n```\r\nRequests      [total, rate, throughput]    60000, 1000.02, 1000.00\r\nDuration      [total, attack, wait]        1m0.000012024s, 59.999069988s, 942.036\u00b5s\r\nLatencies     [mean, 50, 90, 95, 99, max]  1.070524ms, 946.443\u00b5s, 1.414653ms, 1.520775ms, 2.401196ms, 17.650717ms\r\nBytes In      [total, mean]                1320000, 22.00\r\nBytes Out     [total, mean]                1560000, 26.00\r\nSuccess       [ratio]                      100.00%\r\nStatus Codes  [code:count]                 200:60000  \r\n```\r\n\r\nBut after deployed to K8S (with 4 core as CPU request and limit)\r\n\r\nThe performance deteriorates at 100 RPS\r\n```\r\ncat load-1.cfg | vegeta attack -rate 100 -duration 60s | vegeta report\r\n```\r\n\r\n```\r\nRequests      [total, rate, throughput]    6000, 100.02, 89.47\r\nDuration      [total, attack, wait]        1m7.064629048s, 59.989775627s, 7.074853421s\r\nLatencies     [mean, 50, 90, 95, 99, max]  1.524605151s, 3.683733ms, 5.575147148s, 7.007148666s, 7.789309456s, 8.606190814s\r\nBytes In      [total, mean]                132000, 22.00\r\nBytes Out     [total, mean]                192000, 32.00\r\nSuccess       [ratio]                      100.00%\r\nStatus Codes  [code:count]                 200:6000\r\n```\r\n\r\n**What did you expect to happen:**\r\n\r\nThe performance should not be affected too much after deployed to K8S.\r\n\r\n**Anything else you would like to add:**\r\nI suspected the issue come from `nthread` in xgbserver are left to default (i.e. 0). \r\nIn: \r\nhttps://github.com/kubeflow/kfserving/blob/c46a0da8c0cf168dccfcee3b79f306fea135faf4/python/xgbserver/xgbserver/model.py#L36\r\nand\r\nhttps://github.com/kubeflow/kfserving/blob/c46a0da8c0cf168dccfcee3b79f306fea135faf4/python/xgbserver/xgbserver/model.py#L42\r\n\r\nWhich will leave the decision of number of thread to use to xgboost by detecting number of cpu. However, possibly due to this https://bugs.python.org/issue36054, the `nthread` is incorrect and causing the performance issues.\r\n\r\nI did a test with setting `nthread=1` :\r\n```\r\nclass XGBoostModel(kfserving.KFModel):\r\n    def __init__(self, name: str, model_dir: str, booster: XGBModel = None):\r\n        super().__init__(name)\r\n        self.name = name\r\n        self.model_dir = model_dir\r\n        if not booster is None:\r\n            self._booster = booster\r\n            self.ready = True\r\n\r\n    def load(self):\r\n        model_file = os.path.join(\r\n            kfserving.Storage.download(self.model_dir), BOOSTER_FILE)\r\n        self._booster = xgb.Booster(params={\"nthread\" = 1},model_file=model_file)\r\n        self.ready = True\r\n\r\n    def predict(self, request: Dict) -> Dict:\r\n        try:\r\n            # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970\r\n            dmatrix = xgb.DMatrix(request[\"instances\"], nthread=1)\r\n            result: xgb.DMatrix = self._booster.predict(dmatrix)\r\n            return { \"predictions\": result.tolist() }\r\n        except Exception as e:\r\n            raise Exception(\"Failed to predict %s\" % e)\r\n```\r\n\r\n```\r\ncat load.cfg | vegeta attack -rate 1000 -duration 60s | vegeta report\r\n```\r\n\r\nand it performed much better:\r\n\r\n```\r\nRequests      [total, rate, throughput]    60000, 1000.02, 999.99\r\nDuration      [total, attack, wait]        1m0.00064503s, 59.9988685s, 1.77653ms\r\nLatencies     [mean, 50, 90, 95, 99, max]  1.927362ms, 1.73989ms, 1.941139ms, 2.490536ms, 5.660895ms, 56.707145ms\r\nBytes In      [total, mean]                1320000, 22.00\r\nBytes Out     [total, mean]                1920000, 32.00\r\nSuccess       [ratio]                      100.00%\r\nStatus Codes  [code:count]                 200:60000\r\n```\r\n\r\nThe question here is:\r\n1. Is setting `nthread=1` a good idea for other kind of workload?\r\n2. Should this be configurable?\r\n\r\n**Environment:**\r\n\r\n- Istio Version: 1.1.7\r\n- Knative Version: 0.8.1\r\n- KFServing Version: 0.2.1\r\n- Kubeflow version: NA\r\n- Minikube version: NA\r\n- Kubernetes version: (use `kubectl version`):\r\n- OS (e.g. from `/etc/os-release`):\r\n\n", "before_files": [{"content": "# Copyright 2019 kubeflow.org.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#    http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport kfserving\nimport xgboost as xgb\nfrom xgboost import XGBModel\nimport os\nimport numpy as np\nfrom typing import List, Dict\n\nBOOSTER_FILE = \"model.bst\"\n\nclass XGBoostModel(kfserving.KFModel):\n    def __init__(self, name: str, model_dir: str, booster: XGBModel = None):\n        super().__init__(name)\n        self.name = name\n        self.model_dir = model_dir\n        if not booster is None:\n            self._booster = booster\n            self.ready = True\n\n    def load(self):\n        model_file = os.path.join(\n            kfserving.Storage.download(self.model_dir), BOOSTER_FILE)\n        self._booster = xgb.Booster(model_file=model_file)\n        self.ready = True\n\n    def predict(self, request: Dict) -> Dict:\n        try:\n            # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970\n            dmatrix = xgb.DMatrix(request[\"instances\"])\n            result: xgb.DMatrix = self._booster.predict(dmatrix)\n            return { \"predictions\": result.tolist() }\n        except Exception as e:\n            raise Exception(\"Failed to predict %s\" % e)\n", "path": "python/xgbserver/xgbserver/model.py"}, {"content": "# Copyright 2019 kubeflow.org.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#    http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport kfserving\nimport argparse\n\nfrom xgbserver import XGBoostModel\n\nDEFAULT_MODEL_NAME = \"default\"\nDEFAULT_LOCAL_MODEL_DIR = \"/tmp/model\"\n\nparser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser]) #pylint:disable=c-extension-no-member\nparser.add_argument('--model_dir', required=True,\n                    help='A URI pointer to the model directory')\nparser.add_argument('--model_name', default=DEFAULT_MODEL_NAME,\n                    help='The name that the model is served under.')\nargs, _ = parser.parse_known_args()\n\nif __name__ == \"__main__\":\n    model = XGBoostModel(args.model_name, args.model_dir)\n    model.load()\n    kfserving.KFServer().start([model]) #pylint:disable=c-extension-no-member\n", "path": "python/xgbserver/xgbserver/__main__.py"}], "after_files": [{"content": "# Copyright 2019 kubeflow.org.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#    http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport kfserving\nimport xgboost as xgb\nfrom xgboost import XGBModel\nimport os\nimport numpy as np\nfrom typing import List, Dict\n\nBOOSTER_FILE = \"model.bst\"\n\nclass XGBoostModel(kfserving.KFModel):\n    def __init__(self, name: str, model_dir: str, nthread: int, booster: \\\n        XGBModel = None):\n        super().__init__(name)\n        self.name = name\n        self.model_dir = model_dir\n        self.nthread = nthread\n        if not booster is None:\n            self._booster = booster\n            self.ready = True\n\n    def load(self):\n        model_file = os.path.join(\n            kfserving.Storage.download(self.model_dir), BOOSTER_FILE)\n        self._booster = xgb.Booster(params={\"nthread\" : self.nthread},\n                                    model_file=model_file)\n        self.ready = True\n\n    def predict(self, request: Dict) -> Dict:\n        try:\n            # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970\n            dmatrix = xgb.DMatrix(request[\"instances\"], nthread=self.nthread)\n            result: xgb.DMatrix = self._booster.predict(dmatrix)\n            return { \"predictions\": result.tolist() }\n        except Exception as e:\n            raise Exception(\"Failed to predict %s\" % e)\n", "path": "python/xgbserver/xgbserver/model.py"}, {"content": "# Copyright 2019 kubeflow.org.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#    http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport kfserving\nimport argparse\n\nfrom xgbserver import XGBoostModel\n\nDEFAULT_MODEL_NAME = \"default\"\nDEFAULT_LOCAL_MODEL_DIR = \"/tmp/model\"\nDEFAULT_NTHREAD = 1\n\nparser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser]) #pylint:disable=c-extension-no-member\nparser.add_argument('--model_dir', required=True,\n                    help='A URI pointer to the model directory')\nparser.add_argument('--model_name', default=DEFAULT_MODEL_NAME,\n                    help='The name that the model is served under.')\nparser.add_argument('--nthread', default=DEFAULT_NTHREAD,\n                    help='Number of threads to use by XGBoost.')\nargs, _ = parser.parse_known_args()\n\nif __name__ == \"__main__\":\n    model = XGBoostModel(args.model_name, args.model_dir, args.nthread)\n    model.load()\n    kfserving.KFServer().start([model]) #pylint:disable=c-extension-no-member\n", "path": "python/xgbserver/xgbserver/__main__.py"}]}