We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Add underperforming_group issue type among the Datalab defaults 
Test issue manager with different datasets (Image, tabular etc.) to make sure that the underperforming group in the dataset is extracted successfully.  List any failure cases that might need to be addressed before adding this issue type to the defaults.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `cleanlab/datalab/internal/issue_manager_factory.py`
Content:
```
1 # Copyright (C) 2017-2023  Cleanlab Inc.
2 # This file is part of cleanlab.
3 #
4 # cleanlab is free software: you can redistribute it and/or modify
5 # it under the terms of the GNU Affero General Public License as published
6 # by the Free Software Foundation, either version 3 of the License, or
7 # (at your option) any later version.
8 #
9 # cleanlab is distributed in the hope that it will be useful,
10 # but WITHOUT ANY WARRANTY; without even the implied warranty of
11 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
12 # GNU Affero General Public License for more details.
13 #
14 # You should have received a copy of the GNU Affero General Public License
15 # along with cleanlab.  If not, see <https://www.gnu.org/licenses/>.
16 """The factory module provides a factory class for constructing concrete issue managers
17 and a decorator for registering new issue managers.
18 
19 This module provides the :py:meth:`register` decorator for users to register new subclasses of
20 :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`
21 in the registry. Each IssueManager detects some particular type of issue in a dataset.
22 
23 
24 Note
25 ----
26 
27 The :class:`REGISTRY` variable is used by the factory class to keep track
28 of registered issue managers.
29 The factory class is used as an implementation detail by
30 :py:class:`Datalab <cleanlab.datalab.datalab.Datalab>`,
31 which provides a simplified API for constructing concrete issue managers.
32 :py:class:`Datalab <cleanlab.datalab.datalab.Datalab>` is intended to be used by users
33 and provides detailed documentation on how to use the API.
34 
35 Warning
36 -------
37 Neither the :class:`REGISTRY` variable nor the factory class should be used directly by users.
38 """
39 from __future__ import annotations
40 
41 from typing import Dict, List, Type
42 
43 from cleanlab.datalab.internal.issue_manager import (
44     ClassImbalanceIssueManager,
45     DataValuationIssueManager,
46     IssueManager,
47     LabelIssueManager,
48     NearDuplicateIssueManager,
49     NonIIDIssueManager,
50     ClassImbalanceIssueManager,
51     UnderperformingGroupIssueManager,
52     DataValuationIssueManager,
53     OutlierIssueManager,
54     NullIssueManager,
55 )
56 from cleanlab.datalab.internal.issue_manager.regression import RegressionLabelIssueManager
57 from cleanlab.datalab.internal.issue_manager.multilabel.label import MultilabelIssueManager
58 from cleanlab.datalab.internal.task import Task
59 
60 
61 REGISTRY: Dict[Task, Dict[str, Type[IssueManager]]] = {
62     Task.CLASSIFICATION: {
63         "outlier": OutlierIssueManager,
64         "label": LabelIssueManager,
65         "near_duplicate": NearDuplicateIssueManager,
66         "non_iid": NonIIDIssueManager,
67         "class_imbalance": ClassImbalanceIssueManager,
68         "underperforming_group": UnderperformingGroupIssueManager,
69         "data_valuation": DataValuationIssueManager,
70         "null": NullIssueManager,
71     },
72     Task.REGRESSION: {
73         "label": RegressionLabelIssueManager,
74         "outlier": OutlierIssueManager,
75         "near_duplicate": NearDuplicateIssueManager,
76         "non_iid": NonIIDIssueManager,
77         "null": NullIssueManager,
78     },
79     Task.MULTILABEL: {
80         "label": MultilabelIssueManager,
81         "outlier": OutlierIssueManager,
82         "near_duplicate": NearDuplicateIssueManager,
83         "non_iid": NonIIDIssueManager,
84         "null": NullIssueManager,
85     },
86 }
87 """Registry of issue managers that can be constructed from a task and issue type
88 and used in the Datalab class.
89 
90 :meta hide-value:
91 
92 Currently, the following issue managers are registered by default for a given task:
93 
94 - Classification:
95 
96     - ``"outlier"``: :py:class:`OutlierIssueManager <cleanlab.datalab.internal.issue_manager.outlier.OutlierIssueManager>`
97     - ``"label"``: :py:class:`LabelIssueManager <cleanlab.datalab.internal.issue_manager.label.LabelIssueManager>`
98     - ``"near_duplicate"``: :py:class:`NearDuplicateIssueManager <cleanlab.datalab.internal.issue_manager.duplicate.NearDuplicateIssueManager>`
99     - ``"non_iid"``: :py:class:`NonIIDIssueManager <cleanlab.datalab.internal.issue_manager.noniid.NonIIDIssueManager>`
100     - ``"class_imbalance"``: :py:class:`ClassImbalanceIssueManager <cleanlab.datalab.internal.issue_manager.class_imbalance.ClassImbalanceIssueManager>`
101     - ``"underperforming_group"``: :py:class:`UnderperformingGroupIssueManager <cleanlab.datalab.internal.issue_manager.underperforming_group.UnderperformingGroupIssueManager>`
102     - ``"data_valuation"``: :py:class:`DataValuationIssueManager <cleanlab.datalab.internal.issue_manager.data_valuation.DataValuationIssueManager>`
103     - ``"null"``: :py:class:`NullIssueManager <cleanlab.datalab.internal.issue_manager.null.NullIssueManager>`
104     
105 - Regression:
106 
107     - ``"label"``: :py:class:`RegressionLabelIssueManager <cleanlab.datalab.internal.issue_manager.regression.label.RegressionLabelIssueManager>`
108 
109 - Multilabel:
110 
111     - ``"label"``: :py:class:`MultilabelIssueManager <cleanlab.datalab.internal.issue_manager.multilabel.label.MultilabelIssueManager>`
112 
113 Warning
114 -------
115 This variable should not be used directly by users.
116 """
117 
118 
119 # Construct concrete issue manager with a from_str method
120 class _IssueManagerFactory:
121     """Factory class for constructing concrete issue managers."""
122 
123     @classmethod
124     def from_str(cls, issue_type: str, task: Task) -> Type[IssueManager]:
125         """Constructs a concrete issue manager class from a string."""
126         if isinstance(issue_type, list):
127             raise ValueError(
128                 "issue_type must be a string, not a list. Try using from_list instead."
129             )
130 
131         if task not in REGISTRY:
132             raise ValueError(f"Invalid task type: {task}, must be in {list(REGISTRY.keys())}")
133         if issue_type not in REGISTRY[task]:
134             raise ValueError(f"Invalid issue type: {issue_type} for task {task}")
135 
136         return REGISTRY[task][issue_type]
137 
138     @classmethod
139     def from_list(cls, issue_types: List[str], task: Task) -> List[Type[IssueManager]]:
140         """Constructs a list of concrete issue manager classes from a list of strings."""
141         return [cls.from_str(issue_type, task) for issue_type in issue_types]
142 
143 
144 def register(cls: Type[IssueManager], task: str = str(Task.CLASSIFICATION)) -> Type[IssueManager]:
145     """Registers the issue manager factory.
146 
147     Parameters
148     ----------
149     cls :
150         A subclass of
151         :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`.
152 
153     task :
154         Specific machine learning task like classification or regression.
155         See :py:meth:`Task.from_str <cleanlab.datalab.internal.task.Task.from_str>`` for more details,
156         to see which task type corresponds to which string.
157 
158     Returns
159     -------
160     cls :
161         The same class that was passed in.
162 
163     Example
164     -------
165 
166     When defining a new subclass of
167     :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`,
168     you can register it like so:
169 
170     .. code-block:: python
171 
172         from cleanlab import IssueManager
173         from cleanlab.datalab.internal.issue_manager_factory import register
174 
175         @register
176         class MyIssueManager(IssueManager):
177             issue_name: str = "my_issue"
178             def find_issues(self, **kwargs):
179                 # Some logic to find issues
180                 pass
181 
182     or in a function call:
183 
184     .. code-block:: python
185 
186         from cleanlab import IssueManager
187         from cleanlab.datalab.internal.issue_manager_factory import register
188 
189         class MyIssueManager(IssueManager):
190             issue_name: str = "my_issue"
191             def find_issues(self, **kwargs):
192                 # Some logic to find issues
193                 pass
194 
195         register(MyIssueManager, task="classification")
196     """
197 
198     if not issubclass(cls, IssueManager):
199         raise ValueError(f"Class {cls} must be a subclass of IssueManager")
200 
201     name: str = str(cls.issue_name)
202 
203     try:
204         _task = Task.from_str(task)
205         if _task not in REGISTRY:
206             raise ValueError(f"Invalid task type: {_task}, must be in {list(REGISTRY.keys())}")
207     except KeyError:
208         raise ValueError(f"Invalid task type: {task}, must be in {list(REGISTRY.keys())}")
209 
210     if name in REGISTRY[_task]:
211         print(
212             f"Warning: Overwriting existing issue manager {name} with {cls} for task {_task}."
213             "This may cause unexpected behavior."
214         )
215 
216     REGISTRY[_task][name] = cls
217     return cls
218 
219 
220 def list_possible_issue_types(task: Task) -> List[str]:
221     """Returns a list of all registered issue types.
222 
223     Any issue type that is not in this list cannot be used in the :py:meth:`find_issues` method.
224 
225     See Also
226     --------
227     :py:class:`REGISTRY <cleanlab.datalab.internal.issue_manager_factory.REGISTRY>` : All available issue types and their corresponding issue managers can be found here.
228     """
229     return list(REGISTRY.get(task, []))
230 
231 
232 def list_default_issue_types(task: Task) -> List[str]:
233     """Returns a list of the issue types that are run by default
234     when :py:meth:`find_issues` is called without specifying `issue_types`.
235 
236     task :
237         Specific machine learning task supported by Datalab.
238 
239     See Also
240     --------
241     :py:class:`REGISTRY <cleanlab.datalab.internal.issue_manager_factory.REGISTRY>` : All available issue types and their corresponding issue managers can be found here.
242     """
243     default_issue_types_dict = {
244         Task.CLASSIFICATION: [
245             "null",
246             "label",
247             "outlier",
248             "near_duplicate",
249             "non_iid",
250             "class_imbalance",
251         ],
252         Task.REGRESSION: [
253             "null",
254             "label",
255             "outlier",
256             "near_duplicate",
257             "non_iid",
258         ],
259         Task.MULTILABEL: [
260             "null",
261             "label",
262             "outlier",
263             "near_duplicate",
264             "non_iid",
265         ],
266     }
267     if task not in default_issue_types_dict:
268         task = Task.CLASSIFICATION
269     default_issue_types = default_issue_types_dict[task]
270     return default_issue_types
271
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/cleanlab/datalab/internal/issue_manager_factory.py b/cleanlab/datalab/internal/issue_manager_factory.py
--- a/cleanlab/datalab/internal/issue_manager_factory.py
+++ b/cleanlab/datalab/internal/issue_manager_factory.py
@@ -223,6 +223,7 @@
             "near_duplicate",
             "non_iid",
             "class_imbalance",
+            "underperforming_group",
         ],
         "regression": [
             "null",

{"golden_diff": "diff --git a/cleanlab/datalab/internal/issue_manager_factory.py b/cleanlab/datalab/internal/issue_manager_factory.py\n--- a/cleanlab/datalab/internal/issue_manager_factory.py\n+++ b/cleanlab/datalab/internal/issue_manager_factory.py\n@@ -223,6 +223,7 @@\n             \"near_duplicate\",\n             \"non_iid\",\n             \"class_imbalance\",\n+            \"underperforming_group\",\n         ],\n         \"regression\": [\n             \"null\",\n", "issue": "Add underperforming_group issue type among the Datalab defaults \nTest issue manager with different datasets (Image, tabular etc.) to make sure that the underperforming group in the dataset is extracted successfully.  List any failure cases that might need to be addressed before adding this issue type to the defaults.\n", "before_files": [{"content": "# Copyright (C) 2017-2023  Cleanlab Inc.\n# This file is part of cleanlab.\n#\n# cleanlab is free software: you can redistribute it and/or modify\n# it under the terms of the GNU Affero General Public License as published\n# by the Free Software Foundation, either version 3 of the License, or\n# (at your option) any later version.\n#\n# cleanlab is distributed in the hope that it will be useful,\n# but WITHOUT ANY WARRANTY; without even the implied warranty of\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n# GNU Affero General Public License for more details.\n#\n# You should have received a copy of the GNU Affero General Public License\n# along with cleanlab.  If not, see <https://www.gnu.org/licenses/>.\n\"\"\"The factory module provides a factory class for constructing concrete issue managers\nand a decorator for registering new issue managers.\n\nThis module provides the :py:meth:`register` decorator for users to register new subclasses of\n:py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`\nin the registry. Each IssueManager detects some particular type of issue in a dataset.\n\n\nNote\n----\n\nThe :class:`REGISTRY` variable is used by the factory class to keep track\nof registered issue managers.\nThe factory class is used as an implementation detail by\n:py:class:`Datalab <cleanlab.datalab.datalab.Datalab>`,\nwhich provides a simplified API for constructing concrete issue managers.\n:py:class:`Datalab <cleanlab.datalab.datalab.Datalab>` is intended to be used by users\nand provides detailed documentation on how to use the API.\n\nWarning\n-------\nNeither the :class:`REGISTRY` variable nor the factory class should be used directly by users.\n\"\"\"\nfrom __future__ import annotations\n\nfrom typing import Dict, List, Type\n\nfrom cleanlab.datalab.internal.issue_manager import (\n    ClassImbalanceIssueManager,\n    DataValuationIssueManager,\n    IssueManager,\n    LabelIssueManager,\n    NearDuplicateIssueManager,\n    NonIIDIssueManager,\n    ClassImbalanceIssueManager,\n    UnderperformingGroupIssueManager,\n    DataValuationIssueManager,\n    OutlierIssueManager,\n    NullIssueManager,\n)\nfrom cleanlab.datalab.internal.issue_manager.regression import RegressionLabelIssueManager\nfrom cleanlab.datalab.internal.issue_manager.multilabel.label import MultilabelIssueManager\nfrom cleanlab.datalab.internal.task import Task\n\n\nREGISTRY: Dict[Task, Dict[str, Type[IssueManager]]] = {\n    Task.CLASSIFICATION: {\n        \"outlier\": OutlierIssueManager,\n        \"label\": LabelIssueManager,\n        \"near_duplicate\": NearDuplicateIssueManager,\n        \"non_iid\": NonIIDIssueManager,\n        \"class_imbalance\": ClassImbalanceIssueManager,\n        \"underperforming_group\": UnderperformingGroupIssueManager,\n        \"data_valuation\": DataValuationIssueManager,\n        \"null\": NullIssueManager,\n    },\n    Task.REGRESSION: {\n        \"label\": RegressionLabelIssueManager,\n        \"outlier\": OutlierIssueManager,\n        \"near_duplicate\": NearDuplicateIssueManager,\n        \"non_iid\": NonIIDIssueManager,\n        \"null\": NullIssueManager,\n    },\n    Task.MULTILABEL: {\n        \"label\": MultilabelIssueManager,\n        \"outlier\": OutlierIssueManager,\n        \"near_duplicate\": NearDuplicateIssueManager,\n        \"non_iid\": NonIIDIssueManager,\n        \"null\": NullIssueManager,\n    },\n}\n\"\"\"Registry of issue managers that can be constructed from a task and issue type\nand used in the Datalab class.\n\n:meta hide-value:\n\nCurrently, the following issue managers are registered by default for a given task:\n\n- Classification:\n\n    - ``\"outlier\"``: :py:class:`OutlierIssueManager <cleanlab.datalab.internal.issue_manager.outlier.OutlierIssueManager>`\n    - ``\"label\"``: :py:class:`LabelIssueManager <cleanlab.datalab.internal.issue_manager.label.LabelIssueManager>`\n    - ``\"near_duplicate\"``: :py:class:`NearDuplicateIssueManager <cleanlab.datalab.internal.issue_manager.duplicate.NearDuplicateIssueManager>`\n    - ``\"non_iid\"``: :py:class:`NonIIDIssueManager <cleanlab.datalab.internal.issue_manager.noniid.NonIIDIssueManager>`\n    - ``\"class_imbalance\"``: :py:class:`ClassImbalanceIssueManager <cleanlab.datalab.internal.issue_manager.class_imbalance.ClassImbalanceIssueManager>`\n    - ``\"underperforming_group\"``: :py:class:`UnderperformingGroupIssueManager <cleanlab.datalab.internal.issue_manager.underperforming_group.UnderperformingGroupIssueManager>`\n    - ``\"data_valuation\"``: :py:class:`DataValuationIssueManager <cleanlab.datalab.internal.issue_manager.data_valuation.DataValuationIssueManager>`\n    - ``\"null\"``: :py:class:`NullIssueManager <cleanlab.datalab.internal.issue_manager.null.NullIssueManager>`\n    \n- Regression:\n\n    - ``\"label\"``: :py:class:`RegressionLabelIssueManager <cleanlab.datalab.internal.issue_manager.regression.label.RegressionLabelIssueManager>`\n\n- Multilabel:\n\n    - ``\"label\"``: :py:class:`MultilabelIssueManager <cleanlab.datalab.internal.issue_manager.multilabel.label.MultilabelIssueManager>`\n\nWarning\n-------\nThis variable should not be used directly by users.\n\"\"\"\n\n\n# Construct concrete issue manager with a from_str method\nclass _IssueManagerFactory:\n    \"\"\"Factory class for constructing concrete issue managers.\"\"\"\n\n    @classmethod\n    def from_str(cls, issue_type: str, task: Task) -> Type[IssueManager]:\n        \"\"\"Constructs a concrete issue manager class from a string.\"\"\"\n        if isinstance(issue_type, list):\n            raise ValueError(\n                \"issue_type must be a string, not a list. Try using from_list instead.\"\n            )\n\n        if task not in REGISTRY:\n            raise ValueError(f\"Invalid task type: {task}, must be in {list(REGISTRY.keys())}\")\n        if issue_type not in REGISTRY[task]:\n            raise ValueError(f\"Invalid issue type: {issue_type} for task {task}\")\n\n        return REGISTRY[task][issue_type]\n\n    @classmethod\n    def from_list(cls, issue_types: List[str], task: Task) -> List[Type[IssueManager]]:\n        \"\"\"Constructs a list of concrete issue manager classes from a list of strings.\"\"\"\n        return [cls.from_str(issue_type, task) for issue_type in issue_types]\n\n\ndef register(cls: Type[IssueManager], task: str = str(Task.CLASSIFICATION)) -> Type[IssueManager]:\n    \"\"\"Registers the issue manager factory.\n\n    Parameters\n    ----------\n    cls :\n        A subclass of\n        :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`.\n\n    task :\n        Specific machine learning task like classification or regression.\n        See :py:meth:`Task.from_str <cleanlab.datalab.internal.task.Task.from_str>`` for more details,\n        to see which task type corresponds to which string.\n\n    Returns\n    -------\n    cls :\n        The same class that was passed in.\n\n    Example\n    -------\n\n    When defining a new subclass of\n    :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`,\n    you can register it like so:\n\n    .. code-block:: python\n\n        from cleanlab import IssueManager\n        from cleanlab.datalab.internal.issue_manager_factory import register\n\n        @register\n        class MyIssueManager(IssueManager):\n            issue_name: str = \"my_issue\"\n            def find_issues(self, **kwargs):\n                # Some logic to find issues\n                pass\n\n    or in a function call:\n\n    .. code-block:: python\n\n        from cleanlab import IssueManager\n        from cleanlab.datalab.internal.issue_manager_factory import register\n\n        class MyIssueManager(IssueManager):\n            issue_name: str = \"my_issue\"\n            def find_issues(self, **kwargs):\n                # Some logic to find issues\n                pass\n\n        register(MyIssueManager, task=\"classification\")\n    \"\"\"\n\n    if not issubclass(cls, IssueManager):\n        raise ValueError(f\"Class {cls} must be a subclass of IssueManager\")\n\n    name: str = str(cls.issue_name)\n\n    try:\n        _task = Task.from_str(task)\n        if _task not in REGISTRY:\n            raise ValueError(f\"Invalid task type: {_task}, must be in {list(REGISTRY.keys())}\")\n    except KeyError:\n        raise ValueError(f\"Invalid task type: {task}, must be in {list(REGISTRY.keys())}\")\n\n    if name in REGISTRY[_task]:\n        print(\n            f\"Warning: Overwriting existing issue manager {name} with {cls} for task {_task}.\"\n            \"This may cause unexpected behavior.\"\n        )\n\n    REGISTRY[_task][name] = cls\n    return cls\n\n\ndef list_possible_issue_types(task: Task) -> List[str]:\n    \"\"\"Returns a list of all registered issue types.\n\n    Any issue type that is not in this list cannot be used in the :py:meth:`find_issues` method.\n\n    See Also\n    --------\n    :py:class:`REGISTRY <cleanlab.datalab.internal.issue_manager_factory.REGISTRY>` : All available issue types and their corresponding issue managers can be found here.\n    \"\"\"\n    return list(REGISTRY.get(task, []))\n\n\ndef list_default_issue_types(task: Task) -> List[str]:\n    \"\"\"Returns a list of the issue types that are run by default\n    when :py:meth:`find_issues` is called without specifying `issue_types`.\n\n    task :\n        Specific machine learning task supported by Datalab.\n\n    See Also\n    --------\n    :py:class:`REGISTRY <cleanlab.datalab.internal.issue_manager_factory.REGISTRY>` : All available issue types and their corresponding issue managers can be found here.\n    \"\"\"\n    default_issue_types_dict = {\n        Task.CLASSIFICATION: [\n            \"null\",\n            \"label\",\n            \"outlier\",\n            \"near_duplicate\",\n            \"non_iid\",\n            \"class_imbalance\",\n        ],\n        Task.REGRESSION: [\n            \"null\",\n            \"label\",\n            \"outlier\",\n            \"near_duplicate\",\n            \"non_iid\",\n        ],\n        Task.MULTILABEL: [\n            \"null\",\n            \"label\",\n            \"outlier\",\n            \"near_duplicate\",\n            \"non_iid\",\n        ],\n    }\n    if task not in default_issue_types_dict:\n        task = Task.CLASSIFICATION\n    default_issue_types = default_issue_types_dict[task]\n    return default_issue_types\n", "path": "cleanlab/datalab/internal/issue_manager_factory.py"}], "after_files": [{"content": "# Copyright (C) 2017-2023  Cleanlab Inc.\n# This file is part of cleanlab.\n#\n# cleanlab is free software: you can redistribute it and/or modify\n# it under the terms of the GNU Affero General Public License as published\n# by the Free Software Foundation, either version 3 of the License, or\n# (at your option) any later version.\n#\n# cleanlab is distributed in the hope that it will be useful,\n# but WITHOUT ANY WARRANTY; without even the implied warranty of\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n# GNU Affero General Public License for more details.\n#\n# You should have received a copy of the GNU Affero General Public License\n# along with cleanlab.  If not, see <https://www.gnu.org/licenses/>.\n\"\"\"The factory module provides a factory class for constructing concrete issue managers\nand a decorator for registering new issue managers.\n\nThis module provides the :py:meth:`register` decorator for users to register new subclasses of\n:py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`\nin the registry. Each IssueManager detects some particular type of issue in a dataset.\n\n\nNote\n----\n\nThe :class:`REGISTRY` variable is used by the factory class to keep track\nof registered issue managers.\nThe factory class is used as an implementation detail by\n:py:class:`Datalab <cleanlab.datalab.datalab.Datalab>`,\nwhich provides a simplified API for constructing concrete issue managers.\n:py:class:`Datalab <cleanlab.datalab.datalab.Datalab>` is intended to be used by users\nand provides detailed documentation on how to use the API.\n\nWarning\n-------\nNeither the :class:`REGISTRY` variable nor the factory class should be used directly by users.\n\"\"\"\nfrom __future__ import annotations\n\nfrom typing import Dict, List, Type\n\nfrom cleanlab.datalab.internal.issue_manager import (\n    ClassImbalanceIssueManager,\n    DataValuationIssueManager,\n    IssueManager,\n    LabelIssueManager,\n    NearDuplicateIssueManager,\n    NonIIDIssueManager,\n    ClassImbalanceIssueManager,\n    UnderperformingGroupIssueManager,\n    DataValuationIssueManager,\n    OutlierIssueManager,\n    NullIssueManager,\n)\nfrom cleanlab.datalab.internal.issue_manager.regression import RegressionLabelIssueManager\nfrom cleanlab.datalab.internal.issue_manager.multilabel.label import MultilabelIssueManager\n\n\nREGISTRY: Dict[str, Dict[str, Type[IssueManager]]] = {\n    \"classification\": {\n        \"outlier\": OutlierIssueManager,\n        \"label\": LabelIssueManager,\n        \"near_duplicate\": NearDuplicateIssueManager,\n        \"non_iid\": NonIIDIssueManager,\n        \"class_imbalance\": ClassImbalanceIssueManager,\n        \"underperforming_group\": UnderperformingGroupIssueManager,\n        \"data_valuation\": DataValuationIssueManager,\n        \"null\": NullIssueManager,\n    },\n    \"regression\": {\n        \"label\": RegressionLabelIssueManager,\n        \"outlier\": OutlierIssueManager,\n        \"near_duplicate\": NearDuplicateIssueManager,\n        \"non_iid\": NonIIDIssueManager,\n        \"null\": NullIssueManager,\n    },\n    \"multilabel\": {\n        \"label\": MultilabelIssueManager,\n        \"outlier\": OutlierIssueManager,\n        \"near_duplicate\": NearDuplicateIssueManager,\n        \"non_iid\": NonIIDIssueManager,\n        \"null\": NullIssueManager,\n    },\n}\n\"\"\"Registry of issue managers that can be constructed from a string\nand used in the Datalab class.\n\n:meta hide-value:\n\nCurrently, the following issue managers are registered by default:\n\n- ``\"outlier\"``: :py:class:`OutlierIssueManager <cleanlab.datalab.internal.issue_manager.outlier.OutlierIssueManager>`\n- ``\"near_duplicate\"``: :py:class:`NearDuplicateIssueManager <cleanlab.datalab.internal.issue_manager.duplicate.NearDuplicateIssueManager>`\n- ``\"non_iid\"``: :py:class:`NonIIDIssueManager <cleanlab.datalab.internal.issue_manager.noniid.NonIIDIssueManager>`\n\nWarning\n-------\nThis variable should not be used directly by users.\n\"\"\"\n\n\n# Construct concrete issue manager with a from_str method\nclass _IssueManagerFactory:\n    \"\"\"Factory class for constructing concrete issue managers.\"\"\"\n\n    @classmethod\n    def from_str(cls, issue_type: str, task: str) -> Type[IssueManager]:\n        \"\"\"Constructs a concrete issue manager class from a string.\"\"\"\n        if isinstance(issue_type, list):\n            raise ValueError(\n                \"issue_type must be a string, not a list. Try using from_list instead.\"\n            )\n\n        if task not in REGISTRY:\n            raise ValueError(f\"Invalid task type: {task}, must be in {list(REGISTRY.keys())}\")\n        if issue_type not in REGISTRY[task]:\n            raise ValueError(f\"Invalid issue type: {issue_type} for task {task}\")\n\n        return REGISTRY[task][issue_type]\n\n    @classmethod\n    def from_list(cls, issue_types: List[str], task: str) -> List[Type[IssueManager]]:\n        \"\"\"Constructs a list of concrete issue manager classes from a list of strings.\"\"\"\n        return [cls.from_str(issue_type, task) for issue_type in issue_types]\n\n\ndef register(cls: Type[IssueManager], task: str = \"classification\") -> Type[IssueManager]:\n    \"\"\"Registers the issue manager factory.\n\n    Parameters\n    ----------\n    cls :\n        A subclass of\n        :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`.\n\n    task :\n        Specific machine learning task like classification or regression.\n\n    Returns\n    -------\n    cls :\n        The same class that was passed in.\n\n    Example\n    -------\n\n    When defining a new subclass of\n    :py:class:`IssueManager <cleanlab.datalab.internal.issue_manager.issue_manager.IssueManager>`,\n    you can register it like so:\n\n    .. code-block:: python\n\n        from cleanlab import IssueManager\n        from cleanlab.datalab.internal.issue_manager_factory import register\n\n        @register\n        class MyIssueManager(IssueManager):\n            issue_name: str = \"my_issue\"\n            def find_issues(self, **kwargs):\n                # Some logic to find issues\n                pass\n\n    or in a function call:\n\n    .. code-block:: python\n\n        from cleanlab import IssueManager\n        from cleanlab.datalab.internal.issue_manager_factory import register\n\n        class MyIssueManager(IssueManager):\n            issue_name: str = \"my_issue\"\n            def find_issues(self, **kwargs):\n                # Some logic to find issues\n                pass\n\n        register(MyIssueManager, task=\"classification\")\n    \"\"\"\n\n    if not issubclass(cls, IssueManager):\n        raise ValueError(f\"Class {cls} must be a subclass of IssueManager\")\n\n    name: str = str(cls.issue_name)\n\n    if task not in REGISTRY:\n        raise ValueError(f\"Invalid task type: {task}, must be in {list(REGISTRY.keys())}\")\n\n    if name in REGISTRY[task]:\n        print(\n            f\"Warning: Overwriting existing issue manager {name} with {cls} for task {task}.\"\n            \"This may cause unexpected behavior.\"\n        )\n\n    REGISTRY[task][name] = cls\n    return cls\n\n\ndef list_possible_issue_types(task: str) -> List[str]:\n    \"\"\"Returns a list of all registered issue types.\n\n    Any issue type that is not in this list cannot be used in the :py:meth:`find_issues` method.\n\n    See Also\n    --------\n    :py:class:`REGISTRY <cleanlab.datalab.internal.issue_manager_factory.REGISTRY>` : All available issue types and their corresponding issue managers can be found here.\n    \"\"\"\n    return list(REGISTRY.get(task, []))\n\n\ndef list_default_issue_types(task: str) -> List[str]:\n    \"\"\"Returns a list of the issue types that are run by default\n    when :py:meth:`find_issues` is called without specifying `issue_types`.\n\n    See Also\n    --------\n    :py:class:`REGISTRY <cleanlab.datalab.internal.issue_manager_factory.REGISTRY>` : All available issue types and their corresponding issue managers can be found here.\n    \"\"\"\n    default_issue_types_dict = {\n        \"classification\": [\n            \"null\",\n            \"label\",\n            \"outlier\",\n            \"near_duplicate\",\n            \"non_iid\",\n            \"class_imbalance\",\n            \"underperforming_group\",\n        ],\n        \"regression\": [\n            \"null\",\n            \"label\",\n            \"outlier\",\n            \"near_duplicate\",\n            \"non_iid\",\n        ],\n        \"multilabel\": [\n            \"null\",\n            \"label\",\n            \"outlier\",\n            \"near_duplicate\",\n            \"non_iid\",\n        ],\n    }\n    if task not in default_issue_types_dict:\n        task = \"classification\"\n    default_issue_types = default_issue_types_dict[task]\n    return default_issue_types\n", "path": "cleanlab/datalab/internal/issue_manager_factory.py"}]}