We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
add_weighted_adapter() is unusable, throws error: "Invalid type <class 'list'> found in target_modules"
### System Info

- `transformers` version: 4.34.0
- Platform: Linux-6.5.6-arch2-1-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: 0.23.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: false
- Using distributed or parallel set-up in script?: false

### Who can help?

@pacman100 @you

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder
- [X] My own task or dataset (give details below)

### Reproduction

Load the model:
```
model = AutoModelForCausalLM.from_pretrained(
        "RWKV/rwkv-4-169m-pile"
        device_map="auto"
    )
```
Convert it into a PeftModel (to make the `add_weighted_adapter()` method available):
```
model = PeftModel.from_pretrained(
        model, f"{adapter_dir}/{adapter}", adapter_name='adapter_1'
    )
```
Load the 2nd adapter:
```
model.load_adapter(
        f"{adapter_dir}/{adapter}", adapter_name='adapter_2'
    )
```
Attempt to merge the two adapters:
```
model.add_weighted_adapter(
        adapters=['adapter_1', 'adapter_2'],
        weights=[0.5, 0.5],
        adapter_name="combined",
        combination_type="svd",
    )
```
This will result in the error message:
```
Invalid type <class 'list'> found in target_modules
```

### Expected behavior

I have 2 fully trained LoRA adapters. The only configuration difference between them is the rank and alpha used for each.

I would like to merge the two adapters into a new adapter, and set that adapter to the "active" one.

However, trying to use the `add_weighted_adapter()` method always results in the following error:
```
Invalid type <class 'list'> found in target_modules
```

I would expect the `add_weighted_adapter()` method to accept a list, for the "target_modules" argument.

If you can provide any advice, I would greatly appreciate it. I suspect that this is either unsupported and/or not fully-implemented; or, it has something to do with the way I'm attaching adapters. I've tried a bunch of alternate configurations, but I'm not having luck.

Thanks in advance for any help you might provide.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `src/peft/config.py`
Content:
```
1 # coding=utf-8
2 # Copyright 2023-present the HuggingFace Inc. team.
3 #
4 # Licensed under the Apache License, Version 2.0 (the "License");
5 # you may not use this file except in compliance with the License.
6 # You may obtain a copy of the License at
7 #
8 #     http://www.apache.org/licenses/LICENSE-2.0
9 #
10 # Unless required by applicable law or agreed to in writing, software
11 # distributed under the License is distributed on an "AS IS" BASIS,
12 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 # See the License for the specific language governing permissions and
14 # limitations under the License.
15 import inspect
16 import json
17 import os
18 from dataclasses import asdict, dataclass, field
19 from typing import Dict, Optional, Union
20 
21 from huggingface_hub import hf_hub_download
22 from transformers.utils import PushToHubMixin
23 
24 from .utils import CONFIG_NAME, PeftType, TaskType
25 
26 
27 @dataclass
28 class PeftConfigMixin(PushToHubMixin):
29     r"""
30     This is the base configuration class for PEFT adapter models. It contains all the methods that are common to all
31     PEFT adapter models. This class inherits from [`~transformers.utils.PushToHubMixin`] which contains the methods to
32     push your model to the Hub. The method `save_pretrained` will save the configuration of your adapter model in a
33     directory. The method `from_pretrained` will load the configuration of your adapter model from a directory.
34 
35     Args:
36         peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.
37     """
38     peft_type: Optional[PeftType] = field(default=None, metadata={"help": "The type of PEFT model."})
39     auto_mapping: Optional[dict] = field(
40         default=None, metadata={"help": "An auto mapping dict to help retrieve the base model class if needed."}
41     )
42 
43     def to_dict(self) -> Dict:
44         return asdict(self)
45 
46     def save_pretrained(self, save_directory: str, **kwargs) -> None:
47         r"""
48         This method saves the configuration of your adapter model in a directory.
49 
50         Args:
51             save_directory (`str`):
52                 The directory where the configuration will be saved.
53             kwargs (additional keyword arguments, *optional*):
54                 Additional keyword arguments passed along to the [`~transformers.utils.PushToHubMixin.push_to_hub`]
55                 method.
56         """
57         if os.path.isfile(save_directory):
58             raise AssertionError(f"Provided path ({save_directory}) should be a directory, not a file")
59 
60         os.makedirs(save_directory, exist_ok=True)
61         auto_mapping_dict = kwargs.pop("auto_mapping_dict", None)
62 
63         output_dict = asdict(self)
64         # converting set type to list
65         for key, value in output_dict.items():
66             if isinstance(value, set):
67                 output_dict[key] = list(value)
68 
69         output_path = os.path.join(save_directory, CONFIG_NAME)
70 
71         # Add auto mapping details for custom models.
72         if auto_mapping_dict is not None:
73             output_dict["auto_mapping"] = auto_mapping_dict
74 
75         # save it
76         with open(output_path, "w") as writer:
77             writer.write(json.dumps(output_dict, indent=2, sort_keys=True))
78 
79     @classmethod
80     def from_pretrained(cls, pretrained_model_name_or_path: str, subfolder: Optional[str] = None, **kwargs):
81         r"""
82         This method loads the configuration of your adapter model from a directory.
83 
84         Args:
85             pretrained_model_name_or_path (`str`):
86                 The directory or the Hub repository id where the configuration is saved.
87             kwargs (additional keyword arguments, *optional*):
88                 Additional keyword arguments passed along to the child class initialization.
89         """
90         # Avoid circular dependency .. TODO: fix this with a larger refactor
91         from peft.mapping import PEFT_TYPE_TO_CONFIG_MAPPING
92 
93         path = (
94             os.path.join(pretrained_model_name_or_path, subfolder)
95             if subfolder is not None
96             else pretrained_model_name_or_path
97         )
98 
99         hf_hub_download_kwargs, class_kwargs, _ = cls._split_kwargs(kwargs)
100 
101         if os.path.isfile(os.path.join(path, CONFIG_NAME)):
102             config_file = os.path.join(path, CONFIG_NAME)
103         else:
104             try:
105                 config_file = hf_hub_download(
106                     pretrained_model_name_or_path, CONFIG_NAME, subfolder=subfolder, **hf_hub_download_kwargs
107                 )
108             except Exception:
109                 raise ValueError(f"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'")
110 
111         loaded_attributes = cls.from_json_file(config_file)
112 
113         # TODO: this hack is needed to fix the following issue (on commit 702f937):
114         # if someone saves a default config and loads it back with `PeftConfig` class it yields to
115         # not loading the correct config class.
116 
117         # from peft import AdaLoraConfig, PeftConfig
118         # peft_config = AdaLoraConfig()
119         # print(peft_config)
120         # >>> AdaLoraConfig(peft_type=<PeftType.ADALORA: 'ADALORA'>, auto_mapping=None, base_model_name_or_path=None,
121         # revision=None, task_type=None, inference_mode=False, r=8, target_modules=None, lora_alpha=8, lora_dropout=0.0, ...
122         #
123         # peft_config.save_pretrained("./test_config")
124         # peft_config = PeftConfig.from_pretrained("./test_config")
125         # print(peft_config)
126         # >>> PeftConfig(peft_type='ADALORA', auto_mapping=None, base_model_name_or_path=None, revision=None, task_type=None, inference_mode=False)
127         if "peft_type" in loaded_attributes:
128             peft_type = loaded_attributes["peft_type"]
129             config_cls = PEFT_TYPE_TO_CONFIG_MAPPING[peft_type]
130         else:
131             config_cls = cls
132 
133         config = config_cls(**class_kwargs)
134 
135         for key, value in loaded_attributes.items():
136             if hasattr(config, key):
137                 setattr(config, key, value)
138 
139         return config
140 
141     @classmethod
142     def from_json_file(cls, path_json_file: str, **kwargs):
143         r"""
144         Loads a configuration file from a json file.
145 
146         Args:
147             path_json_file (`str`):
148                 The path to the json file.
149         """
150         with open(path_json_file, "r") as file:
151             json_object = json.load(file)
152 
153         return json_object
154 
155     @classmethod
156     def _split_kwargs(cls, kwargs):
157         hf_hub_download_kwargs = {}
158         class_kwargs = {}
159         other_kwargs = {}
160 
161         for key, value in kwargs.items():
162             if key in inspect.signature(hf_hub_download).parameters:
163                 hf_hub_download_kwargs[key] = value
164             elif key in list(cls.__annotations__):
165                 class_kwargs[key] = value
166             else:
167                 other_kwargs[key] = value
168 
169         return hf_hub_download_kwargs, class_kwargs, other_kwargs
170 
171     @classmethod
172     def _get_peft_type(
173         cls,
174         model_id: str,
175         **hf_hub_download_kwargs,
176     ):
177         subfolder = hf_hub_download_kwargs.get("subfolder", None)
178 
179         path = os.path.join(model_id, subfolder) if subfolder is not None else model_id
180 
181         if os.path.isfile(os.path.join(path, CONFIG_NAME)):
182             config_file = os.path.join(path, CONFIG_NAME)
183         else:
184             try:
185                 config_file = hf_hub_download(
186                     model_id,
187                     CONFIG_NAME,
188                     **hf_hub_download_kwargs,
189                 )
190             except Exception:
191                 raise ValueError(f"Can't find '{CONFIG_NAME}' at '{model_id}'")
192 
193         loaded_attributes = cls.from_json_file(config_file)
194         return loaded_attributes["peft_type"]
195 
196     @property
197     def is_prompt_learning(self) -> bool:
198         r"""
199         Utility method to check if the configuration is for prompt learning.
200         """
201         return False
202 
203     @property
204     def is_adaption_prompt(self) -> bool:
205         """Return True if this is an adaption prompt config."""
206         return False
207 
208 
209 @dataclass
210 class PeftConfig(PeftConfigMixin):
211     """
212     This is the base configuration class to store the configuration of a [`PeftModel`].
213 
214     Args:
215         peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.
216         task_type (Union[[`~peft.utils.config.TaskType`], `str`]): The type of task to perform.
217         inference_mode (`bool`, defaults to `False`): Whether to use the Peft model in inference mode.
218     """
219 
220     base_model_name_or_path: str = field(default=None, metadata={"help": "The name of the base model to use."})
221     revision: str = field(default=None, metadata={"help": "The specific model version to use."})
222     peft_type: Union[str, PeftType] = field(default=None, metadata={"help": "Peft type"})
223     task_type: Union[str, TaskType] = field(default=None, metadata={"help": "Task type"})
224     inference_mode: bool = field(default=False, metadata={"help": "Whether to use inference mode"})
225 
226 
227 @dataclass
228 class PromptLearningConfig(PeftConfig):
229     """
230     This is the base configuration class to store the configuration of [`PrefixTuning`], [`PromptEncoder`], or
231     [`PromptTuning`].
232 
233     Args:
234         num_virtual_tokens (`int`): The number of virtual tokens to use.
235         token_dim (`int`): The hidden embedding dimension of the base transformer model.
236         num_transformer_submodules (`int`): The number of transformer submodules in the base transformer model.
237         num_attention_heads (`int`): The number of attention heads in the base transformer model.
238         num_layers (`int`): The number of layers in the base transformer model.
239     """
240 
241     num_virtual_tokens: int = field(default=None, metadata={"help": "Number of virtual tokens"})
242     token_dim: int = field(
243         default=None, metadata={"help": "The hidden embedding dimension of the base transformer model"}
244     )
245     num_transformer_submodules: Optional[int] = field(
246         default=None, metadata={"help": "Number of transformer submodules"}
247     )
248     num_attention_heads: Optional[int] = field(default=None, metadata={"help": "Number of attention heads"})
249     num_layers: Optional[int] = field(default=None, metadata={"help": "Number of transformer layers"})
250 
251     @property
252     def is_prompt_learning(self) -> bool:
253         r"""
254         Utility method to check if the configuration is for prompt learning.
255         """
256         return True
257
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/src/peft/config.py b/src/peft/config.py
--- a/src/peft/config.py
+++ b/src/peft/config.py
@@ -130,12 +130,8 @@
         else:
             config_cls = cls
 
-        config = config_cls(**class_kwargs)
-
-        for key, value in loaded_attributes.items():
-            if hasattr(config, key):
-                setattr(config, key, value)
-
+        kwargs = {**class_kwargs, **loaded_attributes}
+        config = config_cls(**kwargs)
         return config
 
     @classmethod

{"golden_diff": "diff --git a/src/peft/config.py b/src/peft/config.py\n--- a/src/peft/config.py\n+++ b/src/peft/config.py\n@@ -130,12 +130,8 @@\n         else:\n             config_cls = cls\n \n-        config = config_cls(**class_kwargs)\n-\n-        for key, value in loaded_attributes.items():\n-            if hasattr(config, key):\n-                setattr(config, key, value)\n-\n+        kwargs = {**class_kwargs, **loaded_attributes}\n+        config = config_cls(**kwargs)\n         return config\n \n     @classmethod\n", "issue": "add_weighted_adapter() is unusable, throws error: \"Invalid type <class 'list'> found in target_modules\"\n### System Info\n\n- `transformers` version: 4.34.0\r\n- Platform: Linux-6.5.6-arch2-1-x86_64-with-glibc2.35\r\n- Python version: 3.10.12\r\n- Huggingface_hub version: 0.17.3\r\n- Safetensors version: 0.4.0\r\n- Accelerate version: 0.23.0\r\n- Accelerate config:    not found\r\n- PyTorch version (GPU?): 2.1.0+cu121 (True)\r\n- Tensorflow version (GPU?): not installed (NA)\r\n- Flax version (CPU?/GPU?/TPU?): not installed (NA)\r\n- Jax version: not installed\r\n- JaxLib version: not installed\r\n- Using GPU in script?: false\r\n- Using distributed or parallel set-up in script?: false\n\n### Who can help?\n\n@pacman100 @you\n\n### Information\n\n- [ ] The official example scripts\n- [X] My own modified scripts\n\n### Tasks\n\n- [ ] An officially supported task in the `examples` folder\n- [X] My own task or dataset (give details below)\n\n### Reproduction\n\nLoad the model:\r\n```\r\nmodel = AutoModelForCausalLM.from_pretrained(\r\n        \"RWKV/rwkv-4-169m-pile\"\r\n        device_map=\"auto\"\r\n    )\r\n```\r\nConvert it into a PeftModel (to make the `add_weighted_adapter()` method available):\r\n```\r\nmodel = PeftModel.from_pretrained(\r\n        model, f\"{adapter_dir}/{adapter}\", adapter_name='adapter_1'\r\n    )\r\n```\r\nLoad the 2nd adapter:\r\n```\r\nmodel.load_adapter(\r\n        f\"{adapter_dir}/{adapter}\", adapter_name='adapter_2'\r\n    )\r\n```\r\nAttempt to merge the two adapters:\r\n```\r\nmodel.add_weighted_adapter(\r\n        adapters=['adapter_1', 'adapter_2'],\r\n        weights=[0.5, 0.5],\r\n        adapter_name=\"combined\",\r\n        combination_type=\"svd\",\r\n    )\r\n```\r\nThis will result in the error message:\r\n```\r\nInvalid type <class 'list'> found in target_modules\r\n```\n\n### Expected behavior\n\nI have 2 fully trained LoRA adapters. The only configuration difference between them is the rank and alpha used for each.\r\n\r\nI would like to merge the two adapters into a new adapter, and set that adapter to the \"active\" one.\r\n\r\nHowever, trying to use the `add_weighted_adapter()` method always results in the following error:\r\n```\r\nInvalid type <class 'list'> found in target_modules\r\n```\r\n\r\nI would expect the `add_weighted_adapter()` method to accept a list, for the \"target_modules\" argument.\r\n\r\nIf you can provide any advice, I would greatly appreciate it. I suspect that this is either unsupported and/or not fully-implemented; or, it has something to do with the way I'm attaching adapters. I've tried a bunch of alternate configurations, but I'm not having luck.\r\n\r\nThanks in advance for any help you might provide.\n", "before_files": [{"content": "# coding=utf-8\n# Copyright 2023-present the HuggingFace Inc. team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport inspect\nimport json\nimport os\nfrom dataclasses import asdict, dataclass, field\nfrom typing import Dict, Optional, Union\n\nfrom huggingface_hub import hf_hub_download\nfrom transformers.utils import PushToHubMixin\n\nfrom .utils import CONFIG_NAME, PeftType, TaskType\n\n\n@dataclass\nclass PeftConfigMixin(PushToHubMixin):\n    r\"\"\"\n    This is the base configuration class for PEFT adapter models. It contains all the methods that are common to all\n    PEFT adapter models. This class inherits from [`~transformers.utils.PushToHubMixin`] which contains the methods to\n    push your model to the Hub. The method `save_pretrained` will save the configuration of your adapter model in a\n    directory. The method `from_pretrained` will load the configuration of your adapter model from a directory.\n\n    Args:\n        peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.\n    \"\"\"\n    peft_type: Optional[PeftType] = field(default=None, metadata={\"help\": \"The type of PEFT model.\"})\n    auto_mapping: Optional[dict] = field(\n        default=None, metadata={\"help\": \"An auto mapping dict to help retrieve the base model class if needed.\"}\n    )\n\n    def to_dict(self) -> Dict:\n        return asdict(self)\n\n    def save_pretrained(self, save_directory: str, **kwargs) -> None:\n        r\"\"\"\n        This method saves the configuration of your adapter model in a directory.\n\n        Args:\n            save_directory (`str`):\n                The directory where the configuration will be saved.\n            kwargs (additional keyword arguments, *optional*):\n                Additional keyword arguments passed along to the [`~transformers.utils.PushToHubMixin.push_to_hub`]\n                method.\n        \"\"\"\n        if os.path.isfile(save_directory):\n            raise AssertionError(f\"Provided path ({save_directory}) should be a directory, not a file\")\n\n        os.makedirs(save_directory, exist_ok=True)\n        auto_mapping_dict = kwargs.pop(\"auto_mapping_dict\", None)\n\n        output_dict = asdict(self)\n        # converting set type to list\n        for key, value in output_dict.items():\n            if isinstance(value, set):\n                output_dict[key] = list(value)\n\n        output_path = os.path.join(save_directory, CONFIG_NAME)\n\n        # Add auto mapping details for custom models.\n        if auto_mapping_dict is not None:\n            output_dict[\"auto_mapping\"] = auto_mapping_dict\n\n        # save it\n        with open(output_path, \"w\") as writer:\n            writer.write(json.dumps(output_dict, indent=2, sort_keys=True))\n\n    @classmethod\n    def from_pretrained(cls, pretrained_model_name_or_path: str, subfolder: Optional[str] = None, **kwargs):\n        r\"\"\"\n        This method loads the configuration of your adapter model from a directory.\n\n        Args:\n            pretrained_model_name_or_path (`str`):\n                The directory or the Hub repository id where the configuration is saved.\n            kwargs (additional keyword arguments, *optional*):\n                Additional keyword arguments passed along to the child class initialization.\n        \"\"\"\n        # Avoid circular dependency .. TODO: fix this with a larger refactor\n        from peft.mapping import PEFT_TYPE_TO_CONFIG_MAPPING\n\n        path = (\n            os.path.join(pretrained_model_name_or_path, subfolder)\n            if subfolder is not None\n            else pretrained_model_name_or_path\n        )\n\n        hf_hub_download_kwargs, class_kwargs, _ = cls._split_kwargs(kwargs)\n\n        if os.path.isfile(os.path.join(path, CONFIG_NAME)):\n            config_file = os.path.join(path, CONFIG_NAME)\n        else:\n            try:\n                config_file = hf_hub_download(\n                    pretrained_model_name_or_path, CONFIG_NAME, subfolder=subfolder, **hf_hub_download_kwargs\n                )\n            except Exception:\n                raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'\")\n\n        loaded_attributes = cls.from_json_file(config_file)\n\n        # TODO: this hack is needed to fix the following issue (on commit 702f937):\n        # if someone saves a default config and loads it back with `PeftConfig` class it yields to\n        # not loading the correct config class.\n\n        # from peft import AdaLoraConfig, PeftConfig\n        # peft_config = AdaLoraConfig()\n        # print(peft_config)\n        # >>> AdaLoraConfig(peft_type=<PeftType.ADALORA: 'ADALORA'>, auto_mapping=None, base_model_name_or_path=None,\n        # revision=None, task_type=None, inference_mode=False, r=8, target_modules=None, lora_alpha=8, lora_dropout=0.0, ...\n        #\n        # peft_config.save_pretrained(\"./test_config\")\n        # peft_config = PeftConfig.from_pretrained(\"./test_config\")\n        # print(peft_config)\n        # >>> PeftConfig(peft_type='ADALORA', auto_mapping=None, base_model_name_or_path=None, revision=None, task_type=None, inference_mode=False)\n        if \"peft_type\" in loaded_attributes:\n            peft_type = loaded_attributes[\"peft_type\"]\n            config_cls = PEFT_TYPE_TO_CONFIG_MAPPING[peft_type]\n        else:\n            config_cls = cls\n\n        config = config_cls(**class_kwargs)\n\n        for key, value in loaded_attributes.items():\n            if hasattr(config, key):\n                setattr(config, key, value)\n\n        return config\n\n    @classmethod\n    def from_json_file(cls, path_json_file: str, **kwargs):\n        r\"\"\"\n        Loads a configuration file from a json file.\n\n        Args:\n            path_json_file (`str`):\n                The path to the json file.\n        \"\"\"\n        with open(path_json_file, \"r\") as file:\n            json_object = json.load(file)\n\n        return json_object\n\n    @classmethod\n    def _split_kwargs(cls, kwargs):\n        hf_hub_download_kwargs = {}\n        class_kwargs = {}\n        other_kwargs = {}\n\n        for key, value in kwargs.items():\n            if key in inspect.signature(hf_hub_download).parameters:\n                hf_hub_download_kwargs[key] = value\n            elif key in list(cls.__annotations__):\n                class_kwargs[key] = value\n            else:\n                other_kwargs[key] = value\n\n        return hf_hub_download_kwargs, class_kwargs, other_kwargs\n\n    @classmethod\n    def _get_peft_type(\n        cls,\n        model_id: str,\n        **hf_hub_download_kwargs,\n    ):\n        subfolder = hf_hub_download_kwargs.get(\"subfolder\", None)\n\n        path = os.path.join(model_id, subfolder) if subfolder is not None else model_id\n\n        if os.path.isfile(os.path.join(path, CONFIG_NAME)):\n            config_file = os.path.join(path, CONFIG_NAME)\n        else:\n            try:\n                config_file = hf_hub_download(\n                    model_id,\n                    CONFIG_NAME,\n                    **hf_hub_download_kwargs,\n                )\n            except Exception:\n                raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{model_id}'\")\n\n        loaded_attributes = cls.from_json_file(config_file)\n        return loaded_attributes[\"peft_type\"]\n\n    @property\n    def is_prompt_learning(self) -> bool:\n        r\"\"\"\n        Utility method to check if the configuration is for prompt learning.\n        \"\"\"\n        return False\n\n    @property\n    def is_adaption_prompt(self) -> bool:\n        \"\"\"Return True if this is an adaption prompt config.\"\"\"\n        return False\n\n\n@dataclass\nclass PeftConfig(PeftConfigMixin):\n    \"\"\"\n    This is the base configuration class to store the configuration of a [`PeftModel`].\n\n    Args:\n        peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.\n        task_type (Union[[`~peft.utils.config.TaskType`], `str`]): The type of task to perform.\n        inference_mode (`bool`, defaults to `False`): Whether to use the Peft model in inference mode.\n    \"\"\"\n\n    base_model_name_or_path: str = field(default=None, metadata={\"help\": \"The name of the base model to use.\"})\n    revision: str = field(default=None, metadata={\"help\": \"The specific model version to use.\"})\n    peft_type: Union[str, PeftType] = field(default=None, metadata={\"help\": \"Peft type\"})\n    task_type: Union[str, TaskType] = field(default=None, metadata={\"help\": \"Task type\"})\n    inference_mode: bool = field(default=False, metadata={\"help\": \"Whether to use inference mode\"})\n\n\n@dataclass\nclass PromptLearningConfig(PeftConfig):\n    \"\"\"\n    This is the base configuration class to store the configuration of [`PrefixTuning`], [`PromptEncoder`], or\n    [`PromptTuning`].\n\n    Args:\n        num_virtual_tokens (`int`): The number of virtual tokens to use.\n        token_dim (`int`): The hidden embedding dimension of the base transformer model.\n        num_transformer_submodules (`int`): The number of transformer submodules in the base transformer model.\n        num_attention_heads (`int`): The number of attention heads in the base transformer model.\n        num_layers (`int`): The number of layers in the base transformer model.\n    \"\"\"\n\n    num_virtual_tokens: int = field(default=None, metadata={\"help\": \"Number of virtual tokens\"})\n    token_dim: int = field(\n        default=None, metadata={\"help\": \"The hidden embedding dimension of the base transformer model\"}\n    )\n    num_transformer_submodules: Optional[int] = field(\n        default=None, metadata={\"help\": \"Number of transformer submodules\"}\n    )\n    num_attention_heads: Optional[int] = field(default=None, metadata={\"help\": \"Number of attention heads\"})\n    num_layers: Optional[int] = field(default=None, metadata={\"help\": \"Number of transformer layers\"})\n\n    @property\n    def is_prompt_learning(self) -> bool:\n        r\"\"\"\n        Utility method to check if the configuration is for prompt learning.\n        \"\"\"\n        return True\n", "path": "src/peft/config.py"}], "after_files": [{"content": "# coding=utf-8\n# Copyright 2023-present the HuggingFace Inc. team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport inspect\nimport json\nimport os\nfrom dataclasses import asdict, dataclass, field\nfrom typing import Dict, Optional, Union\n\nfrom huggingface_hub import hf_hub_download\nfrom transformers.utils import PushToHubMixin\n\nfrom .utils import CONFIG_NAME, PeftType, TaskType\n\n\n@dataclass\nclass PeftConfigMixin(PushToHubMixin):\n    r\"\"\"\n    This is the base configuration class for PEFT adapter models. It contains all the methods that are common to all\n    PEFT adapter models. This class inherits from [`~transformers.utils.PushToHubMixin`] which contains the methods to\n    push your model to the Hub. The method `save_pretrained` will save the configuration of your adapter model in a\n    directory. The method `from_pretrained` will load the configuration of your adapter model from a directory.\n\n    Args:\n        peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.\n    \"\"\"\n    peft_type: Optional[PeftType] = field(default=None, metadata={\"help\": \"The type of PEFT model.\"})\n    auto_mapping: Optional[dict] = field(\n        default=None, metadata={\"help\": \"An auto mapping dict to help retrieve the base model class if needed.\"}\n    )\n\n    def to_dict(self) -> Dict:\n        return asdict(self)\n\n    def save_pretrained(self, save_directory: str, **kwargs) -> None:\n        r\"\"\"\n        This method saves the configuration of your adapter model in a directory.\n\n        Args:\n            save_directory (`str`):\n                The directory where the configuration will be saved.\n            kwargs (additional keyword arguments, *optional*):\n                Additional keyword arguments passed along to the [`~transformers.utils.PushToHubMixin.push_to_hub`]\n                method.\n        \"\"\"\n        if os.path.isfile(save_directory):\n            raise AssertionError(f\"Provided path ({save_directory}) should be a directory, not a file\")\n\n        os.makedirs(save_directory, exist_ok=True)\n        auto_mapping_dict = kwargs.pop(\"auto_mapping_dict\", None)\n\n        output_dict = asdict(self)\n        # converting set type to list\n        for key, value in output_dict.items():\n            if isinstance(value, set):\n                output_dict[key] = list(value)\n\n        output_path = os.path.join(save_directory, CONFIG_NAME)\n\n        # Add auto mapping details for custom models.\n        if auto_mapping_dict is not None:\n            output_dict[\"auto_mapping\"] = auto_mapping_dict\n\n        # save it\n        with open(output_path, \"w\") as writer:\n            writer.write(json.dumps(output_dict, indent=2, sort_keys=True))\n\n    @classmethod\n    def from_pretrained(cls, pretrained_model_name_or_path: str, subfolder: Optional[str] = None, **kwargs):\n        r\"\"\"\n        This method loads the configuration of your adapter model from a directory.\n\n        Args:\n            pretrained_model_name_or_path (`str`):\n                The directory or the Hub repository id where the configuration is saved.\n            kwargs (additional keyword arguments, *optional*):\n                Additional keyword arguments passed along to the child class initialization.\n        \"\"\"\n        # Avoid circular dependency .. TODO: fix this with a larger refactor\n        from peft.mapping import PEFT_TYPE_TO_CONFIG_MAPPING\n\n        path = (\n            os.path.join(pretrained_model_name_or_path, subfolder)\n            if subfolder is not None\n            else pretrained_model_name_or_path\n        )\n\n        hf_hub_download_kwargs, class_kwargs, _ = cls._split_kwargs(kwargs)\n\n        if os.path.isfile(os.path.join(path, CONFIG_NAME)):\n            config_file = os.path.join(path, CONFIG_NAME)\n        else:\n            try:\n                config_file = hf_hub_download(\n                    pretrained_model_name_or_path, CONFIG_NAME, subfolder=subfolder, **hf_hub_download_kwargs\n                )\n            except Exception:\n                raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'\")\n\n        loaded_attributes = cls.from_json_file(config_file)\n\n        # TODO: this hack is needed to fix the following issue (on commit 702f937):\n        # if someone saves a default config and loads it back with `PeftConfig` class it yields to\n        # not loading the correct config class.\n\n        # from peft import AdaLoraConfig, PeftConfig\n        # peft_config = AdaLoraConfig()\n        # print(peft_config)\n        # >>> AdaLoraConfig(peft_type=<PeftType.ADALORA: 'ADALORA'>, auto_mapping=None, base_model_name_or_path=None,\n        # revision=None, task_type=None, inference_mode=False, r=8, target_modules=None, lora_alpha=8, lora_dropout=0.0, ...\n        #\n        # peft_config.save_pretrained(\"./test_config\")\n        # peft_config = PeftConfig.from_pretrained(\"./test_config\")\n        # print(peft_config)\n        # >>> PeftConfig(peft_type='ADALORA', auto_mapping=None, base_model_name_or_path=None, revision=None, task_type=None, inference_mode=False)\n        if \"peft_type\" in loaded_attributes:\n            peft_type = loaded_attributes[\"peft_type\"]\n            config_cls = PEFT_TYPE_TO_CONFIG_MAPPING[peft_type]\n        else:\n            config_cls = cls\n\n        kwargs = {**class_kwargs, **loaded_attributes}\n        config = config_cls(**kwargs)\n        return config\n\n    @classmethod\n    def from_json_file(cls, path_json_file: str, **kwargs):\n        r\"\"\"\n        Loads a configuration file from a json file.\n\n        Args:\n            path_json_file (`str`):\n                The path to the json file.\n        \"\"\"\n        with open(path_json_file, \"r\") as file:\n            json_object = json.load(file)\n\n        return json_object\n\n    @classmethod\n    def _split_kwargs(cls, kwargs):\n        hf_hub_download_kwargs = {}\n        class_kwargs = {}\n        other_kwargs = {}\n\n        for key, value in kwargs.items():\n            if key in inspect.signature(hf_hub_download).parameters:\n                hf_hub_download_kwargs[key] = value\n            elif key in list(cls.__annotations__):\n                class_kwargs[key] = value\n            else:\n                other_kwargs[key] = value\n\n        return hf_hub_download_kwargs, class_kwargs, other_kwargs\n\n    @classmethod\n    def _get_peft_type(\n        cls,\n        model_id: str,\n        **hf_hub_download_kwargs,\n    ):\n        subfolder = hf_hub_download_kwargs.get(\"subfolder\", None)\n\n        path = os.path.join(model_id, subfolder) if subfolder is not None else model_id\n\n        if os.path.isfile(os.path.join(path, CONFIG_NAME)):\n            config_file = os.path.join(path, CONFIG_NAME)\n        else:\n            try:\n                config_file = hf_hub_download(\n                    model_id,\n                    CONFIG_NAME,\n                    **hf_hub_download_kwargs,\n                )\n            except Exception:\n                raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{model_id}'\")\n\n        loaded_attributes = cls.from_json_file(config_file)\n        return loaded_attributes[\"peft_type\"]\n\n    @property\n    def is_prompt_learning(self) -> bool:\n        r\"\"\"\n        Utility method to check if the configuration is for prompt learning.\n        \"\"\"\n        return False\n\n    @property\n    def is_adaption_prompt(self) -> bool:\n        \"\"\"Return True if this is an adaption prompt config.\"\"\"\n        return False\n\n\n@dataclass\nclass PeftConfig(PeftConfigMixin):\n    \"\"\"\n    This is the base configuration class to store the configuration of a [`PeftModel`].\n\n    Args:\n        peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.\n        task_type (Union[[`~peft.utils.config.TaskType`], `str`]): The type of task to perform.\n        inference_mode (`bool`, defaults to `False`): Whether to use the Peft model in inference mode.\n    \"\"\"\n\n    base_model_name_or_path: str = field(default=None, metadata={\"help\": \"The name of the base model to use.\"})\n    revision: str = field(default=None, metadata={\"help\": \"The specific model version to use.\"})\n    peft_type: Union[str, PeftType] = field(default=None, metadata={\"help\": \"Peft type\"})\n    task_type: Union[str, TaskType] = field(default=None, metadata={\"help\": \"Task type\"})\n    inference_mode: bool = field(default=False, metadata={\"help\": \"Whether to use inference mode\"})\n\n\n@dataclass\nclass PromptLearningConfig(PeftConfig):\n    \"\"\"\n    This is the base configuration class to store the configuration of [`PrefixTuning`], [`PromptEncoder`], or\n    [`PromptTuning`].\n\n    Args:\n        num_virtual_tokens (`int`): The number of virtual tokens to use.\n        token_dim (`int`): The hidden embedding dimension of the base transformer model.\n        num_transformer_submodules (`int`): The number of transformer submodules in the base transformer model.\n        num_attention_heads (`int`): The number of attention heads in the base transformer model.\n        num_layers (`int`): The number of layers in the base transformer model.\n    \"\"\"\n\n    num_virtual_tokens: int = field(default=None, metadata={\"help\": \"Number of virtual tokens\"})\n    token_dim: int = field(\n        default=None, metadata={\"help\": \"The hidden embedding dimension of the base transformer model\"}\n    )\n    num_transformer_submodules: Optional[int] = field(\n        default=None, metadata={\"help\": \"Number of transformer submodules\"}\n    )\n    num_attention_heads: Optional[int] = field(default=None, metadata={\"help\": \"Number of attention heads\"})\n    num_layers: Optional[int] = field(default=None, metadata={\"help\": \"Number of transformer layers\"})\n\n    @property\n    def is_prompt_learning(self) -> bool:\n        r\"\"\"\n        Utility method to check if the configuration is for prompt learning.\n        \"\"\"\n        return True\n", "path": "src/peft/config.py"}]}