We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
`notebook_launcher` cannot use MPS device on M1 or M2 Mac
### System Info

```Shell
- `Accelerate` version: 0.13.0.dev0
- Platform: macOS-12.5.1-arm64-arm-64bit
- Python version: 3.10.4
- Numpy version: 1.23.1
- PyTorch version (GPU?): 1.13.0.dev20220909 (False)
- `Accelerate` default config:
	- compute_environment: LOCAL_MACHINE
	- distributed_type: MPS
	- mixed_precision: no
	- use_cpu: False
	- num_processes: 1
	- machine_rank: 0
	- num_machines: 1
	- main_process_ip: None
	- main_process_port: None
	- rdzv_backend: static
	- same_network: False
	- main_training_function: main
	- deepspeed_config: {}
	- fsdp_config: {}
	- downcast_bf16: no
```


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce:
1. Stable Diffusion Textual Inversion Colab script: https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/sd_textual_inversion_training.ipynb
2. Run script on M1 Max, expecting `accelerate.notebook_launcher` to report running the training on *GPU (or MPS)*
3. Observe it runs the training on *CPU* instead

### Expected behavior

```Shell
With a previous PR, `mps` M1 GPU support was introduced to Accelerate: https://github.com/huggingface/accelerate/pull/596

However `notebook_launcher` needs to be adapted to make use of this change as well.
```


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `src/accelerate/launchers.py`
Content:
```
1 # Copyright 2021 The HuggingFace Team. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import os
16 import sys
17 import tempfile
18 import warnings
19 
20 import torch
21 
22 from .state import AcceleratorState
23 from .utils import PrecisionType, PrepareForLaunch, patch_environment
24 
25 
26 def notebook_launcher(function, args=(), num_processes=None, use_fp16=False, mixed_precision="no", use_port="29500"):
27     """
28     Launches a training function, using several processes if it's possible in the current environment (TPU with
29     multiple cores for instance).
30 
31     Args:
32         function (`Callable`):
33             The training function to execute. If it accepts arguments, the first argument should be the index of the
34             process run.
35         args (`Tuple`):
36             Tuple of arguments to pass to the function (it will receive `*args`).
37         num_processes (`int`, *optional*):
38             The number of processes to use for training. Will default to 8 in Colab/Kaggle if a TPU is available, to
39             the number of GPUs available otherwise.
40         mixed_precision (`str`, *optional*, defaults to `"no"`):
41             If `fp16` or `bf16`, will use mixed precision training on multi-GPU.
42         use_port (`str`, *optional*, defaults to `"29500"`):
43             The port to use to communicate between processes when launching a multi-GPU training.
44     """
45     # Are we in a google colab or a Kaggle Kernel?
46     if any(key.startswith("KAGGLE") for key in os.environ.keys()):
47         in_colab_or_kaggle = True
48     elif "IPython" in sys.modules:
49         in_colab_or_kaggle = "google.colab" in str(sys.modules["IPython"].get_ipython())
50     else:
51         in_colab_or_kaggle = False
52 
53     try:
54         mixed_precision = PrecisionType(mixed_precision.lower())
55     except ValueError:
56         raise ValueError(
57             f"Unknown mixed_precision mode: {args.mixed_precision.lower()}. Choose between {PrecisionType.list()}."
58         )
59 
60     if in_colab_or_kaggle:
61         if os.environ.get("TPU_NAME", None) is not None:
62             # TPU launch
63             import torch_xla.distributed.xla_multiprocessing as xmp
64 
65             if len(AcceleratorState._shared_state) > 0:
66                 raise ValueError(
67                     "To train on TPU in Colab or Kaggle Kernel, the `Accelerator` should only be initialized inside "
68                     "your training function. Restart your notebook and make sure no cells initializes an "
69                     "`Accelerator`."
70                 )
71             if num_processes is None:
72                 num_processes = 8
73 
74             launcher = PrepareForLaunch(function, distributed_type="TPU")
75             print(f"Launching a training on {num_processes} TPU cores.")
76             xmp.spawn(launcher, args=args, nprocs=num_processes, start_method="fork")
77         else:
78             # No need for a distributed launch otherwise as it's either CPU or one GPU.
79             if torch.cuda.is_available():
80                 print("Launching training on one GPU.")
81             else:
82                 print("Launching training on one CPU.")
83             function(*args)
84 
85     else:
86         if num_processes is None:
87             raise ValueError(
88                 "You have to specify the number of GPUs you would like to use, add `num_processes=...` to your call."
89             )
90 
91         if num_processes > 1:
92             # Multi-GPU launch
93             from torch.multiprocessing import start_processes
94 
95             if len(AcceleratorState._shared_state) > 0:
96                 raise ValueError(
97                     "To launch a multi-GPU training from your notebook, the `Accelerator` should only be initialized "
98                     "inside your training function. Restart your notebook and make sure no cells initializes an "
99                     "`Accelerator`."
100                 )
101 
102             if torch.cuda.is_initialized():
103                 raise ValueError(
104                     "To launch a multi-GPU training from your notebook, you need to avoid running any instruction "
105                     "using `torch.cuda` in any cell. Restart your notebook and make sure no cells use any CUDA "
106                     "function."
107                 )
108 
109             if use_fp16:
110                 warnings.warn('use_fp16=True is deprecated. Use mixed_precision="fp16" instead.', DeprecationWarning)
111                 mixed_precision = "fp16"
112 
113             # torch.distributed will expect a few environment variable to be here. We set the ones common to each
114             # process here (the other ones will be set be the launcher).
115             with patch_environment(
116                 world_size=num_processes, master_addr="127.0.01", master_port=use_port, mixed_precision=mixed_precision
117             ):
118                 launcher = PrepareForLaunch(function, distributed_type="MULTI_GPU")
119 
120                 print(f"Launching training on {num_processes} GPUs.")
121                 start_processes(launcher, args=args, nprocs=num_processes, start_method="fork")
122 
123         else:
124             # No need for a distributed launch otherwise as it's either CPU or one GPU.
125             if torch.cuda.is_available():
126                 print("Launching training on one GPU.")
127             else:
128                 print("Launching training on CPU.")
129             function(*args)
130 
131 
132 def debug_launcher(function, args=(), num_processes=2):
133     """
134     Launches a training function using several processes on CPU for debugging purposes.
135 
136     <Tip warning={true}>
137 
138     This function is provided for internal testing and debugging, but it's not intended for real trainings. It will
139     only use the CPU.
140 
141     </Tip>
142 
143     Args:
144         function (`Callable`):
145             The training function to execute.
146         args (`Tuple`):
147             Tuple of arguments to pass to the function (it will receive `*args`).
148         num_processes (`int`, *optional*, defaults to 2):
149             The number of processes to use for training.
150     """
151     from torch.multiprocessing import start_processes
152 
153     with tempfile.NamedTemporaryFile() as tmp_file:
154         # torch.distributed will expect a few environment variable to be here. We set the ones common to each
155         # process here (the other ones will be set be the launcher).
156         with patch_environment(
157             world_size=num_processes,
158             master_addr="127.0.01",
159             master_port="29500",
160             mixed_precision="no",
161             accelerate_debug_rdv_file=tmp_file.name,
162             use_cpu="yes",
163         ):
164             launcher = PrepareForLaunch(function, debug=True)
165             start_processes(launcher, args=args, nprocs=num_processes, start_method="fork")
166
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/src/accelerate/launchers.py b/src/accelerate/launchers.py
--- a/src/accelerate/launchers.py
+++ b/src/accelerate/launchers.py
@@ -121,12 +121,17 @@
                 start_processes(launcher, args=args, nprocs=num_processes, start_method="fork")
 
         else:
-            # No need for a distributed launch otherwise as it's either CPU or one GPU.
-            if torch.cuda.is_available():
+            # No need for a distributed launch otherwise as it's either CPU, GPU or MPS.
+            use_mps_device = "false"
+            if torch.backends.mps.is_available():
+                print("Launching training on MPS.")
+                use_mps_device = "true"
+            elif torch.cuda.is_available():
                 print("Launching training on one GPU.")
             else:
                 print("Launching training on CPU.")
-            function(*args)
+            with patch_environment(use_mps_device=use_mps_device):
+                function(*args)
 
 
 def debug_launcher(function, args=(), num_processes=2):

{"golden_diff": "diff --git a/src/accelerate/launchers.py b/src/accelerate/launchers.py\n--- a/src/accelerate/launchers.py\n+++ b/src/accelerate/launchers.py\n@@ -121,12 +121,17 @@\n                 start_processes(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n \n         else:\n-            # No need for a distributed launch otherwise as it's either CPU or one GPU.\n-            if torch.cuda.is_available():\n+            # No need for a distributed launch otherwise as it's either CPU, GPU or MPS.\n+            use_mps_device = \"false\"\n+            if torch.backends.mps.is_available():\n+                print(\"Launching training on MPS.\")\n+                use_mps_device = \"true\"\n+            elif torch.cuda.is_available():\n                 print(\"Launching training on one GPU.\")\n             else:\n                 print(\"Launching training on CPU.\")\n-            function(*args)\n+            with patch_environment(use_mps_device=use_mps_device):\n+                function(*args)\n \n \n def debug_launcher(function, args=(), num_processes=2):\n", "issue": "`notebook_launcher` cannot use MPS device on M1 or M2 Mac\n### System Info\r\n\r\n```Shell\r\n- `Accelerate` version: 0.13.0.dev0\r\n- Platform: macOS-12.5.1-arm64-arm-64bit\r\n- Python version: 3.10.4\r\n- Numpy version: 1.23.1\r\n- PyTorch version (GPU?): 1.13.0.dev20220909 (False)\r\n- `Accelerate` default config:\r\n\t- compute_environment: LOCAL_MACHINE\r\n\t- distributed_type: MPS\r\n\t- mixed_precision: no\r\n\t- use_cpu: False\r\n\t- num_processes: 1\r\n\t- machine_rank: 0\r\n\t- num_machines: 1\r\n\t- main_process_ip: None\r\n\t- main_process_port: None\r\n\t- rdzv_backend: static\r\n\t- same_network: False\r\n\t- main_training_function: main\r\n\t- deepspeed_config: {}\r\n\t- fsdp_config: {}\r\n\t- downcast_bf16: no\r\n```\r\n\r\n\r\n### Information\r\n\r\n- [ ] The official example scripts\r\n- [X] My own modified scripts\r\n\r\n### Tasks\r\n\r\n- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)\r\n- [X] My own task or dataset (give details below)\r\n\r\n### Reproduction\r\n\r\nSteps to reproduce:\r\n1. Stable Diffusion Textual Inversion Colab script: https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/sd_textual_inversion_training.ipynb\r\n2. Run script on M1 Max, expecting `accelerate.notebook_launcher` to report running the training on *GPU (or MPS)*\r\n3. Observe it runs the training on *CPU* instead\r\n\r\n### Expected behavior\r\n\r\n```Shell\r\nWith a previous PR, `mps` M1 GPU support was introduced to Accelerate: https://github.com/huggingface/accelerate/pull/596\r\n\r\nHowever `notebook_launcher` needs to be adapted to make use of this change as well.\r\n```\r\n\n", "before_files": [{"content": "# Copyright 2021 The HuggingFace Team. All rights reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\nimport sys\nimport tempfile\nimport warnings\n\nimport torch\n\nfrom .state import AcceleratorState\nfrom .utils import PrecisionType, PrepareForLaunch, patch_environment\n\n\ndef notebook_launcher(function, args=(), num_processes=None, use_fp16=False, mixed_precision=\"no\", use_port=\"29500\"):\n    \"\"\"\n    Launches a training function, using several processes if it's possible in the current environment (TPU with\n    multiple cores for instance).\n\n    Args:\n        function (`Callable`):\n            The training function to execute. If it accepts arguments, the first argument should be the index of the\n            process run.\n        args (`Tuple`):\n            Tuple of arguments to pass to the function (it will receive `*args`).\n        num_processes (`int`, *optional*):\n            The number of processes to use for training. Will default to 8 in Colab/Kaggle if a TPU is available, to\n            the number of GPUs available otherwise.\n        mixed_precision (`str`, *optional*, defaults to `\"no\"`):\n            If `fp16` or `bf16`, will use mixed precision training on multi-GPU.\n        use_port (`str`, *optional*, defaults to `\"29500\"`):\n            The port to use to communicate between processes when launching a multi-GPU training.\n    \"\"\"\n    # Are we in a google colab or a Kaggle Kernel?\n    if any(key.startswith(\"KAGGLE\") for key in os.environ.keys()):\n        in_colab_or_kaggle = True\n    elif \"IPython\" in sys.modules:\n        in_colab_or_kaggle = \"google.colab\" in str(sys.modules[\"IPython\"].get_ipython())\n    else:\n        in_colab_or_kaggle = False\n\n    try:\n        mixed_precision = PrecisionType(mixed_precision.lower())\n    except ValueError:\n        raise ValueError(\n            f\"Unknown mixed_precision mode: {args.mixed_precision.lower()}. Choose between {PrecisionType.list()}.\"\n        )\n\n    if in_colab_or_kaggle:\n        if os.environ.get(\"TPU_NAME\", None) is not None:\n            # TPU launch\n            import torch_xla.distributed.xla_multiprocessing as xmp\n\n            if len(AcceleratorState._shared_state) > 0:\n                raise ValueError(\n                    \"To train on TPU in Colab or Kaggle Kernel, the `Accelerator` should only be initialized inside \"\n                    \"your training function. Restart your notebook and make sure no cells initializes an \"\n                    \"`Accelerator`.\"\n                )\n            if num_processes is None:\n                num_processes = 8\n\n            launcher = PrepareForLaunch(function, distributed_type=\"TPU\")\n            print(f\"Launching a training on {num_processes} TPU cores.\")\n            xmp.spawn(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n        else:\n            # No need for a distributed launch otherwise as it's either CPU or one GPU.\n            if torch.cuda.is_available():\n                print(\"Launching training on one GPU.\")\n            else:\n                print(\"Launching training on one CPU.\")\n            function(*args)\n\n    else:\n        if num_processes is None:\n            raise ValueError(\n                \"You have to specify the number of GPUs you would like to use, add `num_processes=...` to your call.\"\n            )\n\n        if num_processes > 1:\n            # Multi-GPU launch\n            from torch.multiprocessing import start_processes\n\n            if len(AcceleratorState._shared_state) > 0:\n                raise ValueError(\n                    \"To launch a multi-GPU training from your notebook, the `Accelerator` should only be initialized \"\n                    \"inside your training function. Restart your notebook and make sure no cells initializes an \"\n                    \"`Accelerator`.\"\n                )\n\n            if torch.cuda.is_initialized():\n                raise ValueError(\n                    \"To launch a multi-GPU training from your notebook, you need to avoid running any instruction \"\n                    \"using `torch.cuda` in any cell. Restart your notebook and make sure no cells use any CUDA \"\n                    \"function.\"\n                )\n\n            if use_fp16:\n                warnings.warn('use_fp16=True is deprecated. Use mixed_precision=\"fp16\" instead.', DeprecationWarning)\n                mixed_precision = \"fp16\"\n\n            # torch.distributed will expect a few environment variable to be here. We set the ones common to each\n            # process here (the other ones will be set be the launcher).\n            with patch_environment(\n                world_size=num_processes, master_addr=\"127.0.01\", master_port=use_port, mixed_precision=mixed_precision\n            ):\n                launcher = PrepareForLaunch(function, distributed_type=\"MULTI_GPU\")\n\n                print(f\"Launching training on {num_processes} GPUs.\")\n                start_processes(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n\n        else:\n            # No need for a distributed launch otherwise as it's either CPU or one GPU.\n            if torch.cuda.is_available():\n                print(\"Launching training on one GPU.\")\n            else:\n                print(\"Launching training on CPU.\")\n            function(*args)\n\n\ndef debug_launcher(function, args=(), num_processes=2):\n    \"\"\"\n    Launches a training function using several processes on CPU for debugging purposes.\n\n    <Tip warning={true}>\n\n    This function is provided for internal testing and debugging, but it's not intended for real trainings. It will\n    only use the CPU.\n\n    </Tip>\n\n    Args:\n        function (`Callable`):\n            The training function to execute.\n        args (`Tuple`):\n            Tuple of arguments to pass to the function (it will receive `*args`).\n        num_processes (`int`, *optional*, defaults to 2):\n            The number of processes to use for training.\n    \"\"\"\n    from torch.multiprocessing import start_processes\n\n    with tempfile.NamedTemporaryFile() as tmp_file:\n        # torch.distributed will expect a few environment variable to be here. We set the ones common to each\n        # process here (the other ones will be set be the launcher).\n        with patch_environment(\n            world_size=num_processes,\n            master_addr=\"127.0.01\",\n            master_port=\"29500\",\n            mixed_precision=\"no\",\n            accelerate_debug_rdv_file=tmp_file.name,\n            use_cpu=\"yes\",\n        ):\n            launcher = PrepareForLaunch(function, debug=True)\n            start_processes(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n", "path": "src/accelerate/launchers.py"}], "after_files": [{"content": "# Copyright 2021 The HuggingFace Team. All rights reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\nimport sys\nimport tempfile\nimport warnings\n\nimport torch\n\nfrom .state import AcceleratorState\nfrom .utils import PrecisionType, PrepareForLaunch, patch_environment\n\n\ndef notebook_launcher(function, args=(), num_processes=None, use_fp16=False, mixed_precision=\"no\", use_port=\"29500\"):\n    \"\"\"\n    Launches a training function, using several processes if it's possible in the current environment (TPU with\n    multiple cores for instance).\n\n    Args:\n        function (`Callable`):\n            The training function to execute. If it accepts arguments, the first argument should be the index of the\n            process run.\n        args (`Tuple`):\n            Tuple of arguments to pass to the function (it will receive `*args`).\n        num_processes (`int`, *optional*):\n            The number of processes to use for training. Will default to 8 in Colab/Kaggle if a TPU is available, to\n            the number of GPUs available otherwise.\n        mixed_precision (`str`, *optional*, defaults to `\"no\"`):\n            If `fp16` or `bf16`, will use mixed precision training on multi-GPU.\n        use_port (`str`, *optional*, defaults to `\"29500\"`):\n            The port to use to communicate between processes when launching a multi-GPU training.\n    \"\"\"\n    # Are we in a google colab or a Kaggle Kernel?\n    if any(key.startswith(\"KAGGLE\") for key in os.environ.keys()):\n        in_colab_or_kaggle = True\n    elif \"IPython\" in sys.modules:\n        in_colab_or_kaggle = \"google.colab\" in str(sys.modules[\"IPython\"].get_ipython())\n    else:\n        in_colab_or_kaggle = False\n\n    try:\n        mixed_precision = PrecisionType(mixed_precision.lower())\n    except ValueError:\n        raise ValueError(\n            f\"Unknown mixed_precision mode: {args.mixed_precision.lower()}. Choose between {PrecisionType.list()}.\"\n        )\n\n    if in_colab_or_kaggle:\n        if os.environ.get(\"TPU_NAME\", None) is not None:\n            # TPU launch\n            import torch_xla.distributed.xla_multiprocessing as xmp\n\n            if len(AcceleratorState._shared_state) > 0:\n                raise ValueError(\n                    \"To train on TPU in Colab or Kaggle Kernel, the `Accelerator` should only be initialized inside \"\n                    \"your training function. Restart your notebook and make sure no cells initializes an \"\n                    \"`Accelerator`.\"\n                )\n            if num_processes is None:\n                num_processes = 8\n\n            launcher = PrepareForLaunch(function, distributed_type=\"TPU\")\n            print(f\"Launching a training on {num_processes} TPU cores.\")\n            xmp.spawn(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n        else:\n            # No need for a distributed launch otherwise as it's either CPU or one GPU.\n            if torch.cuda.is_available():\n                print(\"Launching training on one GPU.\")\n            else:\n                print(\"Launching training on one CPU.\")\n            function(*args)\n\n    else:\n        if num_processes is None:\n            raise ValueError(\n                \"You have to specify the number of GPUs you would like to use, add `num_processes=...` to your call.\"\n            )\n\n        if num_processes > 1:\n            # Multi-GPU launch\n            from torch.multiprocessing import start_processes\n\n            if len(AcceleratorState._shared_state) > 0:\n                raise ValueError(\n                    \"To launch a multi-GPU training from your notebook, the `Accelerator` should only be initialized \"\n                    \"inside your training function. Restart your notebook and make sure no cells initializes an \"\n                    \"`Accelerator`.\"\n                )\n\n            if torch.cuda.is_initialized():\n                raise ValueError(\n                    \"To launch a multi-GPU training from your notebook, you need to avoid running any instruction \"\n                    \"using `torch.cuda` in any cell. Restart your notebook and make sure no cells use any CUDA \"\n                    \"function.\"\n                )\n\n            if use_fp16:\n                warnings.warn('use_fp16=True is deprecated. Use mixed_precision=\"fp16\" instead.', DeprecationWarning)\n                mixed_precision = \"fp16\"\n\n            # torch.distributed will expect a few environment variable to be here. We set the ones common to each\n            # process here (the other ones will be set be the launcher).\n            with patch_environment(\n                world_size=num_processes, master_addr=\"127.0.01\", master_port=use_port, mixed_precision=mixed_precision\n            ):\n                launcher = PrepareForLaunch(function, distributed_type=\"MULTI_GPU\")\n\n                print(f\"Launching training on {num_processes} GPUs.\")\n                start_processes(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n\n        else:\n            # No need for a distributed launch otherwise as it's either CPU, GPU or MPS.\n            use_mps_device = \"false\"\n            if torch.backends.mps.is_available():\n                print(\"Launching training on MPS.\")\n                use_mps_device = \"true\"\n            elif torch.cuda.is_available():\n                print(\"Launching training on one GPU.\")\n            else:\n                print(\"Launching training on CPU.\")\n            with patch_environment(use_mps_device=use_mps_device):\n                function(*args)\n\n\ndef debug_launcher(function, args=(), num_processes=2):\n    \"\"\"\n    Launches a training function using several processes on CPU for debugging purposes.\n\n    <Tip warning={true}>\n\n    This function is provided for internal testing and debugging, but it's not intended for real trainings. It will\n    only use the CPU.\n\n    </Tip>\n\n    Args:\n        function (`Callable`):\n            The training function to execute.\n        args (`Tuple`):\n            Tuple of arguments to pass to the function (it will receive `*args`).\n        num_processes (`int`, *optional*, defaults to 2):\n            The number of processes to use for training.\n    \"\"\"\n    from torch.multiprocessing import start_processes\n\n    with tempfile.NamedTemporaryFile() as tmp_file:\n        # torch.distributed will expect a few environment variable to be here. We set the ones common to each\n        # process here (the other ones will be set be the launcher).\n        with patch_environment(\n            world_size=num_processes,\n            master_addr=\"127.0.01\",\n            master_port=\"29500\",\n            mixed_precision=\"no\",\n            accelerate_debug_rdv_file=tmp_file.name,\n            use_cpu=\"yes\",\n        ):\n            launcher = PrepareForLaunch(function, debug=True)\n            start_processes(launcher, args=args, nprocs=num_processes, start_method=\"fork\")\n", "path": "src/accelerate/launchers.py"}]}