We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
ClipScore errors on captions with more than 77 tokens
## 🐛 Bug

If you run [CLIPScore](https://torchmetrics.readthedocs.io/en/stable/multimodal/clip_score.html) between an image and a caption, where the caption has more than 77 tokens (longer than the max string than CLIP can process) -- the clip score errors.

### To Reproduce
Compute CLIPScore between a caption with 77+ tokens and an image.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

<details>
  <summary>Code sample</summary>

```
metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch32")
metric.to('cuda')
clip_score = metric(image_tensor, caption)
```

```
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/user/scripts/compute_clip_scores.py", line 125, in <module>
    compute_clip_scores(response=response,
  File "/home/user/scripts/compute_clip_scores.py", line 87, in compute_clip_scores
    clip_score = metric(image_tensor, caption)
  File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py", line 288, in forward
    self._forward_cache = self._forward_full_state_update(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py", line 302, in _forward_full_state_update
    self.update(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py", line 456, in wrapped_func
    raise err
  File "/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py", line 446, in wrapped_func
    update(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/torchmetrics/multimodal/clip_score.py", line 123, in update
    score, n_samples = _clip_score_update(images, text, self.model, self.processor)
  File "/home/user/.local/lib/python3.8/site-packages/torchmetrics/functional/multimodal/clip_score.py", line 69, in _clip_score_update
    txt_features = model.get_text_features(
  File "/home/user/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 1017, in get_text_features
    text_outputs = self.text_model(
  File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 730, in forward
    hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
  File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 230, in forward
    embeddings = inputs_embeds + position_embeddings
RuntimeError: The size of tensor a (138) must match the size of tensor b (77) at non-singleton dimension 1
```


<!-- Ideally attach a minimal code sample to reproduce the decried issue.
Minimal means having the shortest code but still preserving the bug. -->

</details>

### Expected behavior

Present a warning to the user and truncate the caption so that the metric can be computed on the first 77 tokens of the provided caption

### Environment

- TorchMetrics version (and how you installed TM, e.g. `conda`, `pip`, build from source): **1.0.3, pip**
- Python & PyTorch Version (e.g., 1.0): **Python 3.8.10, PyTorch 2.0.1+cu118**
- Any other relevant information such as OS (e.g., Linux): **Linux**


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `src/torchmetrics/functional/multimodal/clip_score.py`
Content:
```
1 # Copyright The Lightning team.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 from typing import List, Tuple, Union
15 
16 import torch
17 from torch import Tensor
18 from typing_extensions import Literal
19 
20 from torchmetrics.utilities.checks import _SKIP_SLOW_DOCTEST, _try_proceed_with_timeout
21 from torchmetrics.utilities.imports import _TRANSFORMERS_GREATER_EQUAL_4_10
22 
23 if _TRANSFORMERS_GREATER_EQUAL_4_10:
24     from transformers import CLIPModel as _CLIPModel
25     from transformers import CLIPProcessor as _CLIPProcessor
26 
27     def _download_clip() -> None:
28         _CLIPModel.from_pretrained("openai/clip-vit-large-patch14")
29         _CLIPProcessor.from_pretrained("openai/clip-vit-large-patch14")
30 
31     if _SKIP_SLOW_DOCTEST and not _try_proceed_with_timeout(_download_clip):
32         __doctest_skip__ = ["clip_score"]
33 
34 else:
35     __doctest_skip__ = ["clip_score"]
36     _CLIPModel = None
37     _CLIPProcessor = None
38 
39 
40 def _clip_score_update(
41     images: Union[Tensor, List[Tensor]],
42     text: Union[str, List[str]],
43     model: _CLIPModel,
44     processor: _CLIPProcessor,
45 ) -> Tuple[Tensor, int]:
46     if not isinstance(images, list):
47         if images.ndim == 3:
48             images = [images]
49     else:  # unwrap into list
50         images = list(images)
51 
52     if not all(i.ndim == 3 for i in images):
53         raise ValueError("Expected all images to be 3d but found image that has either more or less")
54 
55     if not isinstance(text, list):
56         text = [text]
57 
58     if len(text) != len(images):
59         raise ValueError(
60             f"Expected the number of images and text examples to be the same but got {len(images)} and {len(text)}"
61         )
62     device = images[0].device
63     processed_input = processor(text=text, images=[i.cpu() for i in images], return_tensors="pt", padding=True)
64 
65     img_features = model.get_image_features(processed_input["pixel_values"].to(device))
66     img_features = img_features / img_features.norm(p=2, dim=-1, keepdim=True)
67 
68     txt_features = model.get_text_features(
69         processed_input["input_ids"].to(device), processed_input["attention_mask"].to(device)
70     )
71     txt_features = txt_features / txt_features.norm(p=2, dim=-1, keepdim=True)
72 
73     # cosine similarity between feature vectors
74     score = 100 * (img_features * txt_features).sum(axis=-1)
75     return score, len(text)
76 
77 
78 def _get_model_and_processor(
79     model_name_or_path: Literal[
80         "openai/clip-vit-base-patch16",
81         "openai/clip-vit-base-patch32",
82         "openai/clip-vit-large-patch14-336",
83         "openai/clip-vit-large-patch14",
84     ] = "openai/clip-vit-large-patch14",
85 ) -> Tuple[_CLIPModel, _CLIPProcessor]:
86     if _TRANSFORMERS_GREATER_EQUAL_4_10:
87         model = _CLIPModel.from_pretrained(model_name_or_path)
88         processor = _CLIPProcessor.from_pretrained(model_name_or_path)
89         return model, processor
90 
91     raise ModuleNotFoundError(
92         "`clip_score` metric requires `transformers` package be installed."
93         " Either install with `pip install transformers>=4.10.0` or `pip install torchmetrics[multimodal]`."
94     )
95 
96 
97 def clip_score(
98     images: Union[Tensor, List[Tensor]],
99     text: Union[str, List[str]],
100     model_name_or_path: Literal[
101         "openai/clip-vit-base-patch16",
102         "openai/clip-vit-base-patch32",
103         "openai/clip-vit-large-patch14-336",
104         "openai/clip-vit-large-patch14",
105     ] = "openai/clip-vit-large-patch14",
106 ) -> Tensor:
107     r"""Calculate `CLIP Score`_ which is a text-to-image similarity metric.
108 
109     CLIP is a reference free metric that can be used to evaluate the correlation between a generated caption for an
110     image and the actual content of the image. It has been found to be highly correlated with human judgement. The
111     metric is defined as:
112 
113     .. math::
114         \text{CLIPScore(I, C)} = max(100 * cos(E_I, E_C), 0)
115 
116     which corresponds to the cosine similarity between visual CLIP embedding :math:`E_i` for an image :math:`i` and
117     textual CLIP embedding :math:`E_C` for an caption :math:`C`. The score is bound between 0 and 100 and the closer
118     to 100 the better.
119 
120     .. note:: Metric is not scriptable
121 
122     Args:
123         images: Either a single [N, C, H, W] tensor or a list of [C, H, W] tensors
124         text: Either a single caption or a list of captions
125         model_name_or_path: string indicating the version of the CLIP model to use. Available models are
126             `"openai/clip-vit-base-patch16"`, `"openai/clip-vit-base-patch32"`, `"openai/clip-vit-large-patch14-336"`
127             and `"openai/clip-vit-large-patch14"`,
128 
129     Raises:
130         ModuleNotFoundError:
131             If transformers package is not installed or version is lower than 4.10.0
132         ValueError:
133             If not all images have format [C, H, W]
134         ValueError:
135             If the number of images and captions do not match
136 
137     Example:
138         >>> import torch
139         >>> _ = torch.manual_seed(42)
140         >>> from torchmetrics.functional.multimodal import clip_score
141         >>> score = clip_score(torch.randint(255, (3, 224, 224)), "a photo of a cat", "openai/clip-vit-base-patch16")
142         >>> print(score.detach())
143         tensor(24.4255)
144 
145     """
146     model, processor = _get_model_and_processor(model_name_or_path)
147     device = images.device if isinstance(images, Tensor) else images[0].device
148     score, _ = _clip_score_update(images, text, model.to(device), processor)
149     score = score.mean(0)
150     return torch.max(score, torch.zeros_like(score))
151
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/src/torchmetrics/functional/multimodal/clip_score.py b/src/torchmetrics/functional/multimodal/clip_score.py
--- a/src/torchmetrics/functional/multimodal/clip_score.py
+++ b/src/torchmetrics/functional/multimodal/clip_score.py
@@ -17,6 +17,7 @@
 from torch import Tensor
 from typing_extensions import Literal
 
+from torchmetrics.utilities import rank_zero_warn
 from torchmetrics.utilities.checks import _SKIP_SLOW_DOCTEST, _try_proceed_with_timeout
 from torchmetrics.utilities.imports import _TRANSFORMERS_GREATER_EQUAL_4_10
 
@@ -65,6 +66,17 @@
     img_features = model.get_image_features(processed_input["pixel_values"].to(device))
     img_features = img_features / img_features.norm(p=2, dim=-1, keepdim=True)
 
+    max_position_embeddings = model.config.text_config.max_position_embeddings
+    if processed_input["attention_mask"].shape[-1] > max_position_embeddings:
+        rank_zero_warn(
+            f"Encountered caption longer than {max_position_embeddings=}. Will truncate captions to this length."
+            "If longer captions are needed, initialize argument `model_name_or_path` with a model that supports"
+            "longer sequences",
+            UserWarning,
+        )
+        processed_input["attention_mask"] = processed_input["attention_mask"][..., :max_position_embeddings]
+        processed_input["input_ids"] = processed_input["input_ids"][..., :max_position_embeddings]
+
     txt_features = model.get_text_features(
         processed_input["input_ids"].to(device), processed_input["attention_mask"].to(device)
     )

{"golden_diff": "diff --git a/src/torchmetrics/functional/multimodal/clip_score.py b/src/torchmetrics/functional/multimodal/clip_score.py\n--- a/src/torchmetrics/functional/multimodal/clip_score.py\n+++ b/src/torchmetrics/functional/multimodal/clip_score.py\n@@ -17,6 +17,7 @@\n from torch import Tensor\n from typing_extensions import Literal\n \n+from torchmetrics.utilities import rank_zero_warn\n from torchmetrics.utilities.checks import _SKIP_SLOW_DOCTEST, _try_proceed_with_timeout\n from torchmetrics.utilities.imports import _TRANSFORMERS_GREATER_EQUAL_4_10\n \n@@ -65,6 +66,17 @@\n     img_features = model.get_image_features(processed_input[\"pixel_values\"].to(device))\n     img_features = img_features / img_features.norm(p=2, dim=-1, keepdim=True)\n \n+    max_position_embeddings = model.config.text_config.max_position_embeddings\n+    if processed_input[\"attention_mask\"].shape[-1] > max_position_embeddings:\n+        rank_zero_warn(\n+            f\"Encountered caption longer than {max_position_embeddings=}. Will truncate captions to this length.\"\n+            \"If longer captions are needed, initialize argument `model_name_or_path` with a model that supports\"\n+            \"longer sequences\",\n+            UserWarning,\n+        )\n+        processed_input[\"attention_mask\"] = processed_input[\"attention_mask\"][..., :max_position_embeddings]\n+        processed_input[\"input_ids\"] = processed_input[\"input_ids\"][..., :max_position_embeddings]\n+\n     txt_features = model.get_text_features(\n         processed_input[\"input_ids\"].to(device), processed_input[\"attention_mask\"].to(device)\n     )\n", "issue": "ClipScore errors on captions with more than 77 tokens\n## \ud83d\udc1b Bug\r\n\r\nIf you run [CLIPScore](https://torchmetrics.readthedocs.io/en/stable/multimodal/clip_score.html) between an image and a caption, where the caption has more than 77 tokens (longer than the max string than CLIP can process) -- the clip score errors.\r\n\r\n### To Reproduce\r\nCompute CLIPScore between a caption with 77+ tokens and an image.\r\n\r\n<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->\r\n\r\n<details>\r\n  <summary>Code sample</summary>\r\n\r\n```\r\nmetric = CLIPScore(model_name_or_path=\"openai/clip-vit-base-patch32\")\r\nmetric.to('cuda')\r\nclip_score = metric(image_tensor, caption)\r\n```\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/usr/lib/python3.8/runpy.py\", line 194, in _run_module_as_main\r\n    return _run_code(code, main_globals, None,\r\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"/home/user/scripts/compute_clip_scores.py\", line 125, in <module>\r\n    compute_clip_scores(response=response,\r\n  File \"/home/user/scripts/compute_clip_scores.py\", line 87, in compute_clip_scores\r\n    clip_score = metric(image_tensor, caption)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\r\n    return forward_call(*args, **kwargs)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py\", line 288, in forward\r\n    self._forward_cache = self._forward_full_state_update(*args, **kwargs)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py\", line 302, in _forward_full_state_update\r\n    self.update(*args, **kwargs)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py\", line 456, in wrapped_func\r\n    raise err\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torchmetrics/metric.py\", line 446, in wrapped_func\r\n    update(*args, **kwargs)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torchmetrics/multimodal/clip_score.py\", line 123, in update\r\n    score, n_samples = _clip_score_update(images, text, self.model, self.processor)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torchmetrics/functional/multimodal/clip_score.py\", line 69, in _clip_score_update\r\n    txt_features = model.get_text_features(\r\n  File \"/home/user/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py\", line 1017, in get_text_features\r\n    text_outputs = self.text_model(\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\r\n    return forward_call(*args, **kwargs)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py\", line 730, in forward\r\n    hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\r\n    return forward_call(*args, **kwargs)\r\n  File \"/home/user/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py\", line 230, in forward\r\n    embeddings = inputs_embeds + position_embeddings\r\nRuntimeError: The size of tensor a (138) must match the size of tensor b (77) at non-singleton dimension 1\r\n```\r\n\r\n\r\n<!-- Ideally attach a minimal code sample to reproduce the decried issue.\r\nMinimal means having the shortest code but still preserving the bug. -->\r\n\r\n</details>\r\n\r\n### Expected behavior\r\n\r\nPresent a warning to the user and truncate the caption so that the metric can be computed on the first 77 tokens of the provided caption\r\n\r\n### Environment\r\n\r\n- TorchMetrics version (and how you installed TM, e.g. `conda`, `pip`, build from source): **1.0.3, pip**\r\n- Python & PyTorch Version (e.g., 1.0): **Python 3.8.10, PyTorch 2.0.1+cu118**\r\n- Any other relevant information such as OS (e.g., Linux): **Linux**\r\n\n", "before_files": [{"content": "# Copyright The Lightning team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nfrom typing import List, Tuple, Union\n\nimport torch\nfrom torch import Tensor\nfrom typing_extensions import Literal\n\nfrom torchmetrics.utilities.checks import _SKIP_SLOW_DOCTEST, _try_proceed_with_timeout\nfrom torchmetrics.utilities.imports import _TRANSFORMERS_GREATER_EQUAL_4_10\n\nif _TRANSFORMERS_GREATER_EQUAL_4_10:\n    from transformers import CLIPModel as _CLIPModel\n    from transformers import CLIPProcessor as _CLIPProcessor\n\n    def _download_clip() -> None:\n        _CLIPModel.from_pretrained(\"openai/clip-vit-large-patch14\")\n        _CLIPProcessor.from_pretrained(\"openai/clip-vit-large-patch14\")\n\n    if _SKIP_SLOW_DOCTEST and not _try_proceed_with_timeout(_download_clip):\n        __doctest_skip__ = [\"clip_score\"]\n\nelse:\n    __doctest_skip__ = [\"clip_score\"]\n    _CLIPModel = None\n    _CLIPProcessor = None\n\n\ndef _clip_score_update(\n    images: Union[Tensor, List[Tensor]],\n    text: Union[str, List[str]],\n    model: _CLIPModel,\n    processor: _CLIPProcessor,\n) -> Tuple[Tensor, int]:\n    if not isinstance(images, list):\n        if images.ndim == 3:\n            images = [images]\n    else:  # unwrap into list\n        images = list(images)\n\n    if not all(i.ndim == 3 for i in images):\n        raise ValueError(\"Expected all images to be 3d but found image that has either more or less\")\n\n    if not isinstance(text, list):\n        text = [text]\n\n    if len(text) != len(images):\n        raise ValueError(\n            f\"Expected the number of images and text examples to be the same but got {len(images)} and {len(text)}\"\n        )\n    device = images[0].device\n    processed_input = processor(text=text, images=[i.cpu() for i in images], return_tensors=\"pt\", padding=True)\n\n    img_features = model.get_image_features(processed_input[\"pixel_values\"].to(device))\n    img_features = img_features / img_features.norm(p=2, dim=-1, keepdim=True)\n\n    txt_features = model.get_text_features(\n        processed_input[\"input_ids\"].to(device), processed_input[\"attention_mask\"].to(device)\n    )\n    txt_features = txt_features / txt_features.norm(p=2, dim=-1, keepdim=True)\n\n    # cosine similarity between feature vectors\n    score = 100 * (img_features * txt_features).sum(axis=-1)\n    return score, len(text)\n\n\ndef _get_model_and_processor(\n    model_name_or_path: Literal[\n        \"openai/clip-vit-base-patch16\",\n        \"openai/clip-vit-base-patch32\",\n        \"openai/clip-vit-large-patch14-336\",\n        \"openai/clip-vit-large-patch14\",\n    ] = \"openai/clip-vit-large-patch14\",\n) -> Tuple[_CLIPModel, _CLIPProcessor]:\n    if _TRANSFORMERS_GREATER_EQUAL_4_10:\n        model = _CLIPModel.from_pretrained(model_name_or_path)\n        processor = _CLIPProcessor.from_pretrained(model_name_or_path)\n        return model, processor\n\n    raise ModuleNotFoundError(\n        \"`clip_score` metric requires `transformers` package be installed.\"\n        \" Either install with `pip install transformers>=4.10.0` or `pip install torchmetrics[multimodal]`.\"\n    )\n\n\ndef clip_score(\n    images: Union[Tensor, List[Tensor]],\n    text: Union[str, List[str]],\n    model_name_or_path: Literal[\n        \"openai/clip-vit-base-patch16\",\n        \"openai/clip-vit-base-patch32\",\n        \"openai/clip-vit-large-patch14-336\",\n        \"openai/clip-vit-large-patch14\",\n    ] = \"openai/clip-vit-large-patch14\",\n) -> Tensor:\n    r\"\"\"Calculate `CLIP Score`_ which is a text-to-image similarity metric.\n\n    CLIP is a reference free metric that can be used to evaluate the correlation between a generated caption for an\n    image and the actual content of the image. It has been found to be highly correlated with human judgement. The\n    metric is defined as:\n\n    .. math::\n        \\text{CLIPScore(I, C)} = max(100 * cos(E_I, E_C), 0)\n\n    which corresponds to the cosine similarity between visual CLIP embedding :math:`E_i` for an image :math:`i` and\n    textual CLIP embedding :math:`E_C` for an caption :math:`C`. The score is bound between 0 and 100 and the closer\n    to 100 the better.\n\n    .. note:: Metric is not scriptable\n\n    Args:\n        images: Either a single [N, C, H, W] tensor or a list of [C, H, W] tensors\n        text: Either a single caption or a list of captions\n        model_name_or_path: string indicating the version of the CLIP model to use. Available models are\n            `\"openai/clip-vit-base-patch16\"`, `\"openai/clip-vit-base-patch32\"`, `\"openai/clip-vit-large-patch14-336\"`\n            and `\"openai/clip-vit-large-patch14\"`,\n\n    Raises:\n        ModuleNotFoundError:\n            If transformers package is not installed or version is lower than 4.10.0\n        ValueError:\n            If not all images have format [C, H, W]\n        ValueError:\n            If the number of images and captions do not match\n\n    Example:\n        >>> import torch\n        >>> _ = torch.manual_seed(42)\n        >>> from torchmetrics.functional.multimodal import clip_score\n        >>> score = clip_score(torch.randint(255, (3, 224, 224)), \"a photo of a cat\", \"openai/clip-vit-base-patch16\")\n        >>> print(score.detach())\n        tensor(24.4255)\n\n    \"\"\"\n    model, processor = _get_model_and_processor(model_name_or_path)\n    device = images.device if isinstance(images, Tensor) else images[0].device\n    score, _ = _clip_score_update(images, text, model.to(device), processor)\n    score = score.mean(0)\n    return torch.max(score, torch.zeros_like(score))\n", "path": "src/torchmetrics/functional/multimodal/clip_score.py"}], "after_files": [{"content": "# Copyright The Lightning team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nfrom typing import List, Tuple, Union\n\nimport torch\nfrom torch import Tensor\nfrom typing_extensions import Literal\n\nfrom torchmetrics.utilities import rank_zero_warn\nfrom torchmetrics.utilities.checks import _SKIP_SLOW_DOCTEST, _try_proceed_with_timeout\nfrom torchmetrics.utilities.imports import _TRANSFORMERS_GREATER_EQUAL_4_10\n\nif _TRANSFORMERS_GREATER_EQUAL_4_10:\n    from transformers import CLIPModel as _CLIPModel\n    from transformers import CLIPProcessor as _CLIPProcessor\n\n    def _download_clip() -> None:\n        _CLIPModel.from_pretrained(\"openai/clip-vit-large-patch14\")\n        _CLIPProcessor.from_pretrained(\"openai/clip-vit-large-patch14\")\n\n    if _SKIP_SLOW_DOCTEST and not _try_proceed_with_timeout(_download_clip):\n        __doctest_skip__ = [\"clip_score\"]\n\nelse:\n    __doctest_skip__ = [\"clip_score\"]\n    _CLIPModel = None\n    _CLIPProcessor = None\n\n\ndef _clip_score_update(\n    images: Union[Tensor, List[Tensor]],\n    text: Union[str, List[str]],\n    model: _CLIPModel,\n    processor: _CLIPProcessor,\n) -> Tuple[Tensor, int]:\n    if not isinstance(images, list):\n        if images.ndim == 3:\n            images = [images]\n    else:  # unwrap into list\n        images = list(images)\n\n    if not all(i.ndim == 3 for i in images):\n        raise ValueError(\"Expected all images to be 3d but found image that has either more or less\")\n\n    if not isinstance(text, list):\n        text = [text]\n\n    if len(text) != len(images):\n        raise ValueError(\n            f\"Expected the number of images and text examples to be the same but got {len(images)} and {len(text)}\"\n        )\n    device = images[0].device\n    processed_input = processor(text=text, images=[i.cpu() for i in images], return_tensors=\"pt\", padding=True)\n\n    img_features = model.get_image_features(processed_input[\"pixel_values\"].to(device))\n    img_features = img_features / img_features.norm(p=2, dim=-1, keepdim=True)\n\n    max_position_embeddings = model.config.text_config.max_position_embeddings\n    if processed_input[\"attention_mask\"].shape[-1] > max_position_embeddings:\n        rank_zero_warn(\n            f\"Encountered caption longer than {max_position_embeddings=}. Will truncate captions to this length.\"\n            \"If longer captions are needed, initialize argument `model_name_or_path` with a model that supports\"\n            \"longer sequences\",\n            UserWarning,\n        )\n        processed_input[\"attention_mask\"] = processed_input[\"attention_mask\"][..., :max_position_embeddings]\n        processed_input[\"input_ids\"] = processed_input[\"input_ids\"][..., :max_position_embeddings]\n\n    txt_features = model.get_text_features(\n        processed_input[\"input_ids\"].to(device), processed_input[\"attention_mask\"].to(device)\n    )\n    txt_features = txt_features / txt_features.norm(p=2, dim=-1, keepdim=True)\n\n    # cosine similarity between feature vectors\n    score = 100 * (img_features * txt_features).sum(axis=-1)\n    return score, len(text)\n\n\ndef _get_model_and_processor(\n    model_name_or_path: Literal[\n        \"openai/clip-vit-base-patch16\",\n        \"openai/clip-vit-base-patch32\",\n        \"openai/clip-vit-large-patch14-336\",\n        \"openai/clip-vit-large-patch14\",\n    ] = \"openai/clip-vit-large-patch14\",\n) -> Tuple[_CLIPModel, _CLIPProcessor]:\n    if _TRANSFORMERS_GREATER_EQUAL_4_10:\n        model = _CLIPModel.from_pretrained(model_name_or_path)\n        processor = _CLIPProcessor.from_pretrained(model_name_or_path)\n        return model, processor\n\n    raise ModuleNotFoundError(\n        \"`clip_score` metric requires `transformers` package be installed.\"\n        \" Either install with `pip install transformers>=4.10.0` or `pip install torchmetrics[multimodal]`.\"\n    )\n\n\ndef clip_score(\n    images: Union[Tensor, List[Tensor]],\n    text: Union[str, List[str]],\n    model_name_or_path: Literal[\n        \"openai/clip-vit-base-patch16\",\n        \"openai/clip-vit-base-patch32\",\n        \"openai/clip-vit-large-patch14-336\",\n        \"openai/clip-vit-large-patch14\",\n    ] = \"openai/clip-vit-large-patch14\",\n) -> Tensor:\n    r\"\"\"Calculate `CLIP Score`_ which is a text-to-image similarity metric.\n\n    CLIP is a reference free metric that can be used to evaluate the correlation between a generated caption for an\n    image and the actual content of the image. It has been found to be highly correlated with human judgement. The\n    metric is defined as:\n\n    .. math::\n        \\text{CLIPScore(I, C)} = max(100 * cos(E_I, E_C), 0)\n\n    which corresponds to the cosine similarity between visual CLIP embedding :math:`E_i` for an image :math:`i` and\n    textual CLIP embedding :math:`E_C` for an caption :math:`C`. The score is bound between 0 and 100 and the closer\n    to 100 the better.\n\n    .. note:: Metric is not scriptable\n\n    Args:\n        images: Either a single [N, C, H, W] tensor or a list of [C, H, W] tensors\n        text: Either a single caption or a list of captions\n        model_name_or_path: string indicating the version of the CLIP model to use. Available models are\n            `\"openai/clip-vit-base-patch16\"`, `\"openai/clip-vit-base-patch32\"`, `\"openai/clip-vit-large-patch14-336\"`\n            and `\"openai/clip-vit-large-patch14\"`,\n\n    Raises:\n        ModuleNotFoundError:\n            If transformers package is not installed or version is lower than 4.10.0\n        ValueError:\n            If not all images have format [C, H, W]\n        ValueError:\n            If the number of images and captions do not match\n\n    Example:\n        >>> import torch\n        >>> _ = torch.manual_seed(42)\n        >>> from torchmetrics.functional.multimodal import clip_score\n        >>> score = clip_score(torch.randint(255, (3, 224, 224)), \"a photo of a cat\", \"openai/clip-vit-base-patch16\")\n        >>> print(score.detach())\n        tensor(24.4255)\n\n    \"\"\"\n    model, processor = _get_model_and_processor(model_name_or_path)\n    device = images.device if isinstance(images, Tensor) else images[0].device\n    score, _ = _clip_score_update(images, text, model.to(device), processor)\n    score = score.mean(0)\n    return torch.max(score, torch.zeros_like(score))\n", "path": "src/torchmetrics/functional/multimodal/clip_score.py"}]}