We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
NMS behaviour w.r.t. fp16 vs fp32
## 🐛 Bug

NMS gives significantly different outputs when switching boxes from FP32 to FP16. I couldn't find any related issue here or on the discussion board, and I didn't see an obvious cause from reading the docs.

## To Reproduce
Call `torchvision.ops.nms(boxes, scores, iou_threshold=0.2)`
boxes and scores (FP32) -> [data.zip](https://github.com/pytorch/vision/files/5961462/data.zip)
NMS has one output.

Change boxes to float16 (I used `.to(torch.float16)`), and NMS gives 37 outputs (no suppression is performed?)
I wasn't expecting type conversion from FP32 to FP16 to dramatically alter results. Let me know if this is just a case of user error.

## Environment
```
PyTorch version: 1.7.1+cu101
Is debug build: False
CUDA used to build PyTorch: 10.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: Tesla V100-SXM2-16GB
GPU 1: Tesla V100-SXM2-16GB
GPU 2: Tesla V100-SXM2-16GB
GPU 3: Tesla V100-SXM2-16GB
GPU 4: Tesla V100-SXM2-16GB
GPU 5: Tesla V100-SXM2-16GB
GPU 6: Tesla V100-SXM2-16GB
GPU 7: Tesla V100-SXM2-16GB

Nvidia driver version: 440.33.01
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.7.1+cu101
[pip3] torchaudio==0.7.2
[pip3] torchvision==0.8.2+cu101
[conda] Could not collect
```

Possible related [issue](https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/issues/153)?

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `torchvision/ops/boxes.py`
Content:
```
1 import torch
2 from torch import Tensor
3 from typing import Tuple
4 from ._box_convert import _box_cxcywh_to_xyxy, _box_xyxy_to_cxcywh, _box_xywh_to_xyxy, _box_xyxy_to_xywh
5 import torchvision
6 from torchvision.extension import _assert_has_ops
7 
8 
9 def nms(boxes: Tensor, scores: Tensor, iou_threshold: float) -> Tensor:
10     """
11     Performs non-maximum suppression (NMS) on the boxes according
12     to their intersection-over-union (IoU).
13 
14     NMS iteratively removes lower scoring boxes which have an
15     IoU greater than iou_threshold with another (higher scoring)
16     box.
17 
18     If multiple boxes have the exact same score and satisfy the IoU
19     criterion with respect to a reference box, the selected box is
20     not guaranteed to be the same between CPU and GPU. This is similar
21     to the behavior of argsort in PyTorch when repeated values are present.
22 
23     Args:
24         boxes (Tensor[N, 4])): boxes to perform NMS on. They
25             are expected to be in (x1, y1, x2, y2) format
26         scores (Tensor[N]): scores for each one of the boxes
27         iou_threshold (float): discards all overlapping boxes with IoU > iou_threshold
28 
29     Returns:
30         keep (Tensor): int64 tensor with the indices
31             of the elements that have been kept
32             by NMS, sorted in decreasing order of scores
33     """
34     _assert_has_ops()
35     return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
36 
37 
38 @torch.jit._script_if_tracing
39 def batched_nms(
40     boxes: Tensor,
41     scores: Tensor,
42     idxs: Tensor,
43     iou_threshold: float,
44 ) -> Tensor:
45     """
46     Performs non-maximum suppression in a batched fashion.
47 
48     Each index value correspond to a category, and NMS
49     will not be applied between elements of different categories.
50 
51     Args:
52         boxes (Tensor[N, 4]): boxes where NMS will be performed. They
53             are expected to be in (x1, y1, x2, y2) format
54         scores (Tensor[N]): scores for each one of the boxes
55         idxs (Tensor[N]): indices of the categories for each one of the boxes.
56         iou_threshold (float): discards all overlapping boxes with IoU > iou_threshold
57 
58     Returns:
59         keep (Tensor): int64 tensor with the indices of
60             the elements that have been kept by NMS, sorted
61             in decreasing order of scores
62     """
63     if boxes.numel() == 0:
64         return torch.empty((0,), dtype=torch.int64, device=boxes.device)
65     # strategy: in order to perform NMS independently per class.
66     # we add an offset to all the boxes. The offset is dependent
67     # only on the class idx, and is large enough so that boxes
68     # from different classes do not overlap
69     else:
70         max_coordinate = boxes.max()
71         offsets = idxs.to(boxes) * (max_coordinate + torch.tensor(1).to(boxes))
72         boxes_for_nms = boxes + offsets[:, None]
73         keep = nms(boxes_for_nms, scores, iou_threshold)
74         return keep
75 
76 
77 def remove_small_boxes(boxes: Tensor, min_size: float) -> Tensor:
78     """
79     Remove boxes which contains at least one side smaller than min_size.
80 
81     Args:
82         boxes (Tensor[N, 4]): boxes in (x1, y1, x2, y2) format
83         min_size (float): minimum size
84 
85     Returns:
86         keep (Tensor[K]): indices of the boxes that have both sides
87             larger than min_size
88     """
89     ws, hs = boxes[:, 2] - boxes[:, 0], boxes[:, 3] - boxes[:, 1]
90     keep = (ws >= min_size) & (hs >= min_size)
91     keep = torch.where(keep)[0]
92     return keep
93 
94 
95 def clip_boxes_to_image(boxes: Tensor, size: Tuple[int, int]) -> Tensor:
96     """
97     Clip boxes so that they lie inside an image of size `size`.
98 
99     Args:
100         boxes (Tensor[N, 4]): boxes in (x1, y1, x2, y2) format
101         size (Tuple[height, width]): size of the image
102 
103     Returns:
104         clipped_boxes (Tensor[N, 4])
105     """
106     dim = boxes.dim()
107     boxes_x = boxes[..., 0::2]
108     boxes_y = boxes[..., 1::2]
109     height, width = size
110 
111     if torchvision._is_tracing():
112         boxes_x = torch.max(boxes_x, torch.tensor(0, dtype=boxes.dtype, device=boxes.device))
113         boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))
114         boxes_y = torch.max(boxes_y, torch.tensor(0, dtype=boxes.dtype, device=boxes.device))
115         boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))
116     else:
117         boxes_x = boxes_x.clamp(min=0, max=width)
118         boxes_y = boxes_y.clamp(min=0, max=height)
119 
120     clipped_boxes = torch.stack((boxes_x, boxes_y), dim=dim)
121     return clipped_boxes.reshape(boxes.shape)
122 
123 
124 def box_convert(boxes: Tensor, in_fmt: str, out_fmt: str) -> Tensor:
125     """
126     Converts boxes from given in_fmt to out_fmt.
127     Supported in_fmt and out_fmt are:
128 
129     'xyxy': boxes are represented via corners, x1, y1 being top left and x2, y2 being bottom right.
130 
131     'xywh' : boxes are represented via corner, width and height, x1, y2 being top left, w, h being width and height.
132 
133     'cxcywh' : boxes are represented via centre, width and height, cx, cy being center of box, w, h
134     being width and height.
135 
136     Args:
137         boxes (Tensor[N, 4]): boxes which will be converted.
138         in_fmt (str): Input format of given boxes. Supported formats are ['xyxy', 'xywh', 'cxcywh'].
139         out_fmt (str): Output format of given boxes. Supported formats are ['xyxy', 'xywh', 'cxcywh']
140 
141     Returns:
142         boxes (Tensor[N, 4]): Boxes into converted format.
143     """
144 
145     allowed_fmts = ("xyxy", "xywh", "cxcywh")
146     if in_fmt not in allowed_fmts or out_fmt not in allowed_fmts:
147         raise ValueError("Unsupported Bounding Box Conversions for given in_fmt and out_fmt")
148 
149     if in_fmt == out_fmt:
150         return boxes.clone()
151 
152     if in_fmt != 'xyxy' and out_fmt != 'xyxy':
153         # convert to xyxy and change in_fmt xyxy
154         if in_fmt == "xywh":
155             boxes = _box_xywh_to_xyxy(boxes)
156         elif in_fmt == "cxcywh":
157             boxes = _box_cxcywh_to_xyxy(boxes)
158         in_fmt = 'xyxy'
159 
160     if in_fmt == "xyxy":
161         if out_fmt == "xywh":
162             boxes = _box_xyxy_to_xywh(boxes)
163         elif out_fmt == "cxcywh":
164             boxes = _box_xyxy_to_cxcywh(boxes)
165     elif out_fmt == "xyxy":
166         if in_fmt == "xywh":
167             boxes = _box_xywh_to_xyxy(boxes)
168         elif in_fmt == "cxcywh":
169             boxes = _box_cxcywh_to_xyxy(boxes)
170     return boxes
171 
172 
173 def box_area(boxes: Tensor) -> Tensor:
174     """
175     Computes the area of a set of bounding boxes, which are specified by its
176     (x1, y1, x2, y2) coordinates.
177 
178     Args:
179         boxes (Tensor[N, 4]): boxes for which the area will be computed. They
180             are expected to be in (x1, y1, x2, y2) format
181 
182     Returns:
183         area (Tensor[N]): area for each box
184     """
185     return (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])
186 
187 
188 # implementation from https://github.com/kuangliu/torchcv/blob/master/torchcv/utils/box.py
189 # with slight modifications
190 def _box_inter_union(boxes1: Tensor, boxes2: Tensor) -> Tuple[Tensor, Tensor]:
191     area1 = box_area(boxes1)
192     area2 = box_area(boxes2)
193 
194     lt = torch.max(boxes1[:, None, :2], boxes2[:, :2])  # [N,M,2]
195     rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:])  # [N,M,2]
196 
197     wh = (rb - lt).clamp(min=0)  # [N,M,2]
198     inter = wh[:, :, 0] * wh[:, :, 1]  # [N,M]
199 
200     union = area1[:, None] + area2 - inter
201 
202     return inter, union
203 
204 
205 def box_iou(boxes1: Tensor, boxes2: Tensor) -> Tensor:
206     """
207     Return intersection-over-union (Jaccard index) of boxes.
208 
209     Both sets of boxes are expected to be in (x1, y1, x2, y2) format.
210 
211     Args:
212         boxes1 (Tensor[N, 4])
213         boxes2 (Tensor[M, 4])
214 
215     Returns:
216         iou (Tensor[N, M]): the NxM matrix containing the pairwise IoU values for every element in boxes1 and boxes2
217     """
218     inter, union = _box_inter_union(boxes1, boxes2)
219     iou = inter / union
220     return iou
221 
222 
223 # Implementation adapted from https://github.com/facebookresearch/detr/blob/master/util/box_ops.py
224 def generalized_box_iou(boxes1: Tensor, boxes2: Tensor) -> Tensor:
225     """
226     Return generalized intersection-over-union (Jaccard index) of boxes.
227 
228     Both sets of boxes are expected to be in (x1, y1, x2, y2) format.
229 
230     Args:
231         boxes1 (Tensor[N, 4])
232         boxes2 (Tensor[M, 4])
233 
234     Returns:
235         generalized_iou (Tensor[N, M]): the NxM matrix containing the pairwise generalized_IoU values
236         for every element in boxes1 and boxes2
237     """
238 
239     # degenerate boxes gives inf / nan results
240     # so do an early check
241     assert (boxes1[:, 2:] >= boxes1[:, :2]).all()
242     assert (boxes2[:, 2:] >= boxes2[:, :2]).all()
243 
244     inter, union = _box_inter_union(boxes1, boxes2)
245     iou = inter / union
246 
247     lti = torch.min(boxes1[:, None, :2], boxes2[:, :2])
248     rbi = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])
249 
250     whi = (rbi - lti).clamp(min=0)  # [N,M,2]
251     areai = whi[:, :, 0] * whi[:, :, 1]
252 
253     return iou - (areai - union) / areai
254
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/torchvision/ops/boxes.py b/torchvision/ops/boxes.py
--- a/torchvision/ops/boxes.py
+++ b/torchvision/ops/boxes.py
@@ -170,6 +170,14 @@
     return boxes
 
 
+def _upcast(t: Tensor) -> Tensor:
+    # Protects from numerical overflows in multiplications by upcasting to the equivalent higher type
+    if t.is_floating_point():
+        return t if t.dtype in (torch.float32, torch.float64) else t.float()
+    else:
+        return t if t.dtype in (torch.int32, torch.int64) else t.int()
+
+
 def box_area(boxes: Tensor) -> Tensor:
     """
     Computes the area of a set of bounding boxes, which are specified by its
@@ -182,6 +190,7 @@
     Returns:
         area (Tensor[N]): area for each box
     """
+    boxes = _upcast(boxes)
     return (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])
 
 
@@ -194,7 +203,7 @@
     lt = torch.max(boxes1[:, None, :2], boxes2[:, :2])  # [N,M,2]
     rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:])  # [N,M,2]
 
-    wh = (rb - lt).clamp(min=0)  # [N,M,2]
+    wh = _upcast(rb - lt).clamp(min=0)  # [N,M,2]
     inter = wh[:, :, 0] * wh[:, :, 1]  # [N,M]
 
     union = area1[:, None] + area2 - inter
@@ -247,7 +256,7 @@
     lti = torch.min(boxes1[:, None, :2], boxes2[:, :2])
     rbi = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])
 
-    whi = (rbi - lti).clamp(min=0)  # [N,M,2]
+    whi = _upcast(rbi - lti).clamp(min=0)  # [N,M,2]
     areai = whi[:, :, 0] * whi[:, :, 1]
 
     return iou - (areai - union) / areai

{"golden_diff": "diff --git a/torchvision/ops/boxes.py b/torchvision/ops/boxes.py\n--- a/torchvision/ops/boxes.py\n+++ b/torchvision/ops/boxes.py\n@@ -170,6 +170,14 @@\n     return boxes\n \n \n+def _upcast(t: Tensor) -> Tensor:\n+    # Protects from numerical overflows in multiplications by upcasting to the equivalent higher type\n+    if t.is_floating_point():\n+        return t if t.dtype in (torch.float32, torch.float64) else t.float()\n+    else:\n+        return t if t.dtype in (torch.int32, torch.int64) else t.int()\n+\n+\n def box_area(boxes: Tensor) -> Tensor:\n     \"\"\"\n     Computes the area of a set of bounding boxes, which are specified by its\n@@ -182,6 +190,7 @@\n     Returns:\n         area (Tensor[N]): area for each box\n     \"\"\"\n+    boxes = _upcast(boxes)\n     return (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])\n \n \n@@ -194,7 +203,7 @@\n     lt = torch.max(boxes1[:, None, :2], boxes2[:, :2])  # [N,M,2]\n     rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:])  # [N,M,2]\n \n-    wh = (rb - lt).clamp(min=0)  # [N,M,2]\n+    wh = _upcast(rb - lt).clamp(min=0)  # [N,M,2]\n     inter = wh[:, :, 0] * wh[:, :, 1]  # [N,M]\n \n     union = area1[:, None] + area2 - inter\n@@ -247,7 +256,7 @@\n     lti = torch.min(boxes1[:, None, :2], boxes2[:, :2])\n     rbi = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])\n \n-    whi = (rbi - lti).clamp(min=0)  # [N,M,2]\n+    whi = _upcast(rbi - lti).clamp(min=0)  # [N,M,2]\n     areai = whi[:, :, 0] * whi[:, :, 1]\n \n     return iou - (areai - union) / areai\n", "issue": "NMS behaviour w.r.t. fp16 vs fp32\n## \ud83d\udc1b Bug\r\n\r\nNMS gives significantly different outputs when switching boxes from FP32 to FP16. I couldn't find any related issue here or on the discussion board, and I didn't see an obvious cause from reading the docs.\r\n\r\n## To Reproduce\r\nCall `torchvision.ops.nms(boxes, scores, iou_threshold=0.2)`\r\nboxes and scores (FP32) -> [data.zip](https://github.com/pytorch/vision/files/5961462/data.zip)\r\nNMS has one output.\r\n\r\nChange boxes to float16 (I used `.to(torch.float16)`), and NMS gives 37 outputs (no suppression is performed?)\r\nI wasn't expecting type conversion from FP32 to FP16 to dramatically alter results. Let me know if this is just a case of user error.\r\n\r\n## Environment\r\n```\r\nPyTorch version: 1.7.1+cu101\r\nIs debug build: False\r\nCUDA used to build PyTorch: 10.1\r\nROCM used to build PyTorch: N/A\r\n\r\nOS: Ubuntu 18.04.5 LTS (x86_64)\r\nGCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\r\nClang version: Could not collect\r\nCMake version: version 3.10.2\r\n\r\nPython version: 3.8 (64-bit runtime)\r\nIs CUDA available: True\r\nCUDA runtime version: Could not collect\r\nGPU models and configuration: \r\nGPU 0: Tesla V100-SXM2-16GB\r\nGPU 1: Tesla V100-SXM2-16GB\r\nGPU 2: Tesla V100-SXM2-16GB\r\nGPU 3: Tesla V100-SXM2-16GB\r\nGPU 4: Tesla V100-SXM2-16GB\r\nGPU 5: Tesla V100-SXM2-16GB\r\nGPU 6: Tesla V100-SXM2-16GB\r\nGPU 7: Tesla V100-SXM2-16GB\r\n\r\nNvidia driver version: 440.33.01\r\ncuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5\r\nHIP runtime version: N/A\r\nMIOpen runtime version: N/A\r\n\r\nVersions of relevant libraries:\r\n[pip3] numpy==1.19.5\r\n[pip3] torch==1.7.1+cu101\r\n[pip3] torchaudio==0.7.2\r\n[pip3] torchvision==0.8.2+cu101\r\n[conda] Could not collect\r\n```\r\n\r\nPossible related [issue](https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/issues/153)?\n", "before_files": [{"content": "import torch\nfrom torch import Tensor\nfrom typing import Tuple\nfrom ._box_convert import _box_cxcywh_to_xyxy, _box_xyxy_to_cxcywh, _box_xywh_to_xyxy, _box_xyxy_to_xywh\nimport torchvision\nfrom torchvision.extension import _assert_has_ops\n\n\ndef nms(boxes: Tensor, scores: Tensor, iou_threshold: float) -> Tensor:\n    \"\"\"\n    Performs non-maximum suppression (NMS) on the boxes according\n    to their intersection-over-union (IoU).\n\n    NMS iteratively removes lower scoring boxes which have an\n    IoU greater than iou_threshold with another (higher scoring)\n    box.\n\n    If multiple boxes have the exact same score and satisfy the IoU\n    criterion with respect to a reference box, the selected box is\n    not guaranteed to be the same between CPU and GPU. This is similar\n    to the behavior of argsort in PyTorch when repeated values are present.\n\n    Args:\n        boxes (Tensor[N, 4])): boxes to perform NMS on. They\n            are expected to be in (x1, y1, x2, y2) format\n        scores (Tensor[N]): scores for each one of the boxes\n        iou_threshold (float): discards all overlapping boxes with IoU > iou_threshold\n\n    Returns:\n        keep (Tensor): int64 tensor with the indices\n            of the elements that have been kept\n            by NMS, sorted in decreasing order of scores\n    \"\"\"\n    _assert_has_ops()\n    return torch.ops.torchvision.nms(boxes, scores, iou_threshold)\n\n\n@torch.jit._script_if_tracing\ndef batched_nms(\n    boxes: Tensor,\n    scores: Tensor,\n    idxs: Tensor,\n    iou_threshold: float,\n) -> Tensor:\n    \"\"\"\n    Performs non-maximum suppression in a batched fashion.\n\n    Each index value correspond to a category, and NMS\n    will not be applied between elements of different categories.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes where NMS will be performed. They\n            are expected to be in (x1, y1, x2, y2) format\n        scores (Tensor[N]): scores for each one of the boxes\n        idxs (Tensor[N]): indices of the categories for each one of the boxes.\n        iou_threshold (float): discards all overlapping boxes with IoU > iou_threshold\n\n    Returns:\n        keep (Tensor): int64 tensor with the indices of\n            the elements that have been kept by NMS, sorted\n            in decreasing order of scores\n    \"\"\"\n    if boxes.numel() == 0:\n        return torch.empty((0,), dtype=torch.int64, device=boxes.device)\n    # strategy: in order to perform NMS independently per class.\n    # we add an offset to all the boxes. The offset is dependent\n    # only on the class idx, and is large enough so that boxes\n    # from different classes do not overlap\n    else:\n        max_coordinate = boxes.max()\n        offsets = idxs.to(boxes) * (max_coordinate + torch.tensor(1).to(boxes))\n        boxes_for_nms = boxes + offsets[:, None]\n        keep = nms(boxes_for_nms, scores, iou_threshold)\n        return keep\n\n\ndef remove_small_boxes(boxes: Tensor, min_size: float) -> Tensor:\n    \"\"\"\n    Remove boxes which contains at least one side smaller than min_size.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes in (x1, y1, x2, y2) format\n        min_size (float): minimum size\n\n    Returns:\n        keep (Tensor[K]): indices of the boxes that have both sides\n            larger than min_size\n    \"\"\"\n    ws, hs = boxes[:, 2] - boxes[:, 0], boxes[:, 3] - boxes[:, 1]\n    keep = (ws >= min_size) & (hs >= min_size)\n    keep = torch.where(keep)[0]\n    return keep\n\n\ndef clip_boxes_to_image(boxes: Tensor, size: Tuple[int, int]) -> Tensor:\n    \"\"\"\n    Clip boxes so that they lie inside an image of size `size`.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes in (x1, y1, x2, y2) format\n        size (Tuple[height, width]): size of the image\n\n    Returns:\n        clipped_boxes (Tensor[N, 4])\n    \"\"\"\n    dim = boxes.dim()\n    boxes_x = boxes[..., 0::2]\n    boxes_y = boxes[..., 1::2]\n    height, width = size\n\n    if torchvision._is_tracing():\n        boxes_x = torch.max(boxes_x, torch.tensor(0, dtype=boxes.dtype, device=boxes.device))\n        boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))\n        boxes_y = torch.max(boxes_y, torch.tensor(0, dtype=boxes.dtype, device=boxes.device))\n        boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))\n    else:\n        boxes_x = boxes_x.clamp(min=0, max=width)\n        boxes_y = boxes_y.clamp(min=0, max=height)\n\n    clipped_boxes = torch.stack((boxes_x, boxes_y), dim=dim)\n    return clipped_boxes.reshape(boxes.shape)\n\n\ndef box_convert(boxes: Tensor, in_fmt: str, out_fmt: str) -> Tensor:\n    \"\"\"\n    Converts boxes from given in_fmt to out_fmt.\n    Supported in_fmt and out_fmt are:\n\n    'xyxy': boxes are represented via corners, x1, y1 being top left and x2, y2 being bottom right.\n\n    'xywh' : boxes are represented via corner, width and height, x1, y2 being top left, w, h being width and height.\n\n    'cxcywh' : boxes are represented via centre, width and height, cx, cy being center of box, w, h\n    being width and height.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes which will be converted.\n        in_fmt (str): Input format of given boxes. Supported formats are ['xyxy', 'xywh', 'cxcywh'].\n        out_fmt (str): Output format of given boxes. Supported formats are ['xyxy', 'xywh', 'cxcywh']\n\n    Returns:\n        boxes (Tensor[N, 4]): Boxes into converted format.\n    \"\"\"\n\n    allowed_fmts = (\"xyxy\", \"xywh\", \"cxcywh\")\n    if in_fmt not in allowed_fmts or out_fmt not in allowed_fmts:\n        raise ValueError(\"Unsupported Bounding Box Conversions for given in_fmt and out_fmt\")\n\n    if in_fmt == out_fmt:\n        return boxes.clone()\n\n    if in_fmt != 'xyxy' and out_fmt != 'xyxy':\n        # convert to xyxy and change in_fmt xyxy\n        if in_fmt == \"xywh\":\n            boxes = _box_xywh_to_xyxy(boxes)\n        elif in_fmt == \"cxcywh\":\n            boxes = _box_cxcywh_to_xyxy(boxes)\n        in_fmt = 'xyxy'\n\n    if in_fmt == \"xyxy\":\n        if out_fmt == \"xywh\":\n            boxes = _box_xyxy_to_xywh(boxes)\n        elif out_fmt == \"cxcywh\":\n            boxes = _box_xyxy_to_cxcywh(boxes)\n    elif out_fmt == \"xyxy\":\n        if in_fmt == \"xywh\":\n            boxes = _box_xywh_to_xyxy(boxes)\n        elif in_fmt == \"cxcywh\":\n            boxes = _box_cxcywh_to_xyxy(boxes)\n    return boxes\n\n\ndef box_area(boxes: Tensor) -> Tensor:\n    \"\"\"\n    Computes the area of a set of bounding boxes, which are specified by its\n    (x1, y1, x2, y2) coordinates.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes for which the area will be computed. They\n            are expected to be in (x1, y1, x2, y2) format\n\n    Returns:\n        area (Tensor[N]): area for each box\n    \"\"\"\n    return (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])\n\n\n# implementation from https://github.com/kuangliu/torchcv/blob/master/torchcv/utils/box.py\n# with slight modifications\ndef _box_inter_union(boxes1: Tensor, boxes2: Tensor) -> Tuple[Tensor, Tensor]:\n    area1 = box_area(boxes1)\n    area2 = box_area(boxes2)\n\n    lt = torch.max(boxes1[:, None, :2], boxes2[:, :2])  # [N,M,2]\n    rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:])  # [N,M,2]\n\n    wh = (rb - lt).clamp(min=0)  # [N,M,2]\n    inter = wh[:, :, 0] * wh[:, :, 1]  # [N,M]\n\n    union = area1[:, None] + area2 - inter\n\n    return inter, union\n\n\ndef box_iou(boxes1: Tensor, boxes2: Tensor) -> Tensor:\n    \"\"\"\n    Return intersection-over-union (Jaccard index) of boxes.\n\n    Both sets of boxes are expected to be in (x1, y1, x2, y2) format.\n\n    Args:\n        boxes1 (Tensor[N, 4])\n        boxes2 (Tensor[M, 4])\n\n    Returns:\n        iou (Tensor[N, M]): the NxM matrix containing the pairwise IoU values for every element in boxes1 and boxes2\n    \"\"\"\n    inter, union = _box_inter_union(boxes1, boxes2)\n    iou = inter / union\n    return iou\n\n\n# Implementation adapted from https://github.com/facebookresearch/detr/blob/master/util/box_ops.py\ndef generalized_box_iou(boxes1: Tensor, boxes2: Tensor) -> Tensor:\n    \"\"\"\n    Return generalized intersection-over-union (Jaccard index) of boxes.\n\n    Both sets of boxes are expected to be in (x1, y1, x2, y2) format.\n\n    Args:\n        boxes1 (Tensor[N, 4])\n        boxes2 (Tensor[M, 4])\n\n    Returns:\n        generalized_iou (Tensor[N, M]): the NxM matrix containing the pairwise generalized_IoU values\n        for every element in boxes1 and boxes2\n    \"\"\"\n\n    # degenerate boxes gives inf / nan results\n    # so do an early check\n    assert (boxes1[:, 2:] >= boxes1[:, :2]).all()\n    assert (boxes2[:, 2:] >= boxes2[:, :2]).all()\n\n    inter, union = _box_inter_union(boxes1, boxes2)\n    iou = inter / union\n\n    lti = torch.min(boxes1[:, None, :2], boxes2[:, :2])\n    rbi = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])\n\n    whi = (rbi - lti).clamp(min=0)  # [N,M,2]\n    areai = whi[:, :, 0] * whi[:, :, 1]\n\n    return iou - (areai - union) / areai\n", "path": "torchvision/ops/boxes.py"}], "after_files": [{"content": "import torch\nfrom torch import Tensor\nfrom typing import Tuple\nfrom ._box_convert import _box_cxcywh_to_xyxy, _box_xyxy_to_cxcywh, _box_xywh_to_xyxy, _box_xyxy_to_xywh\nimport torchvision\nfrom torchvision.extension import _assert_has_ops\n\n\ndef nms(boxes: Tensor, scores: Tensor, iou_threshold: float) -> Tensor:\n    \"\"\"\n    Performs non-maximum suppression (NMS) on the boxes according\n    to their intersection-over-union (IoU).\n\n    NMS iteratively removes lower scoring boxes which have an\n    IoU greater than iou_threshold with another (higher scoring)\n    box.\n\n    If multiple boxes have the exact same score and satisfy the IoU\n    criterion with respect to a reference box, the selected box is\n    not guaranteed to be the same between CPU and GPU. This is similar\n    to the behavior of argsort in PyTorch when repeated values are present.\n\n    Args:\n        boxes (Tensor[N, 4])): boxes to perform NMS on. They\n            are expected to be in (x1, y1, x2, y2) format\n        scores (Tensor[N]): scores for each one of the boxes\n        iou_threshold (float): discards all overlapping boxes with IoU > iou_threshold\n\n    Returns:\n        keep (Tensor): int64 tensor with the indices\n            of the elements that have been kept\n            by NMS, sorted in decreasing order of scores\n    \"\"\"\n    _assert_has_ops()\n    return torch.ops.torchvision.nms(boxes, scores, iou_threshold)\n\n\n@torch.jit._script_if_tracing\ndef batched_nms(\n    boxes: Tensor,\n    scores: Tensor,\n    idxs: Tensor,\n    iou_threshold: float,\n) -> Tensor:\n    \"\"\"\n    Performs non-maximum suppression in a batched fashion.\n\n    Each index value correspond to a category, and NMS\n    will not be applied between elements of different categories.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes where NMS will be performed. They\n            are expected to be in (x1, y1, x2, y2) format\n        scores (Tensor[N]): scores for each one of the boxes\n        idxs (Tensor[N]): indices of the categories for each one of the boxes.\n        iou_threshold (float): discards all overlapping boxes with IoU > iou_threshold\n\n    Returns:\n        keep (Tensor): int64 tensor with the indices of\n            the elements that have been kept by NMS, sorted\n            in decreasing order of scores\n    \"\"\"\n    if boxes.numel() == 0:\n        return torch.empty((0,), dtype=torch.int64, device=boxes.device)\n    # strategy: in order to perform NMS independently per class.\n    # we add an offset to all the boxes. The offset is dependent\n    # only on the class idx, and is large enough so that boxes\n    # from different classes do not overlap\n    else:\n        max_coordinate = boxes.max()\n        offsets = idxs.to(boxes) * (max_coordinate + torch.tensor(1).to(boxes))\n        boxes_for_nms = boxes + offsets[:, None]\n        keep = nms(boxes_for_nms, scores, iou_threshold)\n        return keep\n\n\ndef remove_small_boxes(boxes: Tensor, min_size: float) -> Tensor:\n    \"\"\"\n    Remove boxes which contains at least one side smaller than min_size.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes in (x1, y1, x2, y2) format\n        min_size (float): minimum size\n\n    Returns:\n        keep (Tensor[K]): indices of the boxes that have both sides\n            larger than min_size\n    \"\"\"\n    ws, hs = boxes[:, 2] - boxes[:, 0], boxes[:, 3] - boxes[:, 1]\n    keep = (ws >= min_size) & (hs >= min_size)\n    keep = torch.where(keep)[0]\n    return keep\n\n\ndef clip_boxes_to_image(boxes: Tensor, size: Tuple[int, int]) -> Tensor:\n    \"\"\"\n    Clip boxes so that they lie inside an image of size `size`.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes in (x1, y1, x2, y2) format\n        size (Tuple[height, width]): size of the image\n\n    Returns:\n        clipped_boxes (Tensor[N, 4])\n    \"\"\"\n    dim = boxes.dim()\n    boxes_x = boxes[..., 0::2]\n    boxes_y = boxes[..., 1::2]\n    height, width = size\n\n    if torchvision._is_tracing():\n        boxes_x = torch.max(boxes_x, torch.tensor(0, dtype=boxes.dtype, device=boxes.device))\n        boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))\n        boxes_y = torch.max(boxes_y, torch.tensor(0, dtype=boxes.dtype, device=boxes.device))\n        boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))\n    else:\n        boxes_x = boxes_x.clamp(min=0, max=width)\n        boxes_y = boxes_y.clamp(min=0, max=height)\n\n    clipped_boxes = torch.stack((boxes_x, boxes_y), dim=dim)\n    return clipped_boxes.reshape(boxes.shape)\n\n\ndef box_convert(boxes: Tensor, in_fmt: str, out_fmt: str) -> Tensor:\n    \"\"\"\n    Converts boxes from given in_fmt to out_fmt.\n    Supported in_fmt and out_fmt are:\n\n    'xyxy': boxes are represented via corners, x1, y1 being top left and x2, y2 being bottom right.\n\n    'xywh' : boxes are represented via corner, width and height, x1, y2 being top left, w, h being width and height.\n\n    'cxcywh' : boxes are represented via centre, width and height, cx, cy being center of box, w, h\n    being width and height.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes which will be converted.\n        in_fmt (str): Input format of given boxes. Supported formats are ['xyxy', 'xywh', 'cxcywh'].\n        out_fmt (str): Output format of given boxes. Supported formats are ['xyxy', 'xywh', 'cxcywh']\n\n    Returns:\n        boxes (Tensor[N, 4]): Boxes into converted format.\n    \"\"\"\n\n    allowed_fmts = (\"xyxy\", \"xywh\", \"cxcywh\")\n    if in_fmt not in allowed_fmts or out_fmt not in allowed_fmts:\n        raise ValueError(\"Unsupported Bounding Box Conversions for given in_fmt and out_fmt\")\n\n    if in_fmt == out_fmt:\n        return boxes.clone()\n\n    if in_fmt != 'xyxy' and out_fmt != 'xyxy':\n        # convert to xyxy and change in_fmt xyxy\n        if in_fmt == \"xywh\":\n            boxes = _box_xywh_to_xyxy(boxes)\n        elif in_fmt == \"cxcywh\":\n            boxes = _box_cxcywh_to_xyxy(boxes)\n        in_fmt = 'xyxy'\n\n    if in_fmt == \"xyxy\":\n        if out_fmt == \"xywh\":\n            boxes = _box_xyxy_to_xywh(boxes)\n        elif out_fmt == \"cxcywh\":\n            boxes = _box_xyxy_to_cxcywh(boxes)\n    elif out_fmt == \"xyxy\":\n        if in_fmt == \"xywh\":\n            boxes = _box_xywh_to_xyxy(boxes)\n        elif in_fmt == \"cxcywh\":\n            boxes = _box_cxcywh_to_xyxy(boxes)\n    return boxes\n\n\ndef _upcast(t: Tensor) -> Tensor:\n    # Protects from numerical overflows in multiplications by upcasting to the equivalent higher type\n    if t.is_floating_point():\n        return t if t.dtype in (torch.float32, torch.float64) else t.float()\n    else:\n        return t if t.dtype in (torch.int32, torch.int64) else t.int()\n\n\ndef box_area(boxes: Tensor) -> Tensor:\n    \"\"\"\n    Computes the area of a set of bounding boxes, which are specified by its\n    (x1, y1, x2, y2) coordinates.\n\n    Args:\n        boxes (Tensor[N, 4]): boxes for which the area will be computed. They\n            are expected to be in (x1, y1, x2, y2) format\n\n    Returns:\n        area (Tensor[N]): area for each box\n    \"\"\"\n    boxes = _upcast(boxes)\n    return (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])\n\n\n# implementation from https://github.com/kuangliu/torchcv/blob/master/torchcv/utils/box.py\n# with slight modifications\ndef _box_inter_union(boxes1: Tensor, boxes2: Tensor) -> Tuple[Tensor, Tensor]:\n    area1 = box_area(boxes1)\n    area2 = box_area(boxes2)\n\n    lt = torch.max(boxes1[:, None, :2], boxes2[:, :2])  # [N,M,2]\n    rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:])  # [N,M,2]\n\n    wh = _upcast(rb - lt).clamp(min=0)  # [N,M,2]\n    inter = wh[:, :, 0] * wh[:, :, 1]  # [N,M]\n\n    union = area1[:, None] + area2 - inter\n\n    return inter, union\n\n\ndef box_iou(boxes1: Tensor, boxes2: Tensor) -> Tensor:\n    \"\"\"\n    Return intersection-over-union (Jaccard index) of boxes.\n\n    Both sets of boxes are expected to be in (x1, y1, x2, y2) format.\n\n    Args:\n        boxes1 (Tensor[N, 4])\n        boxes2 (Tensor[M, 4])\n\n    Returns:\n        iou (Tensor[N, M]): the NxM matrix containing the pairwise IoU values for every element in boxes1 and boxes2\n    \"\"\"\n    inter, union = _box_inter_union(boxes1, boxes2)\n    iou = inter / union\n    return iou\n\n\n# Implementation adapted from https://github.com/facebookresearch/detr/blob/master/util/box_ops.py\ndef generalized_box_iou(boxes1: Tensor, boxes2: Tensor) -> Tensor:\n    \"\"\"\n    Return generalized intersection-over-union (Jaccard index) of boxes.\n\n    Both sets of boxes are expected to be in (x1, y1, x2, y2) format.\n\n    Args:\n        boxes1 (Tensor[N, 4])\n        boxes2 (Tensor[M, 4])\n\n    Returns:\n        generalized_iou (Tensor[N, M]): the NxM matrix containing the pairwise generalized_IoU values\n        for every element in boxes1 and boxes2\n    \"\"\"\n\n    # degenerate boxes gives inf / nan results\n    # so do an early check\n    assert (boxes1[:, 2:] >= boxes1[:, :2]).all()\n    assert (boxes2[:, 2:] >= boxes2[:, :2]).all()\n\n    inter, union = _box_inter_union(boxes1, boxes2)\n    iou = inter / union\n\n    lti = torch.min(boxes1[:, None, :2], boxes2[:, :2])\n    rbi = torch.max(boxes1[:, None, 2:], boxes2[:, 2:])\n\n    whi = _upcast(rbi - lti).clamp(min=0)  # [N,M,2]\n    areai = whi[:, :, 0] * whi[:, :, 1]\n\n    return iou - (areai - union) / areai\n", "path": "torchvision/ops/boxes.py"}]}