We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Optimization Suggestion for 'torchaudio.functional.merge_tokens' Function
### 🚀 The feature

Optimization of the merge_tokens function in 'torchaudio.functional.merge_tokens' to leverage PyTorch's tensor operations for improved efficiency and reduced runtime.

Proposed Implementation:
```python
def merge_tokens_optimized(tokens: torch.Tensor, scores: torch.Tensor, blank: int = 0) -> List[TokenSpan]:
    """Removes repeated tokens and blank tokens from the given CTC token sequence

    Args:
        tokens (torch.Tensor): Alignment tokens (unbatched)
        scores (torch.Tensor): Alignment scores (unbatched)
        blank (int, optional): Blank token. Defaults to 0.

    Returns:
        List[TokenSpan]: list of TokenSpan
    """

    # Compute the difference between consecutive tokens. prepend and append a -1 to make sure the first and last
    # tokens are not removed
    diff = torch.diff(
        tokens, prepend=torch.tensor([-1], device=tokens.device), append=torch.tensor([-1], device=tokens.device)
    )
    # Compute the change points and mask out the points where the new value is blank
    changes_wo_blank = torch.nonzero((diff != 0)).squeeze().tolist()

    tokens = tokens.tolist()
    # Create a TokenSpan for each change point.
    spans = [
        TokenSpan(token=token, start=start, end=end, score=scores[start:end].mean().item())
        for start, end in zip(changes_wo_blank[:-1], changes_wo_blank[1:])
        if (token := tokens[start]) != blank
    ]
    return spans
```

### Motivation, pitch

While working on some CTC token sequence operations, I noticed that the current implementation of the merge_tokens function could benefit from tensor operations, potentially offering significant runtime improvements. Specifically, the current loop-based approach might be streamlined by taking advantage of PyTorch's capabilities. My tests indicate that my proposed optimization runs approximately 7 times faster over 1000 runs while producing equivalent outputs.

### Alternatives

Proposed tensor-based optimization: This approach leverages PyTorch's tensor operations, eliminating explicit loops and potentially considerably reducing execution time.

### Additional context

To validate the accuracy of the optimized function, a method was crafted to verify the outputs of both the original and proposed functions:

```python
import torchaudio.functional as F
from torchaudio.functional import TokenSpan

def token_span_outputs_equal(span_list_1: List[TokenSpan], span_list_2: List[TokenSpan]) -> bool:
    """Compares two lists of TokenSpan objects for equality."""
    
    if len(span_list_1) != len(span_list_2):
        return False

    for span_1, span_2 in zip(span_list_1, span_list_2):
        if (
            span_1.token != span_2.token or 
            span_1.start != span_2.start or 
            span_1.end != span_2.end or 
            abs(span_1.score - span_2.score) > 1e-6  # Allowing a small tolerance for floating-point comparisons
        ):
            return False

    return True
    
   
output_original = F.merge_tokens(aligned_tokens, alignment_scores)
output_optimized = merge_tokens_optimized(aligned_tokens, alignment_scores)

assert token_span_outputs_equal(output_original, output_optimized), "Outputs are not equivalent!"
```

Benchmark code:
```python
import timeit

pytorch_time = timeit.timeit(lambda: F.merge_tokens(aligned_tokens, alignment_scores), number=1000)
new_method_time = timeit.timeit(lambda: merge_tokens_optimized(aligned_tokens, alignment_scores), number=1000)

print(f"PyTorch method: {pytorch_time}")
print(f"New method: {new_method_time}")
```

Output:
```
Pytorch method: 7.622203521430492
New method: 1.0753349959850311
```

Integrating this optimization could bring about significant improvements in both performance and maintainability, benefiting a broad spectrum of users.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `torchaudio/functional/_alignment.py`
Content:
```
1 from dataclasses import dataclass
2 from typing import List, Optional, Tuple
3 
4 import torch
5 from torch import Tensor
6 from torchaudio._extension import fail_if_no_align
7 
8 __all__ = []
9 
10 
11 @fail_if_no_align
12 def forced_align(
13     log_probs: Tensor,
14     targets: Tensor,
15     input_lengths: Optional[Tensor] = None,
16     target_lengths: Optional[Tensor] = None,
17     blank: int = 0,
18 ) -> Tuple[Tensor, Tensor]:
19     r"""Align a CTC label sequence to an emission.
20 
21     .. devices:: CPU CUDA
22 
23     .. properties:: TorchScript
24 
25     Args:
26         log_probs (Tensor): log probability of CTC emission output.
27             Tensor of shape `(B, T, C)`. where `B` is the batch size, `T` is the input length,
28             `C` is the number of characters in alphabet including blank.
29         targets (Tensor): Target sequence. Tensor of shape `(B, L)`,
30             where `L` is the target length.
31         input_lengths (Tensor or None, optional):
32             Lengths of the inputs (max value must each be <= `T`). 1-D Tensor of shape `(B,)`.
33         target_lengths (Tensor or None, optional):
34             Lengths of the targets. 1-D Tensor of shape `(B,)`.
35         blank_id (int, optional): The index of blank symbol in CTC emission. (Default: 0)
36 
37     Returns:
38         Tuple(Tensor, Tensor):
39             Tensor: Label for each time step in the alignment path computed using forced alignment.
40 
41             Tensor: Log probability scores of the labels for each time step.
42 
43     Note:
44         The sequence length of `log_probs` must satisfy:
45 
46 
47         .. math::
48             L_{\text{log\_probs}} \ge L_{\text{label}} + N_{\text{repeat}}
49 
50         where :math:`N_{\text{repeat}}` is the number of consecutively repeated tokens.
51         For example, in str `"aabbc"`, the number of repeats are `2`.
52 
53     Note:
54         The current version only supports ``batch_size==1``.
55     """
56     if blank in targets:
57         raise ValueError(f"targets Tensor shouldn't contain blank index. Found {targets}.")
58     if torch.max(targets) >= log_probs.shape[-1]:
59         raise ValueError("targets values must be less than the CTC dimension")
60 
61     if input_lengths is None:
62         batch_size, length = log_probs.size(0), log_probs.size(1)
63         input_lengths = torch.full((batch_size,), length, dtype=torch.int64, device=log_probs.device)
64     if target_lengths is None:
65         batch_size, length = targets.size(0), targets.size(1)
66         target_lengths = torch.full((batch_size,), length, dtype=torch.int64, device=targets.device)
67 
68     # For TorchScript compatibility
69     assert input_lengths is not None
70     assert target_lengths is not None
71 
72     paths, scores = torch.ops.torchaudio.forced_align(log_probs, targets, input_lengths, target_lengths, blank)
73     return paths, scores
74 
75 
76 @dataclass
77 class TokenSpan:
78     """TokenSpan()
79     Token with time stamps and score. Returned by :py:func:`merge_tokens`.
80     """
81 
82     token: int
83     """The token"""
84     start: int
85     """The start time (inclusive) in emission time axis."""
86     end: int
87     """The end time (exclusive) in emission time axis."""
88     score: float
89     """The score of the this token."""
90 
91     def __len__(self) -> int:
92         """Returns the time span"""
93         return self.end - self.start
94 
95 
96 def merge_tokens(tokens: Tensor, scores: Tensor, blank: int = 0) -> List[TokenSpan]:
97     """Removes repeated tokens and blank tokens from the given CTC token sequence.
98 
99     Args:
100         tokens (Tensor): Alignment tokens (unbatched) returned from :py:func:`forced_align`.
101             Shape: `(time, )`.
102         scores (Tensor): Alignment scores (unbatched) returned from :py:func:`forced_align`.
103             Shape: `(time, )`. When computing the token-size score, the given score is averaged
104             across the corresponding time span.
105 
106     Returns:
107         list of TokenSpan
108 
109     Example:
110         >>> aligned_tokens, scores = forced_align(emission, targets, input_lengths, target_lengths)
111         >>> token_spans = merge_tokens(aligned_tokens[0], scores[0])
112     """
113     if tokens.ndim != 1 or scores.ndim != 1:
114         raise ValueError("`tokens` and `scores` must be 1D Tensor.")
115     if len(tokens) != len(scores):
116         raise ValueError("`tokens` and `scores` must be the same length.")
117 
118     t_prev = blank
119     i = start = -1
120     spans = []
121     for t, token in enumerate(tokens):
122         if token != t_prev:
123             if t_prev != blank:
124                 spans.append(TokenSpan(t_prev.item(), start, t, scores[start:t].mean().item()))
125             if token != blank:
126                 i += 1
127                 start = t
128             t_prev = token
129     if t_prev != blank:
130         spans.append(TokenSpan(t_prev.item(), start, len(tokens), scores[start:].mean().item()))
131     return spans
132
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/torchaudio/functional/_alignment.py b/torchaudio/functional/_alignment.py
--- a/torchaudio/functional/_alignment.py
+++ b/torchaudio/functional/_alignment.py
@@ -115,17 +115,14 @@
     if len(tokens) != len(scores):
         raise ValueError("`tokens` and `scores` must be the same length.")
 
-    t_prev = blank
-    i = start = -1
-    spans = []
-    for t, token in enumerate(tokens):
-        if token != t_prev:
-            if t_prev != blank:
-                spans.append(TokenSpan(t_prev.item(), start, t, scores[start:t].mean().item()))
-            if token != blank:
-                i += 1
-                start = t
-            t_prev = token
-    if t_prev != blank:
-        spans.append(TokenSpan(t_prev.item(), start, len(tokens), scores[start:].mean().item()))
+    diff = torch.diff(
+        tokens, prepend=torch.tensor([-1], device=tokens.device), append=torch.tensor([-1], device=tokens.device)
+    )
+    changes_wo_blank = torch.nonzero((diff != 0)).squeeze().tolist()
+    tokens = tokens.tolist()
+    spans = [
+        TokenSpan(token=token, start=start, end=end, score=scores[start:end].mean().item())
+        for start, end in zip(changes_wo_blank[:-1], changes_wo_blank[1:])
+        if (token := tokens[start]) != blank
+    ]
     return spans

{"golden_diff": "diff --git a/torchaudio/functional/_alignment.py b/torchaudio/functional/_alignment.py\n--- a/torchaudio/functional/_alignment.py\n+++ b/torchaudio/functional/_alignment.py\n@@ -115,17 +115,14 @@\n     if len(tokens) != len(scores):\n         raise ValueError(\"`tokens` and `scores` must be the same length.\")\n \n-    t_prev = blank\n-    i = start = -1\n-    spans = []\n-    for t, token in enumerate(tokens):\n-        if token != t_prev:\n-            if t_prev != blank:\n-                spans.append(TokenSpan(t_prev.item(), start, t, scores[start:t].mean().item()))\n-            if token != blank:\n-                i += 1\n-                start = t\n-            t_prev = token\n-    if t_prev != blank:\n-        spans.append(TokenSpan(t_prev.item(), start, len(tokens), scores[start:].mean().item()))\n+    diff = torch.diff(\n+        tokens, prepend=torch.tensor([-1], device=tokens.device), append=torch.tensor([-1], device=tokens.device)\n+    )\n+    changes_wo_blank = torch.nonzero((diff != 0)).squeeze().tolist()\n+    tokens = tokens.tolist()\n+    spans = [\n+        TokenSpan(token=token, start=start, end=end, score=scores[start:end].mean().item())\n+        for start, end in zip(changes_wo_blank[:-1], changes_wo_blank[1:])\n+        if (token := tokens[start]) != blank\n+    ]\n     return spans\n", "issue": "Optimization Suggestion for 'torchaudio.functional.merge_tokens' Function\n### \ud83d\ude80 The feature\r\n\r\nOptimization of the merge_tokens function in 'torchaudio.functional.merge_tokens' to leverage PyTorch's tensor operations for improved efficiency and reduced runtime.\r\n\r\nProposed Implementation:\r\n```python\r\ndef merge_tokens_optimized(tokens: torch.Tensor, scores: torch.Tensor, blank: int = 0) -> List[TokenSpan]:\r\n    \"\"\"Removes repeated tokens and blank tokens from the given CTC token sequence\r\n\r\n    Args:\r\n        tokens (torch.Tensor): Alignment tokens (unbatched)\r\n        scores (torch.Tensor): Alignment scores (unbatched)\r\n        blank (int, optional): Blank token. Defaults to 0.\r\n\r\n    Returns:\r\n        List[TokenSpan]: list of TokenSpan\r\n    \"\"\"\r\n\r\n    # Compute the difference between consecutive tokens. prepend and append a -1 to make sure the first and last\r\n    # tokens are not removed\r\n    diff = torch.diff(\r\n        tokens, prepend=torch.tensor([-1], device=tokens.device), append=torch.tensor([-1], device=tokens.device)\r\n    )\r\n    # Compute the change points and mask out the points where the new value is blank\r\n    changes_wo_blank = torch.nonzero((diff != 0)).squeeze().tolist()\r\n\r\n    tokens = tokens.tolist()\r\n    # Create a TokenSpan for each change point.\r\n    spans = [\r\n        TokenSpan(token=token, start=start, end=end, score=scores[start:end].mean().item())\r\n        for start, end in zip(changes_wo_blank[:-1], changes_wo_blank[1:])\r\n        if (token := tokens[start]) != blank\r\n    ]\r\n    return spans\r\n```\r\n\r\n### Motivation, pitch\r\n\r\nWhile working on some CTC token sequence operations, I noticed that the current implementation of the merge_tokens function could benefit from tensor operations, potentially offering significant runtime improvements. Specifically, the current loop-based approach might be streamlined by taking advantage of PyTorch's capabilities. My tests indicate that my proposed optimization runs approximately 7 times faster over 1000 runs while producing equivalent outputs.\r\n\r\n### Alternatives\r\n\r\nProposed tensor-based optimization: This approach leverages PyTorch's tensor operations, eliminating explicit loops and potentially considerably reducing execution time.\r\n\r\n### Additional context\r\n\r\nTo validate the accuracy of the optimized function, a method was crafted to verify the outputs of both the original and proposed functions:\r\n\r\n```python\r\nimport torchaudio.functional as F\r\nfrom torchaudio.functional import TokenSpan\r\n\r\ndef token_span_outputs_equal(span_list_1: List[TokenSpan], span_list_2: List[TokenSpan]) -> bool:\r\n    \"\"\"Compares two lists of TokenSpan objects for equality.\"\"\"\r\n    \r\n    if len(span_list_1) != len(span_list_2):\r\n        return False\r\n\r\n    for span_1, span_2 in zip(span_list_1, span_list_2):\r\n        if (\r\n            span_1.token != span_2.token or \r\n            span_1.start != span_2.start or \r\n            span_1.end != span_2.end or \r\n            abs(span_1.score - span_2.score) > 1e-6  # Allowing a small tolerance for floating-point comparisons\r\n        ):\r\n            return False\r\n\r\n    return True\r\n    \r\n   \r\noutput_original = F.merge_tokens(aligned_tokens, alignment_scores)\r\noutput_optimized = merge_tokens_optimized(aligned_tokens, alignment_scores)\r\n\r\nassert token_span_outputs_equal(output_original, output_optimized), \"Outputs are not equivalent!\"\r\n```\r\n\r\nBenchmark code:\r\n```python\r\nimport timeit\r\n\r\npytorch_time = timeit.timeit(lambda: F.merge_tokens(aligned_tokens, alignment_scores), number=1000)\r\nnew_method_time = timeit.timeit(lambda: merge_tokens_optimized(aligned_tokens, alignment_scores), number=1000)\r\n\r\nprint(f\"PyTorch method: {pytorch_time}\")\r\nprint(f\"New method: {new_method_time}\")\r\n```\r\n\r\nOutput:\r\n```\r\nPytorch method: 7.622203521430492\r\nNew method: 1.0753349959850311\r\n```\r\n\r\nIntegrating this optimization could bring about significant improvements in both performance and maintainability, benefiting a broad spectrum of users.\n", "before_files": [{"content": "from dataclasses import dataclass\nfrom typing import List, Optional, Tuple\n\nimport torch\nfrom torch import Tensor\nfrom torchaudio._extension import fail_if_no_align\n\n__all__ = []\n\n\n@fail_if_no_align\ndef forced_align(\n    log_probs: Tensor,\n    targets: Tensor,\n    input_lengths: Optional[Tensor] = None,\n    target_lengths: Optional[Tensor] = None,\n    blank: int = 0,\n) -> Tuple[Tensor, Tensor]:\n    r\"\"\"Align a CTC label sequence to an emission.\n\n    .. devices:: CPU CUDA\n\n    .. properties:: TorchScript\n\n    Args:\n        log_probs (Tensor): log probability of CTC emission output.\n            Tensor of shape `(B, T, C)`. where `B` is the batch size, `T` is the input length,\n            `C` is the number of characters in alphabet including blank.\n        targets (Tensor): Target sequence. Tensor of shape `(B, L)`,\n            where `L` is the target length.\n        input_lengths (Tensor or None, optional):\n            Lengths of the inputs (max value must each be <= `T`). 1-D Tensor of shape `(B,)`.\n        target_lengths (Tensor or None, optional):\n            Lengths of the targets. 1-D Tensor of shape `(B,)`.\n        blank_id (int, optional): The index of blank symbol in CTC emission. (Default: 0)\n\n    Returns:\n        Tuple(Tensor, Tensor):\n            Tensor: Label for each time step in the alignment path computed using forced alignment.\n\n            Tensor: Log probability scores of the labels for each time step.\n\n    Note:\n        The sequence length of `log_probs` must satisfy:\n\n\n        .. math::\n            L_{\\text{log\\_probs}} \\ge L_{\\text{label}} + N_{\\text{repeat}}\n\n        where :math:`N_{\\text{repeat}}` is the number of consecutively repeated tokens.\n        For example, in str `\"aabbc\"`, the number of repeats are `2`.\n\n    Note:\n        The current version only supports ``batch_size==1``.\n    \"\"\"\n    if blank in targets:\n        raise ValueError(f\"targets Tensor shouldn't contain blank index. Found {targets}.\")\n    if torch.max(targets) >= log_probs.shape[-1]:\n        raise ValueError(\"targets values must be less than the CTC dimension\")\n\n    if input_lengths is None:\n        batch_size, length = log_probs.size(0), log_probs.size(1)\n        input_lengths = torch.full((batch_size,), length, dtype=torch.int64, device=log_probs.device)\n    if target_lengths is None:\n        batch_size, length = targets.size(0), targets.size(1)\n        target_lengths = torch.full((batch_size,), length, dtype=torch.int64, device=targets.device)\n\n    # For TorchScript compatibility\n    assert input_lengths is not None\n    assert target_lengths is not None\n\n    paths, scores = torch.ops.torchaudio.forced_align(log_probs, targets, input_lengths, target_lengths, blank)\n    return paths, scores\n\n\n@dataclass\nclass TokenSpan:\n    \"\"\"TokenSpan()\n    Token with time stamps and score. Returned by :py:func:`merge_tokens`.\n    \"\"\"\n\n    token: int\n    \"\"\"The token\"\"\"\n    start: int\n    \"\"\"The start time (inclusive) in emission time axis.\"\"\"\n    end: int\n    \"\"\"The end time (exclusive) in emission time axis.\"\"\"\n    score: float\n    \"\"\"The score of the this token.\"\"\"\n\n    def __len__(self) -> int:\n        \"\"\"Returns the time span\"\"\"\n        return self.end - self.start\n\n\ndef merge_tokens(tokens: Tensor, scores: Tensor, blank: int = 0) -> List[TokenSpan]:\n    \"\"\"Removes repeated tokens and blank tokens from the given CTC token sequence.\n\n    Args:\n        tokens (Tensor): Alignment tokens (unbatched) returned from :py:func:`forced_align`.\n            Shape: `(time, )`.\n        scores (Tensor): Alignment scores (unbatched) returned from :py:func:`forced_align`.\n            Shape: `(time, )`. When computing the token-size score, the given score is averaged\n            across the corresponding time span.\n\n    Returns:\n        list of TokenSpan\n\n    Example:\n        >>> aligned_tokens, scores = forced_align(emission, targets, input_lengths, target_lengths)\n        >>> token_spans = merge_tokens(aligned_tokens[0], scores[0])\n    \"\"\"\n    if tokens.ndim != 1 or scores.ndim != 1:\n        raise ValueError(\"`tokens` and `scores` must be 1D Tensor.\")\n    if len(tokens) != len(scores):\n        raise ValueError(\"`tokens` and `scores` must be the same length.\")\n\n    t_prev = blank\n    i = start = -1\n    spans = []\n    for t, token in enumerate(tokens):\n        if token != t_prev:\n            if t_prev != blank:\n                spans.append(TokenSpan(t_prev.item(), start, t, scores[start:t].mean().item()))\n            if token != blank:\n                i += 1\n                start = t\n            t_prev = token\n    if t_prev != blank:\n        spans.append(TokenSpan(t_prev.item(), start, len(tokens), scores[start:].mean().item()))\n    return spans\n", "path": "torchaudio/functional/_alignment.py"}], "after_files": [{"content": "from dataclasses import dataclass\nfrom typing import List, Optional, Tuple\n\nimport torch\nfrom torch import Tensor\nfrom torchaudio._extension import fail_if_no_align\n\n__all__ = []\n\n\n@fail_if_no_align\ndef forced_align(\n    log_probs: Tensor,\n    targets: Tensor,\n    input_lengths: Optional[Tensor] = None,\n    target_lengths: Optional[Tensor] = None,\n    blank: int = 0,\n) -> Tuple[Tensor, Tensor]:\n    r\"\"\"Align a CTC label sequence to an emission.\n\n    .. devices:: CPU CUDA\n\n    .. properties:: TorchScript\n\n    Args:\n        log_probs (Tensor): log probability of CTC emission output.\n            Tensor of shape `(B, T, C)`. where `B` is the batch size, `T` is the input length,\n            `C` is the number of characters in alphabet including blank.\n        targets (Tensor): Target sequence. Tensor of shape `(B, L)`,\n            where `L` is the target length.\n        input_lengths (Tensor or None, optional):\n            Lengths of the inputs (max value must each be <= `T`). 1-D Tensor of shape `(B,)`.\n        target_lengths (Tensor or None, optional):\n            Lengths of the targets. 1-D Tensor of shape `(B,)`.\n        blank_id (int, optional): The index of blank symbol in CTC emission. (Default: 0)\n\n    Returns:\n        Tuple(Tensor, Tensor):\n            Tensor: Label for each time step in the alignment path computed using forced alignment.\n\n            Tensor: Log probability scores of the labels for each time step.\n\n    Note:\n        The sequence length of `log_probs` must satisfy:\n\n\n        .. math::\n            L_{\\text{log\\_probs}} \\ge L_{\\text{label}} + N_{\\text{repeat}}\n\n        where :math:`N_{\\text{repeat}}` is the number of consecutively repeated tokens.\n        For example, in str `\"aabbc\"`, the number of repeats are `2`.\n\n    Note:\n        The current version only supports ``batch_size==1``.\n    \"\"\"\n    if blank in targets:\n        raise ValueError(f\"targets Tensor shouldn't contain blank index. Found {targets}.\")\n    if torch.max(targets) >= log_probs.shape[-1]:\n        raise ValueError(\"targets values must be less than the CTC dimension\")\n\n    if input_lengths is None:\n        batch_size, length = log_probs.size(0), log_probs.size(1)\n        input_lengths = torch.full((batch_size,), length, dtype=torch.int64, device=log_probs.device)\n    if target_lengths is None:\n        batch_size, length = targets.size(0), targets.size(1)\n        target_lengths = torch.full((batch_size,), length, dtype=torch.int64, device=targets.device)\n\n    # For TorchScript compatibility\n    assert input_lengths is not None\n    assert target_lengths is not None\n\n    paths, scores = torch.ops.torchaudio.forced_align(log_probs, targets, input_lengths, target_lengths, blank)\n    return paths, scores\n\n\n@dataclass\nclass TokenSpan:\n    \"\"\"TokenSpan()\n    Token with time stamps and score. Returned by :py:func:`merge_tokens`.\n    \"\"\"\n\n    token: int\n    \"\"\"The token\"\"\"\n    start: int\n    \"\"\"The start time (inclusive) in emission time axis.\"\"\"\n    end: int\n    \"\"\"The end time (exclusive) in emission time axis.\"\"\"\n    score: float\n    \"\"\"The score of the this token.\"\"\"\n\n    def __len__(self) -> int:\n        \"\"\"Returns the time span\"\"\"\n        return self.end - self.start\n\n\ndef merge_tokens(tokens: Tensor, scores: Tensor, blank: int = 0) -> List[TokenSpan]:\n    \"\"\"Removes repeated tokens and blank tokens from the given CTC token sequence.\n\n    Args:\n        tokens (Tensor): Alignment tokens (unbatched) returned from :py:func:`forced_align`.\n            Shape: `(time, )`.\n        scores (Tensor): Alignment scores (unbatched) returned from :py:func:`forced_align`.\n            Shape: `(time, )`. When computing the token-size score, the given score is averaged\n            across the corresponding time span.\n\n    Returns:\n        list of TokenSpan\n\n    Example:\n        >>> aligned_tokens, scores = forced_align(emission, targets, input_lengths, target_lengths)\n        >>> token_spans = merge_tokens(aligned_tokens[0], scores[0])\n    \"\"\"\n    if tokens.ndim != 1 or scores.ndim != 1:\n        raise ValueError(\"`tokens` and `scores` must be 1D Tensor.\")\n    if len(tokens) != len(scores):\n        raise ValueError(\"`tokens` and `scores` must be the same length.\")\n\n    diff = torch.diff(\n        tokens, prepend=torch.tensor([-1], device=tokens.device), append=torch.tensor([-1], device=tokens.device)\n    )\n    changes_wo_blank = torch.nonzero((diff != 0)).squeeze().tolist()\n    tokens = tokens.tolist()\n    spans = [\n        TokenSpan(token=token, start=start, end=end, score=scores[start:end].mean().item())\n        for start, end in zip(changes_wo_blank[:-1], changes_wo_blank[1:])\n        if (token := tokens[start]) != blank\n    ]\n    return spans\n", "path": "torchaudio/functional/_alignment.py"}]}