Upload materials part 1

LICENSE

@@ -0,0 +1,43 @@
+Open Source License
+
+The ten-vad is licensed pursuant to the Apache License v2.0, with the
+following additional conditions. You may reproduce, prepare Derivative Works
+of, publicly display, publicly perform, sublicense, distribute, or otherwise
+make available (together, "Deploy") the ten-vad, for commercial or
+non-commercial purposes, provided that you agree to abide by the terms below:
+
+    1. You may not (i) host the ten-vad or the Derivative Works on any End
+       User devices, including but not limited to any mobile terminal devices
+       or (ii) Deploy the ten-vad in a way that competes with Agora's
+       offerings and/or that allows others to compete with Agora's offerings,
+       including without limitation enabling any third party to develop or
+       deploy Applications.
+
+    2. You may Deploy the ten-vad solely to create and enable deployment
+       of your Application(s) solely for your benefit and the benefit of your
+       direct End Users. If you prefer, you may include the following notice in
+       the documentation of your Application(s): "Powered by ten-vad".
+
+    3. Derivative Works of the ten-vad remain subject to this Open Source
+       License.
+
+    4. "End Users" shall mean the end-users of your Application(s) who access
+       the ten-vad solely to the extent necessary to access and use the
+       Application(s) you create or deploy using ten-vad.
+
+    5. "Application(s)" shall mean your software programs designed or developed
+       by using the ten-vad or where deployment is enabled by the ten-vad.
+
+  Copyright © 2025 Agora
+
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.

README.md

@@ -1,3 +1,399 @@
----
-license: apache-2.0
----
+# **TEN VAD**
+
+***A Low-Latency, Lightweight and High-Performance Streaming VAD***
+
+
+
+## **Introduction**
+**TEN VAD** is a real-time voice activity detection system designed for enterprise use,  providing accurate frame-level speech activity detection. It shows superior precision compared to both WebRTC VAD and Silero VAD, which are commonly used in the industry. Additionally, TEN VAD offers lower computational complexity and reduced memory usage compared to Silero VAD. Meanwhile, the architecture's temporal efficiency enables rapid voice activity detection, significantly reducing end-to-end response and turn detection latency in conversational AI systems.
+
+
+
+## **Key Features**
+
+### **1. High-Performance:** 
+
+The precision-recall curves comparing the performance of WebRTC VAD (pitch-based), Silero VAD, and TEN VAD are shown below. The evaluation is conducted on the precisely manually annotated TEN-VAD-TestSet. The audio files are from librispeech, gigaspeech, DNS Challenge etc. As demonstrated, TEN VAD achieves the best performance. Additionally, cross-validation experiments conducted on large internal real-world datasets demonstrate the reproducibility of these findings. The **TEN-VAD-TestSet with annotated labels** is released in directory "TEN-VAD-TestSet" of this repository.
+
+ <br>
+
+<div style="text-align:">
+  <img src="./images/PR_Curves_TEN-VAD-TestSet.png" width="800">
+</div>
+
+Note that the default threshold of 0.5 is used to generate binary speech indicators (0 for non-speech signal, 1 for speech signal). This threshold needs to be tuned according to your domain-specific task. The precision-recall curve can be obtained by executing the following script on Linux x64. The output figure will be saved in the same directory as the script. Note that only PR curves of Silero VAD and TEN VAD are plotted, we did not plot the one of WebRTC VAD, which is used in the latese version of WebRTC.
+
+```
+cd ./examples
+python plot_pr_curves.py
+```
+<br>
+
+### **2. Agent-Friendly:** 
+As illustrated in the figure below, TEN VAD rapidly detects speech-to-non-speech transitions, whereas Silero VAD suffers from a delay of several hundred milliseconds, resulting in increased end-to-end latency in human-agent interaction systems. In addition, as demonstrated in the 6.5s-7.0s audio segment, Silero VAD fails to identify short silent durations between adjacent speech segments.
+<div style="text-align:">
+  <img src="./images/Agent-Friendly-image.png" width="800">
+</div>
+<br>
+
+### **3. Lightweight:**
+We evaluated the RTF (Real-Time Factor) across five distinct platforms, each equipped with varying CPUs. TEN VAD demonstrates much lower computational complexity and smaller library size than Silero VAD.
+
+<table>
+  <tr>
+    <th align="center" rowspan="2" valign="middle"> Platform </th>
+    <th align="center" rowspan="2" valign="middle"> CPU </th>
+    <th align="center" colspan="2"> RTF </th>
+    <th align="center" colspan="2"> Lib Size </th>
+  </tr>
+  <tr>
+    <th align="center" style="white-space: nowrap;"> TEN VAD </th>
+    <th align="center" style="white-space: nowrap;"> Silero VAD </th>
+    <th align="center"> TEN VAD </th>
+    <th align="center"> Silero VAD </th>
+  </tr>
+  <tr>
+    <th align="center" rowspan="3"> Linux </th>
+    <td style="white-space: nowrap;"> AMD Ryzen 9 5900X 12-Core </td>
+    <td align="center"> 0.0150 </td>
+    <td rowspan="2" style="text-align: center; vertical-align: middle;"> / </td>
+    <td rowspan="3" style="text-align: center; vertical-align: middle;"> 306KB </td>
+    <td rowspan="9" style="text-align: center; vertical-align: middle;"> 2.16MB(JIT) / 2.22MB(ONNX) </td>
+  </tr>
+  <tr>
+    <td style="white-space: nowrap;"> Intel(R) Xeon(R) Platinum 8253 </td>
+    <td align="center"> 0.0136 </td>
+  </tr>
+  <tr>
+    <td style="white-space: nowrap;"> Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz </td>
+    <td align="center"> 0.0086 </td>
+    <td align="center"> 0.0127 </td>
+  </tr>
+  <tr>
+    <th align="center"> Windows </th>
+    <td> Intel i7-10710U </td>
+    <td align="center"> 0.0150 </td>
+    <td rowspan="6" style="text-align: center; vertical-align: middle;"> / </td>
+    <td align="center" style="white-space: nowrap;"> 464KB(x86) / 508KB(x64) </td>
+  </tr>
+  <tr>
+    <th align="center"> macOS </th>
+    <td> M1 </td>
+    <td align="center"> 0.0160 </td>
+    <td align="center"> 731KB </td>
+  </tr>
+  <tr>
+    <th align="center" rowspan="2"> Android </th>
+    <td> Galaxy J6+ (32bit, 425) </td>
+    <td align="center"> 0.0570 </td>
+    <td rowspan="2" style="white-space: nowrap;" style="text-align: center; vertical-align: middle;"> 373KB(v7a) / 532KB(v8a)</td>
+  </tr>
+  <tr>
+    <td> Oppo A3s (450) </td>
+    <td align="center"> 0.0490 </td>
+  </tr>
+  <tr>
+    <th align="center" rowspan="2"> iOS </th>
+    <td> iPhone6 (A8) </td>
+    <td align="center"> 0.0210 </td>
+    <td rowspan="2" style="text-align: center; vertical-align: middle;"> 320KB</td>
+  </tr>
+  <tr>
+    <td> iPhone8 (A11) </td>
+    <td align="center"> 0.0050 </td>
+  </tr> 
+</table>
+
+<style>
+  th, td {
+    border: 1px solid #ddd;
+    padding: 8px;
+  }
+</style>
+<br>
+
+### **4. Multiple programming languages and platforms:**
+TEN VAD provides cross-platform C compatibility across five operating systems (Linux x64, Windows, macOS, Android, iOS), with Python bindings optimized for Linux x64.
+<br>
+<br>
+
+
+### **5. Supproted sampling rate and hop size:**
+TEN VAD operates on 16kHz audio input with configurable hop sizes (optimized frame configurations: 160/256 samples=10/16ms). Other sampling rates must be resampled to 16kHz.
+<br>
+<br>
+
+## **Installation**
+```
+git clone https://huggingface.co/TEN-framework/ten-vad
+```
+<br>
+
+## **Quick Start**
+The project supports five major platforms with dynamic library linking.
+<table>
+  <tr>
+    <th align="center"> Platform </th>
+    <th align="center"> Dynamic Lib </th>
+    <th align="center"> Supported Arch </th>
+    <th align="center"> Interface Language </th>
+    <th align="center"> Header </th>
+    <th align="center"> Comment </v>
+  </tr>
+  <tr>
+    <th align="center"> Linux </th>
+    <td align="center"> libten_vad.so </td>
+    <td align="center"> x64 </td>
+    <td align="center"> Python, C </td>
+    <td rowspan="5" style="text-align: center; vertical-align: middle;">ten_vad.h <br> ten_vad.py</td>
+    <td>  </td>
+  </tr>
+  <tr>
+    <th align="center"> Windows </th>
+    <td align="center"> ten_vad.dll </td>
+    <td align="center"> x64, x86 </td>
+    <td align="center"> C </td>
+    <td>  </td>
+  </tr>
+  <tr>
+    <th align="center"> macOS </th>
+    <td align="center"> ten_vad.framework </td>
+    <td align="center"> arm64, x86_64 </td>
+    <td align="center"> C </td>
+    <td>  </td>
+  </tr>
+  <tr>
+    <th align="center"> Android </th>
+    <td align="center"> libten_vad.so </td>
+    <td align="center"> arm64-v8a, armeabi-v7a </td>
+    <td align="center"> C </td>
+    <td>  </td>
+  </tr>
+  <tr>
+    <th align="center"> iOS </th>
+    <td align="center" style="text-align: center; vertical-align: middle;"> ten_vad.framework </td>
+    <td align="center" style="text-align: center; vertical-align: middle;"> arm64 </td>
+    <td align="center"> C </td>
+    <td> 1. not simulator <br> 2. not iPad </td>
+  </tr>
+</table>
+<br>
+
+
+### **Python Usage**
+#### **1. Linux**
+#### **Requirements**
+- numpy (Version 1.17.4/1.26.4 verified)
+- scipy (Version 1.4.1/1.13.1 verified)
+- scikit-learn (Version 1.2.2/1.5.0 verified, for plotting PR curves)
+- matplotlib (Version 3.1.3/3.10.0 verified, for plotting PR curves)
+- torchaudio (Version 2.2.2 verified, for plotting PR curves)
+
+- Python version 3.8.19/3.10.14 verified
+
+Note: You could use other versions of above packages, but we didn't test other versions. 
+
+You can install the above mentioned dependencies via requirements.txt:
+
+```
+pip install -r requirements.txt
+```
+<br>
+
+#### **Usage**
+Note: For usage in python, you can either use it by **git clone** or **pip**.
+
+##### **By using git clone:**
+
+1. Clone the repository
+```
+git clone https://huggingface.co/TEN-framework/ten-vad
+```
+
+2. Enter examples directory
+```
+cd ./examples
+```
+
+3. Test
+```
+python test.py s0724-s0730.wav out.txt
+```
+<br>
+
+##### **By using pip:**
+
+1. Install via pip 
+
+```
+pip install -U --force-reinstall -v git+https://huggingface.co/TEN-framework/ten-vad
+```
+
+2. Write your own use cases and import the class, the attributes of class TenVAD you can refer to ten_vad.py
+
+```
+from ten_vad import TenVad
+```
+<br>
+
+### **C Usage**
+#### **Build Scripts**
+Located in examples/ directory:
+
+- Linux: build-and-deploy-linux.sh
+- Windows: build-and-deploy-windows.bat
+- macOS: build-and-deploy-mac.sh
+- Android: build-and-deploy-android.sh
+- iOS: build-and-deploy-ios.sh
+
+#### **Dynamic Library Configuration**
+Runtime library path configuration:
+- Linux/Android: LD_LIBRARY_PATH
+- macOS: DYLD_FRAMEWORK_PATH
+- Windows: DLL in executable directory or system PATH
+
+#### **Customization**
+- Modify platform-specific build scripts
+- Adjust CMakeLists.txt
+- Configure toolchain and architecture settings
+
+#### **Overview of Usage**
+- Navigate to examples/
+- Execute platform-specific build script
+- Configure dynamic library path
+- Run demo with sample audio s0724-s0730.wav
+- Processed results saved to out.txt
+
+<br>
+
+The detailed usage methods of each platform are as follows <br> 
+
+####  **1. Linux**
+##### **Requirements**
+- Clang (e.g. 6.0.0-1ubuntu2 verified)
+- CMake
+- Terminal
+
+##### **Usage**
+```
+1) cd ./examples
+2) ./build-and-deploy-linux.sh
+```
+<br>
+
+####  **2. Windows**
+##### **Requirements**
+- Visual Studio (2017, 2019, 2022 verified)
+- CMake (3.26.0-rc6 verified)
+- Terminal (MINGW64 or powershell)
+
+##### **Usage**
+```
+1) cd ./examples
+2) Configure "build-and-deploy-windows.bat" with your preferred:
+    - Architecture (default: x64)
+    - Visual Studio version (default: 2019)
+3) ./build-and-deploy-windows.bat
+```
+<br>
+
+####  **3. macOS**
+##### **Requirements**
+- Xcode (15.2 verified)
+- CMake (3.19.2 verified)
+
+##### **Usage**
+```
+1) cd ./examples
+2) Configure "build-and-deploy-mac.sh" with your target architecture:
+  - Default: arm64 (Apple Silicon)
+  - Alternative: x86_64 (Intel)
+3) ./build-and-deploy-mac.sh
+```
+<br>
+
+####  **4. Android**
+##### **Requirements**
+- NDK (r25b, macOS verified)
+- CMake (3.19.2, macOS verified)
+- adb (1.0.41, macOS verified)
+
+##### **Usage**
+```
+1) cd ./examples
+2) export ANDROID_NDK=/path/to/android-ndk  # Replace it with your NDK installation path
+3) Configure "build-and-deploy-android.sh" with your build settings:
+  - Architecture: arm64-v8a (default) or armeabi-v7a
+  - Toolchain: aarch64-linux-android-clang (default) or custom NDK toolchain
+4) ./build-and-deploy-android.sh
+```
+<br>
+
+####  **5. iOS**
+##### **Requirements**
+Xcode (15.2, macOS verified)
+CMake (3.19.2, macOS verified)
+##### **Usage**
+1. Enter examples directory
+```
+cd ./examples
+```
+
+2. Creates Xcode project files for iOS build
+```
+./build-and-deploy-ios.sh
+```
+
+3. Follow the steps below to build and test on iOS device:
+
+    3.1. Use Xcode to open .xcodeproj files: a) cd ./build-ios, b) open ./ten_vad_demo.xcodeproj
+
+    3.2. In Xcode IDE, select ten_vad_demo target (should check: Edit Scheme → Run → Release), then select your iOS Device (not simulator).
+
+    <div style="text-align:">
+      <img src="./images/ios_image_1.jpg" width="800">
+    </div>
+
+    3.3. Drag ten_vad/lib/iOS/ten_vad.framework  to "Frameworks, Libraries, and Embedded Content"
+
+    - (in TARGETS → ten_vad_demo → ten_vad_demo → General, should set Embed to "Embed & Sign").
+
+    -   or add it directly in this way: "Frameworks, Libraries, and Embedded Content" → "+" → Add Other... → Add Files →...  
+
+    - Note: If this step is not completed, you may encounter the following runtime error: "dyld: Library not loaded: @rpath/ten_vad.framework/ten_vad".
+
+      <div style="text-align:">
+        <img src="./images/ios_image_2.png" width="800">
+      </div>
+
+    3.4. Configure iOS device Signature
+
+    - in TARGETS → ten_vad_demo → Signing & Capabilities → Signing
+
+      - Modify Bundle Identifier: modify "com.yourcompany" to yours;
+
+      - Specify Provisioning Profile
+
+    - In TARGETS → ten_vad_demo → Build Settings → Signing → Code Signing Identity:
+      - Specify your Certification
+
+    3.5. Build in Xcode and run demo on your device.
+<br>
+
+## **Citations**
+```
+@misc{TEN VAD,
+  author = {TEN Team},
+  title = {TEN VAD: A Low-Latency, Lightweight and High-Performance Streaming Voice Activity Detector (VAD)},
+  year = {2025},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {https://github.com/TEN-framework/ten-vad.git},
+  commit = {insert_some_commit_here},
+  email = {TODO}
+}
+```
+<br>
+
+## **License**
+This project is Apache 2.0 licensed.

include/ten_vad.h

@@ -0,0 +1,84 @@
+//
+// This file is part of TEN Framework, an open source project.
+// Licensed under the Apache License, Version 2.0.
+// See the LICENSE file for more information.
+//
+#ifndef TEN_VAD_H
+#define TEN_VAD_H
+
+#if defined(__APPLE__) || defined(__ANDROID__) || defined(__linux__)
+#define TENVAD_API __attribute__((visibility("default")))
+#elif defined(_WIN32) || defined(__CYGWIN__)
+#ifdef TENVAD_EXPORTS
+#define TENVAD_API __declspec(dllexport)
+#else
+#define TENVAD_API __declspec(dllimport)
+#endif
+#else
+#define TENVAD_API
+#endif
+
+#include <stddef.h> /* size_t */
+#include <stdint.h> /* int16_t */
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+  /**
+   * @typedef ten_vad_handle
+   * @brief Opaque handle for ten_vad instance.
+   */
+  typedef void *ten_vad_handle_t;
+
+  /**
+   * @brief Create and initialize a ten_vad instance.
+   *
+   * @param[out] handle       Pointer to receive the vad handle.
+   * @param[in]  hop_size     The number of samples between the start points of
+   * two consecutive analysis frames. (e.g., 256).
+   * @param[in]  threshold    VAD detection threshold ranging from [0.0, 1.0]
+   * (default: 0.5).
+   * @return 0 on success, or -1 error occurs.
+   */
+  TENVAD_API int ten_vad_create(ten_vad_handle_t *handle, size_t hop_size,
+                                float threshold);
+
+  /**
+   * @brief Process one audio frame for voice activity detection.
+   * Must call ten_vad_init() before calling this, and ten_vad_destroy() when done.
+   *
+   * @param[in]  handle           Valid VAD handle returned by ten_vad_create().
+   * @param[in]  audio_data       Pointer to an array of int16_t samples,
+   * buffer length must equal the hop size specified at ten_vad_create.
+   * @param[in]  audio_data_length  size of audio_data buffer, here should be equal to hop_size.
+   * @param[out] out_probability  Pointer to a float (size 1) that receives the
+   * voice activity probability in the range [0.0, 1.0].
+   * @param[out] out_flag         Pointer to an int (size 1) that receives the
+   * detection result: 0 = no voice, 1 = voice detected.
+   * @return 0 on success, or -1 error occurs.
+   */
+  TENVAD_API int ten_vad_process(ten_vad_handle_t handle, const int16_t *audio_data, size_t audio_data_length,
+                                 float *out_probability, int *out_flag);
+
+  /**
+   * @brief Destroy a ten_vad instance and release its resources.
+   *
+   * @param[in,out] handle Pointer to the ten_vad handle; set to NULL on return.
+   * @return 0 on success, or -1 error occurs.
+   */
+  TENVAD_API int ten_vad_destroy(ten_vad_handle_t *handle);
+
+  /**
+   * @brief Get the ten_vad library version string.
+   *
+   * @return The version string (e.g., "1.0.0").
+   */
+  TENVAD_API const char *ten_vad_get_version(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* TEN_VAD_H */

include/ten_vad.py

@@ -0,0 +1,101 @@
+#
+# This file is part of TEN Framework, an open source project.
+# Licensed under the Apache License, Version 2.0.
+# See the LICENSE file for more information.
+#
+from ctypes import c_int, c_int32, c_float, c_size_t, CDLL, c_void_p, POINTER
+import numpy as np
+import os
+
+class TenVad:
+    def __init__(self, hop_size: int = 256, threshold: float = 0.5):
+        self.hop_size = hop_size
+        self.threshold = threshold
+        if os.path.exists(
+            os.path.join(
+                os.path.dirname(os.path.relpath(__file__)),
+                "../lib/Linux/x64/libten_vad.so",
+            )
+        ):
+            self.vad_library = CDLL(
+                os.path.join(
+                    os.path.dirname(os.path.relpath(__file__)), 
+                    "../lib/Linux/x64/libten_vad.so",
+                )
+            )
+        else:
+            self.vad_library = CDLL(
+                os.path.join(
+                    os.path.dirname(
+                        os.path.relpath(__file__)),
+                        "./ten_vad_library/libten_vad.so",
+                    )
+                )
+        self.vad_handler = c_void_p(0)
+        self.out_probability = c_float()
+        self.out_flags = c_int32()
+
+        self.vad_library.ten_vad_create.argtypes = [
+            POINTER(c_void_p),
+            c_size_t,
+            c_float,
+        ]
+        self.vad_library.ten_vad_create.restype = c_int
+
+        self.vad_library.ten_vad_destroy.argtypes = [POINTER(c_void_p)]
+        self.vad_library.ten_vad_destroy.restype = c_int
+
+        self.vad_library.ten_vad_process.argtypes = [
+            c_void_p,
+            c_void_p,
+            c_size_t,
+            POINTER(c_float),
+            POINTER(c_int32),
+        ]
+        self.vad_library.ten_vad_process.restype = c_int
+        self.create_and_init_handler()
+    
+    def create_and_init_handler(self):
+        assert (
+            self.vad_library.ten_vad_create(
+                POINTER(c_void_p)(self.vad_handler),
+                c_size_t(self.hop_size),
+                c_float(self.threshold),
+            ) 
+            == 0
+        ), "[TEN VAD]: create handler failure!"
+
+    def __del__(self):
+        assert (
+            self.vad_library.ten_vad_destroy(
+                POINTER(c_void_p)(self.vad_handler)
+            )
+            == 0
+        ), "[TEN VAD]: destroy handler failure!"
+    
+    def get_input_data(self, audio_data: np.ndarray):
+        audio_data = np.squeeze(audio_data)
+        assert (
+            len(audio_data.shape) == 1 
+            and audio_data.shape[0] == self.hop_size
+        ), "[TEN VAD]: audio data shape should be [%d]" % (
+            self.hop_size
+        )
+        assert (
+            type(audio_data[0]) == np.int16
+        ), "[TEN VAD]: audio data type error, must be int16"
+        data_pointer = audio_data.__array_interface__["data"][0]
+        return c_void_p(data_pointer)
+    
+    def process(self, audio_data: np.ndarray):
+        input_pointer = self.get_input_data(audio_data)
+        self.vad_library.ten_vad_process(
+            self.vad_handler,
+            input_pointer,
+            c_size_t(self.hop_size),
+            POINTER(c_float)(self.out_probability),
+            POINTER(c_int32)(self.out_flags),
+        )
+        return self.out_probability.value, self.out_flags.value
+
+

requirements.txt

@@ -0,0 +1,5 @@
+numpy
+scipy
+scikit-learn
+matplotlib
+torchaudio

setup.py

@@ -0,0 +1,23 @@
+from setuptools import setup
+import os, shutil
+from setuptools.command.install import install
+
+class custom_install_command(install):
+    def run(self):
+        install.run(self)
+        target_dir = os.path.join(self.install_lib, "ten_vad_library")
+        os.makedirs(target_dir, exist_ok=True)
+        shutil.copy("lib/Linux/x64/libten_vad.so", target_dir)
+        print(f"Files installed to: {target_dir}")
+
+root_dir = os.path.dirname(os.path.abspath(__file__))
+shutil.copy(f"{root_dir}/include/ten_vad.py", f"{root_dir}/ten_vad.py")
+setup(
+    name="ten_vad",
+    version="1.0",
+    py_modules=["ten_vad"],
+    cmdclass={
+        "install": custom_install_command,
+    },
+)
+os.remove(f"{root_dir}/ten_vad.py")