Improve model card: Update pipeline tag, add library name, and usage example
Browse filesThis PR enhances the model card for the SeC model by:
- Updating the `pipeline_tag` from `mask-generation` to `image-segmentation` for more precise categorization of the Video Object Segmentation task. This will improve discoverability on the Hugging Face Hub.
- Adding `library_name: transformers` to correctly reflect the model's compatibility and usage with the Hugging Face Transformers library, enabling the "Use in Transformers" widget.
- Including a basic Python usage example to demonstrate how to load and interact with the model, making it easier for users to get started.
These changes will help users better understand the model's capabilities and how to use it within the Hugging Face ecosystem.
README.md
CHANGED
@@ -1,11 +1,12 @@
|
|
1 |
---
|
2 |
-
license: apache-2.0
|
3 |
-
pipeline_tag: mask-generation
|
4 |
base_model:
|
5 |
-
|
6 |
-
|
|
|
|
|
7 |
tags:
|
8 |
-
|
|
|
9 |
---
|
10 |
|
11 |
# SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction
|
@@ -31,6 +32,41 @@ tags:
|
|
31 |
| **SeC (Ours)** | **82.7** | **81.7** | **86.5** | **75.3** | **91.3** | **88.6** | **70.0** |
|
32 |
|
33 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
## Citation
|
35 |
|
36 |
If you find this project useful in your research, please consider citing:
|
|
|
1 |
---
|
|
|
|
|
2 |
base_model:
|
3 |
+
- OpenGVLab/InternVL2.5-4B
|
4 |
+
- facebook/sam2.1-hiera-large
|
5 |
+
license: apache-2.0
|
6 |
+
pipeline_tag: image-segmentation
|
7 |
tags:
|
8 |
+
- SeC
|
9 |
+
library_name: transformers
|
10 |
---
|
11 |
|
12 |
# SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction
|
|
|
32 |
| **SeC (Ours)** | **82.7** | **81.7** | **86.5** | **75.3** | **91.3** | **88.6** | **70.0** |
|
33 |
|
34 |
---
|
35 |
+
|
36 |
+
## Usage
|
37 |
+
|
38 |
+
You can load the SeC model and processor using the `transformers` library with `trust_remote_code=True`. For comprehensive video object segmentation and detailed usage instructions, please refer to the project's [GitHub repository](https://github.com/OpenIXCLab/SeC), particularly `demo.ipynb` for single video inference and `INFERENCE.md` for full inference and evaluation.
|
39 |
+
|
40 |
+
```python
|
41 |
+
import torch
|
42 |
+
from transformers import AutoModel, AutoProcessor
|
43 |
+
from PIL import Image
|
44 |
+
|
45 |
+
# Load model and processor
|
46 |
+
model_name = "OpenIXCLab/SeC-4B"
|
47 |
+
# Ensure your environment has the necessary PyTorch and transformers versions as specified in the GitHub repo.
|
48 |
+
model = AutoModel.from_pretrained(model_name, trust_remote_code=True, torch_dtype=torch.bfloat16).cuda()
|
49 |
+
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
|
50 |
+
|
51 |
+
# Example: Assuming you have an image (e.g., a frame from a video) and a text query
|
52 |
+
# For full video processing, refer to the project's GitHub repository.
|
53 |
+
# Placeholder for an actual image path
|
54 |
+
# image = Image.open("path/to/your/image.jpg").convert("RGB")
|
55 |
+
# text_query = "segment the main object"
|
56 |
+
|
57 |
+
# # Prepare inputs
|
58 |
+
# inputs = processor(images=image, text=text_query, return_tensors="pt").to(model.device)
|
59 |
+
|
60 |
+
# # Perform inference
|
61 |
+
# with torch.no_grad():
|
62 |
+
# outputs = model(**inputs)
|
63 |
+
|
64 |
+
# The output format will vary depending on the model's implementation.
|
65 |
+
# Typically, for segmentation tasks, outputs might include logits or predicted masks.
|
66 |
+
# You will need to process these outputs further to visualize the segmentation.
|
67 |
+
print("Model loaded successfully. For actual inference with video data, please refer to the project's GitHub repository and demo.ipynb.")
|
68 |
+
```
|
69 |
+
|
70 |
## Citation
|
71 |
|
72 |
If you find this project useful in your research, please consider citing:
|