Improve model card

This PR improves the model card by adding a link to the paper, specifying the correct library name, and adding more details about the model from the paper abstract. It also restructures the examples section to be more user-friendly.

README.md

@@ -1,14 +1,37 @@
 ---
-license: gpl-3.0
+base_model:
+- VideoCrafter/VideoCrafter2
 datasets:
 - nkp37/OpenVid-1M
 - TempoFunk/webvid-10M
-base_model:
-- VideoCrafter/VideoCrafter2
+license: gpl-3.0
 pipeline_tag: text-to-video
+library_name: diffusers
 ---
+
 # Advanced text-to-video Diffusion Models
 
+This repository contains the model from the paper [AMD-Hummingbird: Towards an Efficient Text-to-Video Model](https://huggingface.co/papers/2503.18559). Hummingbird is a lightweight text-to-video (T2V) framework that prunes existing models (such as VideoCrafter2) and enhances visual quality through visual feedback learning. It aims to improve the efficiency of T2V generation, making it more suitable for deployment on resource-limited devices while preserving high-quality video generation.
+
+## Table of Contents
+- [Advanced text-to-video Diffusion Models](#advanced-text-to-video-diffusion-models)
+- [Key Features](#key-features)
+- [8-Steps Results](#8-steps-results)
+- [Checkpoint](#checkpoint)
+- [Installation](#installation)
+  - [conda](#conda)
+  - [docker](#docker)
+- [Data Processing](#data-processing)
+  - [VQA](#vqa)
+  - [Remove Dolly Zoom Videos](#remove-dolly-zoom-videos)
+- [Training](#training)
+  - [Model Distillation](#model-distillation)
+  - [Acceleration Training](#acceleration-training)
+- [Inference](#inference)
+- [License](#license)
+
+
+## Key Features
 
 ⚡️ This repository provides training recipes for the AMD efficient text-to-video models, which are designed for high performance and efficiency. The training process includes two key steps:
 
@@ -18,88 +41,111 @@ pipeline_tag: text-to-video
 
 This implementation is released to promote further research and innovation in the field of efficient text-to-video generation, optimized for AMD Instinct accelerators.
 
-You can download the code from our [GitHub Repo](https://github.com/AMD-AIG-AIMA/AMD-Hummingbird-T2V).
-
-<img src="GIFs/vbench.png" alt="Vbench performance" title="Vbench performance" class="vbench-img">
-
-
-**8-Steps Results**
-<style>
-  table {
-    width: auto;
-    border-collapse: collapse;
-  }
-  th, td {
-    border: 1px solid #ddd;
-    text-align: center;
-    padding: 0px;
-    vertical-align: middle;
-    width: 256px; /* 每列宽度固定 */
-  }
-  tr.text-row {
-    height: 30px; /* 文字行高度 */
-  }
-  tr.image-row {
-    height: 160px; /* 图片行高度 */
-  }
-  /* 默认表格中的图片大小 */
-  img {
-    width: 256px;
-    height: 160px;
-    object-fit: cover;
-  }
-  /* 只影响 vbench.png */
-  .vbench-img {
-    width: 785px !important;
-    height: 698px !important;
-    object-fit: contain; /* 让图片完整显示，不裁剪 */
-  }
-</style>
-
-
-<table>
-  <tr class="text-row">
-    <th>A cute happy Corgi playing in park, sunset, pixel.</th>
-    <th>A cute happy Corgi playing in park, sunset, animated style.</th>
-    <th>A cute raccoon playing guitar in the beach.</th>
-    <th>A cute raccoon playing guitar in the forest.</th>
-  </tr>
-  <tr class="image-row">
-    <td><img src="GIFs/A_cute_happy_Corgi_playing_in_park,_sunset,_pixel_.gif"></td>
-    <td><img src="GIFs/A cute happy Corgi playing in park, sunset, animated style.gif"></td>
-    <td><img src="GIFs/A cute raccoon playing guitar in the beach.gif"></td>
-    <td><img src="GIFs/A cute raccoon playing guitar in the forest.gif"></td>
-  </tr>
-  <tr class="text-row">
-    <th>A quiet beach at dawn and the waves gently lapping.</th>
-    <th>A cute teddy bear, dressed in a red silk outfit, stands in a vibrant street, Chinese New Year.</th>
-    <th>A sandcastle being eroded by the incoming tide.</th>
-    <th>An astronaut flying in space, in cyberpunk style.</th>
-  </tr>
-  <tr class="image-row">
-    <td><img src="GIFs/A_quiet_beach_at_dawn_and_the_waves_gently_lapping.gif"></td>
-    <td><img src="GIFs/A cute teddy bear, dressed in a red silk outfit, stands in a vibrant street, chinese new year..gif"></td>
-    <td><img src="GIFs/A sandcastle being eroded by the incoming tide.gif"></td>
-    <td><img src="GIFs/An astronaut flying in space, in cyberpunk style.gif"></td>
-  </tr>
-  <tr class="text-row">
-    <th>A cat DJ at a party.</th>
-    <th>A 3D model of a 1800s victorian house.</th>
-    <th>A drone flying over a snowy forest.</th>
-    <th>A ghost ship navigating through a sea under a moon.</th>
-  </tr>
-  <tr class="image-row">
-    <td><img src="GIFs/A_cat_DJ_at_a_party.gif"></td>
-    <td><img src="GIFs/A 3D model of a 1800s victorian house..gif"></td>
-    <td><img src="GIFs/a_drone_flying_over_a_snowy_forest.gif"></td>
-    <td><img src="GIFs/A_ghost_ship_navigating_through_a_sea_under_a_moon.gif"></td>
-  </tr>
-</table>
-
-
-
-
 
+![Vbench performance](GIFs/vbench.png)
+
+
+
+
+## 8-Steps Results
+
+| Prompt                                       | Generated Video          | Prompt                                         | Generated Video        |
+|--------------------------------------------|--------------------------|----------------------------------------------|------------------------|
+| A cute happy Corgi playing in park, sunset, pixel. | ![GIF](GIFs/A_cute_happy_Corgi_playing_in_park,_sunset,_pixel_.gif) | A cute happy Corgi playing in park, sunset, animated style. | ![GIF](GIFs/A_cute_happy_Corgi_playing_in_park,_sunset,_animated_style.gif) |
+| A quiet beach at dawn and the waves gently lapping. | ![GIF](GIFs/A_quiet_beach_at_dawn_and_the_waves_gently_lapping.gif) | A cute teddy bear, dressed in a red silk outfit, stands in a vibrant street, chinese new year. | ![GIF](GIFs/A_cute_teddy_bear,_dressed_in_a_red_silk_outfit,_stands_in_a_vibrant_street,_chinese_new_year..gif) |
+| A cat DJ at a party.                       | ![GIF](GIFs/A_cat_DJ_at_a_party.gif)     | A 3D model of a 1800s victorian house.        | ![GIF](GIFs/A_3D_model_of_a_1800s_victorian_house..gif)    |
+| A cute raccoon playing guitar in the beach. | ![GIF](GIFs/A_cute_raccoon_playing_guitar_in_the_beach.gif)     | A cute raccoon playing guitar in the forest. | ![GIF](GIFs/A_cute_raccoon_playing_guitar_in_the_forest.gif)|
+| A sandcastle being eroded by the incoming tide. | ![GIF](GIFs/A_sandcastle_being_eroded_by_the_incoming_tide.gif) | An astronaut flying in space, in cyberpunk style. | ![GIF](GIFs/An_astronaut_flying_in_space,_in_cyberpunk_style.gif) |
+| A drone flying over a snowy forest.       | ![GIF](GIFs/a_drone_flying_over_a_snowy_forest.gif)       | A ghost ship navigating through a sea under a moon. | ![GIF](GIFs/A_ghost_ship_navigating_through_a_sea_under_a_moon.gif) |
+
+
+# Checkpoint
+Our pretrained checkpoint can be downloaded from [HuggingFace](https://huggingface.co/amd/AMD-Hummingbird-T2V/tree/main)
+
+# Installation
+We train both 0.9B and 0.7 T2V models on MI250 and evalute them on MI250, MI300, RTX7900xt and RadeonTM 880M RyzenTM AI 9 365 Ubuntu 6.8.0-51-generic.
+
+## conda
+```
+conda create -n AMD_Hummingbird python=3.10
+conda activate AMD_Hummingbird
+pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/rocm6.1
+pip install -r requirements.txt
+```
+For rocm flash-attn, you can install it by this [link](https://github.com/ROCm/flash-attention).
+```
+git clone https://github.com/ROCm/flash-attention.git
+cd flash-attention
+python setup.py install
+```
+It will take about 1.5 hours to install.
+
+## docker
+First, you should use `docker pull` to download the image.
+```
+docker pull rocm/vllm:rocm6.2_mi300_ubuntu20.04_py3.9_vllm_0.6.4
+```
+Second, you can use  `docker run` to run the image, for example:
+```
+docker run \
+        -v "$(pwd):/workspace" \
+        --device=/dev/kfd \
+        --device=/dev/dri \
+        -it \
+        --network=host \
+        --name hummingbird \
+        rocm/vllm:rocm6.2_mi300_ubuntu20.04_py3.9_vllm_0.6.4
+```
+When you in the container, you can use `pip` to install other dependencies:
+```
+pip install -r requirements.txt
+```
+
+# Data Processing
+
+## VQA
+```
+cd data_pre_process/DOVER
+sh run.sh
+```
+Then you can get a score table for all video qualities, sort according to the table, and remove low-scoring videos.
+## Remove Dolly Zoom Videos
+```
+cd data_pre_process/VBench
+sh run.sh 
+```
+According to the motion smoothness score csv file, you can  remove low-scoring videos.
+# Training
+
+## Model Distillation
+
+```
+sh configs/training_512_t2v_v1.0/run_distill.sh
+```
+
+
+## Acceleration Training
+
+```
+cd acceleration/t2v-turbo
+
+# for 0.7 B model
+sh train_07B.sh
+
+# for 0.9 B model
+sh train_09B.sh
+```
+
+
+# Inference
+
+```
+# for 0.7B model
+python inference_command_config_07B.py
+
+# for 0.9B model
+python inference_command_config_09B.py
+```
 
 # License
 Copyright (c) 2024 Advanced Micro Devices, Inc. All Rights Reserved.