Update README.md

README.md

@@ -7,20 +7,35 @@ metrics:
 - bleu
 - meteor
 - rouge
-pipeline_tag: text-generation
+pipeline_tag: video-text-to-text
 inference: false
 tags:
 - video-captioning
 ---
-# SpaceTimeGPT - A Spatiotemporal Video Captioning Model
+<h1 align='center'> SpaceTimeGPT - Video Captioning Model </h1>
 
-![SpaceTimeGPT](https://raw.githubusercontent.com/Neleac/SpaceTimeGPT/main/model.JPG)
+<div align="center">
+  <a href="https://github.com/Neleac/SpaceTimeGPT">
+    <img src="https://img.shields.io/badge/GitHub-Neleac/SpaceTimeGPT-purple.svg">
+  </a>
+  <img src="https://raw.githubusercontent.com/Neleac/SpaceTimeGPT/main/model.JPG" width="75%" height="75%">
+  <p> (partial diagrams from <a href="https://arxiv.org/abs/2103.15691">1</a>, <a href="https://arxiv.org/abs/2102.05095">2</a>, <a href="https://arxiv.org/abs/1706.03762">3</a>) </p>
+</div>
 
-Vision Encoder Model: [timesformer-base-finetuned-k600](https://huggingface.co/facebook/timesformer-base-finetuned-k600) \
-Text Decoder Model: [gpt2](https://huggingface.co/gpt2)
+SpaceTimeGPT is a video description generation model capable of both spatial and temporal reasoning. Given a video, eight frames are sampled and analyzed by the model. The output is a sentence description of the events that occured in the video, generated using autoregression.
 
-#### Evaluation Result: 
-67.2 CIDEr on [VaTeX](https://eric-xw.github.io/vatex-website/index.html) public test set
+## Architecture and Training
+Vision Encoder: [timesformer-base-finetuned-k600](https://huggingface.co/facebook/timesformer-base-finetuned-k600) \
+Text Decoder: [gpt2](https://huggingface.co/gpt2)
+
+The encoder and decoder are initialized using pretrained weights for video classification and sentence completion, respectively. Encoder-decoder cross attention is used to unify the visual and linguistic domains. The model is fine-tuned end-to-end on the video captioning task.
+
+## Dataset and Evaluation
+SpaceTimeGPT is trained on [VATEX](https://eric-xw.github.io/vatex-website/index.html), a large video captioning dataset.
+
+Performance: 67.3 [CIDEr](https://github.com/ramavedantam/cider) on the VATEX test split
+
+Sampling method: 30 $\le$ generated tokens $\le$ 60, beam search with 8 beams
 
 #### Example Inference Code:
 ```python