Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,44 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
widget:
|
| 6 |
+
- text: >
|
| 7 |
+
In a busy city street, an autonomous car navigates around pedestrians and cyclists, making a series of complex decisions to ensure safety.
|
| 8 |
+
example_title: Complex VQA Scenario
|
| 9 |
+
- text: >
|
| 10 |
+
On a clear highway, an autonomous vehicle adjusts its speed based on the flow of traffic and the presence of a slower-moving truck ahead.
|
| 11 |
+
example_title: Simple VQA Scenario
|
| 12 |
+
tags:
|
| 13 |
+
- vision-language
|
| 14 |
+
- autonomous-driving
|
| 15 |
---
|
| 16 |
+
|
| 17 |
+
### What is this?
|
| 18 |
+
Lingo-Judge, a novel evaluation metric that aligns closely with human judgment on the LingoQA evaluation suits.
|
| 19 |
+
|
| 20 |
+
### How to use
|
| 21 |
+
```python
|
| 22 |
+
# Import necessary libraries
|
| 23 |
+
from transformers import pipeline
|
| 24 |
+
|
| 25 |
+
# Define the model name to be used in the pipeline
|
| 26 |
+
model_name = 'wayveai/Lingo-Judge'
|
| 27 |
+
|
| 28 |
+
# Define the question and its corresponding answer and prediction
|
| 29 |
+
question = "Are there any pedestrians crossing the road? If yes, how many?"
|
| 30 |
+
answer = "1"
|
| 31 |
+
prediction = "Yes, there is one"
|
| 32 |
+
|
| 33 |
+
# Initialize the pipeline with the specified model, device, and other parameters
|
| 34 |
+
pipe = pipeline("text-classification", model=model_name)
|
| 35 |
+
# Format the input string with the question, answer, and prediction
|
| 36 |
+
input = f"[CLS]\nQuestion: {question}\nAnswer: {answer}\nStudent: {prediction}"
|
| 37 |
+
|
| 38 |
+
# Pass the input through the pipeline to get the result
|
| 39 |
+
result = pipe(input)
|
| 40 |
+
|
| 41 |
+
# Print the result and score
|
| 42 |
+
score = result[0]['score']
|
| 43 |
+
print(score > 0.5, score)
|
| 44 |
+
|