Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ Data, knowledge store and source code to reproduce the baseline experiments for
|
|
| 9 |
|
| 10 |
## NEWS:
|
| 11 |
- 19.04.2024: The submisstion page (with eval.ai) for the shared-task is alive, you can participate by submitting your predictions [here](https://eval.ai/web/challenges/challenge-page/2285/overview)!
|
| 12 |
-
- 15.07.2024: To facilitate human evaluation we now ask the submission files to include a `scraped_text
|
| 13 |
|
| 14 |
|
| 15 |
## Dataset
|
|
@@ -131,7 +131,7 @@ python -m src.prediction.evaluate_veracity
|
|
| 131 |
|
| 132 |
To facilitate human evaluation, the submission file should include the text of the evidence documents used, retrieved through the `url` field. If external knowledge is utilized, please provide the scraped text. If our provided knowledge store is used, this can be achieved by running the following code block (see [veracity_with_scraped_text.py](https://huggingface.co/chenxwh/AVeriTeC/blob/main/src/prediction/veracity_with_scraped_text.py) for adding the text to the previous prediction file. An example output for the dev set is [here](https://huggingface.co/chenxwh/AVeriTeC/blob/main/data_store/dev_veracity_prediction_for_submission.json).
|
| 133 |
```bash
|
| 134 |
-
python -m src.prediction.veracity_with_scraped_text --knowledge_store_dir <
|
| 135 |
```
|
| 136 |
|
| 137 |
Each line of the final submission file is a json object with the following information:
|
|
|
|
| 9 |
|
| 10 |
## NEWS:
|
| 11 |
- 19.04.2024: The submisstion page (with eval.ai) for the shared-task is alive, you can participate by submitting your predictions [here](https://eval.ai/web/challenges/challenge-page/2285/overview)!
|
| 12 |
+
- 15.07.2024: To facilitate human evaluation we now ask the submission files to include a `scraped_text` field in your submission file, have a look in [here](https://huggingface.co/chenxwh/AVeriTeC#format-for-submission-files) for more information!
|
| 13 |
|
| 14 |
|
| 15 |
## Dataset
|
|
|
|
| 131 |
|
| 132 |
To facilitate human evaluation, the submission file should include the text of the evidence documents used, retrieved through the `url` field. If external knowledge is utilized, please provide the scraped text. If our provided knowledge store is used, this can be achieved by running the following code block (see [veracity_with_scraped_text.py](https://huggingface.co/chenxwh/AVeriTeC/blob/main/src/prediction/veracity_with_scraped_text.py) for adding the text to the previous prediction file. An example output for the dev set is [here](https://huggingface.co/chenxwh/AVeriTeC/blob/main/data_store/dev_veracity_prediction_for_submission.json).
|
| 133 |
```bash
|
| 134 |
+
python -m src.prediction.veracity_with_scraped_text --knowledge_store_dir <directory_of_the_knowledge_store>
|
| 135 |
```
|
| 136 |
|
| 137 |
Each line of the final submission file is a json object with the following information:
|