<jupyter_start><jupyter_text>Process DataImportant: Before running this notebook, make sure that you have downloaded the data from RegDocs (or you have pickled the PDFs and placed them in `data/interim/pickled_pdfs`.)This notebook will grab the text from each PDF, do some preprocessing so that the data is clean to use with elasticsearch and haystack, and then store the pieces of texts in `data/processed/pdf_texts` so that we can grab the text for question-answering.<jupyter_code>from haystack.utils import clean_wiki_text, convert_files_to_dicts, fetch_archive_from_http, print_answers
from haystack.nodes import FARMReader, TransformersReader<jupyter_output>C:\Users\ThibJacq\Anaconda3\envs\cer-qa\lib\site-packages\ray\autoscaler\_private\cli_logger.py:61: FutureWarning: Not all Ray CLI dependencies were found. In Ray 1.4+, the Ray CLI, autoscaler, and dashboard will only be usable via `pip install 'ray[default]'`. Please update your install command.
  "update your install command.", FutureWarning)
C:\Users\ThibJacq\Anaconda3\envs\cer-qa\lib\site-packages\torchaudio\extension\extension.py:13: UserWarning: torchaudio C++ extension is not available.
  warnings.warn('torchaudio C++ extension is not available.')
C:\Users\ThibJacq\Anaconda3\envs\cer-qa\lib\site-packages\torchaudio\backend\utils.py:89: UserWarning: No audio backend is available.
  warnings.warn('No audio backend is available.')
StarcoderdataJupyterStructuredCleanDedup