Jarbas/DrQA_en

DrQA

A pytorch implementation of the ACL 2017 paper Reading Wikipedia to Answer Open-Domain Questions (DrQA).

Reading comprehension is a task to produce an answer when given a question and one or more pieces of evidence (usually natural language paragraphs). Compared to question answering over knowledge bases, reading comprehension models are more flexible and have revealed a great potential for zero-shot learning.

Requirements

• python >=3.5
• pytorch >=0.4. Tested on pytorch 0.4 and pytorch 1.10
• numpy
• msgpack
• spacy 3.x

Quick Start

• download SpaCy English language model python3 -m spacy download en_core_web_md
• download model - https://huggingface.co/Jarbas/DrQA_en (EM: 68.92712550607287 F1: 78.19080821710139)

Usage

Example usage with wikipedia via https://github.com/OpenVoiceOS/DrQA

import argparse

import requests
import torch

from drqa import DrQA
from drqa.utils import str2bool

parser = argparse.ArgumentParser(
    description='Interact with document reader model.'
)
parser.add_argument('--model-file', help='path to model file')
parser.add_argument('--meta-file', help='path to meta.msgpack file')
parser.add_argument("--cuda", type=str2bool, nargs='?',
                    const=True, default=torch.cuda.is_available(),
                    help='whether to use GPU acceleration.')
args = parser.parse_args()


def get_wiki_evidence(search_term):
    try:
        search_url = f"https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch={search_term}&format=json&srlimit=1"
        r = requests.get(search_url).json()
        page_id = r['query']['search'][0]['pageid']
        results_url = f"https://en.wikipedia.org/w/api.php?format=json&action=query&prop=extracts&exintro&explaintext&redirects=1&pageids={page_id}"
        r = requests.get(results_url).json()
        evidence = r['query']['pages'][str(page_id)]['extract']
        return evidence
    except:
        print("wiki search failed")
        return None


dr = DrQA(args.model_file, args.meta_file, cuda=args.cuda)
while True:
    search_term = input("wiki search:")
    evidence = get_wiki_evidence(search_term)
    if not evidence:
        continue
    question = input("question:")
    answer = dr.predict(evidence, question)
    print(">", answer)

Example interactions:

wiki search:dog
question:when were dogs domesticated
> 15,000 years ago
wiki search:abraham lincoln
question:when abraham lincoln was born
> February 12, 1809 – April 15, 1865
wiki search:cat
question:when were cats domesticated
> 3100 BC
wiki search:chicken
question:what is the global population of chickens
> 23.7 billion as of 2018, up from more than 19 billion in 2011.

About

Original Implementation: https://github.com/hitvoice/DrQA

Most of the pytorch model code is borrowed from Facebook/ParlAI under a BSD-3 license.