text: This paper tackles the problem of reading comprehension over long narratives where documents easily span over thousands of tokens. We propose a curriculum learning (CL) based Pointer-Generator framework for reading sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. This can be interpreted as a form of domain randomization and or generative pretraining during training. To this end, the usage of the Pointer-Generator softens the requirement of having the answer within the context, enabling us to construct diverse training samples for learning. Additionally, we propose a new Introspective Alignment Layer (IAL), which reasons over decomposed alignments using block-based self-attention. We evaluate our proposed method on the NarrativeQA reading comprehension benchmark, achieving state-of-the-art performance, improving existing baselines by @math relative improvement on BLEU-4 and @math relative improvement on Rouge-L. Extensive ablations confirm the effectiveness of our proposed IAL and CL components.
  @cite_19: Neural sequence-to-sequence models have provided a viable new approach for abstractive text summarization (meaning they are not restricted to simply selecting and rearranging passages from the original text). However, these models have two shortcomings: they are liable to reproduce factual details inaccurately, and they tend to repeat themselves. In this work we propose a novel architecture that augments the standard sequence-to-sequence attentional model in two orthogonal ways. First, we use a hybrid pointer-generator network that can copy words from the source text via pointing, which aids accurate reproduction of information, while retaining the ability to produce novel words through the generator. Second, we use coverage to keep track of what has been summarized, which discourages repetition. We apply our model to the CNN Daily Mail summarization task, outperforming the current abstractive state-of-the-art by at least 2 ROUGE points.
  @cite_15: Presented on August 28, 2018 at 12:15 p.m. in the Pettit Microelectronics Research Center, Room 102 A B.
  @cite_22: 
summary: Flexible neural mechanisms that learn to point and or generate have been also popular across many NLP tasks. Our model incorporates Pointer-Generator networks @cite_19 which learns to copy or generate new words within the context of neural summarization. Prior to Pointer Generators, CopyNet @cite_22 incorporates a copy mechanism for sequence to sequence learning. Pointer generators have also been recently adopted for learning a universal multi-task architecture for NLP @cite_15 .