  Large Language Models (LLMs) have revolutionized natural language processing
and demonstrated impressive capabilities in various tasks. Unfortunately, they
are prone to hallucinations, where the model exposes incorrect or false
information in its responses, which renders diligent evaluation approaches
mandatory. While LLM performance in specific knowledge fields is often
evaluated based on question and answer (Q&A) datasets, such evaluations usually
report only a single accuracy number for the dataset, which often covers an
entire field. This field-based evaluation, is problematic with respect to
transparency and model improvement. A stratified evaluation could instead
reveal subfields, where hallucinations are more likely to occur and thus help
to better assess LLMs' risks and guide their further development. To support
such stratified evaluations, we propose LLMMaps as a novel visualization
technique that enables users to evaluate LLMs' performance with respect to Q&A
datasets. LLMMaps provide detailed insights into LLMs' knowledge capabilities
in different subfields, by transforming Q&A datasets as well as LLM responses
into an internal knowledge structure. An extension for comparative
visualization furthermore, allows for the detailed comparison of multiple LLMs.
To assess LLMMaps we use them to conduct a comparative analysis of several
state-of-the-art LLMs, such as BLOOM, GPT-2, GPT-3, ChatGPT and LLaMa-13B, as
well as two qualitative user evaluations. All necessary source code and data
for generating LLMMaps to be used in scientific publications and elsewhere is
available on GitHub: https://github.com/viscom-ulm/LLMMaps
