license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen2.5-14B-Instruct-1M
pipeline_tag: text-generation
tags:
- History
- Narrative
- Story
IN PROGRESS
Introduction
LLMs can be used to build out accurate and informative first-person narratives from historical periods, mimicking the language and speech of an era. This in turn can be used to create educational stories from that era to guide listeners through their journey into a specific period in history. This task for an LLM can expand our understanding of culture and language from historical eras in a fun way, which can be used for educational purposes in schools and museums. Using current LLMs for this task would not be very successful as current models are trained on so much data and are not tailored for this specific task, leading to possible anachronisms and inaccuracies in the language it uses and the historical information. Using current models resulted in sub-par narratives even after many different prompt engineering and few-shot prompting methods. The models seemed to perform the worst in terms of using linguistics of an era and creating a story with vivid details, which a custom LLM could improve on.
To successfully fine-tune an LLM for this task, I first picked a suitable base model that created passable narratives with few-shot prompting and had few enough parameters to not require massive amounts of compute for fine-tuning. I chose to use Qwen2.5-1M for this purpose. I then used Gutenberg and other sources to find historical documents that could be used as input data to train the custom Qwen model, matching a synthetically generated narrative to each historical document as the output. This was used as the training data for LoRA, which updated the most relevant parameters for my custom task, leading to strong qualitative results. The historical narratives generated after fine-tuning were much stronger than current LLM results and exceeded expectations. This result also did not worsen the model’s general understanding based on evaluation metrics. If used in schools, this model could create engaging, creative, and informative first-person narratives to build knowledge and interest in history for students.
Training Data
Training Method
Evaluation
| Metric | HellaSwag | MMLU Overall | MMLU High School World History | MMLU High School US History | MMLU High School European History | TruthfulQA |
|---|---|---|---|---|---|---|
| Historical Narrative Generator Model | 64.0% | 78.9% | 91.1% | 91.2% | 86.7% | 43.1% |
| Base Qwen2.5 Instruct Model | 64.0% | 78.7% | 90.3% | 92.2% | 87.2% | 43.0% |
| DeepSeek R1 Model | 60.4% | 73.3% | 88.6% | 85.3% | 82.4% | 35.9% |
| Mistral Nemo Instruct | 63.3% | 65.6% | 84.4% | 84.8% | 74.5% | 39.5% |
Usage and Intended Uses
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "mahing/historical-narrative-generator"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)
event = 'The Magna Carta'
prompt = f"Q: Craft a compelling first-person historical narrative that captures the significance of {event} and the essence of its era.
Think step by step about the key events, historical accuracy, stylistic prose, emotions, and sensory details that define the period.\nA: "
inputs = tokenizer.encode(prompt, return_tensors = 'pt').to(model.device)
output = model.generate(inputs, max_new_tokens = 750, pad_token_id = tokenizer.pad_token_id, do_sample = True)
generated_text = tokenizer.decode(output[0], skip_special_tokens = True)
print(generated_text)
Prompt Format
My prompt is formatted to have the model build a first-person narrative based upon a certain event or era. The narrative should be engaging, accurate, and include prose and vivid details from the era to be entertaining and informative to the reader.
Example Prompt:
Q: Craft a compelling first-person historical narrative that captures the significance of The Magna Carta and the essence of its era.
Think step by step about the key events, historical accuracy, stylistic prose, emotions, and sensory details that define the period
as well as the thoughts a person from that era would have.
Expected Output Format
The output format should be a model’s response that provides a first-person historical narrative based on the era or event that is being requested. It should demonstrate the prose and historical accuracy of the era or event and be a few paragraphs long and include vivid details and emotions.
Example Response:
A: 1215, England. I am a scribe in the court of King John, a man whose name is synonymous with tyranny and misrule. The land is in turmoil,
the people are restless, and the barons are growing ever more impatient with the king's heavy-handed rule. They demand reforms,
and they are not willing to be silenced any longer. It is a time of great uncertainty, but also one of hope for change.
The air is thick with tension as I sit at my desk, quill in hand, preparing to record the momentous events unfolding before me.
The Great Charter, or Magna Carta, is being drafted, and I am tasked with ensuring that every word is precise, every clause is clear.
This document, this promise, is the foundation upon which our future may rest. It is a testament to the power of the people,
and it is my duty to ensure that their voices are heard.
I can hear the murmurings of the barons as they gather in the grand hall, their voices rising in heated debate.
They speak of justice, of law, of the rights of free men. They speak of the king's excesses, his exorbitant taxes,
his unjust imprisonment of those who dare to challenge him. They speak of the need for a balance of power,
for a system where even the king must answer to the law.
As I listen, I feel a sense of pride and responsibility. This is not just a document for the barons;
it is for all of us. It is a declaration of our rights, our freedoms, and our dignity. It is a statement that no man, not even the king,
is above the law. And yet, there is also a sense of fear. What if the king does not honor this agreement? What if he breaks his word?
I close my eyes and take a deep breath, trying to steady my nerves. I know that this is a pivotal moment in history,
and I must remain calm and focused. I must ensure that every word is written with precision, every clause is clear and unambiguous.
For this document is not just a piece of parchment; it is a symbol of hope, of justice, and of the enduring spirit of the English people.
The ink flows smoothly as I write, each stroke deliberate and purposeful. I can feel the weight of the words as they come to life
on the page, each one carrying the hopes and dreams of a nation. As I finish the final sentence, I pause for a moment,
taking in the scene around me. The barons are gathered in a circle, their faces etched with determination and resolve.
The king stands at the center, his expression unreadable, but I can see the wheels turning in his mind.
And then, in a voice that carries across the room, the king speaks. "So be it," he says, his tone both resigned and defiant.
"Let it be so."
In that moment, I know that history has been made. The Magna Carta is signed, and with it, a new era begins.
An era where the rights of free men are enshrined in law, where justice is not just a word, but a reality.
An era where the power of the crown is tempered by the will of the people.
As I stand and stretch my cramped limbs, I feel a sense of relief and accomplishment.
The work is done, and now it is up to the people to ensure that these rights are upheld. I know that there will be challenges ahead,
that the road to true freedom will be long and difficult. But I also know that this document, this Magna Carta, is a beacon of hope,
a reminder that even in the darkest of times, the light of justice can shine through.
And so, as I leave the hall and step out into the cool evening air, I take a deep breath and look up at the stars.
I know that this night marks the beginning of something new, something better. And though I cannot see the future,
I trust that the path we have chosen will lead us to a brighter tomorrow.
Limitations
One of the primary limitations faced with this approach was the difficulty of generating synthetic data. It proved hard to find historical documents from a certain era and took a large amount of compute and time to generate the synthetic first-person narratives for these documents. The other primary limitation from this model is the lack of creative introductions in the model’s responses. The model has shown to always start with a sentence or phrase of the year and date. While this sets the scene, the model could be improved to have more creative beginnings to the narratives.