metadata
license: apache-2.0
license_link: https://huggingface.co/Xtra-Computing/XtraGPT-7B/blob/main/LICENSE
language:
- en
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
- chat
library_name: transformers
XtraGPT-7B
Introduction
XtraGPT is a series of LLMs for Human-AI Collaboration on Controllable Scientific Paper Refinement developed by NUS Xtra Computing Group.
Requirements
The code of Qwen2.5 has been in the latest Hugging face transformers and we advise you to use the latest version of transformers.
Quickstart
Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Xtra-Computing/XtraGPT-7B"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
paper_content="""
In this paper, we propose a metric-based probing method, namely, CAT-probing, to quantitatively evaluate how CodePTMs Attention scores relate to distances between AST nodes.
First, to denoise the input code sequence in the original attention scores matrix, we classify the rows/cols by token types that are pre-defined by compilers,
and then retain tokens whose types have the highest proportion scores to derive a filtered attention matrix (see Figure 1(b)).
Meanwhile, inspired by the works (Wang et al., 2020; Zhu et al., 2022), we add edges to improve the connectivity of AST and calculate the distances between nodes corresponding to the selected tokens,
which generates a distance matrix as shown in Figure 1(c). After that, we define CAT-score to measure the matching degree between the filtered attention matrix and the distance matrix.
Specifically, the point-wise elements of the two matrices are matched if both the two conditions are satisfied:
1) the attention score is larger than a threshold; 2) the distance value is smaller than a threshold. If only one condition is reached, the elements are unmatched.
We calculate the CAT-score by the ratio of the number of matched elements to the summation of matched and unmatched elements.
Finally, the CAT-score is used to interpret how CodePTMs attend code structure, where a higher score indicates that the model has learned more structural information.
"""
selected_content="""
After that, we define CAT-score to measure the matching degree between the filtered attention matrix and the distance matrix.
"""
prompt ="""
help me redefine cat-score based on the context.
"""
content = f"""
Please improve the selected content based on the following. Act as an expert model for improving articles **PAPER_CONTENT**.\n
The output needs to answer the **QUESTION** on **SELECTED_CONTENT** in the input. Avoid adding unnecessary length, unrelated details, overclaims, or vague statements.
Focus on clear, concise, and evidence-based improvements that align with the overall context of the paper.\n
<PAPER_CONTENT>
{paper_content}
</PAPER_CONTENT>\n
<SELECTED_CONTENT>
{selected_content}
</SELECTED_CONTENT>\n
<QUESTION>
{prompt}
</QUESTION>\n
"""
messages = [
{"role": "user", "content": content}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
from openai import OpenAI
model_name = "Xtra-Computing/XtraGPT-7B"
client = OpenAI(
base_url="http://localhost:8088/v1",
api_key="sk-1234567890"
)
paper_content="""
markdown
"""
selected_content="""
After that, we define CAT-score to measure the matching degree between the filtered attention matrix and the distance matrix.
"""
prompt ="""
help me redefine cat-score based on the context.
"""
content = f"""
Please improve the selected content based on the following. Act as an expert model for improving articles **PAPER_CONTENT**.\n
The output needs to answer the **QUESTION** on **SELECTED_CONTENT** in the input. Avoid adding unnecessary length, unrelated details, overclaims, or vague statements.
Focus on clear, concise, and evidence-based improvements that align with the overall context of the paper.\n
<PAPER_CONTENT>
{paper_content}
</PAPER_CONTENT>\n
<SELECTED_CONTENT>
{selected_content}
</SELECTED_CONTENT>\n
<QUESTION>
{prompt}
</QUESTION>\n
"""
response = client.chat.completions.create(
model="xtragpt",
messages=[{"role": "user", "content": content}],
temperature=0.7,
max_tokens=16384
)
print(response.choices[0].message.content)
Citation
If you find our work helpful, feel free to give us a cite.
@misc{xtracomputing2025xtraqa,
title = {XtraQA},
url = {https://huggingface.co/Xtra-Computing/XtraGPT-7B},
author = {Xtra Computing Group},
year = {2025}
}
@article{xtracomputing2025xtragpt,
title={XtraGPT: LLMs for Human-AI Collaboration on Controllable Scientific Paper Refinement},
author={Xtra Computing Group},
journal={arXiv preprint arXiv:abcdefg},
year={2025}
}