Text Generation
Transformers
Safetensors
English
qwen3_moe
conversational
nielsr HF Staff commited on
Commit
faa228a
·
verified ·
1 Parent(s): 08496d3

Improve model card: Add library name, abstract, GitHub link, and sample usage

Browse files

This PR enhances the model card for `MegaScience/Qwen3-30B-A3B-MegaScience` by:

* Adding `library_name: transformers` to the metadata, which enables the "Use in Transformers" widget on the model page for easier interaction.
* Including the full paper abstract to provide immediate context about the model and its underlying research.
* Adding direct links to the primary GitHub repository for the MegaScience project and its associated evaluation system, where users can find the data curation pipeline, evaluation details, and other resources.
* Providing a practical Python code snippet for text generation using the `transformers` library, making it straightforward for users to get started with the model.

These updates aim to make the model card more comprehensive and user-friendly.

Files changed (1) hide show
  1. README.md +52 -4
README.md CHANGED
@@ -1,17 +1,32 @@
1
  ---
2
- license: apache-2.0
 
3
  datasets:
4
  - MegaScience/MegaScience
5
  language:
6
  - en
 
7
  metrics:
8
  - accuracy
9
- base_model:
10
- - Qwen/Qwen3-30B-A3B-Base
11
  pipeline_tag: text-generation
 
12
  ---
 
13
  # [MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning](https://arxiv.org/abs/2507.16812)
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ## Qwen3-30B-A3B-MegaScience
16
 
17
  ### Training Recipe
@@ -39,6 +54,39 @@ pipeline_tag: text-generation
39
  <img src="https://cdn-uploads.huggingface.co/production/uploads/616bfc2b40e2f69baa1c7add/VogIpBbjfNxXFP9DfVMms.png" alt="Data Pipeline" style="width:100%;">
40
  </div>
41
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  ## Citation
43
 
44
  Check out our [paper](https://arxiv.org/abs/2507.16812) for more details. If you use our dataset or find our work useful, please cite
@@ -51,4 +99,4 @@ Check out our [paper](https://arxiv.org/abs/2507.16812) for more details. If you
51
  journal={arXiv preprint arXiv:2507.16812},
52
  url={https://arxiv.org/abs/2507.16812}
53
  }
54
- ```
 
1
  ---
2
+ base_model:
3
+ - Qwen/Qwen3-30B-A3B-Base
4
  datasets:
5
  - MegaScience/MegaScience
6
  language:
7
  - en
8
+ license: apache-2.0
9
  metrics:
10
  - accuracy
 
 
11
  pipeline_tag: text-generation
12
+ library_name: transformers
13
  ---
14
+
15
  # [MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning](https://arxiv.org/abs/2507.16812)
16
 
17
+ This repository contains the `Qwen3-30B-A3B-MegaScience` model, a Qwen3-30B-A3B-Base model fine-tuned on the MegaScience dataset for scientific reasoning.
18
+
19
+ ## Paper Abstract
20
+
21
+ Scientific reasoning is critical for developing AI scientists and supporting human researchers in advancing the frontiers of natural science discovery. However, the open-source community has primarily focused on mathematics and coding while neglecting the scientific domain, largely due to the absence of open, large-scale, high-quality, verifiable scientific reasoning datasets. To bridge this gap, we first present TextbookReasoning, an open dataset featuring truthful reference answers extracted from 12k university-level scientific textbooks, comprising 650k reasoning questions spanning 7 scientific disciplines. We further introduce MegaScience, a large-scale mixture of high-quality open-source datasets totaling 1.25 million instances, developed through systematic ablation studies that evaluate various data selection methodologies to identify the optimal subset for each publicly available scientific dataset. Meanwhile, we build a comprehensive evaluation system covering diverse subjects and question types across 15 benchmarks, incorporating comprehensive answer extraction strategies to ensure accurate evaluation metrics. Our experiments demonstrate that our datasets achieve superior performance and training efficiency with more concise response lengths compared to existing open-source scientific datasets. Furthermore, we train Llama3.1, Qwen2.5, and Qwen3 series base models on MegaScience, which significantly outperform the corresponding official instruct models in average performance. In addition, MegaScience exhibits greater effectiveness for larger and stronger models, suggesting a scaling benefit for scientific tuning. We release our data curation pipeline, evaluation system, datasets, and seven trained models to the community to advance scientific reasoning research.
22
+
23
+ ## Code and Project Resources
24
+ * **Paper**: [MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning](https://arxiv.org/abs/2507.16812)
25
+ * **GitHub Repository**: [https://github.com/GAIR-NLP/MegaScience](https://github.com/GAIR-NLP/MegaScience)
26
+ * **Hugging Face Organization**: [MegaScience](https://huggingface.co/MegaScience)
27
+ * **MegaScience Dataset**: [MegaScience/MegaScience](https://huggingface.co/datasets/MegaScience/MegaScience)
28
+ * **Evaluation System**: [https://github.com/GAIR-NLP/lm-open-science-evaluation](https://github.com/GAIR-NLP/lm-open-science-evaluation)
29
+
30
  ## Qwen3-30B-A3B-MegaScience
31
 
32
  ### Training Recipe
 
54
  <img src="https://cdn-uploads.huggingface.co/production/uploads/616bfc2b40e2f69baa1c7add/VogIpBbjfNxXFP9DfVMms.png" alt="Data Pipeline" style="width:100%;">
55
  </div>
56
 
57
+ ## Usage
58
+
59
+ You can use this model for text generation with the `transformers` library.
60
+
61
+ ```python
62
+ from transformers import AutoModelForCausalLM, AutoTokenizer
63
+ import torch
64
+
65
+ model_id = "MegaScience/Qwen3-30B-A3B-MegaScience"
66
+
67
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
68
+ model = AutoModelForCausalLM.from_pretrained(
69
+ model_id,
70
+ torch_dtype=torch.bfloat16,
71
+ device_map="auto"
72
+ )
73
+
74
+ messages = [
75
+ {"role": "user", "content": "Explain the concept of quantum entanglement in simple terms."},
76
+ ]
77
+
78
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
79
+ model_inputs = tokenizer(text, return_tensors="pt").to(model.device)
80
+
81
+ generated_ids = model.generate(
82
+ model_inputs.input_ids,
83
+ max_new_tokens=512
84
+ )
85
+
86
+ generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
87
+ print(generated_text)
88
+ ```
89
+
90
  ## Citation
91
 
92
  Check out our [paper](https://arxiv.org/abs/2507.16812) for more details. If you use our dataset or find our work useful, please cite
 
99
  journal={arXiv preprint arXiv:2507.16812},
100
  url={https://arxiv.org/abs/2507.16812}
101
  }
102
+ ```