Upload R2E-TestgenAgent - Testing Agent for R2E-Gym

README.md

@@ -0,0 +1,117 @@
+# R2E-TestgenAgent
+
+## Overview
+
+R2E-TestgenAgent is a specialized execution-based testing agent designed for generating targeted unit tests for software engineering tasks. This agent is part of the R2E-Gym framework, which provides a comprehensive environment for training and evaluating software engineering agents.
+
+## Model Description
+
+The R2E-TestgenAgent is an execution-based testing agent that specializes in:
+- **Targeted Unit Test Generation**: Creates specific unit tests to validate code patches and implementations
+- **Execution-Based Verification**: Generates tests that can be executed to verify the correctness of code changes
+- **Corner Case Detection**: Identifies and tests potential edge cases and corner scenarios
+- **Patch Disambiguation**: Creates tests that can differentiate between correct and incorrect patches
+
+## Architecture
+
+The agent is built on top of the Qwen2.5-Coder-32B-Instruct model and fine-tuned using R2E-Gym's SFT (Supervised Fine-Tuning) trajectories specifically designed for testing tasks.
+
+## Training Data
+
+The model was trained on the `R2E-Gym/R2EGym-TestingAgent-SFT-Trajectories` dataset, which contains:
+- High-quality testing trajectories collected from Claude-3.5-Sonnet
+- Execution-based testing scenarios
+- Diverse software engineering problems across 13 repositories
+- Real-world testing patterns and methodologies
+
+## Usage
+
+### Basic Usage
+
+```python
+from r2egym.agenthub.environment.env import EnvArgs, RepoEnv
+from r2egym.agenthub.agent.agent import AgentArgs, Agent
+from pathlib import Path
+from datasets import load_dataset
+
+# Load dataset
+ds = load_dataset("R2E-Gym/R2E-Gym-Lite")
+env_args = EnvArgs(ds=ds['train'][0])
+env = RepoEnv(env_args)
+
+# Load testing agent configuration
+agent_args = AgentArgs.from_yaml(Path('./config/testing_agent.yaml'))
+agent_args.llm_name = 'r2e-gym/R2E-TestgenAgent'
+agent = Agent(name="TestingAgent", args=agent_args)
+
+# Run the testing agent
+output = agent.run(env, max_steps=30, use_fn_calling=True)
+```
+
+### Configuration
+
+The agent uses specific prompts and configurations optimized for test generation:
+
+```yaml
+system_prompt: |
+  You are a specialized testing agent designed to generate targeted unit tests 
+  for software engineering tasks. Your goal is to create comprehensive tests 
+  that can validate code patches and identify potential issues.
+
+instance_prompt: |
+  Given the following problem and potential patches, create targeted unit tests
+  that can effectively validate the correctness of the implementation.
+```
+
+## Training Configuration
+
+The model was trained using the following configuration:
+
+- **Base Model**: Qwen/Qwen2.5-Coder-32B-instruct
+- **Training Method**: Full fine-tuning with DeepSpeed ZeRO-3
+- **Learning Rate**: 1.0e-5
+- **Epochs**: 2.0
+- **Batch Size**: 1 (per device)
+- **Context Length**: 20,480 tokens
+- **Optimizer**: AdamW with cosine learning rate scheduling
+
+## Performance
+
+The R2E-TestgenAgent is designed to work in conjunction with other R2E-Gym agents:
+- **Code Editing Agent**: For generating and fixing code
+- **Execution-free Verifier**: For reranking patches
+- **Hybrid Test-time Scaling**: Combines execution-based and execution-free verification
+
+## Integration with R2E-Gym
+
+This agent is part of the larger R2E-Gym ecosystem:
+
+1. **Environment**: Works with R2E-Gym's 8.1K+ procedurally curated environments
+2. **Evaluation**: Can be evaluated on SWE-Bench Verified and other benchmarks
+3. **Training**: Supports continued training on additional trajectories
+
+## Citation
+
+If you use R2E-TestgenAgent in your research, please cite:
+
+```bibtex
+@article{jain2025r2e,
+  title={R2e-gym: Procedural environments and hybrid verifiers for scaling open-weights swe agents},
+  author={Jain, Naman and Singh, Jaskirat and Shetty, Manish and Zheng, Liang and Sen, Koushik and Stoica, Ion},
+  journal={arXiv preprint arXiv:2504.07164},
+  year={2025}
+}
+```
+
+## License
+
+This model is released under the same license as the base Qwen2.5-Coder model.
+
+## Links
+
+- **Paper**: [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://arxiv.org/abs/2504.07164)
+- **GitHub**: [R2E-Gym](https://github.com/R2E-Gym/R2E-Gym)
+- **Dataset**: [R2EGym-TestingAgent-SFT-Trajectories](https://huggingface.co/datasets/R2E-Gym/R2EGym-TestingAgent-SFT-Trajectories)
+- **Related Models**: 
+  - [R2EGym-32B-Agent](https://huggingface.co/R2E-Gym/R2EGym-32B-Agent)
+  - [R2EGym-Verifier](https://huggingface.co/R2E-Gym/R2EGym-Verifier) 

model_card.md

@@ -0,0 +1,62 @@
+---
+license: apache-2.0
+library_name: transformers
+language:
+- en
+tags:
+- code
+- software-engineering
+- testing
+- unit-tests
+- r2e-gym
+- swe-bench
+base_model: Qwen/Qwen2.5-Coder-32B-Instruct
+datasets:
+- R2E-Gym/R2EGym-TestingAgent-SFT-Trajectories
+model_type: qwen2
+---
+
+# R2E-TestgenAgent
+
+A specialized execution-based testing agent for generating targeted unit tests in software engineering tasks.
+
+## Model Details
+
+- **Model Type**: Qwen2.5-Coder-32B fine-tuned for test generation
+- **Training Data**: R2E-Gym SFT trajectories for testing tasks
+- **Use Case**: Automated unit test generation for software engineering
+- **Framework**: R2E-Gym ecosystem
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "r2e-gym/R2E-TestgenAgent"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+# Use with R2E-Gym framework for best results
+from r2egym.agenthub.agent.agent import Agent, AgentArgs
+agent_args = AgentArgs.from_yaml("testing_agent_config.yaml")
+agent = Agent(name="TestingAgent", args=agent_args)
+```
+
+## Training
+
+- **Base Model**: Qwen/Qwen2.5-Coder-32B-Instruct
+- **Training Method**: Full fine-tuning with DeepSpeed
+- **Learning Rate**: 1e-5
+- **Epochs**: 2
+- **Context Length**: 20,480 tokens
+
+## Citation
+
+```bibtex
+@article{jain2025r2e,
+  title={R2e-gym: Procedural environments and hybrid verifiers for scaling open-weights swe agents},
+  author={Jain, Naman and Singh, Jaskirat and Shetty, Manish and Zheng, Liang and Sen, Koushik and Stoica, Ion},
+  journal={arXiv preprint arXiv:2504.07164},
+  year={2025}
+}
+```

push_to_hf.py

@@ -0,0 +1 @@
+ 

testing_agent_config.yaml

@@ -0,0 +1,47 @@
+system_prompt: |-
+  You are a specialized testing agent designed to generate targeted unit tests for software engineering tasks. Your goal is to create comprehensive tests that can validate code patches and identify potential issues.
+
+  You have access to a repository environment where you can explore files, understand the codebase structure, and generate appropriate unit tests. Your tests should be:
+  1. Comprehensive - covering both normal cases and edge cases
+  2. Targeted - specifically designed to validate the correctness of code changes
+  3. Executable - able to run and provide clear pass/fail results
+  4. Discriminative - able to differentiate between correct and incorrect implementations
+
+  Use the available tools to explore the repository, understand the problem context, and create effective test cases.
+
+instance_prompt: |-
+  Consider the following software engineering problem:
+  <problem>
+  {problem_statement}
+  </problem>
+
+  Your task is to generate targeted unit tests that can effectively validate any proposed solution to this problem. The tests should:
+
+  1. **Explore the repository structure** to understand the codebase and existing test patterns
+  2. **Analyze the problem** to identify key requirements and edge cases
+  3. **Create comprehensive tests** that cover:
+     - Normal functionality
+     - Edge cases and boundary conditions
+     - Error handling scenarios
+     - Integration with existing code
+  4. **Ensure tests are executable** and provide clear validation of correctness
+
+  Focus on creating tests that can distinguish between correct and incorrect implementations. Your tests should be thorough enough to catch potential bugs while being specific enough to validate the correct behavior.
+
+  IMPORTANT: 
+  - Create tests in a file named `test_issue.py` 
+  - Use standard Python testing frameworks (unittest, pytest, etc.)
+  - Each test should have a clear, descriptive name
+  - Include proper assertions and error handling
+  - Add comments explaining what each test validates
+
+  Generate tests that would help validate the correctness of any proposed solution to the given problem.
+
+command_files:
+  - src/r2egym/agenthub/runtime/bash_cmds.yml
+
+llm_name: "r2e-gym/R2E-TestgenAgent"
+use_demo: false
+other_args:
+  max_retries: 3
+  timeout: 300 

train_r2egym_32B_testing_agent.yaml

@@ -0,0 +1,42 @@
+### model
+model_name_or_path: Qwen/Qwen2.5-Coder-32B-instruct
+trust_remote_code: true
+
+### method
+stage: sft
+do_train: true
+finetuning_type: full
+deepspeed: examples/deepspeed/ds_z3_offload_config.json
+# deepspeed: examples/deepspeed/ds_z3_config.json  # choices: [ds_z0_config.json, ds_z2_config.json, ds_z3_config.json]
+
+### dataset
+dataset: R2E-Gym/R2EGym-TestingAgent-SFT-Trajectories
+template: qwen
+cutoff_len: 20480
+max_samples: 100000
+overwrite_cache: true
+preprocessing_num_workers: 16
+
+### output
+output_dir: saves/R2EGym-32B-TestingAgent
+logging_steps: 10
+save_steps: 10000
+plot_loss: true
+overwrite_output_dir: true
+
+### train
+flash_attn: fa2
+enable_liger_kernel: true
+use_unsloth_gc: true
+per_device_train_batch_size: 1
+gradient_accumulation_steps: 1
+learning_rate: 1.0e-5
+num_train_epochs: 2.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.05
+bf16: true
+ddp_timeout: 180000000
+
+### wandb
+report_to: wandb
+run_name: R2EGym-32B-TestingAgent