README.md

---
# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
# Doc / guide: https://huggingface.co/docs/hub/model-cards
#{{ card_data }}

license: mit
language:
- en
base_model: Salesforce/codegen-2B-multi
pipeline_tag: text-generation
library_name: transformers
inference:
  parameters:
    provider: "spaces"
    space_url: "https://huggingface.co/spaces/muhammad-mujtaba-ai/SolidityLLMDemo"
tags:
    - Solidity
    - BlockChain
    - Smart Contracts
    - Code Generation
---

# **Solidity-Code-LLM**
Solidity-Code-LLM is a fine tuned large language model designed to understand, generate, and analyze smart contracts written in Solidity. Developed by ChainGPT—a leader in AI infrastructure for the Web3 and blockchain space—this model is purpose-built for the decentralized development ecosystem. 

- **Developed by:** ChainGPT
- **License:** MIT License
- **Finetuned from model:** Salesforce/codegen-2B-multi

![Training Pipeline](logo.png)


# Model Details

### Model Description
Solidity-Code-LLM is a specialized language model trained in two stages: pre-training on a large, unstructured Solidity dataset, followed by instruction-based fine-tuning on a cleaned, curated dataset. Unlike general-purpose code models, it is exclusively focused on Solidity—the dominant language for Ethereum-compatible blockchains—making it an efficient and accurate assistant for writing and debugging smart contracts across a wide range of use cases, including tokens, DApps, DAOs, and governance protocols.


### Model Features
- **Type**: Code Gen For Causal LLM
- **Tokenizer**: GPT2Tokenizer
- **Number of Parameters**: 2B
- **Number of Layers**: 32 Transformer blocks
- **Context Length**: Full 2048 tokens
- **Dtype**: bfloat16

### Model Sources
For more details, please refer to,
- **Paper [optional]:** {{ paper | default("[More Information Needed]", true)}}
- **Demo:** [Demo On Hugging Face Space](https://huggingface.co/spaces/Chain-GPT/SolidityLLMDemo)


# Model Comparison
We have compared our model with the following models
- Qwen/CodeQwen1.5-7B
- deepseek-ai/deepseek-coder-1.3b-base
- codellama/CodeLlama-7b-hf
- GPT 4o mini

On the following parameters
- Compilation(%)--Percentage of generated contracts that compile successfully without modification.
- OpenZeppelin Compliance(%)--Adherence to OpenZeppelin library usage and standards.
- Gas Efficiency(%)--Degree of gas optimization based on Slither’s suggestions.
- Security(%)--Percentage of code free from common vulnerabilities detected by Slither.

## Benchmark
The figure below presents a detailed comparison of the models across all evaluation criteria
![Benchmark](Benchmark.png)


# Uses
### Direct Use
- Assisting developers in writing Solidity smart contracts.
- Educational tool for learning Solidity.
- Auto-generating documentation or contract templates.

### Downstream Use
- Integrated into IDEs or smart contract development platforms.
- Supporting autonomous agents that interact with blockchains.

### Out-of-Scope Use
- Not suitable for general-purpose code generation in languages other than Solidity.
- Not intended for legal auditing or formal verification without human oversight.
- Should not be used to deploy contracts to production without expert review.

### Bias, Risks, and Limitations
- May reflect biases from web-scraped content (e.g., outdated or insecure coding practices).
- Model might hallucinate code or provide syntactically valid but logically incorrect suggestions.
- Risks associated with using AI-generated code in high-stakes or financial environments without thorough vetting.

### Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Manual code review and testing are strongly recommended before deployment.

# How to Get Started with the Model
The model follows a two-step generation process: it first produces a natural language description of the code, and subsequently generates the corresponding source code based on the given prompt. The complete output is generated internally before being displayed to the user. For scenarios requiring direct code generation without intermediate descriptions, a streaming mode can be utilized to produce code in real time.

Requirements for model.
```Python
pip install transformers torch accelerate
```

Use the code below to get started with the model.

```Python
from transformers import AutoModelForCausalLM, AutoTokenizer

modelpath = "ChainGPT/SolidityLLM"

tokenizer = AutoTokenizer.from_pretrained(modelpath)
model = AutoModelForCausalLM.from_pretrained(modelpath)

prompt = "Write a Solidity function to transfer tokens."
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=1400, pad_token_id=tokenizer.eos_token_id)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(generated_text)
```

## Streaming
To stream the generated code. During streaming, description is not generated.

```Python
import time
import torch
from queue import Empty
from threading import Thread
from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer

model = AutoModelForCausalLM.from_pretrained(
    "Chain-GPT/Solidity-LLM",
    torch_dtype=torch.bfloat16,
    device_map="cuda"
)
tokenizer = AutoTokenizer.from_pretrained("ChainGPT/SolidityLLM")

CodeInstruction = "Develop a Solidity Contract for lottery which requires 1 eth for registration fee and the winner gets a reward of 10 eth."
prompt = (
    f'You are given a coding instruction. Generate only the code that completes the task. Do not include any explanation or description.\n'
    f'Instruction: {CodeInstruction}\nCode:'
)

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
generation_thread = Thread(target=model.generate, kwargs={
    "input_ids": inputs["input_ids"],
    "max_new_tokens": 1800,
    "temperature": 0.7,
    "do_sample": True,
    "streamer": streamer
})
generation_thread.start()

def consume_streamer(streamer: TextIteratorStreamer):
    inside_code_block = False
    buffer = ""
    while True:
        try:
            for token in streamer:
                buffer += token

                while "```" in buffer:
                    parts = buffer.split("```", 1)
                    if not inside_code_block:
                        inside_code_block = True
                        buffer = parts[1]
                    else:
                        for char in parts[0]:
                            yield char
                        return
                if inside_code_block:
                    while buffer:
                        yield buffer[0]
                        buffer = buffer[1:]
            break
        except Empty:
            time.sleep(0.1)

for chunk in consume_streamer(streamer):
    print(chunk, end="", flush=True)
```


# Training Details

### Training Procedure
We uses the following compute for finetuning the model
- 80 GB 4 GPUs Cluster
- Training Duration: 1095 Hours which is approximately 1 and a half month

### Pre-training:
Trained on raw, unstructured data.
-1B tokens, raw corpus

### Fine-tuning:
- Contracts written in Solidity version >= 0.5.
- Token length between 200 and 4000.
- Deduplicated and filtered out interfaces, libraries, contracts with irrelevant comments, or relative imports.
- Only compilable and executable contracts retained.
- 650K instructions finetune


# Results
Prompt
```
Develop a Solidity program for a decentralized library where users can store and retrieve data after paying a small token fee.
```

Output Code
```Solidity
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

import "@openzeppelin/contracts/access/Ownable.sol";
import "@openzeppelin/contracts/token/ERC20/IERC20.sol";

contract DecentralizedLibrary is Ownable(msg.sender) {
    IERC20 public token;
    uint256 public tokenFee;
    
    struct DataEntry {
        string data;
        address owner;
    }
    
    mapping(uint256 => DataEntry) private dataStore;
    mapping(address => uint256) private userBalances;
    uint256 private dataCount;

    event DataStored(uint256 indexed dataId, address indexed owner, string data);
    event DataRetrieved(uint256 indexed dataId, address indexed owner);

    constructor(IERC20 _token, uint256 _initialTokenFee) {
        token = _token;
        tokenFee = _initialTokenFee;
    }

    function storeData(string memory _data) public {
        payTokenFee();
        dataStore[dataCount] = DataEntry(_data, msg.sender);
        emit DataStored(dataCount, msg.sender, _data);
        dataCount++;
    }

    function retrieveData(uint256 _dataId) public {
        payTokenFee();
        require(_dataId < dataCount, "Data does not exist");
        emit DataRetrieved(_dataId, msg.sender);
    }

    function payTokenFee() internal {
        require(token.balanceOf(msg.sender) >= tokenFee, "Insufficient token balance");
        token.transferFrom(msg.sender, address(this), tokenFee);
    }

    function getTokenBalance() public view returns (uint256) {
        return token.balanceOf(msg.sender);
    }

    function setTokenFee(uint256 _newTokenFee) public onlyOwner {
        tokenFee = _newTokenFee;
    }

    function withdrawTokens(uint256 _amount) public onlyOwner {
        require(token.balanceOf(address(this)) >= _amount, "Insufficient contract balance");
        token.transfer(msg.sender, _amount);
    }
}
```

# Evaluation Matrics
To evaluate the performance of our fine-tuned LLM specialized in Solidity smart contract generation, we used **[Slither](https://github.com/crytic/slither)**, a static analysis framework widely used for analyzing Solidity code.

We focused on four key evaluation criteria:

- **Compilation Success Rate**
We measured the percentage of generated smart contracts that compile successfully without modification. This helps assess the syntactic and structural correctness of the model outputs.

- **OpenZeppelin Standards Compliance**
We verified whether the generated contracts adhere to best practices by checking for proper usage of OpenZeppelin libraries. This includes ensuring the latest or stable versions of libraries are used and the overall contract structure aligns with established OpenZeppelin patterns.

- **Gas Optimization Opportunities**
Using Slither’s gas optimization analysis, we identified areas in the generated contracts where gas usage could be reduced. We measured the number and types of optimization suggestions as an indicator of how efficient the generated code is.

- **Security Vulnerabilities**
We analyzed each contract for known security vulnerabilities using Slither’s built-in detectors. We recorded the number and severity of the vulnerabilities detected, providing a measure of the security quality of the model’s outputs.

These evaluation metrics help quantify the practical usability and reliability of the generated smart contracts in real-world scenarios.


# Summary
Model shows improved understanding and generation capabilities in Solidity when compared to baseline LLMs not trained on Solidity data.