Chain-GPT
/

Solidity-LLM

@@ -61,6 +61,8 @@ On the following parameters
 - OpenZeppelin Compliance(%)--Adherence to OpenZeppelin library usage and standards.
 - Gas Efficiency(%)--Degree of gas optimization based on Slither’s suggestions.
 - Security(%)--Percentage of code free from common vulnerabilities detected by Slither.
 ## Benchmark
 Below is a figure summarizing the performance of each model across the four evaluation metrics.
@@ -275,7 +277,7 @@ contract DecentralizedLibrary is Ownable(msg.sender) {
 # Evaluation Matrics
 To evaluate the performance of our fine-tuned LLM specialized in Solidity smart contract generation, we used **[Slither](https://github.com/crytic/slither)**, a static analysis framework widely used for analyzing Solidity code.
-We focused on four key evaluation criteria:
 - **Compilation Success Rate**
 We measured the percentage of generated smart contracts that compile successfully without modification. This helps assess the syntactic and structural correctness of the model outputs.
@@ -289,8 +291,16 @@ Using Slither’s gas optimization analysis, we identified areas in the generate
 - **Security Vulnerabilities**
 We analyzed each contract for known security vulnerabilities using Slither’s built-in detectors. We recorded the number and severity of the vulnerabilities detected, providing a measure of the security quality of the model’s outputs.
 These evaluation metrics help quantify the practical usability and reliability of the generated smart contracts in real-world scenarios.
 # Summary
 Model shows improved understanding and generation capabilities in Solidity when compared to baseline LLMs not trained on Solidity data.

 - OpenZeppelin Compliance(%)--Adherence to OpenZeppelin library usage and standards.
 - Gas Efficiency(%)--Degree of gas optimization based on Slither’s suggestions.
 - Security(%)--Percentage of code free from common vulnerabilities detected by Slither.
+- Average Lines of Code--How lengthy, complete,
+- Correctness of Code--
 ## Benchmark
 Below is a figure summarizing the performance of each model across the four evaluation metrics.
 # Evaluation Matrics
 To evaluate the performance of our fine-tuned LLM specialized in Solidity smart contract generation, we used **[Slither](https://github.com/crytic/slither)**, a static analysis framework widely used for analyzing Solidity code.
+We focused on six key evaluation criteria:
 - **Compilation Success Rate**
 We measured the percentage of generated smart contracts that compile successfully without modification. This helps assess the syntactic and structural correctness of the model outputs.
 - **Security Vulnerabilities**
 We analyzed each contract for known security vulnerabilities using Slither’s built-in detectors. We recorded the number and severity of the vulnerabilities detected, providing a measure of the security quality of the model’s outputs.
+- **Average Lines of Code**
+This metric provides insight into the verbosity or conciseness of the model’s output. Higher LOC may suggest redundancy or complete code, while lower LOC could indicate either efficiency or missing implementation details, depending on context.
+- **Correctness of Code**
+To assess how well the generated code aligns with the given prompt and category, We conducted both manual and OpenAI LLM evaluation of each generated contract. The prompt and the generated code were keenly observed for alignment analysis.
 These evaluation metrics help quantify the practical usability and reliability of the generated smart contracts in real-world scenarios.
 # Summary
 Model shows improved understanding and generation capabilities in Solidity when compared to baseline LLMs not trained on Solidity data.