Intel/Qwen3-235B-A22B-Instruct-2507-gguf-q4km-AutoRound

Model Details

This model is gguf:q4km model of Qwen/Qwen3-235B-A22B-Instruct-2507 generated by intel/auto-round algorithm. Please refer to Section Generate the model for more details.

How To Use

LLamacpp Inference

./llama-cli -hf Intel/Qwen3-235B-A22B-Instruct-2507-gguf-q4km-AutoRound:q4_k_m --conversation

"""
> Hi
Hello! ٩(◕‿◕｡)۶ How can I assist you today?

> Code a flappy bird in python
Sure! Below is a simple implementation of **Flappy Bird** using **Python** and the **Pygame** library. This version includes:

- A bird that flaps when you press the spacebar or click.
- Pipes that move from right to left.
- Collision detection.
- Score tracking.
- Game over when the bird hits a pipe or the ground/ceiling.

---

### ✅ Requirements

Make sure you have `pygame` installed:

```bash
pip install pygame
```

---

### 🐤 Flappy Bird Code

```python
import pygame
import random
import sys

# Initialize Pygame
pygame.init()

# Screen dimensions
SCREEN_WIDTH = 400
SCREEN_HEIGHT = 600
screen = pygame.display.set_mode((SCREEN_WIDTH, SCREEN_HEIGHT))
pygame.display.set_caption("Flappy Bird")

# Colors
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)
GREEN = (0, 255, 0)
BLUE = (0, 0, 255)
SKY_BLUE = (135, 206, 235)

# Game settings
GRAVITY = 0.5
FLAP_STRENGTH = -10
PIPE_SPEED = 3
PIPE_GAP = 150
PIPE_FREQUENCY = 1500  # milliseconds

# Bird settings
bird_x = 100
bird_y = SCREEN_HEIGHT // 2
bird_velocity = 0
bird_width = 40
bird_height = 30

# Pipe list
pipes = []
last_pipe_time = 0

# Score
score = 0
font = pygame.font.SysFont('Arial', 26)

# Clock
clock = pygame.time.Clock()

# Game loop
running = True
game_active = True

def draw_bird(x, y):
    pygame.draw.rect(screen, BLUE, (x, y, bird_width, bird_height), border_radius=10)

def create_pipe():
    height = random.randint(100, SCREEN_HEIGHT - PIPE_GAP - 100)
    top_pipe = pygame.Rect(SCREEN_WIDTH, 0, 60, height)
    bottom_pipe = pygame.Rect(SCREEN_WIDTH, height + PIPE_GAP, 60, SCREEN_HEIGHT)
    return {"top": top_pipe, "bottom": bottom_pipe, "passed": False}

def draw_pipes(pipes):
    for pipe in pipes:
        pygame.draw.rect(screen, GREEN, pipe["top"])
        pygame.draw.rect(screen, GREEN, pipe["bottom"])

def move_pipes(pipes):
    for pipe in pipes:
        pipe["top"].x -= PIPE_SPEED
        pipe["bottom"].x -= PIPE_SPEED

def remove_offscreen_pipes(pipes):
    return [pipe for pipe in pipes if pipe["top"].right > 0]

def check_collision(pipes, bird_y):
    # Check ceiling or floor
    if bird_y < 0 or bird_y + bird_height > SCREEN_HEIGHT:
        return True

    # Check pipes
    bird_rect = pygame.Rect(bird_x, bird_y, bird_width, bird_height)
    for pipe in pipes:
        if bird_rect.colliderect(pipe["top"]) or bird_rect.colliderect(pipe["bottom"]):
            return True
    return False

def draw_score():
    score_text = font.render(f"Score: {score}", True, BLACK)
    screen.blit(score_text, (10, 10))

# Main game loop
while running:
    current_time = pygame.time.get_ticks()
    screen.fill(SKY_BLUE)

    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            running = False
        if event.type == pygame.KEYDOWN:
            if event.key == pygame.K_SPACE and game_active:
                bird_velocity = FLAP_STRENGTH
            if event.key == pygame.K_r and not game_active:
                # Restart game
                bird_y = SCREEN_HEIGHT // 2
                bird_velocity = 0
                pipes = []
                score = 0
                game_active = True
        if event.type == pygame.MOUSEBUTTONDOWN and game_active:
            bird_velocity = FLAP_STRENGTH

    if game_active:
        # Bird physics
        bird_velocity += GRAVITY
        bird_y += bird_velocity

        # Generate pipes
        if current_time - last_pipe_time > PIPE_FREQUENCY:
            pipes.append(create_pipe())
            last_pipe_time = current_time

        # Move and draw pipes
        move_pipes(pipes)
        pipes = remove_offscreen_pipes(pipes)
        draw_pipes(pipes)

        # Check for scoring
        for pipe in pipes:
            if pipe["top"].right < bird_x and not pipe["passed"]:
                score += 1
                pipe["passed"] = True

        # Collision check
        if check_collision(pipes, bird_y):
            game_active = False

    # Draw bird and score
    draw_bird(bird_x, bird_y)
    draw_score()

    # Game over screen
    if not game_active:
        game_over = font.render("Game Over! Press 'R' to Restart", True, BLACK)
        screen.blit(game_over, (SCREEN_WIDTH // 2 - 180, SCREEN_HEIGHT // 2))

    pygame.display.update()
    clock.tick(60)

pygame.quit()
sys.exit()
```

---

### 🎮 How to Play

- Press **Spacebar** or **click the mouse** to make the bird flap.
- Avoid hitting the pipes, ceiling, or floor.
- Your score increases each time you pass a pair of pipes.
- After game over, press **'R'** to restart.

---

### 🧩 Enhancements (Optional)

You can improve this by:
- Adding sound effects.
- Using bird/pipes images instead of rectangles.
- Adding start screen.
- Increasing difficulty over time.

Let me know if you want a version with images or sound! 🎵🐦

"""

Generate the model

auto-round>0.5.1

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from auto_round import AutoRound

model_name = "Qwen/Qwen3-235B-A22B-Instruct-2507" 

model = AutoModelForCausalLM.from_pretrained(model_name,
                                             device_map="cpu", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

autoround = AutoRound(model, tokenizer, iters=0, layer_config=layer_config, nsamples=512)
autoround.quantize_and_save("/dataset/Qwen3-235B-A22B-Instruct-2507-q4km", format="gguf:q4_k_m")

Ethical Considerations and Limitations

The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.

Therefore, before deploying any applications of the model, developers should perform safety testing.

Caveats and Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.

Here are a couple of useful links to learn more about Intel's AI software:

• Intel Neural Compressor link

Disclaimer

The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.

Cite

@article{cheng2023optimize, title={Optimize weight rounding via signed gradient descent for the quantization of llms}, author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi}, journal={arXiv preprint arXiv:2309.05516}, year={2023} }

arxiv github