File size: 792 Bytes
778cecf 1f23060 3a858f3 5a390e2 3a858f3 1f23060 5f7e30a 778cecf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
---
language:
- en
base_model:
- mistralai/Codestral-22B-v0.1
pipeline_tag: text-generation
tags:
- code
- code-generation
---
This repository contains AWS Inferentia2 and neuronx compatible checkpoints for [Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1). You can find detailed information about the base model on its [Model Card](https://huggingface.co/mistralai/Codestral-22B-v0.1).
This model has been exported to the neuron format using specific input_shapes and compiler parameters detailed in the paragraphs below.
It has been compiled to run on an inf2.24xlarge instance on AWS. Note that while the inf2.24xlarge has 12 cores, this compilation uses 12.
- SEQUENCE_LENGTH = 4096
- BATCH_SIZE = 4
- NUM_CORES = 12
- PRECISION = "bf16"
---
license: mnpl
--- |