  The introduction of the transformer architecture and the self-attention
mechanism has led to an explosive production of language models trained on
specific downstream tasks and data domains. With over 200, 000 models in the
Hugging Face ecosystem, users grapple with selecting and optimizing models to
suit multifaceted workflows and data domains while addressing computational,
security, and recency concerns. There is an urgent need for machine learning
frameworks that can eliminate the burden of model selection and customization
and unleash the incredible power of the vast emerging model library for end
users. Here, we propose a context-aware routing system, Tryage, that leverages
a language model router for optimal selection of expert models from a model
library based on analysis of individual input prompts. Inspired by the thalamic
router in the brain, Tryage employs a perceptive router to predict down-stream
model performance on prompts and, then, makes a routing decision using an
objective function that integrates performance predictions with user goals and
constraints that are incorporated through flags (e.g., model size, model
recency). Tryage allows users to explore a Pareto front and automatically
trade-off between task accuracy and secondary goals including minimization of
model size, recency, security, verbosity, and readability. Across heterogeneous
data sets that include code, text, clinical data, and patents, the Tryage
framework surpasses Gorilla and GPT3.5 turbo in dynamic model selection
identifying the optimal model with an accuracy of 50.9% , compared to 23.6% by
GPT 3.5 Turbo and 10.8% by Gorilla. Conceptually, Tryage demonstrates how
routing models can be applied to program and control the behavior of
multi-model LLM systems to maximize efficient use of the expanding and evolving
language model ecosystem.
