how to define the compute_metrics function in fine-tuning
@nferruz Hello Noelia,
Thank you for sharing ProtGPT2 with the protein design community, great work. I'm new to deep learning and only have a few experiences in prediction/classification tasks using pre-trained protein language models, e.g., the ESM-2 model. For fine-tuning, I'm using Hugging Face Trainer.
May I ask how to define the compute_metrics function in fine-tuning ProtGPT2? For the classification task, the compute_metrics function is like:
def compute_metrics(eval_pred):
metrics = load('accuracy')
predictions, labels = eval_pred
predictions = np.argmax(predictions, axis=1)
return metrics.compute(predictions=predictions, references=labels)
I know we can use run_clm.py to fine-tune. I just want to try to use Hugging Face Trainer to make it run.
Don't know if it's appropriate to ask this question. I googled it, but what I got is all about validation for text generation tasks. Any hints would be appreciated.
Sincerely,
Beibei
I’m afraid I don’t know either, but I’ll try to have a look!