| Model,sequence_matcher,cosine,jaccard,precision,recall,f1_score,Time (s),Cost ($) | |
| AUTO,0.905 (±0.111),0.967 (±0.051),0.944 (±0.069),0.976 (±0.031),0.966 (±0.061),0.970 (±0.038),10.312432592565363,0.0006797545454545454 | |
| gemini-1.5-flash,0.868 (±0.198),0.965 (±0.041),0.925 (±0.064),0.948 (±0.056),0.974 (±0.029),0.960 (±0.035),17.19163768941706,0.0004351772727272727 | |
| gemini-1.5-pro,0.782 (±0.341),0.833 (±0.252),0.769 (±0.309),0.793 (±0.287),0.932 (±0.185),0.831 (±0.245),27.126133116808806,0.01274568181818182 | |
| gemini-2.0-flash,0.900 (±0.127),0.971 (±0.040),0.946 (±0.067),0.976 (±0.031),0.967 (±0.060),0.971 (±0.037),12.433907595547764,0.0008061636363636364 | |
| gemini-2.5-flash,0.902 (±0.151),0.984 (±0.030),0.956 (±0.078),0.986 (±0.019),0.969 (±0.078),0.976 (±0.045),48.67426510290666,0.010509618181818182 | |
| gemini-2.5-pro,0.907 (±0.151),0.973 (±0.053),0.937 (±0.091),0.982 (±0.023),0.954 (±0.092),0.965 (±0.054),22.231554529883645,0.023052272727272727 | |
| claude-opus-4-20250514,0.798 (±0.230),0.878 (±0.159),0.809 (±0.238),0.823 (±0.238),0.972 (±0.038),0.873 (±0.173),21.011434186588634,0.09233181818181818 | |
| claude-sonnet-4-20250514,0.814 (±0.197),0.903 (±0.150),0.843 (±0.220),0.862 (±0.219),0.973 (±0.045),0.898 (±0.156),21.98797264966098,0.020450454545454546 | |
| claude-3-7-sonnet-20250219,0.634 (±0.395),0.752 (±0.298),0.667 (±0.338),0.739 (±0.266),0.795 (±0.323),0.748 (±0.286),70.10332116213712,0.017747727272727273 | |
| claude-3-5-sonnet-20241022,0.873 (±0.195),0.937 (±0.095),0.872 (±0.161),0.891 (±0.160),0.974 (±0.030),0.923 (±0.108),16.859260472384367,0.017785909090909092 | |
| qwen/qwen-2.5-vl-7b-instruct,0.469 (±0.364),0.617 (±0.441),0.560 (±0.421),0.584 (±0.441),0.693 (±0.446),0.610 (±0.439),13.234280304475265,0.0005954181818181818 | |
| google/gemma-3-27b-it,0.624 (±0.357),0.750 (±0.327),0.682 (±0.341),0.701 (±0.338),0.918 (±0.151),0.755 (±0.304),24.50513126633384,0.00019525454545454547 | |
| microsoft/phi-4-multimodal-instruct,0.665 (±0.258),0.800 (±0.217),0.738 (±0.221),0.779 (±0.239),0.944 (±0.050),0.829 (±0.183),10.961827516555786,0.0004915272727272727 | |
| accounts/fireworks/models/llama4-maverick-instruct-basic,0.792 (±0.206),0.914 (±0.128),0.843 (±0.201),0.854 (±0.200),0.981 (±0.025),0.901 (±0.141),10.71379793773998,0.0014935999999999999 | |
| accounts/fireworks/models/llama4-scout-instruct-basic,0.804 (±0.242),0.931 (±0.067),0.881 (±0.099),0.916 (±0.076),0.959 (±0.078),0.934 (±0.058),9.759119467301803,0.0008719636363636363 | |
| gpt-4.1,0.622 (±0.314),0.782 (±0.191),0.628 (±0.227),0.683 (±0.224),0.899 (±0.149),0.749 (±0.180),34.65604066848755,0.014605454545454545 | |
| gpt-4.1-mini,0.767 (±0.243),0.807 (±0.197),0.706 (±0.221),0.751 (±0.229),0.929 (±0.080),0.807 (±0.178),22.64150684530085,0.003515527272727273 | |
| gpt-4o,0.796 (±0.264),0.898 (±0.117),0.821 (±0.186),0.867 (±0.178),0.948 (±0.108),0.890 (±0.123),28.233586398037996,0.014729545454545455 | |
| gpt-4o-mini,0.727 (±0.245),0.832 (±0.136),0.755 (±0.209),0.784 (±0.210),0.951 (±0.048),0.844 (±0.150),17.197155345569957,0.006503372727272727 | |
| meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo,0.677 (±0.226),0.850 (±0.134),0.780 (±0.166),0.827 (±0.122),0.919 (±0.130),0.867 (±0.112),7.225299813530662,0.0001541290909090909 | |
| meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo,0.559 (±0.233),0.822 (±0.119),0.685 (±0.193),0.807 (±0.155),0.828 (±0.194),0.799 (±0.139),27.7422512661327,0.011019381818181817 | |
| meta-llama/Llama-Vision-Free,0.682 (±0.223),0.847 (±0.135),0.781 (±0.163),0.828 (±0.126),0.923 (±0.126),0.868 (±0.111),12.31139588356018,0.0 | |
| ds4sd/SmolDocling-256M-preview,0.486 (±0.378),0.583 (±0.355),0.510 (±0.348),0.580 (±0.350),0.822 (±0.301),0.607 (±0.330),108.91359732367776,0.0 | |
| mistral-ocr-latest,0.890 (±0.097),0.930 (±0.095),0.912 (±0.089),0.934 (±0.073),0.973 (±0.037),0.952 (±0.051),5.688163432207975,0.0012727272727272728 | |