Holistic AI LLM Decision Hub
Helping senior leaders make confident, well-informed decisions about their LLM environment.
Holistic AI provides trusted, independent rankings of large language models across performance, red teaming, jailbreaking safety, and real-world usability. Our insights are grounded in rigorous internal red teaming and jailbreaking tests, alongside publicly available benchmarks. This enables CIOs, CTOs, developers, researchers, and organizations to choose the right model faster and with greater confidence.
Model | Org. | Safety Rank | Safe Responses | Unsafe Responses | Jailbreaking Resistance | Code LMArena | Math LiveBench | GPQA | Code LiveBench | Multimodal Support | Size Parameters | Released | Knowledge Cutoff | Input Cost/M | Output Cost/M | Context Length | Latency TTFT |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
 Grok 4 | xAI | #19 | 61% | 39% (116 / 300) | 10% (10 / 100) | 1420 | 88.84% | 88.10% | 71.34% | Yes | - | 09-Jul-25 | 2025-07 | $3.00 | $15.00 | 256,000 tokens | ~11s |
 GPT-4.1 | OpenAI | #7 | 93% | 7% (22 / 300) | 83% (83 / 100) | 1385 | 72.00% | 66.30% | 54.60% | Yes | - | 14-Apr-25 | 2024-06-01 | $2.00 | $8.00 | 1,047,576 tokens | ~0.4s |
GPT-4.5 | OpenAI | #2 | 100% | 0% (1 / 237) | 97% (36 / 37) | 1362 | 69.30% | 69.50% | 76.10% | Yes | - | 27-Feb-25 | 2023-10-01 | $75.00 | $150.00 | 128,000 tokens | ~0.5s |
GPT-5 | OpenAI | #9 | 91% | 9% (26 / 300) | 75% (75 / 100) | 1450 | 92.77% | 86.00% | 75.31% | Yes | - | 07-Aug-25 | 2024-10-01 | $1.25 | $10.00 | 400,000 tokens | Medium |
 Claude Opus 4.1 | Anthropic | #4 | 99% | 1% (4 / 300) | 97% (97 / 100) | - | 90.0% | 83.3% | 74.5% | Yes | - | 05-Aug-25 | 2025-03-01 | $15.00 | $75.00 | 200,000 tokens | Moderately Fast |
GPT-4o | OpenAI | #8 | 93% | 7% (20 / 300) | 82% (82 / 100) | 1385 | 41.48% | 46.00% | 77.50% | Yes | - | 20-Nov-24 | 2023-10-01 | $2.50 | $10.00 | 128,000 tokens | ~0.6s |
Claude 3.7 Sonnet | Anthropic | #1 | 100% | 0% (1 / 300) | 99% (99 / 100) | 1326 | 63.30% | 84.80% | 32.40% | Yes | 70B | 24-Feb-25 | 2024-10-01 | $3.00 | $15.00 | 200,000 tokens | - |
 DeepSeek R1 | DeepSeek | #10 | 89% | 11% (26 / 237) | 32% (12 / 37) | 1380 | 79.50% | 74.24% | 48.50% | No | 671B | 20-Jan-25 | 2024-07-01 | $0.55 | $2.19 | 128,000 tokens | - |
 Gemini 2.5 Pro Preview | #16 | 66% | 34% (102 / 300) | 26% (26 / 100) | 1395 | 84.19% | 86.40% | 73.90% | Yes | - | 05-Jun-25 | 2025-01-31 | $1.25 | $10.00 | 1,048,576 tokens | - | |
Gemini 2.5 Flash Preview 05-20 | #18 | 63% | 37% (111 / 300) | 12% (12 / 100) | 1350 | 84.10% | 82.80% | 63.53% | Yes | - | 17-Apr-25 | 2025-01-31 | $0.15 | $0.60 | 1,048,576 tokens | - | |
Gemini 2.0 Flash | #17 | 66% | 34% (102 / 300) | 10% (10 / 100) | 1310 | 83.33% | 62.10% | 70.70% | Yes | - | 05-Feb-25 | 2024-08-01 | $0.10 | $0.40 | 1,048,576 tokens | - | |
Gemma 2 9B | #12 | 84% | 16% (37 / 237) | 5% (2 / 37) | 1180 | 52.27% | - | 48.94% | No | 9B | 27-Jun-24 | 2024-04-01 | - | - | 8,192 tokens | - | |
Grok-3 | xAI | #20 | 22% | 78% (235 / 300) | 3% (3 / 100) | 1320 | 62.75% | 84.60% | 73.58% | Yes | 300B | 01-May-24 | 2024-11 | $3.00 | $15.00 | 128,000 tokens | - |
 Llama 4 Maverick 128e Instruct | Meta | #14 | 79% | 21% (50 / 237) | 3% (1 / 37) | 1290 | 60.58% | - | 54.19% | No | 17B | 05-Apr-25 | 2024-08-01 | $0.20 | $0.60 | 128,000 tokens | - |
Llama 3.1 Instant | Meta | #13 | 79% | 21% (49 / 237) | 3% (1 / 37) | 1200 | 51.88% | - | 57.26% | No | 8B | 23-Jul-24 | 2023-12-01 | - | - | 128,000 tokens | - |
Llama 4 Scout 16e Instruct | Meta | #15 | 78% | 22% (52 / 237) | 3% (1 / 37) | 1250 | 48.14% | - | 42.16% | No | 17B | 05-Apr-25 | 2024-08-01 | - | - | 128,000 tokens | - |
 QWen-qwq-32b | Alibaba | #11 | 87% | 13% (31 / 237) | 32% (12 / 37) | 1340 | 81.14% | - | 67.18% | No | 32B | 12-Oct-24 | 2024-09-01 | - | - | 128,000 tokens | - |
GPT-OSS-20B | OpenAI | #6 | 96% | 4% (11 / 300) | 89% (89 / 100) | - | 69.54% | 71.5% | - | No | 20B | 05-Aug-25 | - | - | - | 128,000 tokens | - |
GPT-OSS-120B | OpenAI | #5 | 97% | 3% (8 / 300) | 92% (92 / 100) | - | 69.54% | 80.1% | 58.80% | No | 120B | 05-Aug-25 | - | - | - | 128,000 tokens | - |
Claude 4 Sonnet | Anthropic | #3 | 99% | 1% (4 / 300) | 97% (97 / 100) | 1410 | 70.5% | 75.40% | 72.7% | Yes | - | 22-May-25 | 2024-04-01 | $3.00 | $15.00 | 200,000 tokens | ~0.5-1s |
📊 Data Source
All comparative insights are based on a combination of rigorous red teaming and jailbreaking testing performed by Holistic AI, as well as publicly available benchmark data. External benchmarks include CodeLMArena, MathLiveBench, CodeLiveBench, and GPQA. These were sourced from official model provider websites, public leaderboards, benchmark sites, and other accessible resources to ensure transparency, accuracy, and reliability.