Mistral Medium 3.5 on AMD Strix Halo: Painfully Slow (Plan for Overnight Runs)
Mistral Medium 3.5 在 AMD Strix Halo 上跑起來超慢,準備好熬夜吧
Someone actually tested Mistral Medium 3.5 on AMD's new Strix Halo chip, and the results are... not great. For a 48k-token prompt with 4k thinking tokens, it took about 2 hours just to get an answer about code architecture. Yeah, you read that right—two hours. The takeaway: if you want to run this locally on Strix Halo, queue it up before bed. The technical setup involved heavy optimization (Q5_K_XL quantization, GPU acceleration with -ngl 999, cache reuse), but even with all that tuning, it's still a crawl. Not exactly the "instant local AI" dream, but hey, at least it works.
Tech Blogger Take
Someone waited 2 hours for AI to answer a coding question. This is why local AI isn't ready.
Picture this: you ask your local AI a complex coding question, hit enter, and then... wait. For two hours. That's exactly what happened when someone tested Mistral Medium 3.5 on AMD's shiny new Strix Halo chip. We're talking about a 48k-token prompt with 4k thinking tokens — basically asking the AI to really think through a code architecture problem. Even with every optimization trick in the book (Q5_K_XL quantization, maxed-out GPU acceleration, cache reuse), it still crawled along like dial-up internet. This isn't some budget setup either — Strix Halo is AMD's flagship AI chip. The brutal reality? While everyone's hyping 'AI on your laptop,' we're still light-years away from the instant, ChatGPT-like experience people expect. Sure, it works, but 'working' and 'practical' are two very different things.
VerdictQueue up your complex AI tasks before bed and pray your laptop doesn't crash overnight — local AI is still a patience game.
7/10
AI Analysis
Hardware Development
high
Action Required
Start planning AI workload benchmarks for your next chip designs — consumer expectations are shifting from 'can it run' to 'how fast can it run'
Key Insight
Even AMD's flagship Strix Halo with all the optimization tricks still needs 2 hours for what cloud APIs do in seconds — the local AI performance gap is massive
Why It Matters
Your customers are about to discover that 'AI-capable hardware' and 'AI-practical hardware' are completely different things
Job Impact Analysis
AI Engineers
Role Shift
Why It Impacts
Local AI inference is proving painfully slow even on premium hardware, forcing a complete rethink of deployment strategies
How to Adapt
Start building hybrid workflows now — queue long inference jobs locally overnight, keep interactive work in the cloud
Product Managers
At Risk
Why It Impacts
The 2-hour wait time for complex AI tasks on local hardware kills any real-time product experience dreams
How to Adapt
Redesign your AI features around batch processing and overnight workflows, not instant responses
AMD's flagship AI-focused processor chip that this painful 2-hour test exposed as still being way too slow for real-time local AI work.
Q5_K_XL Quantization(Q5_K_XL量化)
A compression method that reduces AI model size to fit on consumer hardware — one of the optimization tricks used in this test that still couldn't save it from the 2-hour crawl.
Thinking Tokens(思考令牌)
The 4k tokens mentioned in this test that represent the AI's internal reasoning process before giving its final answer — basically the AI 'thinking out loud' internally.
Local Inference(本地推論)
Running AI models on your own hardware instead of cloud services — what this 2-hour nightmare demonstrates is still painfully slow compared to cloud APIs.
Related Articles
OpinionsRead
OpenAI Livestream
OpenAI is hosting a livestream event. Details about the specific announcements, product launches, or demonstrations will be revealed during the broadcast.
The last time OpenAI did an unannounced livestream, they dropped GPT-4 Turbo and changed pricing overnight
OpinionsRead
ChatGPT Images 2.0
OpenAI is launching ChatGPT Images 2.0 with major upgrades to image generation capabilities. Watch the livestream announcement at https://openai.com/live/
OpenAI is positioning this as a direct competitor to established image generation tools, suggesting they're confident enough to challenge the current market leaders
OpinionsRead
The "just wait 6 months" argument from 2025 survived exactly one iteration
Throughout 2025 the standard response to any complaint about an LLM was some version of "just wait 3-6 months, the next generation will handle this effortlessly." The argument was everywhere. Every limitation was temporary, every missing capability was a few iterations away, every autonomous agent demo was a preview of imminent reality.
It's April 2026 now and worth checking how that held up.
On r/ClaudeAI this week there's a long thread about Opus 4.7 where multiple users argue it's a regress
OpinionsRead
Anthropic investigates unauthorized access claims against Claude Mythos AI
Anthropic is looking into allegations that Claude Mythos AI experienced unauthorized access. The investigation aims to determine the scope and nature of the potential security breach affecting the AI system.
Tech Blogger Take
Someone waited 2 hours for AI to answer a coding question. This is why local AI isn't ready.
Picture this: you ask your local AI a complex coding question, hit enter, and then... wait. For two hours. That's exactly what happened when someone tested Mistral Medium 3.5 on AMD's shiny new Strix Halo chip. We're talking about a 48k-token prompt with 4k thinking tokens — basically asking the AI to really think through a code architecture problem. Even with every optimization trick in the book (Q5_K_XL quantization, maxed-out GPU acceleration, cache reuse), it still crawled along like dial-up internet. This isn't some budget setup either — Strix Halo is AMD's flagship AI chip. The brutal reality? While everyone's hyping 'AI on your laptop,' we're still light-years away from the instant, ChatGPT-like experience people expect. Sure, it works, but 'working' and 'practical' are two very different things.
AI Analysis
Hardware Development
highStart planning AI workload benchmarks for your next chip designs — consumer expectations are shifting from 'can it run' to 'how fast can it run'
Even AMD's flagship Strix Halo with all the optimization tricks still needs 2 hours for what cloud APIs do in seconds — the local AI performance gap is massive
Your customers are about to discover that 'AI-capable hardware' and 'AI-practical hardware' are completely different things
Job Impact Analysis
AI Engineers
Role ShiftLocal AI inference is proving painfully slow even on premium hardware, forcing a complete rethink of deployment strategies
Start building hybrid workflows now — queue long inference jobs locally overnight, keep interactive work in the cloud
Product Managers
At RiskThe 2-hour wait time for complex AI tasks on local hardware kills any real-time product experience dreams
Redesign your AI features around batch processing and overnight workflows, not instant responses