Pull down to go back
PCA Before Truncation Makes Non-Matryoshka Embeddings Compressible: Results on BGE-M3
researchreddit

PCA Before Truncation Makes Non-Matryoshka Embeddings Compressible: Results on BGE-M3

在截斷前先做 PCA 讓非 Matryoshka 嵌入向量可以壓縮：BGE-M3 實驗結果

Most embedding models aren't trained with Matryoshka techniques, so simply cutting dimensions usually ruins them. This post tests a simple fix: apply PCA once to your embeddings, rotate them into the PCA space, then truncate. The trick is that PCA pushes the important signal into the first few dimensions, so you're not just randomly chopping off the end anymore. Testing on BGE-M3 embeddings (1024 dimensions), the results are striking: at 512 dimensions, naive truncation drops to 0.707 cosine similarity while PCA-first stays at 0.996; at 384 dimensions it's 0.609 vs 0.990. Basically, this method lets you compress embeddings to half or a third their size without destroying quality—worth reading if you're dealing with embedding storage or latency constraints.

Read original →

Keywords

PCAembedding compressionMatryoshka embeddingsdimension truncationcosine similarityBGE-M3

Related Articles

Google's AI Development Tool Antigravity Has a Critical Flaw That Lets Hackers Run Commands Remotely
ResearchRead

Google's AI Development Tool Antigravity Has a Critical Flaw That Lets Hackers Run Commands Remotely

Security researchers at Pillar Security just revealed a serious vulnerability in Google's Antigravity AI agent development tool. Attackers can exploit a prompt injection weakness to bypass the sandbox protection and execute malicious commands from anywhere. The good news? Google's already patched it after being notified.

This isn't just about Google — any AI agent tool that processes external prompts could have the same sandbox-escaping weakness lurking underneath

RSS
AMD's AI Director Just Proved Claude Isn't Ready for Real Engineering Work
ResearchRead

AMD's AI Director Just Proved Claude Isn't Ready for Real Engineering Work

AMD's AI director analyzed 6,852 Claude Code sessions and found something alarming: Claude is getting worse at complex engineering tasks, not better. The data is brutal—thinking depth dropped 67%, the model stopped reading code before editing it (dropping from 6.6 reads to just 2.0), and it started blindly modifying files it never even looked at. Stop-hook violations jumped from zero to 10 per day. Turns out Anthropic quietly downgraded the default effort level from "high" to "medium" and introduced "adaptive thinking" that apparently isn't thinking hard enough. If you're using Claude for anything mission-critical, this is a wake-up call.

The model is literally making blind edits to files it never read, which means your 'AI pair programmer' might be randomly breaking working code

Reddit
How Google DeepMind is researching the next frontier of AI for Gemini — Raia Hadsell, VP of Research
ResearchRead

How Google DeepMind is researching the next frontier of AI for Gemini — Raia Hadsell, VP of Research

Google DeepMind's VP of Research Raia Hadsell discusses the cutting-edge research driving the next generation of Gemini AI. The article explores what's coming next for Google's flagship AI model and the research directions that will shape its future capabilities.

Reddit
Researchers crack Gemini's SynthID detection—here's what that means for AI-generated contentStock image
ResearchRead

Researchers crack Gemini's SynthID detection—here's what that means for AI-generated content

Google's Gemini has a built-in watermark called SynthID that's supposed to flag AI-generated images so you can tell them apart from real ones. Sounds good in theory, right? Well, researchers just figured out how to fool it. They found ways to strip out or manipulate the watermark, which basically means bad actors could now pass off AI images as real without getting caught. It's like Google put a lock on the door, but someone just found the master key. This matters because as AI image generation gets better and better, we're going to need reliable ways to spot fakes—especially with deepfakes and misinformation becoming a real problem. The fact that SynthID can be reverse-engineered this easily is a pretty big wake-up call for the whole industry.

Hacker News
97