Pull down to go back
Run Qwen3.5-397B-A13B with vLLM and 8xR9700
opensourcereddit

Run Qwen3.5-397B-A13B with vLLM and 8xR9700

用 vLLM 和 8 張 R9700 GPU 跑 Qwen3.5-397B-A13B

A guide to running the massive 397B Qwen3.5 model on AMD R9700 GPUs using vLLM. Thanks to u/Sea-Speaker1700 for enabling mxfp4 support on R9700, making it possible to run even 122B models efficiently. The 397B model reportedly runs amazingly fast with this setup. Includes a Dockerfile based on the vllm-rocm-rdna4-mxfp4 image for easy deployment.

Read original →

Keywords

Qwen3.5-397BvLLMR9700mxfp4quantizationlocal inferenceopen source

Related Articles

TRELLIS.2 image-to-3D now runs on Mac (Apple Silicon) - no NVIDIA GPU needed
Open SourceRead

TRELLIS.2 image-to-3D now runs on Mac (Apple Silicon) - no NVIDIA GPU needed

I ported Microsoft's TRELLIS.2 to run on Apple Silicon via PyTorch MPS. The original depends on five CUDA-only compiled extensions (flex_gemm, flash_attn, o_voxel, cumesh, nvdiffrast) that have no Mac equivalent. Wrote replacement backends from scratch: Pure-PyTorch sparse 3D convolution (replacing flex_gemm), Python mesh extraction using spatial hashing (replacing CUDA hashmap ops in o_voxel), SDPA attention for sparse transformers (replacing flash_attn), and GPU-accelerated trilinear interpolation (replacing cumesh and nvdiffrast).

Reddit
Show HN: Run TRELLIS.2 Image-to-3D generation natively on Apple Silicon
Open SourceRead

Show HN: Run TRELLIS.2 Image-to-3D generation natively on Apple Silicon

I ported Microsoft's TRELLIS.2 (4B parameter image-to-3D model) to run on Apple Silicon via PyTorch MPS. The original requires CUDA with flash_attn, nvdiffrast, and custom sparse convolution kernels: none of which work on Mac. I replaced the CUDA-specific ops with pure-PyTorch alternatives: a gather-scatter sparse 3D convolution, SDPA attention for sparse transformers, and a Python-based mesh extraction replacing CUDA hashmap operations. Total changes are a few hundred lines across 9 files.

Hacker News
157
Which Qwen 3.6 Quantization Works Best for M5 Pro with 24GB RAM?
Open SourceRead

Which Qwen 3.6 Quantization Works Best for M5 Pro with 24GB RAM?

A user with an M5 Pro machine featuring 24GB of RAM is trying to figure out the best quantization level for running Qwen 3.6 via Ollama. They're unsure whether to go with Q4 (4-bit quantization) but can't find a solid Q3 (3-bit) alternative. They're looking for recommendations on which version would run smoothly on their setup without compromising too much on quality.

Reddit