下に引いて戻る
Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB

Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB

Mistral Medium 3.5 128B and Qwen 3.5 122B A10B on 4x RTX 3080 20GB

Mistral Medium 3.5 128B with 4x3080 20GB with layer split: CUDA_VISIBLE_DEVICES=0,1,2,3 ./build/bin/llama-bench --model /data/huggingface/Mistral-Medium-3.5-GGUF/Mistral-Medium-3.5-128B-IQ4_XS-00001-of-00003. gguf -ngl 99 -d 0,16384 -fa 1 --split-mode layer ggml_cuda_init: found 4 CUDA devices (Total VRAM: 80211 MiB): Device 0: NVIDIA GeForce RTX 3080, compute capability 8.6, VMM: yes, VRAM: 20052 MiB Device 1: NVIDIA GeForce RTX 3080, compute capability 8.6, VMM: yes, VRAM: 20052 MiB Device 2: