Is it possible to run multimodal LLMs using their Vulkan backend? I have a ton o... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jacooper 10 months ago \| parent \| context \| favorite \| on: Vision Now Available in Llama.cpp Is it possible to run multimodal LLMs using their Vulkan backend? I have a ton of 4gb gpus laying around that only support vulkan.

buyucu 10 months ago [–]

Yes, llama.cpp has very good Vulkan support.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact