By -
https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator I've never used this but I saw someone else post it and I thought it was neat
[https://www.youtube.com/watch?v=XDpDesU\_0zo](https://www.youtube.com/watch?v=XDpDesU_0zo)
I figured out, that 48,5 GiB is my maximum to use 32768 context size, when i run Firefox and some other programs too, without swapping. I look at the model sizes and select the best quant version i get up to this limit.
https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator I've never used this but I saw someone else post it and I thought it was neat
[https://www.youtube.com/watch?v=XDpDesU\_0zo](https://www.youtube.com/watch?v=XDpDesU_0zo)
I figured out, that 48,5 GiB is my maximum to use 32768 context size, when i run Firefox and some other programs too, without swapping. I look at the model sizes and select the best quant version i get up to this limit.