Requesting a line graph that aggregates/adds all GPU VRAM usage into one line graph. The max value would be the total amount of VRAM across all GPUs. This would allow myself and others to see the effects of deploying an LLM across multiple GPUs.
For example, if I deploy a 56GB LLM across x5 RTX 5060 Ti 16GB cards. The total VRAM would be 80GB and the used would be around 56GB. Having this in a graph could let me know how much room is left for increased context (VRAM increases) so as one increases or decreases the context size we could monitor the resulting total VRAM usage.
Another usage would be if one wanted to deploy two or three LLMs across multiple GPUs. The line graph would allow for easy monitoring of VRAM usage and the effects of adding another LLM to the mix.
Thanks for making this application, I am finding it very helpful. Keep up the great work.
Requesting a line graph that aggregates/adds all GPU VRAM usage into one line graph. The max value would be the total amount of VRAM across all GPUs. This would allow myself and others to see the effects of deploying an LLM across multiple GPUs.
For example, if I deploy a 56GB LLM across x5 RTX 5060 Ti 16GB cards. The total VRAM would be 80GB and the used would be around 56GB. Having this in a graph could let me know how much room is left for increased context (VRAM increases) so as one increases or decreases the context size we could monitor the resulting total VRAM usage.
Another usage would be if one wanted to deploy two or three LLMs across multiple GPUs. The line graph would allow for easy monitoring of VRAM usage and the effects of adding another LLM to the mix.
Thanks for making this application, I am finding it very helpful. Keep up the great work.