Request Details on GPU ad memory requirements
#16
by
DragoZatch
- opened
I would i check on if anyone have tried to run the model on the GPU's and find out how much GPU memory is required for this model. Also wanted to know the max memory requirement for full scale and full context length support .
You can check the minimum deployment requirements on our GitHub.
This comment has been hidden (marked as Off-Topic)
I'm testing a distributed cluster to run this full-weights on consumer cards (pooling 4090s) to bypass the VRAM limit. let me know if you want to run a test job.
Hi @YYYAMS That could be helpful if you share any example test runs where you were able to load the model on to the cluster you have.
I am looking into realisting example of model load case.It will be a greate help if anyone has run this model could share the details.