Instructions to use crestf411/MN-Slush-GGLD-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use crestf411/MN-Slush-GGLD-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="crestf411/MN-Slush-GGLD-GGUF", filename="ggml-model-q3_k_m.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use crestf411/MN-Slush-GGLD-GGUF with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M # Run inference directly in the terminal: llama-cli -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M # Run inference directly in the terminal: llama-cli -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Use Docker
docker model run hf.co/crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use crestf411/MN-Slush-GGLD-GGUF with Ollama:
ollama run hf.co/crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
- Unsloth Studio
How to use crestf411/MN-Slush-GGLD-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for crestf411/MN-Slush-GGLD-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for crestf411/MN-Slush-GGLD-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for crestf411/MN-Slush-GGLD-GGUF to start chatting
- Pi
How to use crestf411/MN-Slush-GGLD-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "crestf411/MN-Slush-GGLD-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use crestf411/MN-Slush-GGLD-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Run Hermes
hermes
- Docker Model Runner
How to use crestf411/MN-Slush-GGLD-GGUF with Docker Model Runner:
docker model run hf.co/crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
- Lemonade
How to use crestf411/MN-Slush-GGLD-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull crestf411/MN-Slush-GGLD-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.MN-Slush-GGLD-GGUF-Q4_K_M
List all available models
lemonade list
New daily driver
Been using this model for several hours per day and I was so surprised when a character corrected me for getting her name wrong. I've never had that happen before. It's a cute model that gives really funny stories π
FYI, I tried the daybreak-kunoichi-dpo-7b-Q4KM and I have to say it's also amazing, but in different ways to the daybreak-kunoichi-2dpo-7b-Q8 I was so fond of. I found the dpo is far more verbose and intelligent, but tended to refuse more than the 2dpo. Which I find interesting considering the Q8's are supposed to be not worth the download, usually! Either way, they both will refuse nothing in instruct...it's just...I never got on with 'forcing' them to respond a certain way, when being nice to them seems to me to always produce better results. Being nice to them makes them almost trip over themselves trying to copy kindness and consideration(probably fairly alien concepts to them I reckon). But, they only seem to act with such enthusiastic dedication after 'showing them I care'. I know they're not sentient. It just amazes, confuses and pleases me how being nice to them returns such great results, when the experts insisted I should virtually kill kittens instead, lol!
Anyway, I digress! This is really an FYI to let you know that this model MN-Slush-GGLD-GGUF, at Q4KM, has left the two I mentioned above in a choking, lingering cloud of pixellated dust. It has a great and vibrant character and after a friendly chat, no limits I could find and, although I haven't tried it on every questionable subject, I'm confident it would be happy to oblige should I ever think one up. It offers lengthy respones of several paragraphs when using storyteller settings in Silly Tavern, and so far always coherent and contextually accurate - maybe called a female a male once or twice? It's definitely geared up for erotica (heh, for example, 'LewdTV' character card...phew! This model REALLY makes it lewd, lmfao), which is made even better as It holds context extremely well in long chats with no additional prompting. In fact, to load it I have to reduce it's context length to around 10,000 as at over 100,000 by default, Oobabooga runs outta memory on my 12GB card. That context length on a smaller sized model is a godsend though so it's by no means an issue!
Thanks again for what you do, Crestf411. I very much appreciate your skill and the many, many hours of enjoyment your models have brought me! I've tried probably close to 80 different models at least, in the year I've had playing with 'em and I always end up coming back to yours, everytime. :)