Model Stock: All we need is just a few fine-tuned models
Paper • 2403.19522 • Published • 15
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Undi95/Miqu-MS-70B-GGUF:Q4_0# Run inference directly in the terminal:
llama cli -hf Undi95/Miqu-MS-70B-GGUF:Q4_0Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Undi95/Miqu-MS-70B-GGUF:Q4_0# Run inference directly in the terminal:
./llama-cli -hf Undi95/Miqu-MS-70B-GGUF:Q4_0Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Undi95/Miqu-MS-70B-GGUF:Q4_0# Run inference directly in the terminal:
./build/bin/llama-cli -hf Undi95/Miqu-MS-70B-GGUF:Q4_0Use Docker
docker model run hf.co/Undi95/Miqu-MS-70B-GGUF:Q4_0Quick Links
Miqu-MS-70B
This is a merge of pre-trained language models created using mergekit.
The new MODEL STOCK merge method was used, see below for more information!
Feedback on this model is greatly appreciated! I hope this new merge method will be able to fill some hole Miqu have.
Prompt format
Since it was made with model using different prompt format, the following should work.
Alpaca
### Instruction:
{system prompt}
### Input:
{prompt}
### Response:
{output}
Mistral
[INST] {prompt} [/INST]
Vicuna
SYSTEM: <ANY SYSTEM CONTEXT>
USER:
ASSISTANT:
Merge Details
Merge Method
This model was merged using the Model Stock merge method using 152334H/miqu-1-70b-sf as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: NeverSleep/MiquMaid-v2-70B
- model: sophosympatheia/Midnight-Miqu-70B-v1.0
- model: migtissera/Tess-70B-v1.6
- model: 152334H/miqu-1-70b-sf
merge_method: model_stock
base_model: 152334H/miqu-1-70b-sf
dtype: bfloat16
Support
If you want to support me, you can here.
- Downloads last month
- 46
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Undi95/Miqu-MS-70B-GGUF
Merge model
this model
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf Undi95/Miqu-MS-70B-GGUF:Q4_0# Run inference directly in the terminal: llama cli -hf Undi95/Miqu-MS-70B-GGUF:Q4_0