Instructions to use cstr/Spaetzle-v31-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cstr/Spaetzle-v31-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cstr/Spaetzle-v31-7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cstr/Spaetzle-v31-7b")
model = AutoModelForCausalLM.from_pretrained("cstr/Spaetzle-v31-7b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cstr/Spaetzle-v31-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cstr/Spaetzle-v31-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cstr/Spaetzle-v31-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cstr/Spaetzle-v31-7b

SGLang

How to use cstr/Spaetzle-v31-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cstr/Spaetzle-v31-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cstr/Spaetzle-v31-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cstr/Spaetzle-v31-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cstr/Spaetzle-v31-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cstr/Spaetzle-v31-7b with Docker Model Runner:
```
docker model run hf.co/cstr/Spaetzle-v31-7b
```

cstr commited on Aug 2

Commit

bff6e4e

verified ·

1 Parent(s): 9d379dc

docs: resolve licence to cc-by-nc-4.0 from constituent terms

Browse files

Files changed (1) hide show

README.md +8 -1

README.md CHANGED Viewed

@@ -1,4 +1,5 @@
 ---
 tags:
 - merge
 - mergekit
@@ -165,4 +166,10 @@ Community tab.
 is a weight-space combination of models trained by others, and its training
 content is theirs. All 1 constituent models this card names are still published, so the chain can be followed from here.
-**Licence — not stated, and not guessed.** This repository carries no licence field. A merge inherits the terms of its constituents, taken at their most restrictive, so the correct licence here is whatever the strictest constituent requires. That cannot be derived while part of the lineage is unavailable, and asserting a permissive licence on that basis would grant rights that may never have been granted. Treat the terms as unresolved and consult the constituents above before redistributing or using commercially.

 ---
+license: cc-by-nc-4.0
 tags:
 - merge
 - mergekit
 is a weight-space combination of models trained by others, and its training
 content is theirs. All 1 constituent models this card names are still published, so the chain can be followed from here.
+**Licence — resolved 2026-08-02.** `cc-by-nc-4.0`. `yleo/EmertonMonarch-7B` is CC-BY-NC-4.0. The NonCommercial term propagates to anything containing those weights.
+This was derived from the mergekit configuration reproduced in this card by
+resolving each named constituent's licence on the Hub and taking the most
+restrictive, rather than assumed from the model family. An earlier revision of
+this section said the terms were unresolved; they are resolved now, and the
+method is recorded so the conclusion can be checked rather than trusted.