Instructions to use hussamalafandi/smollm2-sft-rewrite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hussamalafandi/smollm2-sft-rewrite with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="hussamalafandi/smollm2-sft-rewrite")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("hussamalafandi/smollm2-sft-rewrite")
model = AutoModelForCausalLM.from_pretrained("hussamalafandi/smollm2-sft-rewrite")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use hussamalafandi/smollm2-sft-rewrite with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hussamalafandi/smollm2-sft-rewrite"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hussamalafandi/smollm2-sft-rewrite",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/hussamalafandi/smollm2-sft-rewrite

SGLang

How to use hussamalafandi/smollm2-sft-rewrite with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hussamalafandi/smollm2-sft-rewrite" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hussamalafandi/smollm2-sft-rewrite",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hussamalafandi/smollm2-sft-rewrite" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hussamalafandi/smollm2-sft-rewrite",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use hussamalafandi/smollm2-sft-rewrite with Docker Model Runner:
```
docker model run hf.co/hussamalafandi/smollm2-sft-rewrite
```

Model Card for smollm2-sft-results

This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M. It has been trained using TRL.

Quick start

from transformers import pipeline

prompt = [
    {'content': "You're an AI assistant for text re-writing. Rewrite the input "
     'text to make it more concise while preserving its core meaning.',
     'role': 'system'},

    {'content': 'Hey Alex,\n'
     '\n'
     "I hope you're doing well! It's been a while since we met at the "
     'film festival last year. I was the one with the short film about '
     "the old abandoned factory. Anyway, I'm reaching out because I'm "
     'currently working on my thesis film project and I could really '
     'use some advice on cinematography. I remember our conversation '
     'about visual storytelling and I was hoping you might have some '
     'tips or insights to share.\n'
     '\n'
     'My film is a drama set in a small town, and I want to capture '
     'the mood and atmosphere of the location through my '
     "cinematography. I'm planning to shoot on location next month, "
     "but I'm still trying to figure out the best way to approach it. "
     'If you have any suggestions or resources you could point me to, '
     'I would be incredibly grateful.\n'
     '\n'
     "Also, I heard from a mutual friend that you're having a "
     'photography exhibition soon. Congratulations! I would love to '
     "attend if you don't mind sending me the details.\n"
     '\n'
     'Thanks in advance for any help you can provide. I really '
     'appreciate it.\n'
     '\n'
     'Best,\n'
     'Jordan',
     'role': 'user'}]

generator = pipeline("text-generation", model="hussamalafandi/smollm2-sft-rewrite", device="cuda")

output = generator(prompt, max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with SFT.

Framework versions

TRL: 0.16.1
Transformers: 4.51.3
Pytorch: 2.6.0+cu124
Datasets: 3.5.0
Tokenizers: 0.21.1

Citations

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}

Downloads last month: 1

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for hussamalafandi/smollm2-sft-rewrite

Base model

HuggingFaceTB/SmolLM2-135M

Finetuned

(914)

this model

Collection including hussamalafandi/smollm2-sft-rewrite

Generative AI

Collection

6 items • Updated Apr 22, 2025