Instructions to use Vipitis/santacoder-finetuned-Shadertoys with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Vipitis/santacoder-finetuned-Shadertoys with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Vipitis/santacoder-finetuned-Shadertoys", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Vipitis/santacoder-finetuned-Shadertoys", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Vipitis/santacoder-finetuned-Shadertoys", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Vipitis/santacoder-finetuned-Shadertoys with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Vipitis/santacoder-finetuned-Shadertoys"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vipitis/santacoder-finetuned-Shadertoys",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Vipitis/santacoder-finetuned-Shadertoys

SGLang

How to use Vipitis/santacoder-finetuned-Shadertoys with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Vipitis/santacoder-finetuned-Shadertoys" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vipitis/santacoder-finetuned-Shadertoys",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Vipitis/santacoder-finetuned-Shadertoys" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vipitis/santacoder-finetuned-Shadertoys",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Vipitis/santacoder-finetuned-Shadertoys with Docker Model Runner:
```
docker model run hf.co/Vipitis/santacoder-finetuned-Shadertoys
```

santacoder-finetuned-Shadertoys / README.md

Vipitis

add base model metadata

99f9075 over 2 years ago

preview code

raw

history blame contribute delete

2.04 kB

metadata

language:
  - code
license: bigcode-openrail-m
datasets:
  - bigcode/the-stack-dedup
  - Vipitis/Shadertoys
pipeline_tag: text-generation
tags:
  - code
  - shader
base_model: bigcode/santacoder
widget:
  - text: void mainImage( out vec4 fragColor, in vec2 fragCoord )
    example_title: mainImage
    group: Shadertoy
model-index:
  - name: santacoder-finetuned-the-stack-glsl
    results:
      - task:
          type: text-generation
          name: ShaderEval
        dataset:
          type: Vipitis/Shadertoys-fine
          name: Shadertoys-fine
          config: return_completion
          revision: 0.0.2
        metrics:
          - type: exact_match
            value: 0.55
            name: 300 samples, greedy decoding
            verified: false

Santacoder finetuned on Shadertoys for 1000 steps with a batch size of 2 and full sequence length of 2048. adapted finetuning script found here

Try model in the ShaderCoder demo space

Finetuning parameters

python3 train.py --model_path "bigcode/santacoder" \
--dataset_name "Vipitis/Shadertoys" \
--data_column "code" \
--split "train" \
--seq_length 2048 \
--max_steps 1000 \
--batch_size 2 \
--gradient_accumulation_steps 4 \
--learning_rate 5e-5 \
--num_warmup_steps 100 \
--eval_freq 100 \
--save_freq 100 \
--log_freq 1 \
--output_dir "checkpoint_dir" \
--no_fp16

Main purpose of this model is to explore if finetuning models improves performance on ShaderEval, which reached 0.550 with 300 samples.

Disclaimer

While the train/test split is held out, there is a lot of data contamination. The model results can't be trusted for this simple benchmark. Better tasks for the benchmark will be developed and tested against these models.

License carried over from model, however training data has an undefied license. Check details in Shadertoys.