asifdotpy/vetta-granite-2b-gguf-v3

Vetta Granite GGUF Model v3

This repository contains the quantized GGUF version of the Vetta AI interviewer model for efficient inference with Ollama or vLLM.

Usage with Ollama

1. Download the GGUF file
2. Create Modelfile:

FROM ./vetta-granite-2b-gguf-v3.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
SYSTEM "You are Vetta, a professional AI interviewer conducting technical interviews."

3. Create model: ollama create vetta-granite -f Modelfile
4. Run: ollama run vetta-granite

Training Details

• Base Model: ibm-granite/granite-3.0-2b-instruct
• Training Method: LoRA fine-tuning
• Quantization: Q4_K_M
• Dataset: Custom interview conversation dataset
• Training Steps: 450
• Final Loss: 0.2422

Intended Use

This model is designed to conduct professional AI-powered interviews, providing empathetic and technically accurate responses.