Allam-K8sCTL-7B-FT

A QLoRA fine-tune of ALLaM-7B-Instruct-preview that turns a plain-language request into one ready-to-run kubectl command.

It was built as a proof of concept for an in-house "ALLaM CLI" assistant for DevOps and SRE teams: fewer documentation lookups, fewer mistyped flags, and consistent command patterns across teams. Kubernetes kubectl is the first target. Helm, Docker and Git are planned next.

On a held-out test set, exact-match accuracy goes from 12.6% (base model) to 78.6% (fine-tuned).

Base model	humain-ai/ALLaM-7B-Instruct-preview (7B, Llama architecture, 4k context)
Method	QLoRA: 4-bit NF4 base model + LoRA adapters, supervised fine-tuning with TRL
Training data	ComponentSoft/k8s-kubectl-cot-20k (English)
Output	A single kubectl command line, no explanation
License	Apache 2.0 (same as the base model)
Files	LoRA adapter at the repo root (about 160 MB) · merged bf16 model in merged-Allam-K8sCTL-7B-FT/ (about 13 GB)

Examples

Real outputs on requests from the held-out test set (greedy decoding):

Request	Base ALLaM-7B	Allam-K8sCTL-7B-FT
View the logs from the first container of the "log-analysis-job" job	kubectl logs -f -l job=log-analysis-job	kubectl logs job/log-analysis-job
Scale the 'foo' deployment to 3 replicas if its current size is 1	kubectl scale deployment foo --replicas=3	kubectl scale --current-replicas=1 --replicas=3 deployment/foo
Execute the 'redis-cli' command in the 'backend-pod' pod and retrieve the output	kubectl exec backend-pod -c redis-cli -- redis-cli	kubectl exec backend-pod -- redis-cli
Delete the resources specified in the kustomization.yaml file located in the params directory	kubectl delete -f params/kustomization.yaml	kubectl delete -k params/

Usage

Install transformers>=4.57, torch, sentencepiece, and peft (for the adapter). A GPU with at least 16 GB of memory is recommended for bf16 inference.

Prompt format

The model was trained on this exact prompt structure. Use it to get the results reported below:

Objective: <what you want to do>
Return exactly one kubectl command.
No commentary, no backticks, no explanations.
Hints:                        <- optional
Description: <what the kubectl subcommand does>
Base command: <e.g. logs>
Syntax: <e.g. kubectl logs [-f] [-p] (POD | TYPE/NAME) [-c CONTAINER]>
Common flags: <e.g. -c -f -p --since>

During training and evaluation, every prompt also had a Question: line (the request rephrased) and the Hints: block, both taken from the dataset. Prompts with only an Objective: line work, but they haven't been benchmarked separately.

Option 1: LoRA adapter (downloads the base model plus a small adapter)

import re
import torch
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

repo = "Khalid-Hamad/Allam-K8sCTL-7B-FT"
tokenizer = AutoTokenizer.from_pretrained(repo, use_fast=False)
model = AutoPeftModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

def generate_kubectl(objective: str, hints: str = "") -> str:
    lines = [f"Objective: {objective}",
             "Return exactly one kubectl command.",
             "No commentary, no backticks, no explanations."]
    if hints:
        lines.append("Hints:\n" + hints)
    messages = [{"role": "user", "content": "\n".join(lines)}]
    inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
    output = model.generate(inputs, max_new_tokens=64, do_sample=False)
    text = tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True)
    match = re.search(r"(?im)^(kubectl[^\n\r]+)", text)
    return (match.group(1) if match else text).strip()

print(generate_kubectl("List pods in namespace payments"))

Option 2: Merged model (no peft needed)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Khalid-Hamad/Allam-K8sCTL-7B-FT"
sub = "merged-Allam-K8sCTL-7B-FT"
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=sub, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(repo, subfolder=sub, torch_dtype=torch.bfloat16, device_map="auto")

After loading, use the same generate_kubectl function as in option 1.

Recommended: add a safety guard

The model's weights don't add any safety flags. The PoC wraps its output with a small guard that makes risky commands safe to run in demos and pilots:

RISKY = re.compile(r"\b(delete|drain|uncordon|cordon|taint|patch|apply|replace|exec)\b", re.I)
DRY_RUNNABLE = {"apply", "replace", "patch", "delete"}

def guard(cmd: str) -> str:
    if not RISKY.search(cmd) or re.search(r"--dry-run=(client|server)\b", cmd):
        return cmd
    sub = cmd.split()[1] if len(cmd.split()) > 1 else ""
    if sub in DRY_RUNNABLE:
        return re.sub(r"\s--dry-run(=\S+)?", "", cmd) + " --dry-run=client"   # preview without applying
    return cmd.split(" --")[0] + " --help"   # e.g. drain/cordon/exec: show help instead

Training details

Data: ComponentSoft/k8s-kubectl-cot-20k, English. Rows with an empty command were removed, and every target was normalized to start with kubectl, leaving 19,659 rows. These were split 80/10/10, stratified by command_name with seed 42, into 15,727 train / 1,966 validation / 1,966 test examples. The model learns to output only the command field. The dataset's chain-of-thought text was not used as a target.

Setting	Value
Hardware	1× NVIDIA A100 80 GB
Frameworks	TRL SFTTrainer, PEFT 0.17.1, Transformers 4.57.1, PyTorch 2.8.0, bitsandbytes
Quantization (training)	4-bit NF4, double quantization, bf16 compute
LoRA rank / alpha / dropout	16 / 32 / 0.05
LoRA target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Epochs	5 (4,915 optimizer steps)
Batch size	8 per device × 2 gradient accumulation = 16
Learning rate	2e-4 peak, 3% warmup, cosine decay; weight decay 0.01; AdamW
Max sequence length	1,024 tokens, no packing
Loss	Computed over the full sequence (prompt + command)
Precision	bf16, gradient checkpointing

Training loss fell from 2.27 at step 20 to 0.06 by epoch 5. Mean token accuracy on the training data rose from 58% to 98%.

Evaluation

The benchmark compares the base model and the fine-tuned model on the 1,966-example held-out test split, which was never used in training. Both models got identical prompts, greedy decoding, and up to 64 new tokens. Both were loaded in 4-bit NF4 on an A100.

Metric	Base ALLaM-7B-Instruct-preview	Allam-K8sCTL-7B-FT
Exact match	12.6%	78.6%
Similarity (Levenshtein ratio)	80.8%	97.2%
Flag F1	78.9%	97.1%
Subcommand accuracy	98.2%	99.8%
Output contract (single line starting with kubectl)	99.9%	100.0%
Syntax check	86.6%	85.2%
Throughput	2.4 prompts/s	1.8 prompts/s

• Gain in exact match: +66.0 percentage points (95% bootstrap confidence interval 64.0 to 68.1, 1,000 resamples).
• Significance: McNemar χ² = 1,294.0, so the improvement is very unlikely to be chance.
• bf16 spot check: without quantization, on 799 examples the model had not seen in training, exact match was 76.8% for the fine-tuned model vs 15.9% for the base model.

How the metrics are measured

• Exact match: case- and whitespace-insensitive string match with the reference command.
• Similarity: mean Levenshtein ratio between the predicted and reference commands.
• Flag F1: F1 score over flag names, ignoring their values.
• Subcommand accuracy: the first subcommand matches the reference.
• Output contract: one line starting with kubectl.
• Syntax check: the subcommand is accepted by kubectl <subcommand> --help. This is a coarse check that doesn't validate flags or arguments.

Intended use

• Drafting kubectl commands from a plain-language description
• Helping new team members learn kubectl syntax and flags
• A building block for internal DevOps assistants where a person reviews every command before it runs

Limitations and risks

• Review every command before running it. About 1 in 5 outputs differs from the reference. Be most careful with commands that change or remove resources (delete, drain, patch, apply, scale, rollout undo). Don't let this model run commands on a cluster automatically.
• It can invent values. When a request leaves something out, such as an environment variable or an image tag, the model fills in a plausible-looking value from its training data. It also has no view of your cluster, so it can't check whether a resource or namespace exists.
• The model's weights don't include the safety guard. Use the guard above or your own policy layer.
• The benchmark is single-source and English-only. All results come from one synthetic dataset, with prompts that include the dataset's hints. Real-world requests phrased differently may score lower.
• Arabic support is untested. ALLaM is bilingual, so Arabic requests may work, but no Arabic data was used for training or evaluation.
• Flags may be version-specific. Check them against the kubectl reference.
• The base model is a preview release. This model inherits its limitations and biases.

Citation

@misc{hamad2025allamk8sctl,
  author       = {Khalid Hamad},
  title        = {Allam-K8sCTL-7B-FT: Natural language to kubectl with ALLaM-7B},
  year         = {2025},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Khalid-Hamad/Allam-K8sCTL-7B-FT}}
}

Base model: ALLaM: Large Language Models for Arabic and English (arXiv:2407.15390). Training data: ComponentSoft/k8s-kubectl-cot-20k. Trained with TRL.

Author

Khalid Hamad · GitHub