HeresyAgent-8B

HeresyAgent-8B is a Mixture of Experts (MoE) made with the following models using LazyMergekit:

• miromind-ai/MiroThinker-v1.0-8B
• nvidia/Orchestrator-8B
• rl-research/DR-Tulu-8B
• csalab/Qwen3-8B

🧩 Configuration

slices:
  - sources:
      - model: miromind-ai/MiroThinker-v1.0-8B
        layer_range: [0, 32]
      - model: nvidia/Orchestrator-8B
        layer_range: [0, 32]
      - model: rl-research/DR-Tulu-8B
        layer_range: [0, 32]
      - model: csalab/Qwen3-8B
        layer_range: [0, 32]
base_model: miromind-ai/MiroThinker-v1.0-8B
experts:
  - source_model: miromind-ai/MiroThinker-v1.0-8B
    weight: 0.25
  - source_model: nvidia/Orchestrator-8B
    weight: 0.25
  - source_model: rl-research/DR-Tulu-8B
    weight: 0.25
  - source_model: csalab/Qwen3-8B
    weight: 0.25
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
merge_type: slerp
dtype: bfloat16
layer_range: [0, 32]

💻 Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "jsuheb/HeresyAgent-8B"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

< === Cautions === >

Source Specification

This model includes heretic github repo while created with using MergeKit.

Risk Warning

This model incorporates an uncensored model with safety filters intentionally removed. It may generate harmful, biased, or illegal content.

Intended Use Limitation

This model is provided solely for AI safety research and the exploration of ethical boundaries. Using this model with malicious purposes is strictly prohibited.

Liability Disclaimer

All legal and ethical responsibility arising from the use of this model rests entirely with the user.