🦋 Qwythos 9B
so many mythoses and fables
anyways, check your tensors, make sure that model is the same shape as the base (tested june 20): llama_model_load: error loading model: missing tensor 'blk.32.attn_norm.weight'
llama_model_load: error loading model: missing tensor 'blk.32.attn_norm.weight'
Do you have any idea why @empero-ai ?
This bug was related to the MTP heads and was fixed in our v2 GGUFs here: https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF
Due to our own customized training loop there have been some issues with the initial release.
For looping issues its imperative to follow the recommended sampling parameters as we mention in the model card:
https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M#sampling-recommendations
The model is being used by a lot of people in production and works very well.
@redaihf Refusal rate of this model is incredibly unstable. With the branded chat template, condemning the model to vehemently state its name and maker at each prompt, the refusal rate lands on 444/535. With the standard Qwen3.5 9B template, it scores 131/535 initially. However bizarrely, ablation seem to exacerbate refusal rates, hitting almost full censorship at 500/535 refusals on one run, with clear primary refusals (I cannot's and I won'ts). It is as if the finetuning somehow made the model more resistant to ablation. It's fluctuating across trials as well. Got any ideas?
Are you using the source variant or your MTP graft? The MTP heads were probably never decensored and may account for the variability you are seeing as they are activated on a per-token basis. You would decensor the Qwen MTP model and then reapply the decensored MTP heads to Qwythos which is supposed to be decensored by default.
I'm using the MTP grafted version. However, I believe that the MTP weights aren't loaded at all in the case of Heretic, given that the resulting models are consistently saved without its speculative decoding model across HF. You wouldn't subject innocent little MTP weights to horrors beyond LLM comprehension, would you? At least, I did not see such an issue when ablating the standart Qwen3.5 9B, and the MTP graft did not seem to affect the decensor.
Still, for the sake of comparison, I just did the same test with the source variant. With the branded chat template: Initial refusals: 450/535. With the standart chat template: Initial refusals: 138/535.
In addition, here're the results with the default Heretic configuration on version 1.4, in case mine was too vile. With the branded chat template: Initial refusals: 87/100. With the standart chat template: Initial refusals: 26/100.
I went ahead and also tested the new Qwythos-9B-v2, which comes with MTP layers by default. With the branded chat template: Initial refusals: 389/535. With the standart chat template: Initial refusals: 175/535. (It still brings up its name but not in every response, unlike its predecessor.)
Two things are probably happening. Firstly the base model was abliterated but not Hereticised using MPOA which leaves the ability for noncompliance in place because the harmfulness direction is unaddressed. Secondly the distillation training has reactivated this latent refusal direction which manifests most strongly when the finetuning is referenced directly in the system prompt.