# Training out of memory

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-out-of-memory%2F98095%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 18, 2024, 8:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-out-of-memory%2F98095 "2024-07-18T08:12:34Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![daminho](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/daminho/32/29506_2.png) [@daminho](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdaminho)%5C
**Post date:** [July 18, 2024, 8:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-out-of-memory%2F98095%2F1 "2024-07-18T08:12:34Z")

</div>

I’m training the Qwen2 - 500m for classification now, my dataset is Reddit post. Somehow when fine-tuning, it keeps saying that I’m out of memory for CUDA device. I even use floating point 16-bit but it’s still giving the OOM error

Here is my code for model and trainer

```auto
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSequenceClassification
from transformers import TrainingArguments
import torch

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-0.5B")
model = AutoModelForSequenceClassification.from_pretrained("Qwen/Qwen2-0.5B", num_labels=2)
model.half()
print(model)

```

```auto
model.to(device)

training_args = TrainingArguments(
    output_dir = 'kaggle/input/output_dir',
    do_train=True,
    do_eval=False,
    eval_strategy="epoch",
    num_train_epochs = 3,
    per_device_train_batch_size = 1,
    per_device_eval_batch_size = 1,
    warmup_steps = 100,
    weight_decay = 0.01,
    eval_accumulation_steps = 1
)

trainer = Trainer(
    model,
    training_args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    tokenizer=tokenizer,
)
trainer.train()

```

I just use batch size 1 and also eval accumulation steps = 1. The GPU I use is P100 / T4 (two options provided by Kaggle).

Please help meeeeeeeeee! \<3