# Max\_step and generative dataset

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmax-step-and-generative-dataset%2F11400%3E%5C
**Category:** Intermediate\
**Created:** [November 5, 2021, 3:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmax-step-and-generative-dataset%2F11400 "2021-11-05T15:43:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![pkoiralap](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pkoiralap/32/2358_2.png) [@pkoiralap](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpkoiralap)%5C
**Post date:** [November 5, 2021, 3:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmax-step-and-generative-dataset%2F11400%2F1 "2021-11-05T15:43:53Z")

</div>

I am fine tuning BERT for my domain using both MLM and NSP. I have used the `TextDatasetForNextSentencePrediction` for dataset along with `DataCollatorForLanguageModeling` for MLM.

The problem is with `TextDatasetForNextSentencePrediction` as it loads the entire data into the memory. So I tweaked it a bit and now it generates the data rather than storing everything in the memory.

However the problem is with the max\_step for training. Now that it is generative, the ` __len__ ` property can not be added to `TextDatasetForNextSentencePrediction` and I am getting an error regarding the same thing:

```auto
ValueError: train_dataset does not implement __len__ , max_steps has to be specified

```

Can I just specify an upper bound and it will figure it out automatically? Or will it break? Because if it does, I can’t run multiple epochs. (right)?

There are a lot of X, Y questions here and I can see this question is rather open ended. But any sort of advice or direction is much appreciated.

Thanks