# Dataset with no splits

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515%3E%5C
**Category:** 🤗Datasets\
**Created:** [July 20, 2021, 12:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515 "2021-07-20T12:24:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![christopher](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/christopher/32/31595_2.png) [@christopher](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchristopher)%5C
**Post date:** [July 20, 2021, 12:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515%2F1 "2021-07-20T12:24:40Z")

</div>

Hello everyone,

I am currently creating a dataset where the **semantics of a split make no sense**. It’s an Information Retrieval corpus that should not be split. I currently load the entire corpus into the “train” split because I’ve been copying what other dataset loading scripts do, and it works, but would rather drop splits altogether for the `corpus` config.

Can I use other strings to name the splits? Can I leave out splits altogether for a given `self.config.name`?

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [July 21, 2021, 8:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515%2F2 "2021-07-21T08:37:50Z")

</div>

Hi ! By default you can use the train split, but feel free to name it whatever you want.

In practice, when you return the split generators in `_split_generators()`, you can specify the split name you want to `datasets.SplitGenerator(name=...)`

---

<div class="post-metadata">

**Author:** ![Unknown-User](https://avatars.discourse-cdn.com/v4/letter/u/cdc98d/32.png) [@Unknown-User](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FUnknown-User)%5C
**Post date:** [July 27, 2021, 2:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515%2F3 "2021-07-27T14:27:47Z")

</div>

Hi,

Thank you for this information, is it possible not to use any split at all ?

I mean, using " load\_dataset (‘name’) " without any split ? I had an error while trying not to use any split argument.

---

<div class="post-metadata">

**Author:** ![thomwolf](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thomwolf/32/2198_2.png) [@thomwolf](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fthomwolf)%5C
**Post date:** [July 28, 2021, 11:55am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515%2F4 "2021-07-28T11:55:43Z")

</div>

yes we will in the mid term relax this requirement to make the use of splits optional (by merging DatasetDict and Dataset classes basically).

At the moment you can just assume all is in the “train” split for your use-case

---

<div class="post-metadata">

**Author:** ![Sh0g0](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sh0g0/32/27033_2.png) [@Sh0g0](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSh0g0)%5C
**Post date:** [May 16, 2024, 11:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-with-no-splits%2F8515%2F5 "2024-05-16T11:29:39Z")

</div>

@thomwolf, was this requirement relaxed?
