# Loading community JSON based datasets without a script

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-community-json-based-datasets-without-a-script%2F10405%3E%5C
**Category:** 🤗Datasets\
**Created:** [September 30, 2021, 7:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-community-json-based-datasets-without-a-script%2F10405 "2021-09-30T07:38:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vblagoje](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vblagoje/32/20818_2.png) [@vblagoje](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvblagoje)%5C
**Post date:** [September 30, 2021, 7:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-community-json-based-datasets-without-a-script%2F10405%2F1 "2021-09-30T07:38:22Z")

</div>

Hi,

I can easily create and upload JSON community datasets to HF hub (without a script). However, when using the dataset I need to specify data\_files parameter, i.e. `dataset = load_dataset("vblagoje/dataset_abc", data_files={"train": "train.json", "validation": "validation.json", "test": "test.json"})` which is a bit of inconvenience.

Is there a way to avoid data\_files parameters short of providing a loading script as well? If not, why don’t we rely on naming conventions to resolve the data splits anyway?

Cheers,  
Vladimir

---

<div class="post-metadata">

**Author:** ![julien-c](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/julien-c/32/41937_2.png) [@julien-c](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjulien-c)%5C
**Post date:** [September 30, 2021, 11:19am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-community-json-based-datasets-without-a-script%2F10405%2F2 "2021-09-30T11:19:01Z")

</div>

Tagging @lhoestq @albertvillanova in case you hadn’t seen this!

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/64813_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 4, 2021, 9:17am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-community-json-based-datasets-without-a-script%2F10405%2F3 "2021-10-04T09:17:29Z")

</div>

Hi ! I like the idea of using a name convention. By default it could use something like

```auto
data_files = {
    "train": ["*train*"],
    "test": ["*test*"],
    "validation": ["*dev*", "*valid*"]
}

```

Another alternative would be to let the user specify the `data_files` patterns in an additional JSON file (config.json or something like this). Let me know what you think !

---

<div class="post-metadata">

**Author:** ![vblagoje](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vblagoje/32/20818_2.png) [@vblagoje](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvblagoje)%5C
**Post date:** [October 4, 2021, 10:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-community-json-based-datasets-without-a-script%2F10405%2F4 "2021-10-04T10:45:59Z")

</div>

Yes, that’s the perfect default @lhoestq Perhaps it could be expanded later, but for now, this solution suffices IMHO
