Dear Hugging Face Team, I am currently hosting a 3D reconstruction image dataset on the Hugging Face Hub: yinyue27/RefRef · Datasets at Hugging Face . The dataset is in Blender format and contains multiple subsets, where each subset corresponds to a scene. Each scene is further divided into three sp…

I wonder if Croissant is not automatically generated for data set repositories that use loading scripts…? @lhoestq from datasets import load_dataset ds = load_dataset("yinyue27/RefRef", "single-convex", trust_remote_code=True) # error print(ds) Get Croissant metadata We’re on a journe…

Hi John666, Thanks for looking into my problem! I think if you modify the script a little bit by adding this ‘scene’ keyword I customised, you will load the dataset correctly: from datasets import load_dataset ds = load_dataset("yinyue27/RefRef", split="textured_sphere_scene", name="multiple-non-…

It worked! from datasets import load_dataset ds = load_dataset("yinyue27/RefRef", split="textured_sphere_scene", name="multiple-non-convex", scene="beaker", trust_remote_code=True) print(ds) #Generating textured_sphere_scene split: 300 examples [00:06, 47.17 examples/s] #Generating textured_cube_sc…

Yes, so I assume the data loader is good to use for the dataset. Any clues on how I can generate the croissant metadata? :thinking:

The croissant metadata on HF are generated automatically for datasets in supported formats like Parquet or ImageFolder (folder of images and a metadata file ) . If you convert your dataset to Parquet, or if you structure your dataset as a ImageFolder, the croissant metadata will be available. There …

Hi John, I’m now trying to load my dataset and use push_to_hub to push it to a new dataset. This is the script I’m using: from datasets import load_dataset dataset = load_dataset( # path="eztao/RefRef_test", path="yinyue27/RefRef", name="single-convex", scene="ball", split="tex…

(I’m assuming you’re passing the token using login() or something similar) The version of the library in charge of serialization and uploading may be out of date. pip install -U huggingface_hub

I used huggingface-cli login and generated a token to login, and I’m still getting the error after using pip to update hf_hub (I’m already at the latest version actually) :smiling_face_with_tear:

If it happens with .save_to_disk(), it’s definitely a bug. [image] Process We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Generating Croissant Metadata for Custom Image Dataset

🤗Datasets

John6666 April 15, 2025, 2:31am 10

self.info(path).get(“size”, None)

It’s strange that it returns None… in other words, it means that this path cannot be found.
I think it’s a bug, but I wonder what kind of bug it is…
It’s different from the case below, and it’s probably looking for a local path, so it’s not related to networking, is it…?

Topic		Replies	Views
Possible issue of contentUrl in croissant file of the dataset 🤗Datasets	2	69	October 18, 2025
Help making object detection dataset Beginners	4	196	April 26, 2025
Custom loading dataset script 🤗Datasets	4	568	January 3, 2023
Dataset preview not showing for uploaded DatasetDict 🤗Datasets	6	2237	December 7, 2021
Create Dataset with metadata 🤗Datasets	1	1426	November 28, 2022

Generating Croissant Metadata for Custom Image Dataset

Related topics