# How to filter Datasets object?
**URL:**
**Author:**  [@kuliop](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkuliop)%5C
**Post date:** [June 6, 2024, 3:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-filter-datasets-object%2F90718%2F1 "2024-06-06T15:48:39Z")
I have a hugging face dataset object and I want to filter it as I would a pandas dataframe:
```auto
train[train['language']=="English"]
```
‘language’ is one of the features in the train split.
I also tried:
```auto
train.select(train['language']=='English')
```
and got error:
TypeError: ‘bool’ object is not iterable
And similarly, boolean masking, with the same error code:
```auto
is_english = dataset['train']['language'] == 'English'
eng_convos = dataset['train'].select(is_english)
```
Or a list comprehnsion:
```auto
eng_convos = [dataset['train']['conversation'] for dataset['train']['conversation'] in dataset['train'] if dataset['train']['language']=='English']
```
I got:
TypeError: ‘Dataset’ object does not support item assignment
Lastly, I tried follwoing advice from this discussion post [Filtering Dataset](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffiltering-dataset%2F10228)
```auto
import pyarrow as pa
import pyarrow.compute as compute
table = dataset.data
flags = compute.is_in(train['language'], value_set=pa.array(['English'], pa.string()))
filtered_table = train.filter(flags)
filtered_table.to_pandas()
```
and got error: 557 # apply actual function
→ 558 out: Union[“Dataset”, “DatasetDict”] = func(self, \*args, \*\*kwargs)
559 datasets: List[“Dataset”] = list(out.values()) if isinstance(out, dict) else [out]
560 # re-apply format to the output
File c:\Users\Admin\HC3 EDA.venv\Lib\site-packages\datasets\fingerprint.py:482, in fingerprint\_transform..\_fingerprint..wrapper(\*args, \*\*kwargs)
478 validate\_fingerprint(kwargs[fingerprint\_name])
480 # Call actual function
→ 482 out = func(dataset, \*args, \*\*kwargs)
484 # Update fingerprint of in-place transforms + update in-place history of transforms
…
6260 else:
6261 # inputs is a list of columns
6262 columns: List[List] = inputs
TypeError: ‘pyarrow.lib.BooleanArray’ object is not callable
---
**Author:**  [@kuliop](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkuliop)%5C
**Post date:** [June 6, 2024, 3:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-filter-datasets-object%2F90718%2F2 "2024-06-06T15:52:30Z")
All I want to do is essentially filter my HF Datasets object as I would a pandas dataframe i.e. only return the dataset where a specific column value meets my condition…
---
**Author:**  [@kuliop](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkuliop)%5C
**Post date:** [June 6, 2024, 5:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-filter-datasets-object%2F90718%2F3 "2024-06-06T17:04:12Z")
Found a solution:
1. converted the datasets object to pandas DataFrame
```auto
import pandas as pd
df = pd.DataFrame(dataset['train'])
```
1. filtered it as usual
```auto
english_only = df[df['language']=="English"]
```
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [June 7, 2024, 5:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-filter-datasets-object%2F90718%2F4 "2024-06-07T05:04:31Z")
Closed on June 7, 2024, 5:04am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.