Upload folder using huggingface_hub

README.md

@@ -0,0 +1,106 @@
+---
+license: apache-2.0
+datasets:
+  - Sleep-EDF
+  - TUAB
+  - MOABB
+language:
+  - en
+tags:
+  - eeg
+  - brain
+  - timeseries
+  - self-supervised
+  - transformer
+  - biomedical
+  - neuroscience
+---
+
+# BENDR: BErt-inspired Neural Data Representations
+
+Pretrained BENDR model for EEG classification tasks. This is the official Braindecode implementation
+of BENDR from Kostas et al. (2021).
+
+## Model Details
+
+- **Model Type**: Transformer-based EEG encoder
+- **Pretraining**: Self-supervised learning on masked sequence reconstruction
+- **Architecture**: 
+  - Convolutional Encoder: 6 blocks with 512 hidden units
+  - Transformer Contextualizer: 8 layers, 8 attention heads
+  - Total Parameters: ~157M
+- **Input**: Raw EEG signals (20 channels, variable length)
+- **Output**: Contextualized representations or class predictions
+
+## Usage
+
+```python
+from braindecode.models import BENDR
+import torch
+
+# Load pretrained model
+model = BENDR(n_chans=20, n_outputs=2)
+
+# Load pretrained weights from Hugging Face
+from huggingface_hub import hf_hub_download
+checkpoint_path = hf_hub_download(repo_id="braindecode/bendr-pretrained-v1", filename="pytorch_model.bin")
+checkpoint = torch.load(checkpoint_path)
+model.load_state_dict(checkpoint["model_state_dict"], strict=False)
+
+# Use for inference
+model.eval()
+with torch.no_grad():
+    eeg_data = torch.randn(1, 20, 600)  # (batch, channels, time)
+    predictions = model(eeg_data)
+```
+
+## Fine-tuning
+
+```python
+import torch
+from torch.optim import Adam
+
+# Freeze encoder for transfer learning
+for param in model.encoder.parameters():
+    param.requires_grad = False
+
+# Fine-tune on downstream task
+optimizer = Adam(model.parameters(), lr=0.0001)
+```
+
+## Paper
+
+[BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://doi.org/10.3389/fnhum.2021.653659)
+
+Kostas, D., Aroca-Ouellette, S., & Rudzicz, F. (2021).
+Frontiers in Human Neuroscience, 15, 653659.
+
+## Citation
+
+```bibtex
+@article{kostas2021bendr,
+  title={BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data},
+  author={Kostas, Demetres and Aroca-Ouellette, St{\'e}phane and Rudzicz, Frank},
+  journal={Frontiers in Human Neuroscience},
+  volume={15},
+  pages={653659},
+  year={2021},
+  publisher={Frontiers}
+}
+```
+
+## Implementation Notes
+
+- Start token is correctly extracted at index 0 (BERT [CLS] convention)
+- Uses T-Fixup weight initialization for stability
+- Includes LayerDrop for regularization
+- All architectural improvements from original paper maintained
+
+## License
+
+Apache 2.0
+
+## Authors
+
+- Braindecode Team
+- Original paper: Kostas et al. (2021)

config.json

@@ -0,0 +1,32 @@
+{
+  "model_type": "bendr",
+  "n_chans": 20,
+  "encoder_h": 512,
+  "contextualizer_hidden": 3076,
+  "transformer_heads": 8,
+  "transformer_layers": 8,
+  "position_encoder_length": 25,
+  "drop_prob": 0.1,
+  "layer_drop": 0.0,
+  "start_token": -5,
+  "final_layer": true,
+  "projection_head": false,
+  "enc_width": [
+    3,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "enc_downsample": [
+    3,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "notes": "Pretrained BENDR model for EEG classification",
+  "paper": "https://doi.org/10.3389/fnhum.2021.653659"
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58696d59ae4fb3d041837746c6c6225fa851841f68753e8cc28d4ecd4383d828
+size 628594288