SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 2048 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "(\u041d\u0430\u0437\u0432\u0430\u043d\u0438\u0435:) \u0420\u0415\u0419\u0413\u0410\u041d-\u041f\u0420\u041e\u0412\u041e\u041a\u0410\u0422\u041e\u0420 \n(\u041f\u043e\u044d\u0442:) \u0410\u043d\u0434\u0440\u0435\u0439 \u201c\u0421\u0432\u0438\u043d\u201d \u041f\u0430\u043d\u043e\u0432 \n\n\u041e\u043f\u044f\u0442\u044c \u043d\u0435\u043b\u0451\u0442\u043d\u0430\u044f \u043f\u043e\u0433\u043e\u0434\u0430, \n\u0418 \u043e\u0442 \u0441\u0443\u0434\u044c\u0431\u044b \u043d\u0435 \u0443\u0431\u0435\u0436\u0430\u0442\u044c. \n\u041c\u044b \u043e\u0431\u044a\u044f\u0432\u043b\u044f\u0435\u043c \u043c\u043e\u0440\u0430\u0442\u043e\u0440\u0438\u0439 \u2014 \n\u0410\u043c\u0435\u0440\u0438\u043a\u0430\u043d\u0446\u0430\u043c \u043d\u0430\u043f\u043b\u0435\u0432\u0430\u0442\u044c\u2026 \n\u0412 \u0416\u0435\u043d\u0435\u0432\u0435 \u0432\u0441\u0435 \u043f\u0435\u0440\u0435\u0433\u043e\u0432\u043e\u0440\u044b \n\u0414\u0430\u0432\u043d\u043e \u0443\u0436\u0435 \u0437\u0430\u0448\u043b\u0438 \u0432 \u0442\u0443\u043f\u0438\u043a. \n\u0420\u0435\u0439\u043a\u044c\u044f\u0432\u0438\u043a \u0434\u0430\u043b \u043f\u043e\u043d\u044f\u0442\u044c \u043d\u0430\u0440\u043e\u0434\u0443 \u2014 \n\u041a\u0430\u043a \u0413\u043e\u0440\u0431\u0430\u0447\u0451\u0432 \u0443 \u043d\u0430\u0441 \u0432\u0435\u043b\u0438\u043a. \n\n\u0410\u2026 \n\u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \n\u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \n\n\u0412 \u041d\u0435\u0432\u0430\u0434\u0435 \u0442\u0430\u043a \u0436\u0435, \u043a\u0430\u043a \u0438 \u0440\u0430\u043d\u044c\u0448\u0435, \n\u0412\u0437\u0440\u044b\u0432\u0430\u044e\u0442 \u043c\u043d\u043e\u0433\u043e \u043a\u0438\u043b\u043e\u0442\u043e\u043d\u043d. \n\u0420\u044d\u0439-\u0433\u0430\u043d \u0443\u0441\u0442\u0440\u043e\u0438\u043b \u0438\u0437 \u043f\u043b\u0430\u043d\u0435\u0442\u044b \n\u041e\u0433\u0440\u043e\u043c\u043d\u044b\u0439 \u043c\u043e\u0449\u043d\u044b\u0439 \u043f\u043e\u043b\u0438\u0433\u043e\u043d. \n\u041d\u0430\u0440\u043e\u0434\u044b \u0432 \u0433\u043e\u043b\u043e\u0434\u0435 \u0438 \u0432 \u0432\u043e\u0439\u043d\u0430\u0445 \n\u0422\u0435\u0440\u044f\u044e\u0442 \u0442\u044b\u0441\u044f\u0447\u0438 \u0436\u0438\u0437\u043d\u0435\u0301\u0439; \n\u0410 \u0432 \u0412\u0430\u0448\u0438\u043d\u0433\u0442\u043e\u043d\u0435, \u0411\u0435\u043b\u043e\u043c \u0434\u043e\u043c\u0435 \n\u0420\u0435\u0448\u0430\u044e\u0442 \u043a\u0430\u043a \u0443\u0431\u0438\u0442\u044c \u0434\u0435\u0442\u0435\u0439. \n\n\u0422\u043e\u0442\u2026 \n\u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \n\u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440! \u0420\u0435\u0439\u0433\u0430\u043d-\u043f\u0440\u043e\u0432\u043e\u043a\u0430\u0442\u043e\u0440!",
]
documents = [
    "(Title:) REAGAN PROVOCATEUR \n(Poet:) Andrey “Swine” Panov \n\nAgain a weather barring travel, \nBut fate not easily escaped, \nOur moratorium is blaring — \nBut USA don't give a shit… \nNegotiations in Geneva, \nAchieve an impasse again, \nReykjavik really showed the people — \nOur Gorbachev the greater man. \n\nAnd… \nReagan — provocateur! Reagan — provocateur! \nReagan — provocateur! Reagan — provocateur! \n\nWhile in Nevada, same as ever, \nThey blow up many kilotons, \nRay-gun is splitting up our planet \nIn mighty weapons testing zones. \nAmid the war and hunger, peoples \nAre losing millions of lives; \nWhile back in Washington, the White House\nTries to ensure no child survives. \n\nExcept… \nReagan — provocateur! Reagan — provocateur! \nReagan — provocateur! Reagan — provocateur!",
    "THAT PROVOCATIVE REAGAN \n(Poet:) Andrew “The Pig” Panoff \n\nOnce again, the weather is unfavorable for flying, \nAnd there is no escaping fate. \nWe declare a moratorium— \nThe Americans don't care… \nIn Geneva, all negotiations \nHave long since reached an impasse. \nReykjavik made it clear to the people — \nHow great Gorbachev is to us. \n\nAh... \nReagan the provocateur! Provocateur Reagan! \nReagan the provocateur! Provocateur Reagan! \n\nIn Nevada, just like before, \nThey detonate thousands of kilotons. \nReagan has turned the planet \nInto a huge, powerful testing ground. \nPeople are starving and fighting wars, \nLosing thousands of lives; \nAnd in Washington, in the White House, \nThey decide how to kill children. \n\nThat... \nProvocateur Reagan! Reagan the provocateur! \nHe... Provocateur Reagan! Reagan the provocateur!",
    '(Title:) I GROW BITCHY \n(Songwriter: Yanka Dyagileva) \n\nI irrevocably grow bitchy \nEvery night, with every chuckle, \nEvery emptied out glass cup \nI go on boarding up the doors \nAnd letting mean and hungry dogs \nFrom all the chains run freely \nWhat else could we do – \nWe who inherit only kneecaps blistered over \nI irrevocably grow bitchy every time I \n\nI’m educated \nTo be iron barrel’s latched continuation \nOf a rifle the arm shaft \nSit if you wanna \nHave a smoke beside me on a little bench – into the ground staring \n\nWhere else could we go – we who inherit only dirtiest of pathways \nI irrevocably grow bitchy by the hour \n\nI’m irrevocably made bitchy \nEvery sighting of a cop hat, or a fancy mink fur hat \nOut where the wartime never ends, \nWhere springtime never really sets, where childhood never continues, \nWhere else could we turn – we who are left with only dreams and conversations, \nI irrevocably grow bitchy by the hour \nI irrevocably grow bitchy every step I \nI irrevocably grow bitchy every time I',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.1512, -0.5952,  0.8725]])

Training Details

Training Dataset

Unnamed Dataset

Size: 43 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 43 samples:

	anchor	positive	negative
type	string	string	string
details	min: 8 tokens mean: 235.65 tokens max: 448 tokens	min: 8 tokens mean: 219.33 tokens max: 442 tokens	min: 8 tokens mean: 193.65 tokens max: 361 tokens

Samples:

anchor	positive	negative
(Название:) ЗВЕЗДА ПО ИМЕНИ СОЛНЦЕ (Поэт:) Виктор Цой Белый снег, серый лёд На растрескавшейся земле Одеялом лоскутным на ней Город в дорожной петле А над городом – плывут облака Закрывая небесный свет А над городом жёлтый дым Городу две тысячи лет Прожитых под светом звезды по имени Солнце И две тысячи лет война Война без особых причин Война дело молодых Лекарство против морщин Красная-красная кровь Через час уже просто земля Через два — на ней цветы и трава Через три — она снова жива И согрета лучами звезды по имени Солнце И мы знаем, что так было всегда Что судьбою больше любим Кто живёт по законам другим И кому умирать молодым Он не помнит слово «да» и слово «нет» Он не помнит ни чинов, ни имён И способен дотянуться до звёзд Не считая, что это сон И упасть опалённым звездой по имени Солнце	(Title:) STAR KNOWN AS THE SUN (Poet:) Viktor Tsoy White of snow, grey of ice, Over earth splintered dry with cracks As a blanket unrolling on there — Noosed in roads, a metropolis. While above it – rolling clouds’ slow tide All the heavenly light conceals. On the city fall yellow fumes – It had stood for two thousand years. Lived in the light of that star known as the sun there. And for two thousand years, there’s war – And this war has no special cause. To war – is the work for the young, Common treatment against wrinkles. Blood runs crimson and blood runs red – In an hour, as it were, mere earth, After two – grass, flowers on it, And in three it’s, once more alive, And is warmed by the light of that star known as the sun still. And we know it had always been thus, That the fates tend to love and approve Those who live by differing laws, And reach death while still with youth. Knowing not of words like “yes" or “no", Who forget all the ranks and roles A...	(Title:) THAT SUN NAMED STAR (Poet:) Viktor Tsoi Snow is white, while ice is gray When it’s on cracking ground A big blanket covering it A city in a traffic loop And clouds swim above the city Blocking tout he light of the sky And the yellow smoke above the city This city has already turned two thousand years old Lived under the light of a star called the Sun And thousand years long also is a war War without any particular reason War is an enterprise perfect for the young A cure for wrinkles Red, red seems the blood An hour later, just simply soil Two hours later, flowers and grass on it Three hours later, it is alive again And warmed by the rays of a star called Sun And we know that it has always been something like this That by fate is categorically preferred That fellow who lives by other sorts of laws And who is supposed to die young He does not recall what is that word “yes” or that word “no” He does not remember hierarchical formal ranks or names A...

(Название:) “Вечером желтым как зрелый колос…“ 
(Поэт:) Константин Вагинов 

Вечером желтым как зрелый колос, 
Средь случайных дорожных берез, 
Цыганенок плакал голый, 
Вспоминал он имя свое; 

Но не мог никак он вспомнить – 
Кто, откуда, зачем он здесь; 
Слышал матери шепот любовный; 
Но не видел ее нигде. 

На дороге воробьи чирикают – 
Чирик, чирик и по дороге скок; 
И девушки уносят землянику; 
Но завтра солнце озарит восток.

(Title:) “Come one evening, as yellow as ripe grain…“    (Poet:) Konstantin Vaginov 

Come one evening, as yellow as ripe grain, 
Between birches there strewn by the road, 
Wept a small gypsy boy, bare-naked, 
For his name he no longer recalled; 

How he tried to, but couldn’t remember – 
Wherefrom, or wherefore, he’d fared; 
Heard his mother whispering tender; 
He would look, but she wasn’t there. 

On the road, baby sparrows chirp past him – 
Chirpy-chirpy, up the road they would frisk; 
Girls are passing with strawberry baskets; 
But tomorrow, the sun would flood the east.

At Night… 
(Poetry:) Vaginoff 

At night, as yellowy as ripened ears of corn, 
Amidst random roadside birches, 
A naked gypsy-wanderer child cried. 
He remembered his name, 

But he couldn't remember 
Who he was, where he came from, why he was here. 
He heard his mommy’s loving speaking under her breath, 
But he couldn't see her around him. 

Sparrows squeak on the highway  
Squeaking, squeaking, and bounce along the highway – 
And women carry away strawberries, 
But the very next day the sun will shine in the eastern direction.

| |

(Название:) НА ЧËРНЫЙ ДЕНЬ 
(Поэт:) Янка Дягилева 

На чёрный день усталый танец пьяных глаз, дырявых рук 
Второй упал, четвёртый сел, восьмого вывели на круг 
На провода из-под колёс да на три буквы из-под асфальта 
В тихий омут буйной головой 
В холодный пот — расходятся круги 

Железный конь, защитный цвет, резные гусеницы в ряд 
Аттракцион для новичков — по кругу лошади летят 
А заводной калейдоскоп гремит кривыми зеркалами 
Колесо вращается быстрей 
Под звуки марша головы долой 

Поела моль цветную шаль, на картах тройка и семёрка 
Бык хвостом сгоняя мух с тяжёлым сердцем лезет в горку 
Лбов бильярдные шары от столкновения раскатились 
Пополам по обе стороны 
Да по углам просторов и широт 

А за осколками витрин обрывки праздничных нарядов 
Под полозьями саней живая плоть чужих раскладов 
За прилавком попугай из шапки достаёт билеты на трамвай 
До ближнего моста 
На вертолёт без окон и дверей — 
В тихий омут буйной головой — 
Колесо вращается быстрей...

(Title:) COME RAINY DAY 
(Poet:) Yanka Dyaghileva 

Come rainy day a weary dance of drunken eyes, of clumsy hands 
Odd person fell, each fourth was penned, each eighth was rounded for a spin 
Across the wiring under wheels and triple letters under asphalt 
Quiet washers whirlpool rowdy heads 
Unto cold sweat — their spinners gather steam 

The iron horse, its armored shade, etched rows of caterpillar tracks  
A ride for novices designed — such horses flying endless rounds 
While the kaleidoscope on cogs flips grating crooked fun house mirrors 
And the wheel is spinning faster yet 
Off with their heads, off to the marching band 

A moth consumed the rainbow shawl, a three then seven in the cards 
While swatting flies under its tail a bull grave hearted climbs a mount 
Of forehead lobes, like billiard balls from a collision scattered    Over either side in equal portioned parts 
And into nooks of latitudes and views 

While over shattered store displays hang scraps of celebration wear 
And...

(Title:) ON A DARK DAY 
(Poet:) Yanka Dyagileva 

On a dark day, the exhausted jig of intoxicated looks, hole-covered hands 
The second collapsed, the fourth sat down, every eighth was led into the circle 
On wires from under the tracks and on three letters from under the concrete 
Into a quiet pool with a wild head 
In chilled sweat frothing — circles spread 

An metallic horsey, protective color, carved caterpillars in a row 
An attraction for beginners — horses fly in a circle 
And a wind-up kaleidoscope rattles with crooked mirrors 
The wheel spins faster 
To the sounds of the march, heads down 

The moth ate the colorful fabric, on the cards appear a three and then a seven 
The bull brushes flies with its tail and climbs the hill with a heavy spirit 
Pool balls roll away from the bounce 
In half on both sides 
And in the corners of spaces and meridians 

And behind the shards of shop windows, scraps of festive costumes 
Under the runners of the sleigh, the living flesh of stranger...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 1
learning_rate: 2e-05
num_train_epochs: 4
warmup_ratio: 0.1
prompts: task: classification | query:

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 1
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: task: classification | query:
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
1.0	43	0.231
2.0	86	0.0008
3.0	129	0.0002
4.0	172	0.0

Framework Versions

Python: 3.13.12
Sentence Transformers: 5.2.3
Transformers: 4.57.0.dev0
PyTorch: 2.10.0+cu128
Accelerate: 1.12.0
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}