Instructions to use IQuestLab/IQuest-Q1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use IQuestLab/IQuest-Q1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="IQuestLab/IQuest-Q1") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("IQuestLab/IQuest-Q1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use IQuestLab/IQuest-Q1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IQuestLab/IQuest-Q1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IQuestLab/IQuest-Q1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/IQuestLab/IQuest-Q1
- SGLang
How to use IQuestLab/IQuest-Q1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "IQuestLab/IQuest-Q1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IQuestLab/IQuest-Q1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "IQuestLab/IQuest-Q1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IQuestLab/IQuest-Q1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use IQuestLab/IQuest-Q1 with Docker Model Runner:
docker model run hf.co/IQuestLab/IQuest-Q1
Upload folder using huggingface_hub
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +1 -0
- chat_template.jinja +193 -0
- config.json +67 -0
- generation_config.json +12 -0
- model-00001-of-00175.safetensors +3 -0
- model-00002-of-00175.safetensors +3 -0
- model-00003-of-00175.safetensors +3 -0
- model-00004-of-00175.safetensors +3 -0
- model-00005-of-00175.safetensors +3 -0
- model-00006-of-00175.safetensors +3 -0
- model-00007-of-00175.safetensors +3 -0
- model-00008-of-00175.safetensors +3 -0
- model-00009-of-00175.safetensors +3 -0
- model-00010-of-00175.safetensors +3 -0
- model-00011-of-00175.safetensors +3 -0
- model-00012-of-00175.safetensors +3 -0
- model-00013-of-00175.safetensors +3 -0
- model-00014-of-00175.safetensors +3 -0
- model-00015-of-00175.safetensors +3 -0
- model-00016-of-00175.safetensors +3 -0
- model-00017-of-00175.safetensors +3 -0
- model-00018-of-00175.safetensors +3 -0
- model-00019-of-00175.safetensors +3 -0
- model-00020-of-00175.safetensors +3 -0
- model-00021-of-00175.safetensors +3 -0
- model-00022-of-00175.safetensors +3 -0
- model-00023-of-00175.safetensors +3 -0
- model-00024-of-00175.safetensors +3 -0
- model-00025-of-00175.safetensors +3 -0
- model-00026-of-00175.safetensors +3 -0
- model-00027-of-00175.safetensors +3 -0
- model-00028-of-00175.safetensors +3 -0
- model-00029-of-00175.safetensors +3 -0
- model-00030-of-00175.safetensors +3 -0
- model-00031-of-00175.safetensors +3 -0
- model-00032-of-00175.safetensors +3 -0
- model-00033-of-00175.safetensors +3 -0
- model-00034-of-00175.safetensors +3 -0
- model-00035-of-00175.safetensors +3 -0
- model-00036-of-00175.safetensors +3 -0
- model-00037-of-00175.safetensors +3 -0
- model-00038-of-00175.safetensors +3 -0
- model-00039-of-00175.safetensors +3 -0
- model-00040-of-00175.safetensors +3 -0
- model-00041-of-00175.safetensors +3 -0
- model-00042-of-00175.safetensors +3 -0
- model-00043-of-00175.safetensors +3 -0
- model-00044-of-00175.safetensors +3 -0
- model-00045-of-00175.safetensors +3 -0
- model-00046-of-00175.safetensors +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,193 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- if tools -%}
|
| 2 |
+
{%- macro tool_to_json(tool) -%}
|
| 3 |
+
{%- set ns_tool = namespace(first=true) -%}
|
| 4 |
+
{{ '{' -}}
|
| 5 |
+
{%- for k, v in tool.items() -%}
|
| 6 |
+
{%- if k != 'defer_loading' and k != 'strict' -%}
|
| 7 |
+
{%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
|
| 8 |
+
{%- set ns_tool.first = false -%}
|
| 9 |
+
"{{ k }}": {{ v | tojson(ensure_ascii=False) }}
|
| 10 |
+
{%- endif -%}
|
| 11 |
+
{%- endfor -%}
|
| 12 |
+
{{- '}' -}}
|
| 13 |
+
{%- endmacro -%}
|
| 14 |
+
{% macro render_extra_keys(json_dict, handled_keys) %}
|
| 15 |
+
{%- if json_dict is mapping %}
|
| 16 |
+
{%- for json_key in json_dict if json_key not in handled_keys %}
|
| 17 |
+
{%- if json_dict[json_key] is mapping or (json_dict[json_key] is sequence and json_dict[json_key] is not string) %}
|
| 18 |
+
{{- '\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | tojson | safe) ~ '</' ~ json_key ~ '>' }}
|
| 19 |
+
{%- else %}
|
| 20 |
+
{{-'\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | string) ~ '</' ~ json_key ~ '>' }}
|
| 21 |
+
{%- endif %}
|
| 22 |
+
{%- endfor %}
|
| 23 |
+
{%- endif %}
|
| 24 |
+
{%- endmacro -%}
|
| 25 |
+
{{- '<|iquest_system|>\n# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 26 |
+
{%- for tool in tools %}
|
| 27 |
+
{%- if tool.function is defined %}
|
| 28 |
+
{%- set tool = tool.function %}
|
| 29 |
+
{%- endif %}
|
| 30 |
+
{%- set _req = tool.parameters.required if (tool.parameters is defined and tool.parameters is mapping and tool.parameters.required is defined and tool.parameters.required is iterable and tool.parameters.required is not string) else [] %}
|
| 31 |
+
{{- '\ndef ' ~ tool.name ~ '(' }}
|
| 32 |
+
{%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
|
| 33 |
+
{%- set _ns = namespace(first=true) %}
|
| 34 |
+
{%- for param_name, param_fields in tool.parameters.properties|items %}
|
| 35 |
+
{%- if not _ns.first %}{{- ', ' }}{%- endif %}
|
| 36 |
+
{{- param_name }}
|
| 37 |
+
{%- if param_fields.type is defined %}{{- ': ' ~ (param_fields.type | string) }}{%- endif %}
|
| 38 |
+
{%- if param_name not in _req %}{{- ' = None' }}{%- endif %}
|
| 39 |
+
{%- set _ns.first = false %}
|
| 40 |
+
{%- endfor %}
|
| 41 |
+
{%- endif %}
|
| 42 |
+
{{- ') -> any:' }}
|
| 43 |
+
{{- '\n """' }}
|
| 44 |
+
{%- if tool.description is defined %}
|
| 45 |
+
{{- (tool.description | trim) }}
|
| 46 |
+
{%- endif %}
|
| 47 |
+
{%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
|
| 48 |
+
{{- '\n\n Args:' }}
|
| 49 |
+
{%- for param_name, param_fields in tool.parameters.properties|items %}
|
| 50 |
+
{{- '\n ' ~ param_name }}
|
| 51 |
+
{{- ' (' }}
|
| 52 |
+
{%- if param_fields.type is defined %}{{- (param_fields.type | string) }}{%- else %}{{- 'any' }}{%- endif %}
|
| 53 |
+
{%- if param_name not in _req %}{{- ', optional' }}{%- endif %}
|
| 54 |
+
{{- ')' }}
|
| 55 |
+
{%- if param_fields.description is defined %}
|
| 56 |
+
{{- ': ' ~ (param_fields.description | trim) }}
|
| 57 |
+
{%- endif %}
|
| 58 |
+
{%- set handled_keys = ['name', 'type', 'description'] %}
|
| 59 |
+
{{- render_extra_keys(param_fields, handled_keys) }}
|
| 60 |
+
{%- endfor %}
|
| 61 |
+
{%- endif %}
|
| 62 |
+
{% set handled_keys = ['type', 'properties', 'required'] %}
|
| 63 |
+
{{- render_extra_keys(tool.parameters, handled_keys) }}
|
| 64 |
+
{{- '\n """' }}
|
| 65 |
+
{%- set handled_keys = ['type', 'name', 'description', 'parameters'] %}
|
| 66 |
+
{{- render_extra_keys(tool, handled_keys) }}
|
| 67 |
+
{%- endfor %}
|
| 68 |
+
{{- "\n</tools>" }}
|
| 69 |
+
{{- '\n\nFor each function call, output the function name and arguments within the following XML format:\n<iquest_tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</iquest_tool_call>' }}
|
| 70 |
+
{{- '<|iquest_end|>' -}}
|
| 71 |
+
{%- endif -%}
|
| 72 |
+
{%- macro visible_text(content) -%}
|
| 73 |
+
{%- if content is undefined or content is none -%}
|
| 74 |
+
{{- '' }}
|
| 75 |
+
{%- elif content is string -%}
|
| 76 |
+
{{- content }}
|
| 77 |
+
{%- elif content is iterable and content is not mapping -%}
|
| 78 |
+
{%- for item in content -%}
|
| 79 |
+
{%- if item is mapping and item.type == 'text' -%}
|
| 80 |
+
{{- item.text }}
|
| 81 |
+
{%- elif item is string -%}
|
| 82 |
+
{{- item }}
|
| 83 |
+
{%- elif item is mapping and item.type in ['image', 'image_url', 'video', 'video_url', 'audio', 'audio_url', 'input_audio'] -%}
|
| 84 |
+
{%- set media_type = item.type | replace('_url', '') | replace('input_', '') -%}
|
| 85 |
+
{{- "<reminder>You are unable to process this " ~ media_type ~ " because you don't have multi-modal input ability. Try different methods.</reminder>" }}
|
| 86 |
+
{%- endif -%}
|
| 87 |
+
{%- endfor -%}
|
| 88 |
+
{%- else -%}
|
| 89 |
+
{{- content }}
|
| 90 |
+
{%- endif -%}
|
| 91 |
+
{%- endmacro -%}
|
| 92 |
+
{%- set ns = namespace(last_user_index=-1) -%}
|
| 93 |
+
{%- for m in messages %}
|
| 94 |
+
{%- if m.role == 'user' %}
|
| 95 |
+
{%- set ns.last_user_index = loop.index0 -%}
|
| 96 |
+
{%- endif %}
|
| 97 |
+
{%- endfor %}
|
| 98 |
+
{%- for m in messages -%}
|
| 99 |
+
{%- if m.role == 'user' -%}{{- '<|iquest_user|>' -}}{{ visible_text(m.content) }}{{- '<|iquest_end|>' -}}
|
| 100 |
+
{%- elif m.role == 'assistant' -%}
|
| 101 |
+
{{- '<|iquest_assistant|>' -}}
|
| 102 |
+
{%- set content = visible_text(m.content) %}
|
| 103 |
+
{%- if m.reasoning_content is string %}
|
| 104 |
+
{%- set reasoning_content = m.reasoning_content %}
|
| 105 |
+
{%- elif m.reasoning_content is none %}
|
| 106 |
+
{%- set reasoning_content = '' %}
|
| 107 |
+
{%- elif '</think>' in content %}
|
| 108 |
+
{%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
|
| 109 |
+
{%- set content = content.split('</think>')[-1] %}
|
| 110 |
+
{%- endif %}
|
| 111 |
+
{%- if (clear_thinking is not defined or not clear_thinking or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
|
| 112 |
+
{%- if m.reasoning_content_mask is defined and not m.reasoning_content_mask -%}
|
| 113 |
+
{%- generation -%}{{- '<think>' + reasoning_content + '</think>' -}}{%- endgeneration -%}
|
| 114 |
+
{%- else -%}
|
| 115 |
+
{{- '<think>' -}}
|
| 116 |
+
{{- reasoning_content -}}
|
| 117 |
+
{%- if m.boundary_token_mask is defined and not m.boundary_token_mask -%}
|
| 118 |
+
{%- generation -%}{{- '</think>' -}}{%- endgeneration -%}
|
| 119 |
+
{%- else -%}
|
| 120 |
+
{{- '</think>' -}}
|
| 121 |
+
{%- endif %}
|
| 122 |
+
{%- endif -%}
|
| 123 |
+
{%- else -%}
|
| 124 |
+
{{- '<think>' -}}
|
| 125 |
+
{%- if m.boundary_token_mask is defined and not m.boundary_token_mask -%}
|
| 126 |
+
{%- generation -%}{{- '</think>' -}}{%- endgeneration -%}
|
| 127 |
+
{%- else -%}
|
| 128 |
+
{{- '</think>' -}}
|
| 129 |
+
{%- endif %}
|
| 130 |
+
{%- endif -%}
|
| 131 |
+
{%- if m.content_mask is defined and not m.content_mask -%}
|
| 132 |
+
{%- generation -%}
|
| 133 |
+
{%- if content.strip() -%}{{- content.strip() -}}{%- endif -%}
|
| 134 |
+
{%- if m.tool_calls -%}
|
| 135 |
+
{%- for tc in m.tool_calls -%}
|
| 136 |
+
{%- if tc.function -%}{%- set tc = tc.function -%}{%- endif -%}
|
| 137 |
+
{{- '<iquest_tool_call>' + tc.name -}}
|
| 138 |
+
{%- set _args = (tc.arguments | fromjson) if tc.arguments is string else tc.arguments -%}{%- if _args is not mapping -%}{{- raise_exception('tool call arguments must be a JSON object') -}}{%- endif -%}{%- for k, v in _args.items() -%}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{%- endfor -%}</iquest_tool_call>{%- endfor -%}
|
| 139 |
+
{%- endif -%}
|
| 140 |
+
{{- '<|iquest_end|>' -}}
|
| 141 |
+
{%- endgeneration -%}
|
| 142 |
+
{%- else -%}
|
| 143 |
+
{%- if content.strip() -%}{{- content.strip() -}}{%- endif -%}
|
| 144 |
+
{%- if m.tool_calls -%}
|
| 145 |
+
{%- for tc in m.tool_calls -%}
|
| 146 |
+
{%- if tc.function -%}{%- set tc = tc.function -%}{%- endif -%}
|
| 147 |
+
{{- '<iquest_tool_call>' + tc.name -}}
|
| 148 |
+
{%- set _args = (tc.arguments | fromjson) if tc.arguments is string else tc.arguments -%}{%- if _args is not mapping -%}{{- raise_exception('tool call arguments must be a JSON object') -}}{%- endif -%}{%- for k, v in _args.items() -%}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{%- endfor -%}</iquest_tool_call>{%- endfor -%}
|
| 149 |
+
{%- endif -%}
|
| 150 |
+
{%- if m.boundary_token_mask is defined and not m.boundary_token_mask -%}
|
| 151 |
+
{%- generation -%}{{- '<|iquest_end|>' -}}{%- endgeneration -%}
|
| 152 |
+
{%- else -%}
|
| 153 |
+
{{- '<|iquest_end|>' -}}
|
| 154 |
+
{%- endif %}
|
| 155 |
+
{%- endif -%}
|
| 156 |
+
{%- elif m.role == 'tool' -%}
|
| 157 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 158 |
+
{{- '<|iquest_tool|>' -}}
|
| 159 |
+
{%- endif %}
|
| 160 |
+
{%- if m.content is string -%}
|
| 161 |
+
{{- '<tool_response>' + m.content + '</tool_response>' -}}
|
| 162 |
+
{%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0.type == "tool_reference" -%}
|
| 163 |
+
{{- '<tool_response><tools>\n' -}}
|
| 164 |
+
{% for tr in m.content %}
|
| 165 |
+
{%- for tool in tools -%}
|
| 166 |
+
{%- if 'function' in tool -%}
|
| 167 |
+
{%- set tool = tool['function'] -%}
|
| 168 |
+
{%- endif -%}
|
| 169 |
+
{%- if tool.name == tr.name -%}
|
| 170 |
+
{{- tool_to_json(tool) + '\n' -}}
|
| 171 |
+
{%- endif -%}
|
| 172 |
+
{%- endfor -%}
|
| 173 |
+
{%- endfor -%}
|
| 174 |
+
{{- '</tools></tool_response>' -}}
|
| 175 |
+
{%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0 is mapping and m.content.0.output is defined -%}
|
| 176 |
+
{%- for tr in m.content -%}
|
| 177 |
+
{{- '<tool_response>' + tr.output + '</tool_response>' -}}
|
| 178 |
+
{%- endfor -%}
|
| 179 |
+
{%- else -%}
|
| 180 |
+
{{- '<tool_response>' + visible_text(m.content) + '</tool_response>' -}}
|
| 181 |
+
{% endif -%}
|
| 182 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") -%}
|
| 183 |
+
{{- '<|iquest_end|>' -}}
|
| 184 |
+
{%- endif -%}
|
| 185 |
+
{%- elif m.role == 'system' -%}
|
| 186 |
+
{{- '<|iquest_system|>' -}}{{ visible_text(m.content) }}{{- '<|iquest_end|>' -}}
|
| 187 |
+
{%- elif m.role == 'developer' -%}
|
| 188 |
+
{{- '<|iquest_developer|>' -}}{{ visible_text(m.content) }}{{- '<|iquest_end|>' -}}
|
| 189 |
+
{%- endif -%}
|
| 190 |
+
{%- endfor -%}
|
| 191 |
+
{%- if add_generation_prompt -%}
|
| 192 |
+
{{- '<|iquest_assistant|>' -}}{{- '<think></think>' if (enable_thinking is defined and not enable_thinking) else '<think>' -}}
|
| 193 |
+
{%- endif -%}
|
config.json
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"IQuestQ1ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"attn_out_scale": 1.0,
|
| 8 |
+
"dense_intermediate_size": 12288,
|
| 9 |
+
"dtype": "bfloat16",
|
| 10 |
+
"embed_scale": 1.0,
|
| 11 |
+
"enable_sink_attention": true,
|
| 12 |
+
"eos_token_id": 0,
|
| 13 |
+
"ffn_out_scale": 0.53881590608,
|
| 14 |
+
"first_layer_attn_out_scale": 1.0,
|
| 15 |
+
"first_layer_ffn_out_scale": 1.0,
|
| 16 |
+
"first_layers_types": [
|
| 17 |
+
"full_attention"
|
| 18 |
+
],
|
| 19 |
+
"fuse_sink_attention": true,
|
| 20 |
+
"head_dim": 128,
|
| 21 |
+
"hidden_act": "silu",
|
| 22 |
+
"hidden_size": 3072,
|
| 23 |
+
"hybrid_layers_types_block": [
|
| 24 |
+
"full_attention",
|
| 25 |
+
"sliding_attention",
|
| 26 |
+
"sliding_attention",
|
| 27 |
+
"sliding_attention"
|
| 28 |
+
],
|
| 29 |
+
"intermediate_size": 1536,
|
| 30 |
+
"last_layers_types": [
|
| 31 |
+
"full_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"full_attention"
|
| 34 |
+
],
|
| 35 |
+
"logit_scale": 1.0,
|
| 36 |
+
"max_position_embeddings": 524288,
|
| 37 |
+
"mlp_only_layers": [
|
| 38 |
+
0
|
| 39 |
+
],
|
| 40 |
+
"model_type": "iquest_q1",
|
| 41 |
+
"moe_router_dtype": "fp32",
|
| 42 |
+
"enable_lm_head_fp32": true,
|
| 43 |
+
"no_rope_layers": [],
|
| 44 |
+
"num_attention_heads": 48,
|
| 45 |
+
"num_experts": 256,
|
| 46 |
+
"num_experts_per_tok": 8,
|
| 47 |
+
"num_hidden_layers": 88,
|
| 48 |
+
"num_hybrid_layers_block": 21,
|
| 49 |
+
"num_key_value_heads": 8,
|
| 50 |
+
"num_mtp_layers": 0,
|
| 51 |
+
"rms_norm_eps": 1e-06,
|
| 52 |
+
"rope_theta": 1000000,
|
| 53 |
+
"rotary_dim": 32,
|
| 54 |
+
"shared_kv_num_layers": 0,
|
| 55 |
+
"shared_kv_source_begin": 0,
|
| 56 |
+
"shared_kv_target_begin": 0,
|
| 57 |
+
"sliding_window": 4096,
|
| 58 |
+
"softmax_scale": null,
|
| 59 |
+
"swa_rope_theta": 10000.0,
|
| 60 |
+
"tie_word_embeddings": false,
|
| 61 |
+
"transformers_version": "4.57.1",
|
| 62 |
+
"use_cache": true,
|
| 63 |
+
"use_hybrid_layers": true,
|
| 64 |
+
"use_over_encoding": false,
|
| 65 |
+
"use_sliding_window": true,
|
| 66 |
+
"vocab_size": 160000
|
| 67 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_sample": true,
|
| 3 |
+
"eos_token_id": [
|
| 4 |
+
0
|
| 5 |
+
],
|
| 6 |
+
"pad_token_id": 0,
|
| 7 |
+
"top_k": 20,
|
| 8 |
+
"top_p": 0.95,
|
| 9 |
+
"transformers_version": "4.57.1",
|
| 10 |
+
"trust_remote_code": true,
|
| 11 |
+
"temperature": 1.0
|
| 12 |
+
}
|
model-00001-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:22059c1cc63f72190372438935c523b944fa3e73af52d50c494cc879f508248c
|
| 3 |
+
size 4831838352
|
model-00002-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6abaee94f41cad481a112e41e8f0deaf3ebcbf8c49885885f0ced3e91d17f45c
|
| 3 |
+
size 2415919256
|
model-00003-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3ee3d3fb2fac3147a27f9e6f763edcedb86b475288865421895d11737dacc830
|
| 3 |
+
size 4831838352
|
model-00004-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:00a76e2d6e3112e2cab4f5f563341c9c141b295a13932baa36a7ff1cca218f22
|
| 3 |
+
size 2415919256
|
model-00005-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a0c470502b0c9694adc05ef08fb0a85b105fc3d6fa3606a273bb3286b79756ae
|
| 3 |
+
size 4869587208
|
model-00006-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8847a6594de99d7937aa5611fc555fe0d7490cfdd77f21fee5b19e6e695ca4cf
|
| 3 |
+
size 2417492232
|
model-00007-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6918ceaeafa051acbbf18b524b36ec7af65b3d32563257278af6e248733877e2
|
| 3 |
+
size 4871160184
|
model-00008-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e911125b1adca14e156ec5c9709f73d9bdfc69231a5895c680e5077fedc76ea6
|
| 3 |
+
size 2417492232
|
model-00009-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6c88f6b98f5885cca7ca23e7f51ddd89f5b18067c1a230ec151b9d4cd1b0c528
|
| 3 |
+
size 4869587208
|
model-00010-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:09031c38202a55d6cb17919130067e6e12eccf51ed889f54f9b4533bf8972260
|
| 3 |
+
size 2417492232
|
model-00011-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7382120c79dc78e3b542bf95ef7b2f0355b34cc23c9e9b924db4a91557cced38
|
| 3 |
+
size 4871160184
|
model-00012-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a591dfab89f8c2fccce597016a48c122f5953d0143dfed2750b0fa9f07c1f158
|
| 3 |
+
size 2417492232
|
model-00013-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e50f3a5c8081210fcfec184f82b6d6821ca00c511e4bc287e1718e795bfdb24b
|
| 3 |
+
size 4869587208
|
model-00014-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1fa6b13de1b8cdc8146c89787f4a5177c6a74e6b1f744eec1948478bcf09a7dc
|
| 3 |
+
size 2417492232
|
model-00015-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e0050e527e23ba89a8874a41ecd1773450554d81979f12e0bf19ca79cd013195
|
| 3 |
+
size 4871160184
|
model-00016-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:43872c01866beec659d4f9f8b687762861c020c67dd3a9da15fb8c4a96ff50a2
|
| 3 |
+
size 2417492232
|
model-00017-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95c406e592a5e474acad993c513f15f2c072e2144456076b545b67dd8fe767dc
|
| 3 |
+
size 4869587208
|
model-00018-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e807e89a6e4186d44512134432cf584aed2e181995400950e022fb8771bb909c
|
| 3 |
+
size 2417492232
|
model-00019-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6400428d1a498f4511b3285e831eb85e4cf11dc0bd392d12a3159fa349f4199c
|
| 3 |
+
size 4869587208
|
model-00020-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dcb213f870c15df1ff71f61b3c9ddad6677847c85cf5fcddd7a2a18c2e08dcac
|
| 3 |
+
size 2417492232
|
model-00021-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:65eee68ddb928e1711d7d082fff2c052e3e206c680ea364d255fb3d36ae3b6d6
|
| 3 |
+
size 4869587208
|
model-00022-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:87dae448a722f4b31586fb69b36c8612b5398c6bd11e7f6097fb2c83ca020d8f
|
| 3 |
+
size 2417492232
|
model-00023-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2ac77a6f9296c1795eeb95a03931f84ba1a8450b840cb825526b4691b3dc6278
|
| 3 |
+
size 4869587208
|
model-00024-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1352a8fa22c77702cd24a2a07f3b60af8c45ec51bb8c8ce5ef283f9f48accf36
|
| 3 |
+
size 2417492232
|
model-00025-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4867ac92c69b8be1b6366d621a3c7cba4e88b394dc09e80ee809d604d841116d
|
| 3 |
+
size 4869587208
|
model-00026-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d26e4015ec4ed971e167ddaf822154b6bddec48f61d69c0bdc8e65e40f60e52b
|
| 3 |
+
size 2417492232
|
model-00027-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d0f00d2948db6c972778f56cbca64799ead1d226b35dd26e854729bbd6d2d128
|
| 3 |
+
size 4869587208
|
model-00028-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c2005abfb9bcf14bd044eabe6ef673c5877c67626fafd41397be5cc7367079d2
|
| 3 |
+
size 2417492232
|
model-00029-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cbd1df28f1622087d70327ad16ff79a5ab8af53b71670043e244a90625be0f29
|
| 3 |
+
size 4869587208
|
model-00030-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:46d062b992daa79bdf013640eed453c8128dc14e893edae1992ff7ab01daf005
|
| 3 |
+
size 2415919256
|
model-00031-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:00c698c4609b8f49e9d7c0f1ad2b3c1a9b3a0cf11063b87019a8bcdefdb6a8f9
|
| 3 |
+
size 4831865504
|
model-00032-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aab16b4e71ba84202a5168d72acbc7a8bb0d7627e8882a937f25f0509be80003
|
| 3 |
+
size 2415933912
|
model-00033-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9a2fcc472889b58a81158e67c05eada8f2e1415a8a79e8a38324ee76ed0a955d
|
| 3 |
+
size 4831844608
|
model-00034-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:56da7ea02169f62ff0aaaff143a556cb61b7e290144c93a4408895495c95c337
|
| 3 |
+
size 2415940912
|
model-00035-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:41d08c4b43f3a3b8948a99cf9eef27b9da82ed822fb3d539c835f3a5e733568a
|
| 3 |
+
size 4869593464
|
model-00036-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2ca3bebff98e2ba099f3a4ce57a38dee5f547b658fa73524227f287010a786ef
|
| 3 |
+
size 2417507632
|
model-00037-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b7e68180aaf3de2c8a6df6d6ca36ac1422302840f6a77bf83934d9dbf08c9dec
|
| 3 |
+
size 4871166808
|
model-00038-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ebbdb0cc62b8ceb1e7720c658a4df3d62997eb4ee3fc9af832caaab00bfe85be
|
| 3 |
+
size 2417513520
|
model-00039-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a8bec27ef8b09f28a51623a056012966d5bcd6fe390eaa7d8aceeb1b1713df4d
|
| 3 |
+
size 4869595984
|
model-00040-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3d465cc542ae4adc877aedaf80a4e10f742f655eca11fcd7b2894e45b209372f
|
| 3 |
+
size 2417505480
|
model-00041-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:205f10da95194c7cd51c59e4ac41c5198893130975fdbcb664d58d6b091281f0
|
| 3 |
+
size 4871175224
|
model-00042-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c305723df9bc7c14a6e844635ad8ad5773f4089bfc45e5ed003fb639262d2331
|
| 3 |
+
size 2417507264
|
model-00043-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:548a95c5b3aa41e231b24d424f32bb52ee7e57e38db27c996ec5fc5432d20d47
|
| 3 |
+
size 4869593832
|
model-00044-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:75ea05dc0f356de4011a4d0d7da2d0534437de2bd7d1549083c666bc05585355
|
| 3 |
+
size 2417513888
|
model-00045-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c5ab92dc4cfbfc2ae9a827305b9f5b2cd4bef77547c832fc0ea5ce56ed6ca174
|
| 3 |
+
size 4871166440
|
model-00046-of-00175.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:48ac4b7cd157f95612b9a033f8d40be9328890243435f4c276596e7177982a0b
|
| 3 |
+
size 2417507632
|