|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: apache-2.0 |
|
|
library_name: mlc-llm |
|
|
base_model: unsloth/Qwen2.5-0.5B-Instruct |
|
|
tags: |
|
|
- conversational |
|
|
- chat |
|
|
- fine-tuned |
|
|
- mlc |
|
|
- webllm |
|
|
- quantized |
|
|
- q4f16_1 |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
# Groupchat Model - Qwen2.5 0.5B Fine-tuned |
|
|
|
|
|
A fine-tuned Qwen2.5-0.5B model trained to mimic individual users in a group chat, converted to MLC format for browser deployment via WebLLM. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base Model**: [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) |
|
|
- **Fine-tuning Method**: Unsloth |
|
|
- **Quantization**: q4f16_1 (4-bit weights, float16 activations) |
|
|
- **Format**: MLC-LLM for WebLLM deployment |
|
|
|
|
|
## Dataset |
|
|
|
|
|
- **Source**: Private GroupMe group chat |
|
|
- **Size**: ~60,000 messages |
|
|
- **Time Period**: 2013-2016 |
|
|
- **Participants**: 10 users with varying activity levels |
|
|
|
|
|
## Prompt Format |
|
|
|
|
|
The model was fine-tuned using the ChatML format with the following structure: |
|
|
|
|
|
``` |
|
|
You are mimicking users in a group chat. Given the conversation history, respond as the specified user. |
|
|
|
|
|
Recent messages: |
|
|
[Speaker1]: [Message1] |
|
|
[Speaker2]: [Message2] |
|
|
... |
|
|
|
|
|
Respond as [TargetSpeaker]: |
|
|
``` |
|
|
|
|
|
### Example Usage |
|
|
|
|
|
```javascript |
|
|
const prompt = `You are mimicking users in a group chat. Given the conversation history, respond as the specified user. |
|
|
|
|
|
Recent messages: |
|
|
Jackson: what's in a bento box |
|
|
Jackson: That's like a pupu platter right |
|
|
Nick: What's a pupu platter |
|
|
|
|
|
Respond as Spencer:`; |
|
|
``` |
|
|
|
|
|
## Recommended Generation Parameters |
|
|
|
|
|
For best results, use these settings: |
|
|
|
|
|
```javascript |
|
|
{ |
|
|
temperature: 0.8, |
|
|
top_p: 0.9, |
|
|
max_tokens: 32-64, |
|
|
} |
|
|
``` |
|
|
|
|
|
## WebLLM Integration |
|
|
|
|
|
This model is optimized for browser deployment using [WebLLM](https://github.com/mlc-ai/web-llm). |
|
|
|
|
|
### Loading in WebLLM |
|
|
|
|
|
```javascript |
|
|
import * as webllm from "@mlc-ai/web-llm"; |
|
|
|
|
|
const appConfig = webllm.prebuiltAppConfig; |
|
|
|
|
|
// Borrow model_lib from native Qwen2.5-0.5B |
|
|
const qwenLib = appConfig.model_list.find(m => m.model_id === "Qwen2.5-0.5B-Instruct-q4f16_1-MLC"); |
|
|
|
|
|
appConfig.model_list.push({ |
|
|
...qwenLib, |
|
|
model_id: "Groupchat-Qwen2.5-0.5B-q4f16_1", |
|
|
model: "https://huggingface.co/brimtown/Groupchat-Qwen2.5-0.5B-MLC", |
|
|
low_resource_required: true, |
|
|
overrides: { |
|
|
context_window_size: 512 |
|
|
} |
|
|
}); |
|
|
|
|
|
const engine = await webllm.CreateMLCEngine("Groupchat-Qwen2.5-0.5B-q4f16_1", { |
|
|
appConfig: appConfig |
|
|
}); |
|
|
``` |
|
|
|
|
|
## License |
|
|
|
|
|
This model is released under the same license as the base Qwen2.5 model. The training data is private and not included. |
|
|
|
|
|
## Acknowledgments |
|
|
|
|
|
- Fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) |
|
|
- Converted to MLC format using [MLC-LLM](https://github.com/mlc-ai/mlc-llm) |
|
|
- Deployed with [WebLLM](https://github.com/mlc-ai/web-llm) |
|
|
|
