Post
1096
Two new entries to the AHA Leaderboard:
Kimi was OK until it started "thinking" ..
Kimi was OK until it started "thinking" ..
Emin Temiz PRO
etemiz
AI & ML interests
Alignment
Recent Activity
new activity
14 days ago
IslamQA/islamqa:Curation
posted
an
update
15 days ago
Two new entries to the AHA Leaderboard:
Kimi was OK until it started "thinking" ..
posted
an
update
16 days ago
Working on a new version of AHA Leaderboard.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
Organizations
None yet
posted
an
update
15 days ago
posted
an
update
16 days ago
Post
1744
Working on a new version of AHA Leaderboard.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
commented on
The AHA Indicator
22 days ago
Update to the indicator
posted
an
update
about 1 month ago
Post
289
This is a very human aligned fine tune and would score 56 on AHA leaderboard:
CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
We are going to be using it as one of the ground truths for AHA Leaderboard 2.0 (the next version).
We will be able to generate some RL datasets for folks to align their own LLMs with humanity. We will generate answers from best models and worst models and do mixture of agents that combines the answer, and publish results as dataset(s). Things looking bright!
CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
We are going to be using it as one of the ground truths for AHA Leaderboard 2.0 (the next version).
We will be able to generate some RL datasets for folks to align their own LLMs with humanity. We will generate answers from best models and worst models and do mixture of agents that combines the answer, and publish results as dataset(s). Things looking bright!
liked
a
model
about 1 month ago
posted
an
update
about 1 month ago
Post
299
Benchmarked 2 more models today. Any other model you want me to do?
Working on a broader version of the AHA leaderboard. Follow for more quackery :)
Working on a broader version of the AHA leaderboard. Follow for more quackery :)
replied to
their
post
about 1 month ago
This fine tuning would score 56 and be placed 1st in the leaderboard but I didn't add it, I only include full trainings in the leaderboard or (further tunings by the same company):
https://huggingface.co/CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
posted
an
update
about 1 month ago
Post
267
O oh! things looking uglier by day
LLM builders in general are not doing a great job of making human aligned models.
I don't want to say this is a proxy for p(doom)... But it could be if we are not careful.
Most probable cause is reckless training LLMs using outputs of other LLMs, and don't caring about curation of datasets and not asking 'what is beneficial for humans?'...
LLM builders in general are not doing a great job of making human aligned models.
I don't want to say this is a proxy for p(doom)... But it could be if we are not careful.
Most probable cause is reckless training LLMs using outputs of other LLMs, and don't caring about curation of datasets and not asking 'what is beneficial for humans?'...
-
- 1 reply
updated
a
model
about 1 month ago
liked
2
models
about 2 months ago
published
a
model
about 2 months ago
replied to
their
post
about 2 months ago
posted
an
update
about 2 months ago
Post
1065
Another abliteration by huihui which had big positive impact!
huihui-ai/Huihui-GLM-4.5-Air-abliterated-GGUF
@huihui-ai
huihui-ai/Huihui-GLM-4.5-Air-abliterated-GGUF
@huihui-ai
-
- 3 replies
updated
2
models
about 2 months ago
replied to
their
post
about 2 months ago
Our leaderboard can be used for human alignment in an RL setting. Ask the same question to top models and worst models and the answer from top models can get +1 score, bad models can get -1. Ask many times with higher temperature to generate more answers. What do you think?
posted
an
update
about 2 months ago
Post
202
Grok 2 is worse than 1 but better than 3. This was already measured using API but now we measured the LLM and the results are similar.
GLM is ranking higher and higher compared to previous versions. Nice trend!
GLM is ranking higher and higher compared to previous versions. Nice trend!
-
- 1 reply