OpenMOSE/RWKV-Qwen3-30B-A3B-Instruct-cxa07c

this is full RWKV

hf (pretrained=/workspace/output/qwen3-30b/hf_temp_cxa07c,trust_remote_code=True), gen_kwargs: (max_gen_toks=10), limit: None, num_fewshot: None, batch_size: 4
|    Tasks    |Version|Filter|n-shot|Metric|   |Value|   |Stderr|
|-------------|------:|------|-----:|-----:|---|----:|---|------|
|niah_single_1|      1|none  |     0| 16384|↑  |0.378|±  |   N/A|
|             |       |none  |     0|  2048|   |0.996|±  |0.0028|
|             |       |none  |     0|  4096|↑  |0.942|±  |   N/A|
|             |       |none  |     0|  8192|↑  |0.840|±  |   N/A|

hf (pretrained=/workspace/output/qwen3-30b/hf_temp_cxa07c,trust_remote_code=True), gen_kwargs: (max_gen_toks=64), limit: None, num_fewshot: None, batch_size: 4
| Tasks  |Version|Filter|n-shot|Metric|   |Value |   |Stderr|
|--------|------:|------|-----:|-----:|---|-----:|---|------|
|ruler_vt|      1|none  |     0|  4096|↑  |0.4144|±  |   N/A|

|      Groups      |Version|Filter|n-shot|Metric|   |Value |   |Stderr|
|------------------|------:|------|------|------|---|-----:|---|-----:|
|mmlu              |      2|none  |      |acc   |↑  |0.7477|±  |0.0034|
| - humanities     |      2|none  |      |acc   |↑  |0.6227|±  |0.0063|
| - other          |      2|none  |      |acc   |↑  |0.8072|±  |0.0068|
| - social sciences|      2|none  |      |acc   |↑  |0.8560|±  |0.0062|
| - stem           |      2|none  |      |acc   |↑  |0.7697|±  |0.0073|