Merging is cheap. Winning is not.

Gate

hidden — Derives the gate from the hidden-state representations of the positive and negative prompts. The documented default and the best of the three. Runs every prompt through the base model. --load-in-8bit or --load-in-4bit reduces the VRAM this needs.

Routing prompts

The gate is derived from these. One line per prompt. Under gate_mode: hidden, every expert needs at least one positive prompt or mergekit has nothing to build a gate from.

Mechanical findings

Each row names the API field it read. There is no overall score and no "compatible" verdict, because the only published predictor of merge collapse is representational distance, which no metadata field carries.

Nothing here blocks the merge. Every one of these findings still costs you something later. Licence: every member declares apache-2.0, and the output inherits it.

Merged size

An exact figure is shared_weights + experts x ffn_weights + router. Splitting a checkpoint into those two parts requires the layer dimensions in the repository's full config.json, which the Hub's config summary omits. This project will not publish a point estimate it cannot derive.

To compute it exactly this page would need, from each member's full config.json: config.json → hidden_size · config.json → intermediate_size · config.json → num_hidden_layers · config.json → vocab_size. The Hub's config summary does not carry them, so no point estimate is offered.

# mergekit-moe configuration generated by MOEModels.ai
# Sources are pinned to the commit revisions indexed in this catalogue snapshot.
base_model: Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
gate_mode: hidden
dtype: bfloat16
experts_per_token: 2
experts:
  - source_model: Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
    # positive_prompts: required by gate_mode "hidden" and not yet written for this expert
  - source_model: Qwen/Qwen3-8B@b968826d9c46dd6066d109eabc6255188de91218
    # positive_prompts: required by gate_mode "hidden" and not yet written for this expert
  - source_model: Qwen/Qwen3-Embedding-0.6B@97b0c614be4d77ee51c0cef4e5f07c00f9eb65b3
    # positive_prompts: required by gate_mode "hidden" and not yet written for this expert

mergekit-moe config.yaml ./out --copy-tokenizer

• Qwen/Qwen3-0.6B has no positive prompts. gate_mode "hidden" derives the router from them, so mergekit cannot build a gate without at least one.
• Qwen/Qwen3-8B has no positive prompts. gate_mode "hidden" derives the router from them, so mergekit cannot build a gate without at least one.
• Qwen/Qwen3-Embedding-0.6B has no positive prompts. gate_mode "hidden" derives the router from them, so mergekit cannot build a gate without at least one.

• gate_mode hidden runs every prompt through the base model. Add --load-in-8bit or --load-in-4bit if the base does not fit in the VRAM you have.

Sources are pinned with mergekit's path@revision syntax to the commits this snapshot indexed on 2026-09-05, so the merge is reproducible against the exact artifacts described above rather than against whatever the default branches hold later. mergekit-moe documentation ↗

The evaluation that would settle it

Not "how does the merge score". Whether it beats each of the models already inside it, on the same questions, on one pinned harness.

1. Pin the harnessOne harness at one commit, one prompt template, one decoding configuration, for every arm. A merged model and its experts evaluated on different harness versions are not comparable, and the difference between harness versions is routinely larger than the effect being chased.
2. Run the arms on identical questions4 arms — the merged artifact and 3 references — over the same question set, in the same order, at temperature 0.
3. Record per-question outcomesStore the score for every question and every arm, not just the aggregate. The paired analysis that makes this affordable is impossible to run afterwards from summary numbers alone.
4. Report the paired differencePublish the difference, its standard error and the per-question correlation between arms — not two averages side by side. Two averages with no error bar cannot distinguish a real gain from sampling noise.
5. Compare against the base rateThe claim is only interesting if the merged artifact beats its best expert. mlabonne/Beyonder-4x7B-v3 did not: 61.91 against 62.74 for mlabonne/AlphaMonarch-7B, one of its own four experts.

• Detecting a 3-point difference at 80% power and alpha 0.05 needs 2,181 questions under these assumptions.
• GPQA Diamond has 198 questions. At that size the smallest difference the comparison can resolve is about 9.9 points, so a 3-point result on it is structurally unresolvable — no amount of careful running fixes a sample size.
• These are sample sizes, not permission. A benchmark large enough to resolve the effect still says nothing about whether the effect transfers to the work you actually do.

No merge, no GPUs, no derivative artifact and therefore no licence chain on an output. The weights stay where they are; a policy decides which endpoint answers. This is the only tier of the three that a browser can carry end to end, and it is measurably under-served.

# Inference-time routing policy generated by MOEModels.ai
# A policy document, not a vendor configuration. No router product reads this format.
# Weights are never modified: every candidate is served from its own endpoint.
kind: routing_policy
version: 1
escalation:
  - order: 1
    model: Qwen/Qwen3-Embedding-0.6B@97b0c614be4d77ee51c0cef4e5f07c00f9eb65b3
    role: first-attempt
    parameters: 595776512
    accept_when: "TODO: calibrate — see calibration below"
  - order: 2
    model: Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
    role: escalation
    parameters: 751632384
    accept_when: "TODO: calibrate — see calibration below"
  - order: 3
    model: Qwen/Qwen3-8B@b968826d9c46dd6066d109eabc6255188de91218
    role: escalation
    parameters: 8190735360
    accept_when: "TODO: calibrate — see calibration below"

1. Collect a query set from your own traffic. A router calibrated on someone else's distribution is a router calibrated on someone else's problem.
2. Score every candidate on every query. The escalation order above is by size; whether size predicts quality on your queries is a measurement, not an assumption.
3. Compute the oracle: the accuracy you would reach by always picking the best candidate per query. That is the ceiling your policy is competing against, and no published router reaches it.
4. Set the accept threshold from the measured accuracy-versus-cost curve, then re-measure. A threshold copied from a blog post is a guess wearing a number.

All twelve evaluated routers fall short of the oracle, chiefly by failing to notice when a smaller, cheaper model would have been sufficient. Of the 12 evaluated, the commercial entrant NotDiamond placed 12th on the paper's composite score, which prices accuracy against cost: it reaches 68% accuracy at $9.34 per thousand queries. Routing is the tier a browser can honestly ship end to end; it is not a solved problem, and buying one does not make it solved. RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers ↗