ironbench leaderboard - firmware-agent benchmark
Model
pass@k
Tasks
control
fsm
io
protocol
resilience
qwen2.5-7b-instruct
31%
13
2/8 (25%)
2/4 (50%)
2/2 (100%)
0/8 (0%)
2/8 (25%)
qwen2.5-coder-14b-instruct
31%
13
1/8 (12%)
2/4 (50%)
2/2 (100%)
0/8 (0%)
2/8 (25%)