StructureBench

A benchmark for evaluating structured output capabilities of on-device models.

StructureBench: A Unified Benchmark Suite for Multi-Scenario Structured Generation Tasks with On-Device Models

StructureBench Overview Figure

How to Use This Benchmark

Step 1

Choose your model and dataset

Step 2

Choose a constrained decoder and generate

Step 3

Evaluate the model output

Step 4

Analyze with our metrics

Model Family	Modality / Task	Specific Models	Sizes

Domain	Dataset	Text	Image	Output	Constraint	Input Modality / Description	Expected Output	Size (Samples)

Dataset	Model	Constraint	SFR	SBR	cSGR	CIR