Tech Graphics Benchmark Ratings
Tech Graphics Benchmark: annotations Write-up: Can image models spell the labels? A first look at my Tech Graphics Benchmark Raw human evaluation data for Tech-Graphics-Benchmark: one JSON file per annotated generation, written by the annotation Space. The model was hidden from the annotator until the file was saved; cost and size were shown. Prompts: danielrosehill/Tech-Graphics-Benchmark Generations (images + manifests): danielrosehill/Tech-Graphics-Benchmark-Runs Layout:… See the full description on the dataset page: https://huggingface.co/datasets/danielrosehill/Tech-Graphics-Benchmark-Ratings.
About This Project
Tech-Graphics-Benchmark-Ratings serves as a repository for raw human evaluation data derived from the Tech-Graphics-Benchmark project. It stores individual JSON files for each annotated generation, capturing specific quality metrics while maintaining a blind evaluation process where the underlying model remains hidden from the annotator until data submission. The dataset is structured to map directly to the corresponding runs and prompts, ensuring traceability between input instructions and final outputs.
The project utilizes a standardized directory layout to organize annotations by run and generation IDs, facilitating integration with the broader Tech-Graphics-Benchmark ecosystem. This resource is intended for researchers and developers focused on benchmarking the performance of generative models in specialized tasks such as technical diagram creation and precise text rendering within images. It provides the necessary ground-truth data for evaluating model efficacy in complex visual generation scenarios.