feat: img-saver-extended#490

Closed

Marius-Graml wants to merge 45 commits intomainfrom

feat/img-saver-extended

Commits on Jan 19, 2026

feat: 2 vbench dimensions and vbench dependencies

begumcig
authored and
Marius-Graml
committed
test: vbench metric tests

begumcig
authored and
Marius-Graml
committed
docs: add more comprehensive docstring explanations for important parameters

begumcig
authored and
Marius-Graml
committed
feat: add additional helper tools to utilities

begumcig
authored and
Marius-Graml
committed
refactor: small updates to utilities and docstrings

begumcig
authored and
Marius-Graml
committed
refactor: add support for more calltypes in video eval utils

begumcig
authored and
Marius-Graml
committed
refactor: make utilities more vbench independent and fix small things in the metric implementations.

begumcig
authored and
Marius-Graml
committed
refactor: address PR comments

begumcig
authored and
Marius-Graml
committed
test: adding more tests for dynamic degree and background consistency

begumcig
authored and
Marius-Graml
committed
feat: artifact saving and vbench related agent updates

begumcig
authored and
Marius-Graml
committed
test: add tests for the artifact savers

begumcig
authored and
Marius-Graml
committed
test: add artifact related evaluation tests and task modality tests

begumcig
authored and
Marius-Graml
committed
refactor: add some comments

begumcig
authored and
Marius-Graml
committed
refactor: better initialization for artifact savers

begumcig
authored and
Marius-Graml
committed
test: add more dtype tests for artifact saver

begumcig
authored and
Marius-Graml
committed
feat: metric modalities as sets

begumcig
authored and
Marius-Graml
committed
refactor: comments tests task modality

begumcig
authored and
Marius-Graml
committed
feat: add video inference support and seeding strategies to inference handler

begumcig
authored and
Marius-Graml
committed
feat: remove per evaluation seed and add tests

begumcig
authored and
Marius-Graml
committed
chore: add comments

begumcig
authored and
Marius-Graml
committed
fix: bfloats cannot be moved to cpu error in cmmd metric

begumcig
authored and
Marius-Graml
committed
fix: pre commit file fix

begumcig
authored and
Marius-Graml
committed
refactor: configure seeding and tests

begumcig
authored and
Marius-Graml
committed
refactor: algorithm compatibility (#401)

johannaSommer
authored and
Marius-Graml
committed
feat:stratification to vbench datasets

begumcig
authored and
Marius-Graml
committed
feat: data stratification by indexing

begumcig
authored and
Marius-Graml
committed
Add image artifactsaver and modified utils to use it for algo sweeper in prime intellect

Marius Graml
authored and
Marius-Graml
committed
Changes to vbench-utils via ruff

Marius Graml
authored and
Marius-Graml
committed
Add filename sanitizer which modifies invalid filename´aliases

Marius Graml
authored and
Marius-Graml
committed
Change file format via ruff

Marius Graml
authored and
Marius-Graml
committed
Add helper function for creating aliases as prompt names for outputs

Marius Graml
authored and
Marius-Graml
committed
feat: 0 vbench dimensions and vbench dependencies

begumcig
authored and
Marius-Graml
committed
Undo limiting prompt length as name for image logging

Marius Graml
authored and
Marius-Graml
committed
Integrate uncommented method from task.py

Marius Graml
authored and
Marius-Graml
committed
Update function in evaluation agent for generating metadata-json file

Marius Graml
authored and
Marius-Graml
committed
Add evaluation-agent parameter for optional JSON metadata creation; include batch_idx in metadata
Marius-Graml
committed
Adjust doc string for _maybe_create_input_output_metadat() in evaluation_agent.py
Marius-Graml
committed
Add model role in metadata json when applying pairwise metrics
Marius-Graml
committed
Change name and if logic of json file creation function in evaluation_agent.py
Marius-Graml
committed
Copy the tests from image-artifactsaver to feat/img-saver-extended to have one branch for optimization agent and image artifact saver
Marius-Graml
committed
Remove unused import
Marius-Graml
committed
Kid metric added (#435)

minettekaum
authored and
Marius-Graml
committed
Pin `torchao==0.12.0` to avoid PyTorch ABI warnings, also pin `numpydoc>=1.6.0` and `ty==0.0.1a21` for compatibility. (#417)

authored and
Marius-Graml
committed
feat: metric modalities as sets

begumcig
authored and
Marius-Graml
committed
Make prepare_inputs in diffuser handler more robust. Models such as Flux Klein expect the prompt as second positional argument. However, the current implementation always expects it as first argume…
Marius-Graml
committed