Skip to content

feat: add general synthetic data iterator and examples for WAN and FLUX.#310

Open
jianhan-amd wants to merge 2 commits intoAI-Hypercomputer:mainfrom
jianhan-amd:jianhan/synthetic_data_iterator
Open

feat: add general synthetic data iterator and examples for WAN and FLUX.#310
jianhan-amd wants to merge 2 commits intoAI-Hypercomputer:mainfrom
jianhan-amd:jianhan/synthetic_data_iterator

Conversation

@jianhan-amd
Copy link

In the PR, a data iterator for synthetic image/video data is implemented with following features.

  • A general synthetic data iterator for all the supported models for quick model testing and benchmark.
  • Support to load input dimensions from the model pipeline.
  • Support to override input dimensions from the config yaml.
  • Examples for WAN and FLUX models are provided.

@google-cla
Copy link

google-cla bot commented Jan 20, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@jianhan-amd jianhan-amd deleted the jianhan/synthetic_data_iterator branch January 20, 2026 09:05
@jianhan-amd jianhan-amd restored the jianhan/synthetic_data_iterator branch January 20, 2026 14:26
@jianhan-amd jianhan-amd reopened this Jan 20, 2026
@jianhan-amd jianhan-amd force-pushed the jianhan/synthetic_data_iterator branch from 3e96b99 to e8bdd82 Compare January 20, 2026 14:57
@jianhan-amd jianhan-amd reopened this Jan 20, 2026
@entrpn
Copy link
Collaborator

entrpn commented Jan 20, 2026

@jianhan-amd thanks for the submission. Can you add unit tests to this workflow. You can add them inside this file.

Then make sure to run ruff check --fix. I'll kick off the tests again once submitted. Thanks!

@cpersson-amd cpersson-amd force-pushed the jianhan/synthetic_data_iterator branch from b3fcc99 to d09b02f Compare February 25, 2026 11:31
@cpersson-amd cpersson-amd requested a review from entrpn as a code owner February 25, 2026 11:31
@cpersson-amd
Copy link
Contributor

@entrpn Hello, I've gone ahead and added the unit tests. The code has been rebased on main and ruffed. Please let me know if something needs changed!

@entrpn
Copy link
Collaborator

entrpn commented Feb 25, 2026

@jianhan-amd can you squash your commits into 1? Besides that, everything looks good.

@cpersson-amd cpersson-amd force-pushed the jianhan/synthetic_data_iterator branch from d09b02f to cf3788e Compare February 26, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants