Getting noisy video

[AvisP]

I have a container setup where I am getting proper video when using the script you provided. But when I am trying to create the video using the commands in sample_video.py script then getting a random noise output similar to folllowing. I checked the arguments section and didn't find any issues there. Please let me know what might be the cause of it.

Namespace(model='HYVideo-T/2-cfgdistill', latent_channels=16, precision='bf16', rope_theta=256, vae='884-16c-hy', vae_precision='fp16', vae_tiling=True, text_encoder='llm', text_encoder_precision='fp16', text_states_dim=4096, text_len=256, tokenizer='llm', prompt_template='dit-llm-encode', prompt_template_video='dit-llm-encode-video', hidden_state_skip_layer=2, apply_final_norm=False, text_encoder_2='clipL', text_encoder_precision_2='fp16', text_states_dim_2=768, tokenizer_2='clipL', text_len_2=77, denoise_type='flow', flow_shift=7.0, flow_reverse=False, flow_solver='euler', use_linear_quadratic_schedule=False, linear_schedule_end=25, model_base='ckpts', dit_weight='/__modal/volumes/vo-YJYY48yHayfN1pWsceD9iT/Hunyuan/ckpts/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states.pt', model_resolution='540p', load_key='module', use_cpu_offload=False, batch_size=1, infer_steps=20, disable_autocast=False, save_path='./results', save_path_suffix='', name_suffix='', num_videos=1, video_size=(720, 1280), video_length=129, prompt='a cat is running, realistic.', seed_type='auto', seed=None, neg_prompt=None, cfg_scale=1.0, embedded_cfg_scale=6.0, use_fp8=False, reproduce=False, ulysses_degree=1, ring_degree=1)

Code used

hunyuan_video_sampler = HunyuanVideoSampler.from_pretrained(models_root_path, args=args)

results = hunyuan_video_sampler.predict(
                prompt=prompt, 
                height=self.args.video_size[0],
                width=self.args.video_size[1],
                video_length=self.args.video_length,
                seed=self.args.seed,
                negative_prompt=self.args.neg_prompt,
                infer_steps=self.args.infer_steps,
                guidance_scale=self.args.cfg_scale,
                num_videos_per_prompt=self.args.num_videos,
                flow_shift=self.args.flow_shift,
                batch_size=self.args.batch_size,
                embedded_guidance_scale=self.args.embedded_cfg_scale
            )
        samples = results['samples']

        mp4_name = modify(prompt)

        for i, sample in enumerate(samples):
            sample = samples[i].unsqueeze(0)
            save_path = f"{OUTPUTS_PATH}/"+mp4_name
            save_videos_grid(sample, save_path, fps=24)
            logger.info(f'Sample save to: {save_path}')

mp4_name_lowres.mp4

[HaozheZhao]

also run into same issue

[AvisP]

They never respond to the issues posted. If you manage to find a solution to this issue let me know.

[chengmengli06]

maybe you guys forget to add the flow-reverse option

[HaozheZhao]

maybe you guys forget to add the flow-reverse option

That works! Thanks a lot!

[ckczzj]

maybe you guys forget to add the flow-reverse option

does this solve your problem?

[AvisP]

Yes that fixed the problem. I had to set the args.flow_reverse to True as it is by default set to False. You're awesome @chengmengli06 !