RunwayML seem to have pulled SD 1.5 from HF #23
Description
Hi,
I thought I would draw attention to the fact that the install instructions do not work for me, since the link to the HF SD 1.5 resources is now giving a 404.
As a work-around, I attempted to use a repo which appears to be a fork of the original SD 1.5, although I'm not clear if this is the case:
https://huggingface.co/pt-sk/stable-diffusion-1.5
In any case, running the demo against this alternate repo gives an error
model_config = {'base_model_type': 'realisticVisionV60B1_v51VAE', 'model_path': 'resources/models', 'text_to_video_mm_path': 'resources/models/fancyvideo_ckpts/vae_3d_61_frames/mp_rank_00_model_states.pt', 'base_model_path': 'resources/models/sd_v1-5_base_models/realisticVisionV60B1_v51VAE.safetensors', 'res_adapter_type': 'res_adapter_v2', 'trained_keys': ['motion_modules.', 'conv_in.weight', 'fps_embedding.', 'motion_embedding.'], 'vae_type': 'vae_3d', 'use_fps_embedding': True, 'use_motion_embedding': True, 'common_positive_prompt': 'Best quality, masterpiece, ultra high res, photorealistic, Ultra realistic illustration, hyperrealistic, 8k', 'common_negative_prompt': '(low quality:1.3), (worst quality:1.3),poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face,Facial blurring,a large crowd, many people,advertising, information, news, watermark, text, username, signature,out of frame, low res, error, cropped, worst quality, low quality, artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck, nsfw, breast, naked, eroticism'}
infer_config = {'infer_mode': 'i2v', 'resolution': [768, 768], 'video_length': 16, 'output_fps': 25, 'cond_fps': 25, 'cond_motion_score': 3.0, 'use_noise_scheduler_snr': True, 'seed': 22, 'prompt_path': 'resources/demos/test_prompts/test_i2v_prompt.txt', 'reference_image_folder': 'resources/demos/reference_images/768x768', 'output_folder': 'resources/demos/samples/i2v/realisticVisionV60B1_v51VAE/768x768'}
Start loading infer pipeline...
device = cuda
unet trained keys = ['motion_modules.', 'conv_in.weight', 'fps_embedding.', 'motion_embedding.']
vae_type = vae_3d
/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/diffusers/schedulers/scheduling_ddim.py:142: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.betas = torch.tensor(trained_betas, dtype=torch.float32)
load scheduler done
Traceback (most recent call last):
File "/mnt/storage2/workspace/FancyVideo/scripts/demo.py", line 77, in <module>
main(args)
File "/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/mnt/storage2/workspace/FancyVideo/scripts/demo.py", line 22, in main
infer_pipeline = InferPipeline(
File "/mnt/storage2/workspace/FancyVideo/fancyvideo/pipelines/fancyvideo_infer_pipeline.py", line 151, in __init__
tokenizer = CLIPTokenizer.from_pretrained(pretrained_model_path, subfolder="tokenizer")
File "/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1760, in from_pretrained
resolved_vocab_files[file_id] = cached_file(
File "/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/transformers/utils/hub.py", line 409, in cached_file
resolved_file = hf_hub_download(
File "/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
return f(*args, **kwargs)
File "/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
validate_repo_id(arg_value)
File "/mnt/storage2/workspace/FancyVideo/env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'resources/models/stable-diffusion-v1-5'. Use `repo_type` argument if needed.
Thank you for all the hard work that went into this, I look forward to trying it out