Doesn't work on M1 Mac : please add MPS support

[kayzen-ml]

When ran on a Mac, the extension says that it doesn't detect any CUDA and will switch to CPU. Still, nothing happens. Could you please add MPS support, like webUI does, in order to use Apple's powerful chips? Thank you

[edwios]

What model and config you are using? Except for video, I can generate depth map on a M1 Max with 64GB RAM. It's slow (60s for a 512x512 image) but it works.

HOWEVER, video generation seems to have problem with EGL support. I am trying glfw to see if it would work but it looks like the video generation process is taking forever so I am not sure if it would make any difference.

[jmy12k3]

What model and config you are using? Except for video, I can generate depth map on a M1 Max with 64GB RAM. It's slow (60s for a 512x512 image) but it works.

HOWEVER, video generation seems to have problem with EGL support. I am trying glfw to see if it would work but it looks like the video generation process is taking forever so I am not sure if it would make any difference.

The current project for MAC users is only available on cpu. It does not utilize the mps in pytorch module. Of course, you could run at cpu and compile your 512x512 depthmap for 60 seconds. However, I think what this issue really means is that add native mps support.

[jmy12k3]

For example, you could change torch.cuda.is_available() to torch.backends.mps.is_available() and stuff like that and torch.devices("mps") also. Therefore, technically it is possible for a mps support.

[BrianBing]

I'm getting an error message while loading on my M1. I have installed FFMPEG with homebrew, not entirely sure if I'm doing something wrong.

Error loading script: depthmap.py Traceback (most recent call last): File "/Users/briandavison/stable-diffusion-webui/modules/scripts.py", line 229, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "/Users/briandavison/stable-diffusion-webui/modules/script_loading.py", line 11, in load_module module_spec.loader.exec_module(module) File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/Users/briandavison/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 54, in <module> from inpaint.mesh import write_ply, read_ply, output_3d_photo File "/Users/briandavison/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/inpaint/mesh.py", line 16, in <module> from moviepy.editor import ImageSequenceClip File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/moviepy/editor.py", line 36, in <module> from .video.io.VideoFileClip import VideoFileClip File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/moviepy/video/io/VideoFileClip.py", line 3, in <module> from moviepy.audio.io.AudioFileClip import AudioFileClip File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/moviepy/audio/io/AudioFileClip.py", line 3, in <module> from moviepy.audio.AudioClip import AudioClip File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/moviepy/audio/AudioClip.py", line 7, in <module> from moviepy.audio.io.ffmpeg_audiowriter import ffmpeg_audiowrite File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/moviepy/audio/io/ffmpeg_audiowriter.py", line 7, in <module> from moviepy.config import get_setting File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/moviepy/config.py", line 36, in <module> FFMPEG_BINARY = get_exe() File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/imageio/plugins/ffmpeg.py", line 173, in get_exe return imageio_ffmpeg.get_ffmpeg_exe() File "/Users/briandavison/stable-diffusion-webui/venv/lib/python3.10/site-packages/imageio_ffmpeg/_utils.py", line 33, in get_ffmpeg_exe raise RuntimeError( RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

[FateIsNotForFree]

i have the model installed and its working except when I try to make 4 video from the mash its going with this error
u have some ideas ?
RuntimeError: Could not import backend "EGL":
dlopen(EGL.framework/EGL, 0x0006): tried: 'EGL.framework/EGL' (no such file), '/System/Volumes/Preboot/Cryptexes/OSEGL.framework/EGL' (no such file), '/opt/homebrew/lib/EGL.framework/EGL' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/EGL.framework/EGL' (no such file), '/usr/lib/EGL.framework/EGL' (no such file, not in dyld cache), 'EGL.framework/EGL' (no such file), '/Library/Frameworks/EGL.framework/EGL' (no such file), '/System/Library/Frameworks/EGL.framework/EGL' (no such file, not in dyld cache)

Traceback (most recent call last):
File "/Users/germannalobin/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 337, in run_predict
output = await app.get_blocks().process_api(
File "/Users/germannalobin/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1018, in process_api
data = self.postprocess_data(fn_index, result["prediction"], state)
File "/Users/germannalobin/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 935, in postprocess_data
if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range

[FateIsNotForFree]

@BrianBing try to fallow this instruction

1. Open Terminal.
2. Install Homebrew by typing the following command and pressing Enter:
3. /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
4. Once Homebrew is installed, type the following command and press Enter to install ffmpeg:
5. brew install ffmpeg
6. Wait for the installation to complete.
7. After installation, run the "which ffmpeg" command in Terminal to confirm that ffmpeg is now installed and the path to the executable file is displayed.(copy that)
8. Type the following command: "export IMAGEIO_FFMPEG_EXE=" (without the quotes, but make sure to include the space at the end).
9. Paste the path to the ffmpeg executable file that you copied in step 7 at the end of the command from step 8. For example, if the path to ffmpeg is "/usr/local/bin/ffmpeg", you would type: "export IMAGEIO_FFMPEG_EXE=/usr/local/bin/ffmpeg"
10. Press Enter to run the command.
11. Try running the "depthmap.py" script again to see if the error has been resolved.

After you run the command "export IMAGEIO_FFMPEG_EXE=" in the Terminal, you should not see any output. The command sets the environment variable "IMAGEIO_FFMPEG_EXE" to the path of the ffmpeg executable file, but it does not produce any output.
To verify that the environment variable has been set correctly, you can run the command "echo $IMAGEIO_FFMPEG_EXE" in the Terminal. If the output is the path to the ffmpeg executable that you set earlier, then the environment variable has been set correctly. If the output is empty or an error message, then the environment variable has not been set correctly and you should try setting it again.

[B00Berry]

Just adding a voice to the wish for MPS support. Processing on M1 Ultra uses all CPUs well, but it's sad to see the GPU sitting idle - particularly during the lovely "generating inpainted mesh... (go make some coffee)" phase.

[thygate]

@B00Berry
Generating the inpainted mesh runs mostly on CPU only, only the pix2pix parts can use GPU, but those are just a tiny part of the whole process. It would likely not be much faster with GPU support.

Also for generating stereo images, this runs on CPU completely.

Only the depth map generation would benefit a lot from GPU.

[B00Berry]

Only the depth map generation would benefit a lot from GPU.

Great info! Thanks for the reply. I ran a test with a folder of video frames, and indeed, a speed improvement would be nice. I also like being able to walk away while the computer works. So either way I'm happy. Thanks for such a useful tool!