Unsupported type in MLMultiArray when running with Swift CLI

[davidw0311]

Error:
CoreML/MLShapedArray.swift:557: Fatal error: MLMultiArray of data type Float16 is not supported.

After generation of the model in using

python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --model-version runwayml/stable-diffusion-v1-5 --bundle-resources-for-swift-cli --attention-implementation SPLIT_EINSUM -o swift-original-coreml-sd-v1-5

and running with

swift run StableDiffusionSample "a photo of an astronaut riding a horse on mars" --resource-path swift-original-coreml-sd-v1-5/Resources/ --seed 93 --output-path swift_images

the above error occurs.

Seems to be occuring during execution of line line 89 in TextEncoder.swift StableDiffusion/pipeline/TextEncoder.swift

return MLShapedArray<Float32>(converting: embeddingFeature!.multiArrayValue!)

[atiorh]

Hello! Could you please report your detailed macOS version ($sw_vers) and confirm that you are running from the latest main branch commit?

[atiorh]

and your Mac device's chip information please

[ZhouXiaolin]

The issue arises on Intel chips. I adjusted the data types of the input and output for the ct.convert function, which allowed it to run smoothly.

[davidw0311]

I am running on a mac pro M1 chip,

sw_vers:
ProductName: macOS
ProductVersion: 13.5.1
BuildVersion: 22G90

Error shown below:

[atiorh]

@davidw0311 I am looking into the Core ML input dtype changes to isolate this issue though I see that you closed the issue: Do you have a solution?

[davidw0311]

I have not yet been able to find a resolution to the issue for the models which were generated locally. However, I am able to run inference on the model with no problems using the checkpoints downloaded from https://huggingface.co/apple/coreml-stable-diffusion-v1-5-palettized, using swift cli

[ZachNagengast]

@davidw0311 might be related to some part of the python environment, can you try this command and paste the output?

pip show torch coremltools diffusers | grep 'Name\|Version'

[qlmaster]

Hi,
I have same issue

sw_vers

ProductName:		macOS
ProductVersion:		13.5.2
BuildVersion:		22G91

pip show torch coremltools diffusers | grep 'Name|Version'

Name: torch
Version: 2.0.0
Name: coremltools
Version: 7.0b2
Name: diffusers
Version: 0.20.1

python script python -m python_coreml_stable_diffusion.pipeline ... and *.mlpackage works very well

[ZachNagengast]

I think I've narrowed this down to a change in default behavior with compute_precision=None in coremltools 7.0b2, which started defaulting to float16 instead of float32 if unspecified. If you are looking for a quick solution, you could go with forcing float32 as the compute precision by manually editing the convert function. Here's the place to do it:

ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py

Line 129 in 7b59c2b

compute_precision=precision,

Just change precision to ct.precision.FLOAT32

The reason the old models work is because they defaulted to float32 at the time they were made.

[atiorh]

Thanks for the find @ZachNagengast! cc: @TobyRoseman @aseemw for coremltools visibility

[qlmaster]

@ZachNagengast Thank you, this solves the issue.

[jrittvo]

I think I've narrowed this down to a change in default behavior with compute_precision=None in coremltools 7.0b2, which started defaulting to float16 instead of float32 if unspecified. If you are looking for a quick solution, you could go with forcing float32 as the compute precision by manually editing the convert function. Here's the place to do it:

ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py

Line 129 in 7b59c2b

compute_precision=precision,

Just change precision to ct.precision.FLOAT32

The reason the old models work is because they defaulted to float32 at the time they were made.

I haven't had a chance to try this fix but it feels like it is the real thing. Thank you for running this down. It needed someone looking at it from outside of any silos to find the bug in the interaction between packages. Very much appreciated.