Skip to content

model compilation fails ? #3

@Edouard2laire

Description

@Edouard2laire
./siamize -i sub-Edouard_ses-01_T1w.nii -o labels.nii -F nii
[hint]     lowmem preset applied (memory-tight host detected; RAM 16384 MB, VRAM 8192 MB):
[hint]       --cudnn-max-workspace 0
[hint]       --gpu-mem-limit 6G
[hint]     (override by passing the flags explicitly; see --help)
[siamize]  v0.2.0 (Agate)  device=auto  threads=8 (auto)  format=nii
[models]   -M/--models not given; defaulting to single-fold fold_0_fp16.onnx (auto-downloaded if missing).
[input]    sub-Edouard_ses-01_T1w.nii
           canon shape (Z,Y,X) (320,300,256)  voxel size (X,Y,Z) (0.800,0.800,0.800) mm
[crop]     nonzero bbox (Z,Y,X) (319,299,256)
[resample] (Z,Y,X) (340,319,273) @ 0.750 mm
[weights]  fetching https://neurojson.org/io/stat.cgi?action=get&db=siam_v03&doc=coreml&file=fold_0_fp16.onnx.gz
                                      -=O=-                            #  # ##  
[weights]  decompressed fold_0_fp16.onnx (95375852 -> 283175765 bytes)
[infer]    image (Z,Y,X) (340,319,273)  patch (256,256,192)  step 0.50  tiles 8  weights 1  backend onnxruntime
[weight]   1/1  /Users/edelaire1/.cache/siamize/models/coreml/fold_0_fp16.onnx
[coreml]   probing CoreML EP...
[coreml]   enabled (units=all, cache=/Users/edelaire1/.cache/siamize/coreml, static_shapes=yes)
           first run compiles ONNX -> .mlmodelc (~10-30 s, cached after)
[warn]     ORT: Non-zero status code returned while running 16428538389238306980_CoreML_16428538389238306980_2 node. Name:'CoreMLExecutionProvider_16428538389238306980_CoreML_16428538389238306980_2_2' Status Message: Error executing model: Unable to compute the prediction using ML Program. It can be an invalid input data or broken/unsupported model.
[warn]     CoreML model compile failed: Non-zero status code returned while running 16428538389238306980_CoreML_16428538389238306980_2 node. Name:'CoreMLExecutionProvider_16428538389238306980_CoreML_16428538389238306980_2_2' Status Message: Error executing model: Unable to compute the prediction using ML Program. It can be an invalid input data or broken/unsupported model.
[hint]     -c auto falling back to CPU. Common cause: op-coverage gap.
             - rank-5 InstanceNormalization (SIAM v0.3 uses 3D InstanceNorm,
               which CoreML's MLProgram format supports only at ranks 3-4)
             - 3D ConvTranspose with non-standard stride/output_padding
           workaround: re-export the ONNX with rank-5 InstanceNorm rewritten
           as Reshape -> InstanceNorm2D -> Reshape (see tools/onnx_export/)


On an Apple with M1 cpu. Downloaded from siamize-0.2.0-macos-arm64-coreml.zip

it then failed with the following error:


./siamize -i sub-Edouard_ses-01_T1w.nii -o labels.nii -F nii
[hint]     lowmem preset applied (memory-tight host detected; RAM 16384 MB, VRAM 8192 MB):
[hint]       --cudnn-max-workspace 0
[hint]       --gpu-mem-limit 6G
[hint]     (override by passing the flags explicitly; see --help)
[siamize]  v0.2.0 (Agate)  device=auto  threads=8 (auto)  format=nii
[models]   -M/--models not given; defaulting to single-fold fold_0_fp16.onnx (auto-downloaded if missing).
[input]    sub-Edouard_ses-01_T1w.nii
           canon shape (Z,Y,X) (320,300,256)  voxel size (X,Y,Z) (0.800,0.800,0.800) mm
[crop]     nonzero bbox (Z,Y,X) (319,299,256)
[resample] (Z,Y,X) (340,319,273) @ 0.750 mm
[weights]  fetching https://neurojson.org/io/stat.cgi?action=get&db=siam_v03&doc=coreml&file=fold_0_fp16.onnx.gz
                                      -=O=-                            #  # ##  
[weights]  decompressed fold_0_fp16.onnx (95375852 -> 283175765 bytes)
[infer]    image (Z,Y,X) (340,319,273)  patch (256,256,192)  step 0.50  tiles 8  weights 1  backend onnxruntime
[weight]   1/1  /Users/edelaire1/.cache/siamize/models/coreml/fold_0_fp16.onnx
[coreml]   probing CoreML EP...
[coreml]   enabled (units=all, cache=/Users/edelaire1/.cache/siamize/coreml, static_shapes=yes)
           first run compiles ONNX -> .mlmodelc (~10-30 s, cached after)
[warn]     ORT: Non-zero status code returned while running 16428538389238306980_CoreML_16428538389238306980_2 node. Name:'CoreMLExecutionProvider_16428538389238306980_CoreML_16428538389238306980_2_2' Status Message: Error executing model: Unable to compute the prediction using ML Program. It can be an invalid input data or broken/unsupported model.
[warn]     CoreML model compile failed: Non-zero status code returned while running 16428538389238306980_CoreML_16428538389238306980_2 node. Name:'CoreMLExecutionProvider_16428538389238306980_CoreML_16428538389238306980_2_2' Status Message: Error executing model: Unable to compute the prediction using ML Program. It can be an invalid input data or broken/unsupported model.
[hint]     -c auto falling back to CPU. Common cause: op-coverage gap.
             - rank-5 InstanceNormalization (SIAM v0.3 uses 3D InstanceNorm,
               which CoreML's MLProgram format supports only at ranks 3-4)
             - 3D ConvTranspose with non-standard stride/output_padding
           workaround: re-export the ONNX with rank-5 InstanceNorm rewritten
           as Reshape -> InstanceNorm2D -> Reshape (see tools/onnx_export/)
[infer]    image (Z,Y,X) (340,319,273)  patch (256,256,192)  step 0.50  tiles 8  weights 1  backend onnxruntime
[weight]   1/1  /Users/edelaire1/.cache/siamize/models/coreml/fold_0_fp16.onnx
[tile]     [####################] 8/8 (100%)        
[saved]    labels.nii  in 1471.0s
siamize: fatal: mutex lock failed: Invalid argument


Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions