Change between GPU+NeuralEngine / NeuralEngine will error generate image.

[TimYao18]

I'm testing the Neural Engine supported models. When I use the SD1.5 model with Neural Engine first time, it works well with log "ANE model load has failed for on-device compiled macho. Must re-compile the E5 bundle. @ GetANEFModel".

Then I switch to GPU and Neural Engine, it will load the pipeline about 330 seconds, and then click generate it runs into error.

The error image is as the link.

Then I close the diffusers app and restart it again.
It starts with GPU and Neural Engine, and it load the pipeline about 143.7 seconds, and It can generate images.
But when I switch it back to Neural Engine, it runs into error when generate images.

This can be reproduced easily, but I don't know how to fix this.

[SpiraMira]

same here ... getting an ANE Program inference error when switching to any of the ANE options. Using 2.0 or 2.1 SD modules.

ANE inference operation failed with error message = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error} @ EvaluateANERequest
E5RT: ANE inference operation failed with error message = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error} (11)

Any guidance on this is appreciated.

[SpiraMira]

Here is the full log and more system details for the issue above:

MacBook Pro M1 Pro 16GB
Sonoma 14.2
Xcode 15.2
latest Diffuser app v1.1
ml_stable_diffusion v1.1 (latest package update)

running the app with SD 2.1 with GPU+ANE

Pipeline loaded in 6.93523097038269

ANE_ProgramSendRequest:1768 status=e00002bd
ERROR: H11ANEProgramProcessRequestDirect: pH11ANEFrameReceiver->ProgramProcessRequest failed status=0xe00002bd
processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9 lModel=_ANEModel: { modelURL=(null) : sourceURL= (null) : key=(null) : identifierSource=2 : cacheURLIdentifier=FA4124F35E61581018734B4C5DACC495B4FE785C4385AFE533B6CDDA3B235985_D10B201EB55CAC2AD2273CEAFA3A04FB6F87723AE99E4B940DDCA5E921119FBB : string_id=0x00000000 : program=_ANEProgramForEvaluation: { programHandle=133076128061594 : intermediateBufferHandle=133076166163913 : queueDepth=127 } : state=3 : programHandle=133076128061594 : intermediateBufferHandle=133076166163913 : queueDepth=127 : attr={
    ANEFModelDescription =     {
        ANEFModelInput16KAlignmentArray =         (
        );
        ANEFModelOutput16KAlignmentArray =         (
        );
        ANEFModelProcedures =         (
                        {
                ANEFModelInputSymbolIndexArray =                 (
                    0,
                    1,
                    2
                );
                ANEFModelOutputSymbolIndexArray =                 (
                    0,
                    1,
                    2,
                    3,
                    4,
                    5,
                    6,
                    7,
                    8,
                    9,
                    10,
                    11,
                    12
                );
                ANEFModelProcedureID = 0;
            }
        );
        kANEFModelInputSymbolsArrayKey =         (
            "emb_cast_ane",
            "encoder_hidden_states_tensor_ane",
            "sample_tensor_ane"
        );
        kANEFModelOutputSymbolsArrayKey =         (
            "hidden_states_149_cast_1_ane@output",
            "input_115_cast_1_ane@output",
            "input_117_cast_1_ane@output",
            "input_143_cast_1_ane@output",
            "input_15_cast_1_ane@output",
            "input_169_cast_1_ane@output",
            "input_171_cast_1_ane@output",
            "input_253_cast_1_ane@output",
            "input_35_cast_1_ane@output",
            "input_61_cast_1_ane@output",
            "input_63_cast_1_ane@output",
            "input_7_cast_1_ane@output",
            "input_89_cast_1_ane@output"
        );
        kANEFModelProcedureNameToIDMapKey =         {
            "main_ane" = 0;
        };
    };
    NetworkStatusList =     (
                {
            LiveInputList =             (
                                {
                    BatchStride = 1280;
                    Batches = 1;
                    Channels = 1;
                    Depth = 1;
                    DepthStride = 1280;
                    Height = 2;
                    Interleave = 1;
                    Name = "emb_cast_ane";
                    PlaneCount = 1;
                    PlaneStride = 1280;
                    RowStride = 640;
                    Symbol = "emb_cast_ane";
                    Type = Float16;
                    Width = 320;
                },
                                {
                    BatchStride = 196608;
                    Batches = 2;
                    Channels = 1024;
                    Depth = 1;
                    DepthStride = 196608;
                    Height = 1;
                    Interleave = 1;
                    Name = "encoder_hidden_states_tensor_ane";
                    PlaneCount = 2048;
                    PlaneStride = 192;
                    RowStride = 192;
                    Symbol = "encoder_hidden_states_tensor_ane";
                    Type = Float16;
                    Width = 77;
                },
                                {
                    BatchStride = 32768;
                    Batches = 2;
                    Channels = 4;
                    Depth = 1;
                    DepthStride = 32768;
                    Height = 64;
                    Interleave = 1;
                    Name = "sample_tensor_ane";
                    PlaneCount = 8;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "sample_tensor_ane";
                    Type = Float16;
                    Width = 64;
                }
            );
            LiveOutputList =             (
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 8;
                    Interleave = 1;
                    Name = "hidden_states_149_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 512;
                    RowStride = 64;
                    Symbol = "hidden_states_149_cast_1_ane@output";
                    Type = Float16;
                    Width = 8;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 640;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 32;
                    Interleave = 1;
                    Name = "input_115_cast_1_ane@output";
                    PlaneCount = 1280;
                    PlaneStride = 2048;
                    RowStride = 64;
                    Symbol = "input_115_cast_1_ane@output";
                    Type = Float16;
                    Width = 32;
                },
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 640;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 16;
                    Interleave = 1;
                    Name = "input_117_cast_1_ane@output";
                    PlaneCount = 1280;
                    PlaneStride = 1024;
                    RowStride = 64;
                    Symbol = "input_117_cast_1_ane@output";
                    Type = Float16;
                    Width = 16;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 16;
                    Interleave = 1;
                    Name = "input_143_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 1024;
                    RowStride = 64;
                    Symbol = "input_143_cast_1_ane@output";
                    Type = Float16;
                    Width = 16;
                },
                                {
                    BatchStride = 81920;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 81920;
                    Height = 1;
                    Interleave = 1;
                    Name = "input_15_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 64;
                    RowStride = 64;
                    Symbol = "input_15_cast_1_ane@output";
                    Type = Float16;
                    Width = 1;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 16;
                    Interleave = 1;
                    Name = "input_169_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 1024;
                    RowStride = 64;
                    Symbol = "input_169_cast_1_ane@output";
                    Type = Float16;
                    Width = 16;
                },
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 8;
                    Interleave = 1;
                    Name = "input_171_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 512;
                    RowStride = 64;
                    Symbol = "input_171_cast_1_ane@output";
                    Type = Float16;
                    Width = 8;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 2560;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 8;
                    Interleave = 1;
                    Name = "input_253_cast_1_ane@output";
                    PlaneCount = 5120;
                    PlaneStride = 512;
                    RowStride = 64;
                    Symbol = "input_253_cast_1_ane@output";
                    Type = Float16;
                    Width = 8;
                },
                                {
                    BatchStride = 2621440;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 2621440;
                    Height = 64;
                    Interleave = 1;
                    Name = "input_35_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "input_35_cast_1_ane@output";
                    Type = Float16;
                    Width = 64;
                },
                                {
                    BatchStride = 2621440;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 2621440;
                    Height = 64;
                    Interleave = 1;
                    Name = "input_61_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "input_61_cast_1_ane@output";
                    Type = Float16;
                    Width = 64;
                },
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 32;
                    Interleave = 1;
                    Name = "input_63_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 2048;
                    RowStride = 64;
                    Symbol = "input_63_cast_1_ane@output";
                    Type = Float16;
                    Width = 32;
                },
                                {
                    BatchStride = 2621440;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 2621440;
                    Height = 64;
                    Interleave = 1;
                    Name = "input_7_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "input_7_cast_1_ane@output";
                    Type = Float16;
                    Width = 64;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 640;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 32;
                    Interleave = 1;
                    Name = "input_89_cast_1_ane@output";
                    PlaneCount = 1280;
                    PlaneStride = 2048;
                    RowStride = 64;
                    Symbol = "input_89_cast_1_ane@output";
                    Type = Float16;
                    Width = 32;
                }
            );
            Name = "main_ane";
        }
    );
} : perfStatsMask=0}
processRequest:model:qos:qIndex:modelStringID:options:error:: Could not process request ret=0x1 lModel=_ANEModel: { modelURL=(null) : sourceURL= (null) : key=(null) : identifierSource=2 : cacheURLIdentifier=FA4124F35E61581018734B4C5DACC495B4FE785C4385AFE533B6CDDA3B235985_D10B201EB55CAC2AD2273CEAFA3A04FB6F87723AE99E4B940DDCA5E921119FBB : string_id=0x00000000 : program=_ANEProgramForEvaluation: { programHandle=133076128061594 : intermediateBufferHandle=133076166163913 : queueDepth=127 } : state=3 : programHandle=133076128061594 : intermediateBufferHandle=133076166163913 : queueDepth=127 : attr={
    ANEFModelDescription =     {
        ANEFModelInput16KAlignmentArray =         (
        );
        ANEFModelOutput16KAlignmentArray =         (
        );
        ANEFModelProcedures =         (
                        {
                ANEFModelInputSymbolIndexArray =                 (
                    0,
                    1,
                    2
                );
                ANEFModelOutputSymbolIndexArray =                 (
                    0,
                    1,
                    2,
                    3,
                    4,
                    5,
                    6,
                    7,
                    8,
                    9,
                    10,
                    11,
                    12
                );
                ANEFModelProcedureID = 0;
            }
        );
        kANEFModelInputSymbolsArrayKey =         (
            "emb_cast_ane",
            "encoder_hidden_states_tensor_ane",
            "sample_tensor_ane"
        );
        kANEFModelOutputSymbolsArrayKey =         (
            "hidden_states_149_cast_1_ane@output",
            "input_115_cast_1_ane@output",
            "input_117_cast_1_ane@output",
            "input_143_cast_1_ane@output",
            "input_15_cast_1_ane@output",
            "input_169_cast_1_ane@output",
            "input_171_cast_1_ane@output",
            "input_253_cast_1_ane@output",
            "input_35_cast_1_ane@output",
            "input_61_cast_1_ane@output",
            "input_63_cast_1_ane@output",
            "input_7_cast_1_ane@output",
            "input_89_cast_1_ane@output"
        );
        kANEFModelProcedureNameToIDMapKey =         {
            "main_ane" = 0;
        };
    };
    NetworkStatusList =     (
                {
            LiveInputList =             (
                                {
                    BatchStride = 1280;
                    Batches = 1;
                    Channels = 1;
                    Depth = 1;
                    DepthStride = 1280;
                    Height = 2;
                    Interleave = 1;
                    Name = "emb_cast_ane";
                    PlaneCount = 1;
                    PlaneStride = 1280;
                    RowStride = 640;
                    Symbol = "emb_cast_ane";
                    Type = Float16;
                    Width = 320;
                },
                                {
                    BatchStride = 196608;
                    Batches = 2;
                    Channels = 1024;
                    Depth = 1;
                    DepthStride = 196608;
                    Height = 1;
                    Interleave = 1;
                    Name = "encoder_hidden_states_tensor_ane";
                    PlaneCount = 2048;
                    PlaneStride = 192;
                    RowStride = 192;
                    Symbol = "encoder_hidden_states_tensor_ane";
                    Type = Float16;
                    Width = 77;
                },
                                {
                    BatchStride = 32768;
                    Batches = 2;
                    Channels = 4;
                    Depth = 1;
                    DepthStride = 32768;
                    Height = 64;
                    Interleave = 1;
                    Name = "sample_tensor_ane";
                    PlaneCount = 8;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "sample_tensor_ane";
                    Type = Float16;
                    Width = 64;
                }
            );
            LiveOutputList =             (
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 8;
                    Interleave = 1;
                    Name = "hidden_states_149_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 512;
                    RowStride = 64;
                    Symbol = "hidden_states_149_cast_1_ane@output";
                    Type = Float16;
                    Width = 8;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 640;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 32;
                    Interleave = 1;
                    Name = "input_115_cast_1_ane@output";
                    PlaneCount = 1280;
                    PlaneStride = 2048;
                    RowStride = 64;
                    Symbol = "input_115_cast_1_ane@output";
                    Type = Float16;
                    Width = 32;
                },
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 640;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 16;
                    Interleave = 1;
                    Name = "input_117_cast_1_ane@output";
                    PlaneCount = 1280;
                    PlaneStride = 1024;
                    RowStride = 64;
                    Symbol = "input_117_cast_1_ane@output";
                    Type = Float16;
                    Width = 16;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 16;
                    Interleave = 1;
                    Name = "input_143_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 1024;
                    RowStride = 64;
                    Symbol = "input_143_cast_1_ane@output";
                    Type = Float16;
                    Width = 16;
                },
                                {
                    BatchStride = 81920;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 81920;
                    Height = 1;
                    Interleave = 1;
                    Name = "input_15_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 64;
                    RowStride = 64;
                    Symbol = "input_15_cast_1_ane@output";
                    Type = Float16;
                    Width = 1;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 16;
                    Interleave = 1;
                    Name = "input_169_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 1024;
                    RowStride = 64;
                    Symbol = "input_169_cast_1_ane@output";
                    Type = Float16;
                    Width = 16;
                },
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 1280;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 8;
                    Interleave = 1;
                    Name = "input_171_cast_1_ane@output";
                    PlaneCount = 2560;
                    PlaneStride = 512;
                    RowStride = 64;
                    Symbol = "input_171_cast_1_ane@output";
                    Type = Float16;
                    Width = 8;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 2560;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 8;
                    Interleave = 1;
                    Name = "input_253_cast_1_ane@output";
                    PlaneCount = 5120;
                    PlaneStride = 512;
                    RowStride = 64;
                    Symbol = "input_253_cast_1_ane@output";
                    Type = Float16;
                    Width = 8;
                },
                                {
                    BatchStride = 2621440;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 2621440;
                    Height = 64;
                    Interleave = 1;
                    Name = "input_35_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "input_35_cast_1_ane@output";
                    Type = Float16;
                    Width = 64;
                },
                                {
                    BatchStride = 2621440;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 2621440;
                    Height = 64;
                    Interleave = 1;
                    Name = "input_61_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "input_61_cast_1_ane@output";
                    Type = Float16;
                    Width = 64;
                },
                                {
                    BatchStride = 655360;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 655360;
                    Height = 32;
                    Interleave = 1;
                    Name = "input_63_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 2048;
                    RowStride = 64;
                    Symbol = "input_63_cast_1_ane@output";
                    Type = Float16;
                    Width = 32;
                },
                                {
                    BatchStride = 2621440;
                    Batches = 2;
                    Channels = 320;
                    Depth = 1;
                    DepthStride = 2621440;
                    Height = 64;
                    Interleave = 1;
                    Name = "input_7_cast_1_ane@output";
                    PlaneCount = 640;
                    PlaneStride = 8192;
                    RowStride = 128;
                    Symbol = "input_7_cast_1_ane@output";
                    Type = Float16;
                    Width = 64;
                },
                                {
                    BatchStride = 1310720;
                    Batches = 2;
                    Channels = 640;
                    Depth = 1;
                    DepthStride = 1310720;
                    Height = 32;
                    Interleave = 1;
                    Name = "input_89_cast_1_ane@output";
                    PlaneCount = 1280;
                    PlaneStride = 2048;
                    RowStride = 64;
                    Symbol = "input_89_cast_1_ane@output";
                    Type = Float16;
                    Width = 32;
                }
            );
            Name = "main_ane";
        }
    );
} : perfStatsMask=0}

ANE inference operation failed with error message = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error" 
UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error} @ EvaluateANERequest

E5RT: ANE inference operation failed with error message = Error Domain=com.apple.appleneuralengine Code=8 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error" 
UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0x1 : statusType=0x9: Program Inference error} (11)