Error during quantizing the executorch model

[VEERA-NAGENDRA-KUMAR-EETHA]

I have followed the quantization documentation for the XNN backend.

Using that document, I have successfully quantized a Matmul executorch model.
Then, I have started with llm model with XNN backend.
For it, I am using the export script as :
quantization_script.py

From the above script, I have successfully generated a quantized model.
As a next step, I perform inference for FP32 as well as the int8 quantized model, and the output logs are as follows.
Inference code: inference.py
For FP32:

For int8 quantized model:

Here, I can't understand why I am getting the error. Can you please suggest a solution for this?

cc @GregoryComer @digantdesai @cbilgin

[metascroy]

@GregoryComer can you take a look at this XNNPACK runtime failure?

[GregoryComer]

@VEERA-NAGENDRA-KUMAR-EETHA It looks like there's something internally wrong in the lowering. I can take a look. There's unfortunately not enough information in the error message to diagnose it.

Would it be possible to share a code snippet of how you instantiate the model? I see from the export and quantization script, it's lowering mamba. If you can share the model definition (random weights is fine), I can try to repro it locally.

[nil-is-all]

Hi @VEERA-NAGENDRA-KUMAR-EETHA, could you share a code snippet for us to investigate this issue?

[nil-is-all]

@VEERA-NAGENDRA-KUMAR-EETHA closing this issue considering this is resolved. Feel free to re-open and share a code snippet for us to investigate.