Deal with keras embedding layers

[zkysfls]

I was trying to use DeepLIFT to analyse my text classification model but it will throw key error about embedding layers since there is no key about embedding in layer_name_to_conversion_function function. So it is impossible to use DeepLIFT to model that use keras embedding layers?

[akundaje]

You can use the DeepSHAP/DeepExplainer implementation of DeepLIFT which does not require you to write additional conversion functions. https://github.com/slundberg/shap

…

On Wed, Jul 8, 2020 at 11:59 AM Kangyu Zheng ***@***.***> wrote: I was trying to use DeepLIFT to analyse my text classification model but it will throw key error about embedding layers since there is no key about embedding in layer_name_to_conversion_function function. So it is impossible to use DeepLIFT to model that use keras embedding layers? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#108>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABDWEJ52V2P6XCFMZOYFMDR2S6XNANCNFSM4OU3NPKA> .

[AvantiShri]

Another thing that could be done is to subset the model to only include the layers that come after the embedding layer, and then supply the output of the embedding layer as the input to this subsetted model. This would give you the importance scores at the level of the embedding layer, which could then be mapped back to the individual words. But it would be worth checking that the architecture of the model after the embedding layer is supported by this deeplift implementation; if the architecture is not supported, then the shap library’s implementation of DeepLIFT would probably be the better way to go as Anshul said.