We may use the shap package and provide a utility function to make things much simpler for users. See #147 for a discussion.
Alternatively, we may be able to take advantage of the tree representation and obtain a faster method as we did for partial dependence plots. See https://hughchen.github.io/its_blog/index.html for example