Uninstalling via add-remove programs should delete the entire package directory in ~/.jdeploy/gh-packages if the app is a github install.
Perhaps it would be better to generate a .bat script for the uninstaller (not sure if this is possible) rather than running the uninstaller exe, as it's also disconcerting if the uninstaller launches the updater for the jdeploy installer - which contains the uninstall logic. Perhaps a headless install would help with this - but still the concept of updating the installer to uninstall the app itself seems wrong.