When a controller is disconnected and a game is currently running, it doesn't get re-assigned and it would appear that you have to restart to assign the controller back to the right port. The cable for my controller is unreliable, and this is sort of a requirement for usability for me. Messing around with the settings doesn't seem to let me fix it manually either.