Previously I was using Spigot's serialization of locations into the last-locations.yml file. There would be an error deserializing if a world was changed. Now I'm using my own serialization in which I store the UUID of the world, and if the world is deleted, only a 1 line warning will be displayed.
If you were having this problem, it will likely be fixed after the second restart.