No. Provided that the domain is registered in your name (as it should be), you can simply update your namservers (from where you manage your domain names) to that of your new host.