Do I have to license different language versions extra?

No, you don’t have to license the royalty free music differently, whether you are creating one version of a video, or several, e.g. different language versions.