The language differences might remain, despite cultural similarities, because of landscape. People in the mountains or in other geographically-separated areas, or those who don't get down to cities often, may be more likely to retain the language they grew up speaking. It's also possible that the common culture includes terminology in both languages, so that some people end up speaking a combined version that becomes Barrayaran, as opposed to the earlier separate languages.

(disclaimer: not a linguist.)