The Balkans are a very diverse and mixed place and it's the oldest part of Europe, this is a fact. So it's not that strange you have a lot of territorial pissing contests. It's not some genetic hatred we have for one another, it's basically all politics and bickering over history.
From my experince Balkan people in diaspora get along with each other very well, regardless of ethnicity. Here in the states I've had Albanian friends, Kosovar Albanias, Bosniak, Croats, Serbs. None of them hate each other to the contrary they're always friendly and respectful and get along real well. It's like we know we're from the same region and we share similar cultures and customs and that brings us closer. Even religion doesn't create obstacles. It's a shame we can't get along better in our native lands. I blame politics for that and difficult economic conditions and a lot of propaganda. The current nationalism and racism has spread like a plague in Balkans especially since the late 80's and most of the people got nothing positive from it, only a very small percentage capitalized on it.
I agree with much of what you mention. Sadly(im not advocating for abolishing religion) religion is one of the biggest problems. Its no wonder that historically most disputes between balkan peoples have been of a religious nature. Really, nothing but religion separates Bosnians from Serbs. That is why Serbs butchered Bosnian men women and children. Of course as you say, politics and personal agendas play a big role, as most people follow their leaders blindly.
For instance(being Albanian), many Albanians I encounter who identify with Greek orthodoxy, have expressed their disdain and hate for muslim Albanians. Honestly cant tell on what grounds, because most of our people are godless. It is evident however, that balkan countries are either far right wing, or have some type of religious institution propagating shit to these people.
During the population exchange between Greece and Turkey, Greece only took orthodox folk, calling them Greek. They forgot some of their brothers in the black sea coast of Turkey. Romaika speaking Greek Muslims.
You would think they would want to preserve a dying classical greek dialect, but they instead considered them turks and left them with Turkey.
This is the problem with Balkan peoples. They are still in the middle aged clan system. What the Balkans need is a separation of church and state. A lingua franca(I would say English) where they can communicate and do business, whilst maintaining their regional languages and dialects. With an open system of freedom, no one would impede on another. Rather than one balkan union, it should function as a loose federation, whereby a council representing the interests of all peoples is appointed to ensure peace and stability.
Tribal mentalities would need to be quelled. Its just inevitable on the course to progress. The future has no place for minds of the past. Only when we reach this state could we consider joining the rest of Europe. Lest we pollute it further.
The reason why Balkan union doesn't work, is because it usually heavily dominated by one group, forcing their culture and language down anothers throat(Yugoslavia). The reason America works(not counting recent events leading to the decline of peoples mental capacity), is due to an open cultural system. It welcomes change, and input. It has the established language of its country, but does not force name changes, or forbid the people practice of their religion(within reason), or maintenance of their culture/language.
For instance this can be seen in Croatia, where Albanians since the 1400s migrated. They have equal rights, and preservation of their language. Their government even went out of their way to fund schools to teach and preserve a dying Dalmatian dialect of Albanian. No problem..
Italy, you have Arbereshe who still have equal rights and language. No problem. However, in Macedonia, there are problems. It really comes down to us learning to live and let live. All the "demons" balkan people fear, are things they have no way of knowing could happen. So instead, they feed their fears and hatreds, and all act based upon fear. The balkans(while it wont happen over night) needs to step out of the dark ages. They will find this "peace" that they were searching for is very attainable with cooperation and compromise. Working to benefit each other. Until they stop thinking they are ancient and everyone else an intruder, they will still keep butchering each other. It really doesnt matter how old the place is. All nations have come and gone. Ours will be no different. History is in constant change and flux. It cannot be stopped.
The root problem of all conflict is ignorance. When we stand for justice and eschew evil, then we can think about moving forward as a peoples in the world.