Exactly. What makes a German in the Rhine closer to a Bavarian or Baltic German, and further from a German-border Dutchman? History, not culture. They could behave exactly the same culturally as they do now and be different "nations" than they are now if history had gone differently far enough back.
The German
in the Rhine is considerably more wet than the others.
You're right though, a relatively recent example of this is Austria. Only a very small minority of Austrians today would consider themselves German, but that used to be different before the World Wars. Austria might very vell have ended up part of Germany (or technically speaking more likely the other way round), but didn't and has since formed it's own national (as opposed to regional) identity, while the other German states grew together into one nation.
The idea of the Soviet nation as a conglomerate of formerly separate nations of the USSR isn't a unique one - many multinational empires (like the Roman and Byzantine Empires, for example) put the imperial national identity over the national identities of its residents and tried to unify its nationalities with various degrees of success.
In a sense, all national identities are this. A nation is the unification of territory under a banner of shared identity and, usually, an imposed national language: before there were countries like France or Germany or India, there were thousands of tiny kingdoms, fiefdoms, wilderness, warring tribes etc. all speaking in different tongues. We've got used to thinking in terms of contiguous nation states but for most of human history this wasn't the case.
National identity in the modern sense is for the most part an invention of the early 19. century, as a reaction to the French Revolution. Since then most truly "multi-national" states have pretty much failed, most notably Austria-Hungary. Germany was much more culturally homogenous at that point than even France, which also was no longer as culturally diverse as it used to be. A unifying language seems to be the most important factor to me, any state that lacks something like that is pretty much bound to fail. That's even true for the New World countries, which are populated by immigrants, but all have a dominant culture/language. Trying to impose a common language on already formed national identities - as was the case with the Soviet Union - is also only going to re-enforce national sentiment.