Isn't it pretty likely that Kim's brother (which was apparently his father's favourite, according to some documentary I half watched the other day) was assassinated because there might have been a high chance of there brewing a plot to replace him?
Speaking as Bay 12's foremost amateur self-declared Pyongyangologist: Not really. The clearest motive for the assassination has to do with why Kim Jong-nam was exiled in the first place: the Disneyland incident and his ideological shift. For the sons of most dictators, this wouldn't really be an issue and it definitely wouldn't lead to exile and assassination, but the style of the Kim family going all the way back to Kim Il-sung.
Kim Il-sung in his memoirs admits a very telling anecdote, one which starts to put a lot of the Kims actions in a more clear light. As a child, Sung's father was frequently tasked with bringing wine to one of his teachers, who was in fact a habitual alcoholic. This teacher eventually overdid his habit to the point he fell into a ditch in a drunken stupor, and at that moment, Sung's father lost all respect for the man and started to shame and order him around to the point that he gave up alcohol entirely.
Whether that actually happened to Sung's father is irrelevant, but it showcases the reasons for the Kims exaggerating their cult of personality to blatently absurd levels. They all, through a style passed down by Kim Il-sung, believe that the truest essence of leadership is is national myth and hype. Furthermore, they can
never allow other people to see them as being just people like they are, because if they're of the same sort then they can be judged and overthrown.
The relevance to Kim Jong-nam is that he embarrassed the family, he cracked the myth by getting caught like a "commoner" trying to sneak into Tokyo Disneyland. If he had actually gotten in and out without being stopped it probably would have ended up in his own cult as an anecdote of how stupid and easily fooled capitalists are by an enlightened Juche revolutionary. The other aspect, the reformism, isn't so much because he "became a capitalist" like Kim Jong-il claimed but because he did things like talk to the media and expose "playboy" activities like drinking, gambling, and womanizing to the public eye. Other reformists in North Korea are usually tolerated, what Nam did differently was be public about his lifestyle.
By damaging the myth he presented an ideological threats to the Kims inside North Korea (by making them publicly rearrange the line of succession, introducing doubt) and outside as well (by showing the world that the Kims are not true madmen anymore than Nixon was). Kim Jong-Il may or may not have spared him due to the family connection and settled for exile, but Kim Jong-un had no such sentiments. Killing Nam, and killing him in such a supervillian fashion instead of with guns or knives like normal people would, repairs damage done to the grand myth that is the Kim family.