I really have stopped complaining about Linear stories in RPGs mostly because I've seen what a mess the Sandbox has truely been.
The reason most RPGs don't do Branching Story Lines is simply this. It is a lot of work. They often feel like they can either tell one great story or a lot of less good ones.
This doesn't hold up in all cases because many RPGs are based upon branching storylines and in fact do them all very well (Soul Nomad is an interesting case because its main story is kinda boring... but its ONE branching storyline is so good that it should have been a game onto itself). Others don't do them well at all or only give the illusion of branching.
Now Linear however isn't nessisarily the opposite of Sandbox but is often considered so. Oblivion I would argue is in fact incredibly linear because the main questline and all quests for that matter are on very strict tracks.
However let me answer you dirrectly
But I wonder, is it really that hard? Might there be some shortcuts or logic systems that would allow the players to have a rich set of choices on any given problem without the full hassle to write each piece individually?
In truth. Yes. The shortcut is simply that because you are just altering a single storyline that you don't have to do all the work. Many games are based on this premise where all the branches are required for a full understanding of the plot. The first Way of the Samurai game (not an RPG) in fact was based upon learning from the game in order to achieve the best ending (a sad feat never replicated).
However what those games do is they make one story and allow you to explore only a small part of it. They are games about being one story.
No logic system can accurately replace a "Branching storyline" because in altering the story you are changing the characters, the setting, and the plot. These kinds of changes are not simple they are very complex. When you are creating a game that is multiple stories you are essentially adding a new layer onto a game.
So the shortcuts are as follows
1) Make one story in which each branch is just a small part of the collective story
and
2) Make each branch similar yet different from the seperate branches
and
3) Give the illusion of choice. Make only slight alterations and give different endings anyhow.
First of all RPGs have to lose the concept of two tier "moral choices", it's easily the biggest factor in "rail-roading".
I disagree. It is actually more psychological then anything (IGNORING games like Infamos and the second Black and White). You don't have to lean to the extremes. In fact I think what hurts it is that we as gamers have a pathological need to go to the extremes of the morality meter. Even though it is ultimately cosmetic.
You can blame Bioware for having us stuck with "being a dick" or "being a saint" sliding scales. Have a much more interesting tier system like D&D's (pre 4.0 abortion) four tiers system (evil, good, neutral, chaotic). If you've played Planescape: Torment you can see that this is generally a much more interesting system. This can help to create an illusion of non-linearity
They both are vomit enducing systems in their own right.
The Bioware morality system breaks down simply because the moral choices in the game are moral choices in the sense that you often can decide to do something terrible for the sake of being terrible or good for the sake of being good. It takes the character out of being a logical person who may or may not want to do good in this world into someone who is vying for the Saint of Fluffy Bunnies. It simplified morality.
Dungeons and Dragons fails however simply by how the rules are just uttar nonsense. In order to understand this you have to know something about logic and human psychology but I'll give it to you quick: People are flawed beings who even the most depraved of us wish to do good but not nessisarily good onto others.
Alignment often fails to understand that good people do bad things. It fails to understand the basic premise of evil. As well it fails to understand the differences between action and motivation as well as the many grey areas of morality. Dungeons and dragons itself doesn't know its own rules and constantly betrays them.
The idea of why the Monk needs to be lawful is that a chaotic person lacks the mental dicipline in order to do so. Yet it cannot support this premise within the gameplay or story. Chaotic characters are not chaotic people they are people who consider freedom and autonomy to be what should be the foundations of society. A Chaotic Good character for example may believe that people need to be free to do good and all laws do is get in the way of goodness. YET Chaotic characters do not nessissarily lack dicipline.
The easiest proof of how broken the game is, is the idea that a person who is a good person but not "That good" (as in someone who will do bad things if the need calls for it) is often treated as a Neutral character. Which is a fundemental problem. You cannot put characters who are good but recognise that their own morals do not hold up in all situations into a neutral allignment because by all means they are good.
Good isn't flawless yet it is treated as such.
Extend this to a society. Do Lawful Good societies have capital punishment? What kinds of capital punishment can they perform? If they had a particularly cruel form of capital punishment that had to be executed because of the tenants of their allignment would that still count?
Thus ultimately Dungeon and dragons allignment is simply illogical nonsense stupid talk that no one should take seriously. It feigns the image of being comprehensive but lacks anything to do so.