My group of friends has been working on a system that fits us for more than a decade. We tried lots of different things so there are quite some ideas considering non combat interaction.
But first, in the end we decided to not have an intricate sytem for social interactions. Because it is the one thing we can do best without many rules. We are not going to hit each other to find out who will win a swordfight, but we can argue pretty well when playing out social interactions. If the character you play does lack social skill you'll come up with something inept or outright stupid which is just as much fun as setting up a fine argument. If there will be fights or not will depend on the setting a lot. When you need to get rid of a dictator and lack the weapons and skills to do it by force you will automatically have to look for allies, for weaknesses in his life or lifestyle, for weaknesses in the personalty and lifestyle of his lackeys and allies - and exploit those. Setting up a revolution can take many evenings with plannings, talking to lots of different people, solving many smaller problems, and be lots of fun without involving a single fight.
But now to ideas for rules to strenghten the social interactions in the game.
One thing you can do is discourage fighting. Let fights be bloody and bad. Describe how people suffer when they lose a friend or family member - have it clear that the players made them suffer. Have limbs flying around. Do not have simple healing for your players, if they get a cut they could very well die of infection, if they lose a limb they're done. People usually do not want to get injured or die, let the people in the game world be like real people. Enemies and players might rather just give up a fight instead of dieing. If somebody does make a stand till the end do not be kind, describe how he is hacked to pieces, how much pain is involved, and let him see in the end what a wonderful day it actually is and that he might just have sat with his family or some girl and enjoyed his life instead of dieing for some stupid cause. That might discourage the player to try to solve everything by force the next time. Try to give them PTSD.
Another thing is to flesh out the rules for social interactions. The rules for fighting are very intricate in most games. You have lots of stats and different skills, you will have to think quick and come up with solutions for game situations, you will also have many little successes in the fights to keep you going. And then there is all the nerdy optimizing of numbers to get the perfect fighter or whatever, which too can be fun. Have a similar system for arguments. Don't just have the skill 'Talk'. Have defensive and offensive social skills, have willpower points that act like hitpoints, you might have to grind somebody down by argueing with them, threatening them or flirting with them. Have attacks and counterstrikes with words and arguments, have reasoning and diatribes. There are many different ways to interact with people, more than there are weaponskills.
But in the end it is really up to the story you play out and the mood of the players. If they really want to play a fight it is stupid to force them to not play it, they will not have fun and everybody will just waste their time. To play a nonviolent story takes a lot of effort for the gamemaster. It is not enough to write down ten orcs with attack +10, you will have to come up with lots of different people, their interests, their standing towards other people in the world. There will have to be a path or several paths for the players to follow to achieve their goals, even if they will do something completely different in the end. Then the gamemaster will have to be quick and improvise things which will still have to make sense and fit to the overall story, situations and characters. This will not always work out, so throwing some orcs or gang members at them might give you the time to come up with a good idea.
----
We played again a few days ago, everybody being back in town for christmas. And the story somewhat fits the theme. I will elaborate, but this might get a little bit long.
The setting is the close future, some country in south america. Our guys were working for a mid level security firm, doing simple guard jobs after failing to succeed in the military or police.
But we got our big chance, a bodyguard job for some important guy from a huge corporation. His bodyguard got ill, nobody was around and we were chosen to guard him when he went to a big party with lots of important figures of the country. But our guys were inexperienced, so we let no do them everything perfectly. While the client was in a backroom with a whore we had some drinks that were brought to us and got drugged. Somebody killed the client and made it look like we did it. The guards of the building went after us. Before we reached our car they shot at us, we shot back and hit one guy. He ended up dieing, this was going to haunt us later.
We lost our jobs and had the police after us. However one of us had a camera running and we got the face of the assassin. After quite some detective work, which filled most of that play session, we found her. She just gave up, agreed to go to the police with us if we left everybody else alone. That was good enough for us. She also tried to influence us, while the group she was part of had fucked us, they were fighting an important cause. Several corporations and part of the elite of the country were planning a coup. Work camps and cheap labor for the corporations would be the result.
After delivering her to the police we expected things to calm down. We were arguing what to do now, being without a job and knowing of the big conspiracy. Then somebody called us, we had done good work finding the assassin. They needed those kind of guys, there was a whole terrorist network that had to be uncovered. In regard to the money they offered we agreed to work for them.
Soon the other side contacted us, we really had to stop going after them and help them instead. We ended up deciding to be double agents, feeding the conspirators with false data, while also sacrificing some people to make our information look real enough. We would fuck other people over, just like we had been.
While we were working on the fake information and one guy was doing some errants unexpected things started to happen. The guy running errants was run over by a truck, then shot. Soon later some guys with masks and guns showed up at our place, good thing we had a security system and some cameras running. We torched the appartment complex and got away in the ensuing chaos. After some more investigations and visiting our guy at the hospital we found out that those guys were members of the security firm that guarded the party our client had been killed at. We had killed their friend and they were a close knit group. Now they were going to kill us. One of us argued that we had more important things to work on. We had to get rid of the guys, fast. They would not leave us alone, in the end it meant them or us. As they were more experienced, had better equipment and were twice as many guys an open confrontation was out of the question. In the end we decided to set off an I.E.D. It was lots of detective work again to find out who the guys were, where they would be at and how we could get all of them to one location at one time without anybody else being involved.
We ended up building an impovised bomb made of a mobile, a toy car motor and battery and some old russian grenades. However we were no specialists in building bombs, they heard the motor, the click of the grenades being unlocked and started moving before it went off. Some of those guys stumpled out of the building, horribly burnt and maimed by grenade fragments. We decided to go after them and killing them for good. But one of them was still unharmed. And he was better than us, being faster, with better aim and being better at taking cover he shot us several times before his revolver was empty. His lack of ammunition was the only reason we got him in the end. Then we still had to murder the horribly burnt guys. We knew they had family, it was gruesome. A horrible situation.
We had to get away, shot, heavily bleeding. We had not planned to be in a firefight, we had no doctor. Going to a hospital meant going to jail. We ended up at a veterinarian, forcing him at gunpoint to stitch us up. But he insisted this would not be enough, we had internal bleeding, bullet fragments in the wounds, we wouldn't last hours without going to a hospital.
There we were, some pale shocked guys in an old dirty van full of blood phoning frantically to get hold of some doctor who would not talk to the police. We ended up getting somebody, one of us had a friend who owed him a lot, that guy knew a doctor involved in drugs who could be coerced to be quite about the whole thing.
Even with the help of the doctor we were going to be out of the loop for weaks. Things will progress, the rest of the world will not wait for us. Stopping the coup will get much harder the longer we wait. We had fucked up, killing those guys had been a horrible idea, trying to talk them out of killing us or just hiding would have been a much better approach.
But roleplaying is like df, it is more fun if you fuck up and have !!fun!!.