Sockets, multithreading, any method other than a fine grained integration of the interface with the engine will result in a slow down. Or do you expect Toady to resend all the creatures and items every frame?
The API method obviously offers the best chance at usability so I'm going to ignore any future mention of sockets.
Oooh... the merchant just got deleted while I was trying to access it... gotta add some synchronization. Wtf. Why is it running slower now? Thread synchronization is a costly operation, don't kid yourself.
You're confusing the interface with something that would have direct access to the game's logic. A third party interface would have no more capability than the official interface. Do you currently have the ability to delete a merchant in the interface as it stands now? Of course not. The interface would have two roles. Sending requests and outputting the current state of the game.
Let's start with sending requests. Say the player wants to build a construction at a specific location. Say a wall. Well the player clicks/types whatever has to be to select a wall and the location for it. The interface then sends a request to the game that a wall be placed at that location. The game then responds whether or not such a request was successful. Thus you've either placed a wall or the game responds back that it couldn't be done and the interface either refuses to place it or tells you that the attempt failed.
Now for output. The interface has decided it is time to render a portion of the game to the screen. The interface sends a request to the game. This request would amount to returning information on a visible tile/object at a certain location. The game would comply sending the data for that location. If the interface is set to render at a higher rate than the game currently updates then the interface could choose to cache the results it gets and keep using them over and over until a new game cycle occurs at which point the cached copies are labeled "dirty" and a fresh copy has to be requested.
As for threading, well threading the graphics wouldn't be too difficult at all. Simply have a thread that sits around doing nothing until receiving a command from the interface to render the current contents of the cache to the screen. The thread then immediately springs into action trying to render as fast as it possibly can. With hardware acceleration such a render shouldn't take very long at all. In fact in most cases the interface should be able to render fast than the game updates.
When it comes to threading I honestly have about as much experience as Toady does (though I do have some knowledge I picked up at college it isn't anywhere near as useful as experience). Perhaps even less. Thus I would probably avoid threading more than what can be easily threaded (like video, audio, events, etc). And you are right, syncing stuff can be slow. However the thread I mentioned above for graphics wouldn't have need of syncing beyond detecting if the cache has become "dirty" and possibly halting the current render if so.
Edit: I should mention that shortly after my last programming course ended (Java and it was last since I had run out of funds) I had started development of a text adventure game engine that involved a server/client like system using an API to communicate. It basically consisted of two commands. One fetched output and the other sent input. Not anywhere near as flexible as the interface for DF would have to be, but at least I can say I've tried it myself.