If H.P. Lovecraft writes a book, and I like the story, I might end up writing a pretty awesome book based on it. Maybe even better than Lovecraft's. But I can't because Lovecraft holds the rights to the specific creatures he talks about. He could sue me, and maybe he could win. So my book doesn't get written and society is slightly impoverished for it.
Or, if you have the capacity to write awesome books, you could create your own characters instead of stealing them from someone else. Or, after getting a note from your publisher that you don't have Mr. Lovecraft's permission to use his stuff, change all the names. Or, having already rejected the notion of the author having say in what his characters are used for, release it illegally, without telling anyone your real name. It will be nearly impossible to trace back to you. Of course, this way you don't get the recognition and money that usually comes with writing great books. And if someone decided to take your book, change all the names and sell it for money, the irony would probably destroy the ozone layer. But since we are striving for public good here, you might be willing to accept that.
And calling it fan fiction is just a meaningless put-down to make the problem seem silly. I'm talking about, for example, the first fully 3D FPS game capturing the patent for the genre and that company being the only one that can make them. The idea is perfectly simple to explain when you're talking about designs for physical inventions as intellectual property, less well for video games, and even less well for books and paintings. But the ideas are the same.
No, I don't think the ideas are the same at all. Copyright only applies to creative works. You cannot copyright
concepts. For example: while they are some detective stories from before it, "The Murders in the Rue Morgue" by Edgar Allan Poe essentially started the genre in the English-speaking world. While the short story itself, and all the characters belonged to E.A. Poe, there was nothing stopping anyone from writing
other detective stories. The story was published in the 19th century, before international copyright existed, but I don't think the situation would be any different today.