Hell emulation would be stupid easy (lol libraries FTW)
It's a plethora of issues that combine into one hideous, mind melting beast that makes backwards compatibility less enticing for big businesses.
On the PS3, they used to include an extra processor so you could run PS2 games. They since stopped, partly because it was cheaper, but also because the PS3 had the new digital store come out where you could buy all the old PS2 games digitally. The downloadable PS2 games would be optimized for the new system, unlike the disc version. Boom, Sony saves money and makes money. Expect to see PS3 games on sale in the PS4 digital marketplace some time in the future.
Note, of the games that have made it over from ps2 to ps3 via store, from what i was told by a coder the games were either running on a crap emulator or the original dev team rebuilt the code for the ps3 usage with software limiter to conform to ps2 speeds. Sony knew they would be pissing people off when they cut out the ps2 hardware (still pissed ever since they cut out OtherOS!). They never bothered to even phase out the ps2 they just devs continue working on it until they might bother looking at the ps3. Ps2 emulation hands down will go down as one of the toughest things to ever do, hell even the current pc version still requires a powerful quad-core to be able to play full speed.
PS3 architecture is hard. With one CPU core and 6 parallel Cell units, how the hell can anyone be able to emulate that on a x86 arch unless you got a crapton of HLE going on. This is why they said the ps3 games on the ps4 market will be done via streaming from their cloud servers.