I have no idea how Ultima RR is doing it, but the usual answer is to make the AI pretty dumb.
Chances are each entity doesn't really need to know about every single tile to do what it needs to do. Its generally just something like
1) Check my status
2) Check a target or two near me
3) Decide what to do with that, which is usually:
- run away,
- run towards
- stab with the stabby sword
If you have a bunch of entities, then it'll probably look smart enough.
That's actually exactly what my AI does right now (although the commander AI issues some pretty silly commands to the squad leaders, who either move where the commander tells them, or have their men charge). The issue is that when I have a squad of 10 vs another squad of 10, the pathfinding AI creates a traffic jam, where there will only be 1 guy from each side fighting, with the rest of the squad piled up behind them, each one trying to get at that "closest enemy" and not recognizing that there's another guy blocking their path. I can (and will, soon) change them to look for something other than the closest enemy, but that doesn't solve the core issue of them not realizing objects obstruct their paths.
You know, I'm not sure individual level path finding is that way to go in games like these. I mean historically anyone more tactically advanced than a tavern brawl fought with some level of coordination, even if it's only something as simple as "I'll stick with my mates so I'll have some to watch my back". To that end I think an AI descison tree like this:
-Check status
-Check for squadmates
-check for threats
-Make grade dependent descisions. For the grunts this would be:
-If overwhelmed or cut of, run away
-if ranged threat, respond/hold ranks/charge
-if if opponents are close by attempt to engage while maintaining squad coherency.
For squad leaders you could add in slightly more advanced path finding, such are moving towards the enemy.