Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[FinalSin]

Haha. To tell you the truth, I have little experience in database design myself, but even if they just replicated the XML layouts (one table for each top-level element) it would be so handy.

What I wanted to do was use the generated worlds to generate new content from (things like stories or jokes including people from the world, or even small games). It's an experiment, really, in using one game's content to create new things, that I wanted to hack out. I'm rather unfairly burdening you with a million requests beforehand though!

Text export would be great anyway, because I'm sure people would be interested in a more human-readable format. From my perspective, a database (or even a CSV-style file) would be a boon, but it's really up to you. You've done great work so far.

[Kaelem Gaen]

Hopefully I didn't miss this when I jumped to the end but, I was wondering if you plan to allow a  filter for any race. 
I was parsing a history off a NrthAmer map, but I was trying to find info about a human city (Well ruin) to see how it fell,  I forgot to turn on "Cull Unimportant Historic figures"  (actually since it was all inhabiting I was just lazy I guess).    But you can only remove Non-Dwarven Civ/sites.
 

Though I'm wondering if It didn't bring up the ruins as they'd be covered in the History bit that didn't get parsed.

[Mason11987]

Haha. To tell you the truth, I have little experience in database design myself, but even if they just replicated the XML layouts (one table for each top-level element) it would be so handy.

What I wanted to do was use the generated worlds to generate new content from (things like stories or jokes including people from the world, or even small games). It's an experiment, really, in using one game's content to create new things, that I wanted to hack out. I'm rather unfairly burdening you with a million requests beforehand though!

Text export would be great anyway, because I'm sure people would be interested in a more human-readable format. From my perspective, a database (or even a CSV-style file) would be a boon, but it's really up to you. You've done great work so far.

Doing a straight replacement of the xml with a table would be difficult on a few levels. 

• Sites aren't tied to civs in the XML, they could be though through intelligent use of the events (in XML) though
• Using just the XML it's impossible to distinguish among races for anything
• The events table would need to have a large number of columns, most would be empty for any individual event.
• I'd need to be able to append items to the event table, because I can't put all the events in at once

The first two could be solved by intelligently using the XML and sites/history files together to make the database (like they are used in the program).  The third isn't a big problem, but I don't know how manageable a 2.5million record x 15 field table would be.  The last may or may not be possible.  I suspect it is though.

I'll work first on a text export of some sort.  It's just hard to know what's important to other people.

If you want a straight list of HistFigs/Sites/Regions stripped from the XML with their IDs that's not a problem.  If you want the events, I'll have to think of a meaningful way to export them (perhaps by event type?)

Hopefully I didn't miss this when I jumped to the end but, I was wondering if you plan to allow a  filter for any race. 
I was parsing a history off a NrthAmer map, but I was trying to find info about a human city (Well ruin) to see how it fell,  I forgot to turn on "Cull Unimportant Historic figures"  (actually since it was all inhabiting I was just lazy I guess).    But you can only remove Non-Dwarven Civ/sites.
 

Though I'm wondering if It didn't bring up the ruins as they'd be covered in the History bit that didn't get parsed.

If the events didn't get parsed then you wouldn't see what happened to it, but my understanding is that sites, once founded, are a permanent fixture on the world, so they should be in that list somewhere.  If it was destroyed it wouldn't be listed as "Human" under the sites, because race is tied to the race of the currently occupying civ, which there wouldn't be one.  In the future I'll (possibly) tie race of a destroyed site to the race that founded it, or last reclaimed or captured it, but for now I think it makes sense as is.

You can also sort by name to find it.

But to answer your question, I am planning a general filter for a future release.  It's something I would find personally really cool and also really useful for debugging.  It won't be exactly as efficient as the name filter in legends mode, but it'll be more extensive (only births > 100, only the dead, only sites that have been reclaimed, etc.)

[FinalSin]

Event type would work. I feel like I"m tossing a lot of work out there for my benefit only - anyone else think this might be useful to them?

I don't know how manageable a database that big would be either, but I think it wouldn't be any worse than storing it as XML. Maybe.

The other alternative could be to re-engineer part of your code so it becomes a sort of API that could be used to query the XML.

[Ratbert_CP]

re: db design

Normalize, normalize, normalize!

Quick, partial, off-the-top-of-my-head schema:

Spoiler (click to show/hide)

tbl_Race:

Code: [Select]

INT RaceKey
STRING Name
tbl_Deity:

Code: [Select]

INT DeityKey
STRING Name
STRING Spheres
tbl_Civ

Code: [Select]

INT CivKey
STRING Name
INT RaceKey 
  tbl_Site

Code: [Select]

INT SiteKey
STRING Name
INT XLoc
INT YLoc
INT FoundingCivKey
INT Founded
tbl_HistoricalFigs:

Code: [Select]

INT HFkey
STRING Name
INT RaceKey
INT Born
INT Died
INT ParentKey
INT Parent2Key
INT BirthCivKey
INT BirthSiteKey
INT DeityKey
tbl_Title

Code: [Select]

INT TitleKey
STRING Name
INT RaceKey
tbl_Leader

Code: [Select]

INT HFKey
INT SiteKey
INT TitleKey
INT Begin
INT End

[FinalSin]

I like this guy.

Any recommendations for handling events, though?

[Mason11987]

Event type would work. I feel like I"m tossing a lot of work out there for my benefit only - anyone else think this might be useful to them?

I don't know how manageable a database that big would be either, but I think it wouldn't be any worse than storing it as XML. Maybe.

The other alternative could be to re-engineer part of your code so it becomes a sort of API that could be used to query the XML.

No worries, if I can export to a database of some sort, I'll handle that when I finish some other more requested (and easier) things.  I always liked the idea of databases and linking tables together so I think I might refresh myself on my database course in college and do a database design.

*dusts off copy of Visio*

I don't know about manageability either, but we'll see.  I don't know about API stuff though.

re: db design

Normalize, normalize, normalize!

Quick, partial, off-the-top-of-my-head schema:

Spoiler (click to show/hide)

tbl_Race:

Code: [Select]

INT RaceKey
STRING Name
tbl_Deity:

Code: [Select]

INT DeityKey
STRING Name
STRING Spheres
tbl_Civ

Code: [Select]

INT CivKey
STRING Name
INT RaceKey 
  tbl_Site

Code: [Select]

INT SiteKey
STRING Name
INT XLoc
INT YLoc
INT FoundingCivKey
INT Founded
tbl_HistoricalFigs:

Code: [Select]

INT HFkey
STRING Name
INT RaceKey
INT Born
INT Died
INT ParentKey
INT Parent2Key
INT BirthCivKey
INT BirthSiteKey
INT DeityKey
tbl_Title

Code: [Select]

INT TitleKey
STRING Name
INT RaceKey
tbl_Leader

Code: [Select]

INT HFKey
INT SiteKey
INT TitleKey
INT Begin
INT End

I like it, definitely a good start.  And I know all about normalizing, don't worry there.

I like this guy.

Any recommendations for handling events, though?

I'll probably do an eventtype table.  But should I do a table for each type of event?  Since they each have different attributes?

[FinalSin]

I guess you'll have to. And then - I literally have done no databases since a first-year uni course four years ago - then maybe a master table that uses the event id to identify what kind of type it is?

[Mason11987]

I guess you'll have to. And then - I literally have done no databases since a first-year uni course four years ago - then maybe a master table that uses the event id to identify what kind of type it is?

Yeah, but then I have foreign keys in a column on the eventtypes table that refer to multiple different tables, and that's not right.  I guess the simplest, and the correct solution is to just do all events in the same table, and to just leave null the fields a record doesn't use.

[FinalSin]

Yeah, could easily do that. I would almost certainly marry you if any of this happens.

[Mason11987]

Yeah, could easily do that. I would almost certainly marry you if any of this happens.

So I've found that it SEEMS like exporting to sqlite is possible, here's some plans.

Spoiler: Thoughts? (click to show/hide)

There is a many-to-many between civs and deities too (which doesn't quite get covered by the deity -> Site -> Civ relationship) that isn't shown. 

Specifically I'm not outputting events, because since I can display information in the program without storing events, I think it can be just as useful in the database without a table (by putting stuff like founders in sites and killed by in HF.

[FinalSin]

That's essentially doing an extra step of work for me. Bloody brilliant; if you can manage that then that would be fantastic. I'd totally be willing to donate a little bit too, seeing as it's a fair bit of extra work.

[darius]

hmm shouldn't the father/mother go to tbl_HistoricalFigs, not leader?

[Mason11987]

hmm shouldn't the father/mother go to tbl_HistoricalFigs, not leader?

Actually no. there isn't any way of telling which historical figure gave birth to which historical figure (at least not yet).  Given the current state of the XML you can only determine parents under a very small set of circumstances.  Only if the father was a leader, and he died and his child became the leader afterwards, otherwise it's impossible to know.  All of this you can get just from the worldhistory file which only shows information on leaders (kings/queen/princess/law-giver/master/etc.)

Some other glaring omissions (from the XML, and therefore, here) is that there isn't a birth year for historical figures.  I think I can go from the first event referring to the HF to be their birth, but I haven't tested this out yet.

[dennislp3]

---------------------------
WorldViewer
---------------------------
Unknown Event type!

<historical_event>

      <id>147570</id>

      <year>161</year>

      <seconds72>-1</seconds72>

      <type>plundered site</type>

      <attacker_civ_id>781</attacker_civ_id>

      <defender_civ_id>797</defender_civ_id>

      <site_civ_id>797</site_civ_id>

      <site_id>88</site_id>

   </historical_event>

Saved to clipboard, please paste to development thread.  Thanks!
---------------------------
OK   
---------------------------


Dont shoot me im just the messenger =P

Let me know if you need my worldgen files

*EDIT* Also under the world details tab it reads that I have -119210 living HF and 119210 dead ones (total HF count says 0) I assume this is because it did not parse HF (which you already stated will be fixed next release)