Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: [Rubble] Rubblized Masterwork: Planning and progress reports  (Read 3753 times)

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile

So ever since I made Rubble I have been wanting to get more mods to use it, but due to modders having better things to do than learning a new syntax and porting lots of legacy raws it has not become quite the hit I had hoped for. After some consideration I have decided that if I want Rubble to become popular I needed users, and the best way to get lots of users is to have a popular mod use it.

But, as most modders have better things to do than spend all their time porting, I knew I would have to do the bulk of the work myself and so it would have to wait until I had time.
And now I have time.

Basically this is an effort to port Masterwork so that it uses Rubble to handle all the settings that are currently handled by the GUI. This is not a replacement for the GUI, it is a replacement for the backend.

The current system setting system uses raws that look something like this:
Code: [Select]
YES_SETTING[TAG]
YES_SETTING[OTHERTAG]
when a setting is active and:
Code: [Select]
!NO_SETTING!TAG]
!NO_SETTING!OTHERTAG]
when a setting is deactivated.

With rubble this would change to:
Code: [Select]
{@IF;$MW_SETTING;YES;
[TAG]
[OTHERTAG]
}
or possibly:
Code: [Select]
{SOME_SETTING}
which would expand to the proper raws only if the setting is active.

If this was all Rubble would do then it would be of only minor value, but Rubble also has builtin support for tilesets and creature graphics as well as significant support for DFHack.

I have an early version of the port working, but the only thing it does is handle tilesets, none of the other settings are handled yet. I would still upload this partial result except for one thing, it uses the new version of Rubble (pre4) that is not yet available, so no one would be able to do anything with it yet.

In any case here are my "port notes"
Spoiler (click to show/hide)

Main Goals:
Switching tilesets on worlds in progress (planned Rubble feature, NYI but not too hard to do)
Reformatting all raw files to make the formatting consistent (time to write that pretty printer...).
Templating all repetitive raw sequences (makes raws shorter, easier to edit).
Making Masterwork easily extensible via the Rubble addon system (easy to use third party mini-mods? where do I sign!).
Removing many "system inorganics" (eg transformation rocks for autosyndrome) and replacing them with automatic templates (like those used by the "Libs/DFHack/Command" default template).

Preview Download: At DFFD (1.1)
« Last Edit: June 04, 2014, 03:36:59 pm by milo christiansen »
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #1 on: May 21, 2014, 05:48:47 pm »

Reserve, because you never know when you'll need more space :p
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Meph

  • Bay Watcher
    • View Profile
    • worldbicyclist
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #2 on: May 22, 2014, 05:57:28 am »

I could fully understand taking any other mod into Rubble, to enable settings, additions and tileset options, but MDF has these functions already. At least in Windows. Does Rubble work on Mac Os and Linux? That would be great. :)
Logged
::: ☼Meph Tileset☼☼Map Tileset☼- 32x graphic sets with TWBT :::
::: ☼MASTERWORK DF☼ - A comprehensive mod pack now on Patreon - 250.000+ downloads and counting :::
::: WorldBicyclist.com - Follow my bike tours around the world - 148 countries visited :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #3 on: June 02, 2014, 10:25:36 am »

Rubble works on every OS that a go program can be compiled for, which includes OS X and Linux, but the main benefit is making the setting easier to handle (no need for the GUI to know how to handle a setting aside from whether it should set a config var or activate an addon). Making things like third part addons easy and providing a unified system for all OSes is just a bonus.

Any way I finally am ready to offer an initial trial version. Not nearly everything works yet, but tilesets and entity settings are complete, as well as a few of the others. Settings are handled via a combination of an ini file and a batch file for now, but expect a quick and dirty GUI for the next version :)

See the first post for the new port notes and the download link.

I am sticking with 4j for now, as I have a lot of work done with it, but as things progress I will have to update (I hope the changelog is detailed!)
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Meph

  • Bay Watcher
    • View Profile
    • worldbicyclist
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #4 on: June 02, 2014, 10:29:13 am »

The Changelogs are detailed in so far that they list every change, but from a players point of view. I dont say which ID in which file I changed.

I will have a look at the download asap. :) Edit: Cant find any download link. O.o
Logged
::: ☼Meph Tileset☼☼Map Tileset☼- 32x graphic sets with TWBT :::
::: ☼MASTERWORK DF☼ - A comprehensive mod pack now on Patreon - 250.000+ downloads and counting :::
::: WorldBicyclist.com - Follow my bike tours around the world - 148 countries visited :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #5 on: June 02, 2014, 10:32:28 am »

The download is there now :) I posted before I edited if you know what I mean...
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Meph

  • Bay Watcher
    • View Profile
    • worldbicyclist
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #6 on: June 02, 2014, 10:52:07 am »

Yes, happened to me too in my early days. ;) All good now. :)
Logged
::: ☼Meph Tileset☼☼Map Tileset☼- 32x graphic sets with TWBT :::
::: ☼MASTERWORK DF☼ - A comprehensive mod pack now on Patreon - 250.000+ downloads and counting :::
::: WorldBicyclist.com - Follow my bike tours around the world - 148 countries visited :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #7 on: June 02, 2014, 12:07:29 pm »

I just found something that pertains to my problem: https://xkcd.com/1319/

That is me and DF, I spend all my time working on my utilities and mods and no time playing anymore... Oh well I guess the result is worth the effort :)

Anyway I hope to have at least half of the remaining options ported by the next time I log on, which may not be for a few weeks.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Meph

  • Bay Watcher
    • View Profile
    • worldbicyclist
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #8 on: June 02, 2014, 12:09:34 pm »

That also fits Urist McTeellox. :D

Just a note: Splinterz and me will continue working on the GUI and change tilesets this way, but Rubble is perfect for Linux and Mac and people requested a GUI version for these OS since the mod existed. They will be thrilled, and Rubble could be the basis for the official Mac and Linux release of the mod. :)
Logged
::: ☼Meph Tileset☼☼Map Tileset☼- 32x graphic sets with TWBT :::
::: ☼MASTERWORK DF☼ - A comprehensive mod pack now on Patreon - 250.000+ downloads and counting :::
::: WorldBicyclist.com - Follow my bike tours around the world - 148 countries visited :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #9 on: June 02, 2014, 12:22:58 pm »

The problem is that to make full use of Rubble the raws need SIGNIFICANT changes! So the only way using Rubble makes sense is if the current GUI is ported to use Rubble as a backend at some point (Linux/mac users will have to make do with some other front end, but the backend will be the same everywhere)

Take a look at the entity files, right at the top of each one is a small taste of what porting to Rubble will mean for the raws.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Meph

  • Bay Watcher
    • View Profile
    • worldbicyclist
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #10 on: June 02, 2014, 12:32:00 pm »

Quote
So the only way using Rubble makes sense is if the current GUI is ported to use Rubble as a backend at some point (Linux/mac users will have to make do with some other front end, but the backend will be the same everywhere)
Which... makes no sense? Porting it to Rubble is more work than merging the tileset versions manually, and it adds another utility that needs to be updated, which neither me nor splinterz can do. You yourself havent been online the last 12 days.

Maybe you should make a poll? See if there is any interest in a Rubble-version of Masterwork, before you do all the work of making one. Especially if there is a lot of updating to do. I am on V.5.03 here at home, and I will do rapid releases in future, now that I am back in Germany. Like I said, it makes perfect sense as an alternative for Linux and Mac, but for Windows and the main release I will stick with the GUI we currently have. Splinterz is doing amazing work on it, improving it nearly every day, much more than I expected.

Could this somehow be done parallel? Both the same time, and I include the GUI for Win, and Rubble for Linux/Mac ? (wait, that makes no sense, because other files are different, the libraries and .exes etc.)
Logged
::: ☼Meph Tileset☼☼Map Tileset☼- 32x graphic sets with TWBT :::
::: ☼MASTERWORK DF☼ - A comprehensive mod pack now on Patreon - 250.000+ downloads and counting :::
::: WorldBicyclist.com - Follow my bike tours around the world - 148 countries visited :::

jcd

  • Bay Watcher
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #11 on: June 02, 2014, 01:19:29 pm »

i 'd love to stop having to use the gui exe on linux.
I think i can speak for a measurable portion of linux users when saying that there needs to be no GUI for linux, just a well-documented .config text file from where we will be able to modify all the options the gui currently has, and then the backend could run at game launch or as requested to apply the changes, if the .config has changed.
That 'd be even better than a new gui or a ported gui for linux as far i'm concerned. Faster and clearer about what is going on behind the scene.
From what i hear here (pun intended) Rubble could do that!

But, on the other hand, i never spent the time to understand how raws really work, and especially what the MWDF+Dfhack does, and i have a feeling that porting this (not to mention that new versions are usually coming out frequently and one has to not only catch up, but make the developers learn and adher to the new stadard) could be a stupendus effort.

Could this somehow be done parallel? Both the same time, and I include the GUI for Win, and Rubble for Linux/Mac ? (wait, that makes no sense, because other files are different, the libraries and .exes etc.)

Yup, many other files are different so we have to take all the linux variants of the components (vanilladf, dfhack, dtherapist), apply the MWDF raws and changes with each new build, then control it via the windows GUI.
The thing is, a port can be done and its not too hard to do (the script i mention in another thread could maybe become a base for a valid linux 'build' system for the developers. But it still requires lots of polishing and a linux box (that you've said somewhere you lack iirc)

So i think a solution for this could be 'building' each new linux version the way we do now (maybe improve the procedure when someone who understands how everything DF related works together works on it), then test it on 1-2 more machines and if its all ok just link it below the offical windows version.

Tbh the 'rapid' release circle you mention in your post scares me a little WRT linux porting. But its great for MWDF in general!!
« Last Edit: June 02, 2014, 01:26:17 pm by jcd »
Logged

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #12 on: June 02, 2014, 04:42:09 pm »

i've been exporting all the options and their details for a while now. if you don't want to use the exe you could use this to determine what settings you'd like changed and write your own scripts.

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #13 on: June 04, 2014, 03:34:40 pm »

Porting it to Rubble is more work than merging the tileset versions manually, and it adds another utility that needs to be updated, which neither me nor splinterz can do. You yourself havent been online the last 12 days.
Porting to Rubble would be WAY more work, in the short term. In the long term it would save huge amounts of time all around (and I am over half way done).

One prime example of how it would save time is the many autosyndrome rocks that Masterwork uses, if Rubble is used then they can be generated automatically from a simple {DFHACK_COMMAND;some_command;and;it's;args} in the reaction, the material is never referenced directly and is automatially shared by all identical templates. Fast and simple.
Another example of something gained is the fact that Rubble (at least the upcomming version of Rubble) can regenerate a world, allowing you to switch tilesets on a world in progress seamlessly.
And then there is the prep addon. If you have several worlds with different tilesets (say some sucession games) then you can "prep" a world to install the worlds tileset into your DF, without needing anything but the save and a copy of Rubble. Really the benafits are far too numerous to list.

Rubble is not going to stop being supported for the forseable future, and even if it is there is nothing hard coded that would bind it to a particuler DF version.

Anyway I am going to upload the latest version of the port in a little bit, and it has everything ported except the buildings/reactions (they work, they just can't be turned on and off yet)
Not everything is ported in the best way possible, but internal polish can come after everything works :)

Linux users will be hapy to know that Rubble works exactly the same on Linux, Windows, and Mac, and I gave lots of thought to running Rubble with no frontend/GUI so it works quite well from the command line (that's how I use it on windows).

I don't know if anyone has looked at the raws for the last version I uploaded, but the one you want are not in "raw/objects", they are in "rubble/addons/Masterwork" (mostly in the "Base" sub folder). The raws in "raw/objects" are the generated ones, eg not what you want to edit.

EDIT: New version is now up, check first post.
« Last Edit: June 04, 2014, 03:37:41 pm by milo christiansen »
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: [Rubble] Rubblized Masterwork: Planning and progress reports
« Reply #14 on: June 04, 2014, 09:23:33 pm »

Quote
So the only way using Rubble makes sense is if the current GUI is ported to use Rubble as a backend at some point (Linux/mac users will have to make do with some other front end, but the backend will be the same everywhere)
Which... makes no sense? Porting it to Rubble is more work than merging the tileset versions manually, and it adds another utility that needs to be updated, which neither me nor splinterz can do. You yourself havent been online the last 12 days.

Maybe you should make a poll? See if there is any interest in a Rubble-version of Masterwork, before you do all the work of making one. Especially if there is a lot of updating to do. I am on V.5.03 here at home, and I will do rapid releases in future, now that I am back in Germany. Like I said, it makes perfect sense as an alternative for Linux and Mac, but for Windows and the main release I will stick with the GUI we currently have. Splinterz is doing amazing work on it, improving it nearly every day, much more than I expected.

Could this somehow be done parallel? Both the same time, and I include the GUI for Win, and Rubble for Linux/Mac ? (wait, that makes no sense, because other files are different, the libraries and .exes etc.)

Couldn't the .ini files be parsed by a utility that you just run on every version?

I guess it's more than just that, but I was thinking if ! needs to be turned into {... couldn't a script (supplied by Milo) be ran that does this that could be pre-included with mwdf.

update:

on second thought.  I guess you would have to write a set of code for each module conversion you were planning on porting to rubble.

For example, I noticed the embark profiles in v5.0.4 have these ! tags associated to them now.  So I see each ! flag would need to be evaluated and have custom code written for it.

However.  Identifying those !flags, and then working with Meph on those and how/what has changed between versions (oh man, the headache of working on v4 when v5 has all the raw changes... not to mention the upcoming raw changes...)

could your tool be used to help convert the raws to the new update that is about to be pushed out (Toady's update)
« Last Edit: June 04, 2014, 10:25:29 pm by thistleknot »
Logged
Pages: [1] 2