Topic: Rubble 8.5.5 - DF 44.7 - New DF, new Rubble. (Read 71040 times)

Abadrausar · « **Reply #120 on:** May 09, 2016, 03:42:25 am »

Quote from: milo christiansen on May 07, 2016, 12:23:55 pm

If you want that kind of feature you have two choices: 1) you can already format the description line however you want (markdown allows embedded HTML) 2) You can edit the HTML templates to add the extra abilities you want.

Milo: Could you give me some more insights about in which directories and files are these HTML templates? I would like to have a look to see if I can do something...

Report of a future bug, not entirely sound actual implementation or some hours of unnecessary testing in the future, solution proposed.
Description: The content of the file material_template_user_generic_animal_mats.txt in the generic material addon, should be changed to

Quote

{@FOREACH;=BONE=0|HAIR=0|TOOTH=0|LEATHER=0|HORN=0|HOOF=0|PEARL=0|SILK=0|SHELL=0|
   SWEAT=0|TEARS=0|VENOM=0|WOOD=0|THREAD=0;   #   Those are actually unimplemented as generics but maybe they should, specially wood as Meph has recently done in his retaken Masterwork Mod
   {!SHARED_OBJECT_DUPLICATE;%{key}_TEMPLATE;%{key}_GENERIC_TEMPLATE;true;true}}
#   For those materials where genericity is not desired like blood of civilized creatures (thirsty vampires) or venom (syndromes)
#   we should also define
#   {@FOREACH;=BLOOD=0|VENOM=0;
#      {!SHARED_OBJECT_DUPLICATE;%{key}__TEMPLATE;%{key}_UNGENERIC_TEMPLATE;true;true}}

Why? Because for working correctly these templates must be guaranteed copies of the last ToadyOne version for those materials, in case contrary the generic material will have in the future different behavior to the base material.
Example: Alcohol material has recently received a syndrome to model inebriation, this kind of "activation" of materials can be done by Toady in any new version.
This fact forces us to preemptively solve this problem either with the clean and automatic solution proposed or alternatively by means of an excruciating and error prone process where for each new version of vanilla a manual verification (or copy-paste directly) of each one of those materials is done, to be sure that the behavior of the generic material is synchronized with the up to date base vanilla object, a 3rd way could be to do nothing until the first bug reports from the users concerning incoherent behavior of the generic materials begin to appear at some point in the future...
The inconvenient that I see with the lazy solution is that, if at the moment the bugs appears, we have forgotten all that, then some unnecessary hours of debugging will be necessary to track down the problem.
Another item in the wish list
I have remarked that rubble know internally the filename.txt where each new DF object must be generated, could be possible to have {@READ_LIST;<MASTERKEY>;<INDEX>} redefined as {@READ_LIST;<MASTERKEY>;<INDEX>[;<FILENAME>=true]}? and then when INDEX is defined as being a filename, MASTERKEY will return to us the list of DF objectsIDs that will be generated inside the file INDEX.

If that were possible, how the elements of one list can be removed or added to another list?
Could you give me one working example of the Rubble macro or Lua code doing that?
Thanks in advance! Not pressed at all, I will understand if you do not find the time...

I think that the list structure is very recurrent in our doings and that disposing of some generic operations over it, in adition to the list-like ,ADD, REMOVE or even INTERSECTION could help and clarify a lot our algorithms. Maybe the basics are already there but I simply do not see them.

milo christiansen · « **Reply #121 on:** May 11, 2016, 01:57:20 pm »

Quote from: Abadrausar on May 09, 2016, 03:42:25 am

Milo: Could you give me some more insights about in which directories and files are these HTML templates? I would like to have a look to see if I can do something...

After you run Rubble for the first time they appear in "other/webUI". Any HTML file in that directory is a template for the web UI. You'll probably have to dig though the source code for the web UI to see exactly what each template is given in the way of data, see "other/src/rubble8/rblutil/actions/webui/action.go". Rubble Basics has a section about this (web UI customization).

Quote from: Abadrausar on May 09, 2016, 03:42:25 am

(some stuff about generic materials)

Take another look at the generic animal materiel addons. They don't have copies of any materials, they simply define a new creature that has every common animal material (and a few more). These materials are mostly added via the default body detail plans, so if toady changes anything it will be automatically taken care of. The scripts simply change "LOCAL_CREATURE_MAT" into "CREATURE:ANIMAL" or some such. Other generic materials are of no interest to me right now.

Quote from: Abadrausar on May 09, 2016, 03:42:25 am

Another item in the wish list
I have remarked that rubble know internally the filename.txt where each new DF object must be generated,

Actually it doesn't. Rubble simply holds the data until a template requests it. The whole system is remarkably "dumb", it doesn't really have much special logic powering it.

As for more list operations: You can use the default Lua table library for many things. The values aren't really tables, but they act enough like them for any functions that operate on a table "in place" to work.

Look, your pet project is all well and good, but you can't expect me to keep adding special features just for you. If you have a good idea for something that fits in with the way Rubble already does things I'll probably add it, but for other stuff you'll have to prove it is worth my time (or provide a working implementation). HDFPS looks interesting, but it is radically different from what Rubble was intended to be. I'll help as I can, but I won't go out of my way to change Rubble just to support it. I simply don't have time to work on more projects.

8.1.0 is now up!

This version breaks the "old" auto-update system (you know the one I just added), but it is replaced by two much better ones. See the full changelog.

If anyone can run a dedicated content server I would be very grateful! PM me for details if you are interested.

Dirst · « **Reply #122 on:** May 14, 2016, 01:02:33 am »

Hi Milo, I'm kind of stuck on documentation. Apparenty there is now a maximum length that the GUI will render, because my file just stopped in the middle. So I split things up logically and tried to add documentation links. This first version "works," but it renders the file as plain text with no CSS or back button:

[Reference](/addonfile/addon_reference.md?addon=The%20Earth%20Strikes%20Back%21)

This is my best guess at how a doc-link is supposed to work, but it can't find the file:

[Reference](/doc/addon_reference.md?addon=The%20Earth%20Strikes%20Back%21)

Spoiler: error log (click to show/hide)

I've tried prepending the filename with addons/ and addondata/ and addons/The%20Earth%20Strikes%20Back%21 and addondata/The%20Earth%20Strikes%20Back%21 to no avail.

I'm probably doing something very stupid. The only thing I've established is that it can't parse "The Earth Strikes Back!" as an addon name without the HTML %codes for special characters.

milo christiansen · « **Reply #123 on:** May 16, 2016, 01:05:53 pm »

Quote from: Dirst on May 14, 2016, 01:02:33 am

Apparenty there is now a maximum length that the GUI will render, because my file just stopped in the middle.

Bizarre... There is no max length that I know of, does the log say anything or is there obviously bad markup where it ends? It could be a (HTML) template error of some kind, as that would trigger a log page redirect under normal circumstances, but won't work if the template was partly parsed before the error was hit.

Quote from: Dirst on May 14, 2016, 01:02:33 am

[Reference](/doc/addon_reference.md?addon=The%20Earth%20Strikes%20Back%21)

If the first path works, but this one doesn't then it's a Rubble bug of some kind, I'll try to figure out what's wrong when I get home...

Edit: Good news! You won't have to wait until I get home, as I remembered I have my Linux test version on this system.

The URL problem appears to be in the template, one of the functions that "fixes" links is chopping off any existing query. Any doc links with a query in the URL is borked in this version (and probably the last few versions as well).

I'll have this fixed and a new version up soon (a day or so).

If I manually copy the URL and add the "&addon=" part to the end it works fine, except the end of your reference document is chopped off too. This turns out to be caused by invalid HTML, you are missing a end tag for one of your spoiler DIVs.

Fixing this one is up to you

milo christiansen · « **Reply #124 on:** May 18, 2016, 01:47:23 pm »

OK, 8.2.0 is up now! This version fixes the doc link bug and adds a new feature that is 100% useless until I get a content server up and running...

Dirst · « **Reply #125 on:** May 22, 2016, 10:12:49 pm »

Hi milo!

The FileBank works like a charm for Stonesense content, except for one thing that you mentioned. Adding content to Stonesense requires overwriting at least one file, typically df/stonesense/index.txt.

I'd like my Rubble module to scan the existing index file and append a line if and only if that line is not already present. This is a relatively simple task, but I haven't tried to run a Lua script during the "build raws" phase before. First, how do I make a Lua script execute at the right time? Second, do I simply use an AXIS path in the Lua statement? For example, io.open('df/stonesense/index.txt')

milo christiansen · « **Reply #126 on:** May 23, 2016, 04:07:16 pm »

The "io" module is not available in Rubble Lua scripts (for security reasons), you will need to use the "axis" module. This module is documented in the Rubble Lua documentation.

From memory (and skipping error handling that you will definitely want to do):

Code: [Select]

local content, ok = axis.read("df/stonesense/index.txt") -- Return values may not be correct here, I can't remember how they are ordered
content = content.."Your stuff"
local err = axis.write("df/stonesense/index.txt") -- Same deal with the return values, look it up.

It would be best to make this a "post" script (runs after raws are done, but before files are written), to do this simply name your file "<filename>.post.lua". If you are using scripts to register the file bank this one would be a good place to put that too (but if you are using templates there is no reason to change).

Dirst · « **Reply #127 on:** May 23, 2016, 09:52:45 pm »

Thanks. FYI, the Lua Scripting Reference doesn't appear to be linked from the main documentation page. Fortunately markdown is human-readable in a text editor, but it would be nice to see it as intended.

Edit: So the code without any guardrails would be

Code: [Select]

local ok, content = axis.read("df/stonesense/index.txt")
content = content.."Your stuff"
local ok, err = axis.write("df/stonesense/index.txt")

Thundercraft · « **Reply #128 on:** May 25, 2016, 12:44:27 am »

Yes, I could just dive into the documentation and try it out. Also, I realize that I'm asking for opinions using highly subjective terms like "easier". But, I thought I'd ask first:

a) How much of a learning curve is there?
b) Generally, is it more tedious to develop a mod with Rubble, or does it make things easier?
c) If it is rather easy to create a small mod with Rubble, would developing large mod with Rubble be a different story?

I did read the part about "Rubble automates many common tasks". So it sounds like it should shorten dev time. Maybe my hesitation has to do with how I'm more visually-oriented than the average person. I could not find any screenshots...

Dirst · « **Reply #129 on:** May 25, 2016, 11:06:45 am »

Quote from: Thundercraft on May 25, 2016, 12:44:27 am

a) How much of a learning curve is there?

Most of what I do in Rubble is template-based, and that is at about the same level of complexity as raw modding. Not nearly as many valid tokens to learn, and Milo's stuff is marginally more internally consistent than Toady's

Normal raws are passed through Rubble unchanged, so you don't need to figure out a new way of doing everything if you already know a neat trick with the raws. This also means that Milo's syntax needs to avoid all of the common symbols used in raws, so he uses a lot of curly braces and semicolons while the raw text uses square brackets and colons.

Quote from: Thundercraft on May 25, 2016, 12:44:27 am

b) Generally, is it more tedious to develop a mod with Rubble, or does it make things easier?

There are two cases where Rubble saves you a boatload of time.
1. If you'd like to implement features that Milo has already built (such as powered workshops).
2. If you end up doing a lot of copy-and-paste-and-edit-slightly of raws. You can write the meat of it once as a custom template and call it with parameters.
Both of these reduce the chance for errors to creep into your mod.

Quote from: Thundercraft on May 25, 2016, 12:44:27 am

c) If it is rather easy to create a small mod with Rubble, would developing large mod with Rubble be a different story?

It's not really the size of the mod that matters, it's how much you'd be leveraging the two strengths above. I use the hell out of the second one, and I still haven't quite de-duplicated everything yet.

milo christiansen · « **Reply #130 on:** June 02, 2016, 11:22:17 am »

What Dirst said. In particular the ability to avoid duplication is the defining feature of Rubble.

If you want, you can use Rubble to generate your mod, then distribute the generated files instead of the Rubble sources, as this appeals to a certain kind of user (you know, the kind that doesn't want to learn anything new

). AFAIK Dirst does this for the vanilla versions of TESB.

I was planning on pushing a new version today, but I forgot to bring it with me

This version was mostly for the new DF, but it also has a minor new feature for file banks, namely they are mounted to a special location on the AXIS filesystem so you can easily read single files from them.

Oh well, it'll be up in a few days...

EDIT: I haven't been working on Rubble much lately because I am writing a building editor instead. That was the other thing I was going to upload today, but forgot...

Dirst · « **Reply #131 on:** June 02, 2016, 08:51:20 pm »

Hi Milo, yes I've been composing The Earth Strikes Back! in Rubble and publishing two distributions since v2.00 of the mod.

When working on de-duplicating code for v2.07, I ran into an odd timing issue with the Rubble parser. This might be intended behavior, but it's not what I expected.

Given a simplified building file like this:

Code: [Select]

{!TEMPLATE;TESB_WSHOP;name;
{@PARSE_TO;TEMP;{@STR_TO_ID;%{name}}}
[BUILDING_WORKSHOP:TESB_TRIBUTE_${TEMP}]
	[NAME:Tribute to %{name}]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:${TEMP}]
}

{TESB_WSHOP;Andesite}

{TESB_WSHOP;Basalt}

{TESB_WSHOP;Chalk}

You end up with:

Code: [Select]

[BUILDING_WORKSHOP:TESB_TRIBUTE_]
	[NAME:Tribute to Andesite]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:]

[BUILDING_WORKSHOP:TESB_TRIBUTE_ANDESITE]
	[NAME:Tribute to Basalt]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:ANDESITE]

[BUILDING_WORKSHOP:TESB_TRIBUTE_BASALT]
	[NAME:Tribute to Chalk]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:BASALT]

The @PARSE_TO seems to happen after the raw code is parsed, which doesn't mesh with the "soon as possible" definition of @.

The following code works as expected:

Code: [Select]

{!TEMPLATE;TESB_WSHOP;name;
{@PARSE_TO;TEMP;{@STR_TO_ID;%{name}}}
{SHARED_OBJECT_EXISTS;INORGANIC:${TEMP};
[BUILDING_WORKSHOP:TESB_TRIBUTE_${TEMP}]
	[NAME:Tribute to %{name}]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:${TEMP}]
;${TEMP} does not exist.}
}

{TESB_WSHOP;Andesite}

{TESB_WSHOP;Basalt}

{TESB_WSHOP;Chalk}

Producing:

Code: [Select]

[BUILDING_WORKSHOP:TESB_TRIBUTE_ANDESITE]
	[NAME:Tribute to Andesite]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:ANDESITE]

[BUILDING_WORKSHOP:TESB_TRIBUTE_BASALT]
	[NAME:Tribute to Basalt]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:BASALT]

[BUILDING_WORKSHOP:TESB_TRIBUTE_CHALK]
	[NAME:Tribute to Chalk]
	[NAME_COLOR:7:0:0]
	[DIM:3:3]
	[WORK_LOCATION:2:2]
	[BUILD_LABOR:MASON]
	Snipped out tiles and colors.
	[BUILD_ITEM:3:BLOCKS:NONE:INORGANIC:CHALK]

I can also get this output if I modify the original file to dispense with the @PARSE_TO and just put the @STR_TO_ID in both places. Of course the SHARED_OBJECT_EXISTS test should have been in there all along, but the timing between ! and @ has me confused. It looks like {!foo{@bar}} will run in the pre-parse stage, but {!foo{@bar{@baz}}} will not. Is this a limitation on nested templates that I just need to watch out for?

milo christiansen · « **Reply #132 on:** June 04, 2016, 11:46:46 am »

Those examples seem very weird... I have no idea why you are getting the results you are getting, but it is a safe bet it has something to do with variable expansion timings.

When the Rubble template parser finds a template it reads it's ID and decides what to do with it based on it's first character, to that end "@" is completely identical to "!" when it is found in the preparse stage, etc. So that is definitely not your problem...

Oh, wait! The problem is that variables are expanded before the template body is parsed, so of course $TEMP is expanding to nothing in the first template. (lucky you it didn't expand into some random garbage) Now how to fix it, hmmm.... Try "{!;$TEMP}" instead, that should work.

8.3.0 is up now!

This version is very minor, with only one new feature (the ability to read single files from a file bank).

Dirst · « **Reply #133 on:** June 12, 2016, 11:59:47 pm »

Quote from: milo christiansen on June 04, 2016, 11:46:46 am

Oh, wait! The problem is that variables are expanded before the template body is parsed, so of course $TEMP is expanding to nothing in the first template. (lucky you it didn't expand into some random garbage) Now how to fix it, hmmm.... Try "{!;$TEMP}" instead, that should work.

Okay, took me a bunch of times reading that for it to sink in. (And it's a moot point in this mod because the SHARED_INORGANIC test really does belong there.)

Let me know if this is correct: the parsing order is $VAR !PRE PARSE #POST, with early-phase stuff delayed if it's nested inside something later. In that case, @ is effectively synonymous with the parent structure's prefix. My guess is that %ARG expansion is on-par with @, but can it be delayed at all? (perhaps to build an indirect reference to one of several different arguments, just avoiding @IF out of snarkiness)

milo christiansen · « **Reply #134 on:** June 14, 2016, 08:56:50 am »

Quote from: Dirst on June 12, 2016, 11:59:47 pm

... with early-phase stuff delayed if it's nested inside something later.

The only template type that can be delayed to a later stage this way is "@" templates. If a template with an explicit parse stage (eg, not an "@" template) is delayed to a stage later than its specified stage generation will abort.

The following details how a template body is parsed:

Quote from: Rubble Basics

1. The template's body is loaded and these transforms are run on it:
   1. Text of the forms `%<NAME>` or `%{<NAME>}` is replaced with the value of the param named
     `<NAME>` or the default value specified for that param.
   2. Text of the forms `$<NAME>` or `${<NAME>}` *that is not in a child template* is replaced
     with the value of the variable `<NAME>`.
   3. Text of the forms `&<NAME>` or `&{<NAME>}` is replaced with the value of the variable
     `<NAME>`.
2. The template body is passed to the stage parser and the result is returned

This is where all variable expansion happens, see Rubble Basics for more info.

Quote from: Dirst on June 12, 2016, 11:59:47 pm

In that case, @ is effectively synonymous with the parent structure's prefix.

Exactly.

Quote from: Dirst on June 12, 2016, 11:59:47 pm

My guess is that %ARG expansion is on-par with @, but can it be delayed at all?

No. You can store the value for later (by inserting it into an ECHO call or similar), but you can't delay expansion. If you need to nest template definitions (or use @FOREACH inside a definition) Rubble Basics has some techniques that work for delaying expansion, but it's not the same.

Here is an example of storing a value for later:

Code: [Select]

# is a alias for #ECHO
{!TEMPLATE;arg;{#;%arg}}

This example doesn't do anything useful, but the technique may be used for all sorts of things (delayed variable expansion mostly).

News:

Author Topic: Rubble 8.5.5 - DF 44.7 - New DF, new Rubble. (Read 71040 times)

Abadrausar

Re: Rubble 8.0.0 - DF 42.6 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.0.0 - DF 42.6 - Addon Pack Automatic Update (and other cool stuff)!

Dirst

Re: Rubble 8.1.1 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.1.1 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

Dirst

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

Dirst

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

Thundercraft

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

Dirst

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

Dirst

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

Dirst

Re: Rubble 8.2.0 - DF 43.2 - Addon Pack Automatic Update (and other cool stuff)!

milo christiansen

Re: Rubble 8.3.0 - DF 43.3 - Addon Pack Automatic Update (and other cool stuff)!