Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 5 6 [7] 8 9 ... 14

Author Topic: Rubble 8.5.5 - DF 44.7 - New DF, new Rubble.  (Read 71043 times)

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.0 - DF 42.6 - Automatic remote dependency download!
« Reply #90 on: April 07, 2016, 06:49:15 pm »

...
Apology accepted Milo.

Even if taking care of the feelings of the people about things is important to have a strong and united community, in reality what were worrying me the most were the feeling of lost opportunity...

What I mean is that any successful software project needs a bug database as complete and detailed as posible, because that is like the intelligence reports for an army, if you have it good, you can take your objetives, you can save your people and you can assign your resources effectively, if you have it bad, surely your army is about to be overwhelmed because without good infos the army is blind.

In this sense you can even measure the success of a software by the diversity, quality and quantity of their intelligence reports (their bug databases), if anyone doubt it only has to consult the bug databases of projects like Dwarf Fortress, DFhack or Linux.

But reporting a bug requires more work than grumbling mildly and deciding than a tool is not helping us, it is sad to say that, but most of us are really lazy, have little time or both.

That means that the process of bug reporting, so valuable for us, should be done as much as possible without drama ni confrontation, in an aseptic way because if we fail in that, people will report scarcely or not at all, and that is against our interest.

Only my opinion, but open to other views...

I just have downloaded First Landing 1.7 and Rubble 7.7, I will compare them with their precedent versions with gitextension and if I detect some problem the relevant info will be at your disposal.
« Last Edit: April 07, 2016, 06:58:34 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.0 - DF 42.6 - Automatic remote dependency download!
« Reply #91 on: April 07, 2016, 07:32:31 pm »

There will never be a bug tracker so long as I do not have internet,  there will also not be a public repository, fast turnaround times for bugfixes, and a dozen other things that would be really nice to have.

Without easy access to both the internet and my development/game machine, and with turnaround time measured in days or weeks between me finding out about a bug and me being able to do something about it, bug reporting and fixing will both remain frustrating.

But, I may get internet this summer (I hope), and that would help a lot (faster communications loop).
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Dirst

  • Bay Watcher
  • [EASILY_DISTRA
    • View Profile
Re: Rubble 7.7.0 - DF 42.6 - Automatic remote dependency download!
« Reply #92 on: April 09, 2016, 12:47:33 am »

Since it's just a plain text file, is there a way to get the module to spit out a .webload file in the right place at raw-generation time?

Once the system is stable (that is, functions when the module itself is not yet installed), I can add instructions on making a .webload into my mod's OP.
Logged
Just got back, updating:
(0.42 & 0.43) The Earth Strikes Back! v2.15 - Pay attention...  It's a mine!  It's-a not yours!
(0.42 & 0.43) Appearance Tweaks v1.03 - Tease those hippies about their pointy ears.
(0.42 & 0.43) Accessibility Utility v1.04 - Console tools to navigate the map

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.0 - DF 42.6 - Automatic remote dependency download!
« Reply #93 on: April 11, 2016, 08:27:52 am »

Milo I have tried your rubble conversion addon SO Insert and it seems to work, at least at a basic level as you say, luckily this is what is needed most of the time.
Proposal to streamline the process of converting Dwarf Fortress raws to the rubble system:
RFC: Adding a new File Extension and Tag .DF with every file of a mod containing unmodified Dwarf Fortress objects
In a stage anterior to any other in rubble, any files with extension .DF will be feeded to the SO Insert subsystem that will transform them into the correspondent  .txt files with the Dwarf Fortress objects already rubblized, following that, will begin the normal rubble process as we know it now.

Bonus points if most of this registration process can be done in lua lands instead of in go lands, mostly because I would like to learn to contribute myself some way to declare a file extension as being .world generation parameters and thus being effectively added to the end of the data/init/world_gen.txt file or to data/init/embark_profiles.txt for the extension .embark, or many other as speech...

I could try to do those little additions with lua, but hardly with goland...
There are between us more lua noobs, like me for example, than go noobs.
« Last Edit: April 11, 2016, 08:42:38 am by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.0 - DF 42.6 - Automatic remote dependency download!
« Reply #94 on: April 12, 2016, 07:16:04 am »

If you want to handle more extensions (via Lua instead of adding native code to handle them) it is certainly possible. Currently you will have to parse the file name yourself (which, considering the state of the Lua string library, is a real pain) and decide how you want to to handle each file individually. Depending on the file's existing last part extension it may be possible to use the file action API (look at the script that applies extra file tags to raw files for a great example), but if the file type you want to handle does not have any tags then you will have to walk the addons and their files yourself.

Once you find a file of interest you have a choice, handle it now, or tag it and handle it later. I personally recommend tagging files in a load script, then writing them out in a post script with a sequence of "zzz" (as this way the vast majority of possible failures will be past).

Obviously this is cumbersome, but it would be impossible for Lua scripts to add rules to the file tagger, as this is run at load time, long before it is possible to run scripts.

Since it's just a plain text file, is there a way to get the module to spit out a .webload file in the right place at raw-generation time?

Nope. Rubble has no way to know where you will upload the addon pack or where you downloaded the pack from. Oh, wait. You want it to install a .webload so that users will have automatic update without needing to install a .webload too? In that case yes, but you will have to use a script to write a file to "addons:dir:" and it won't be guaranteed to work properly (what if a user installs your addon under a non-default file name?).

I am currently working on a new version of AXIS VFS that will allow a more flexible addon loader, in particular I plan to change the way Rubble stores loaded addons so that addons are loaded once at startup instead of being loaded fresh every time a new State is created. Obviously this will require the ability to add more addons to the database at any time prior to generation. Also obvious is that it will be possible to add an automatic update feature that does not depend on the existing web load system (which is more useful for initial installation).

The new AXIS and loader will probably not hit soon, but when it does it will bring major changes to addon loading. This (better addon loading) will probably be the main feature of "Rubble 8". I do have to stress that I won't be making such a large jump for a while...



New version! 7.7.1 brings a minor bug fix and basic support for "spoilers". Spoiler support is very basic, but it works. For details see the change log (and there is an example in Rubble Basics, read it in the web UI to see it in action).
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Dirst

  • Bay Watcher
  • [EASILY_DISTRA
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #95 on: April 12, 2016, 10:03:09 am »

Thanks Milo.

What I was looking for in generating a .webload was something like (in completely made-up pseudocode)

IF NOT EXISTS("addons:dir:The Earth Strikes Back!.webload") THEN
   CREATE WLFILE AS "addons:dir:The Earth Strikes Back!.webload"
   WRITE TO WLFILE "http://dffd.bay12games.com/download.php?id=11912&f=The+Earth+Strikes+Back%21.zip"
   CLOSE WLFILE
END IF


If the user changed the name of the zip, then they'll end up with two copies.  But that's better than a really complicated and brittle round of exception handling.
Logged
Just got back, updating:
(0.42 & 0.43) The Earth Strikes Back! v2.15 - Pay attention...  It's a mine!  It's-a not yours!
(0.42 & 0.43) Appearance Tweaks v1.03 - Tease those hippies about their pointy ears.
(0.42 & 0.43) Accessibility Utility v1.04 - Console tools to navigate the map

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #96 on: April 14, 2016, 04:22:24 am »

Report of active BUG in the webload system:

The Rubble changelog for version 7.7.1 says
Spoiler (click to show/hide)
What happens in reality is
Spoiler (click to show/hide)
If we compare that with the desired behavior
Milo when a .webload exist in the local dir for an addon but no the related .zip file of the addon, could the actual implementation of the .webload system be slightly modified? in order to:
1) Redownload the .zip when the fileserver is accesible.
2) If not, log that the needed file is offline for any reasons and also one message saying that the file will be redownloaded when the file will be accesible again.
3) Never the rubble generation process should fail or block only because one zip is missing when it can be automatically redownloaded later. (with the actual system, if the user did not find the missing .zip he is forced to delete the .webload file  >:( to deblock the rubble generation )
The situation has advanced a little, because now, at least, the downloaded zipped file is created in the local dir, but it has always size 0, at the source of the error, apparently, a problem with the write permissions of the zipped file in the local dir, at least that is what is stated by the error message.
This blocks rubble generation as described in point 3)

In resume, the bug in the webload system is not fixed: Now the downloaded file is created at the local dir, but empty, so the downloaded zipped file is not really saved in the local dir as it should.

I use windows 7 and google chrome as navigator for testing but I have tried also with others (firefox and opera) and the behavior is the same.
« Last Edit: April 14, 2016, 04:56:19 am by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #97 on: April 15, 2016, 07:42:45 am »

Damn... I have no idea why that doesn't work, it did in my (admitedly not very comprehensive) testing. I'll do some more extensive testing across my LAN and see what happens.

Spoiler: Technical Stuff (click to show/hide)

If the user changed the name of the zip, then they'll end up with two copies.  But that's better than a really complicated and brittle round of exception handling.

Actually it's worse, as then Rubble will find two addons with the same name and abort. I'll have working automatic-update soon (powered by an addon.meta key and a new AXIS version), so it's probably better to just wait and not worry about it right now.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #98 on: April 16, 2016, 06:37:50 am »

...Damn... I have no idea why that doesn't work...
I wanted to wait until the webload system were usable before reporting more bugs, but if you are going to rewrite all the AXYS system, and considering that the two problems are probably related to the elusive frontiers between real and mounted virtual filesystem:

Report of a bug: ANY zipped hierarchy of addons should actually be interpreted by the AXYS system in Rubble exactly the same, independently of if it is zipped or unzipped, when the path of access real (unzipped) or virtual (zipped) are the same by means of the tags->name of the root directory of the addonpack in the addon.meta file.

This is not actually true.
Please ask me more detailed info if you need it to clearly identify the bug described.

« Last Edit: April 16, 2016, 01:26:41 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #99 on: April 18, 2016, 11:33:37 am »

OK, I figured out what was wrong with the webload system (there was a goof in the "Create" method in the "Multiplexer", which is only used by the addons directory), but I probably won't actully upload a fix, as Rubble 8 (with the all new loader and AXIS 2) is almost ready. Currently it is "finished", but I want to do more testing.

As a special treat you can have a log :) See if you can spot the newest features... (the random startup line is unintentionally awesome!)

Code: [Select]
Rubble v8.0.0 Internal Test Build
Long hours of debugging later...
==============================================
Attempting to Read Config File: ./rubble.ini
  Read OK, loading options from file.
==============================================
Loading Addons...
  Loading Addons From Local Directories...
  Loading Addons From Local and Remote Addon Packs...
  Loading Pack: FL-Mawrings
    Trying to find update URL...
      No update URL found.
  Loading Pack: The Earth Strikes Back!
    Trying to find update URL...
      No update URL found.
  Checking Loaded Data...
==============================================
Creating New State...
  Loading Special Global Files...
    Reading DF Init Files...
  Creating Script Runners...
==============================================
Starting the Default Web Browser...
  Attempting to Open: "http://127.0.0.1:2120/menu"
Starting Server...
UI Transition: "/menu"
==============================================
Creating New State...
  Loading Special Global Files...
    Reading DF Init Files...
  Creating Script Runners...
UI Transition: "/tsetaddons"
Toggling addon: "Tilesets/MLC/TWBT - 24px" to true
UI Transition: "/tsetrun"
==============================================
Entering Tileset Mode for Region: raw
==============================================
Creating New State...
  Loading Special Global Files...
    Reading DF Init Files...
  Creating Script Runners...
==============================================
Activating...
  Clearing Leftover Addon State Information...
  Loading Config Variables...
    No variables specified.
  Generating Active Addon List...
  Activating Addons from Generated List...
  Pruning Library Addons...
  Activating Required Addons from Meta Data...
  Running Loader Scripts...
    aaa_init_scripting.load.lua
  Active Addons:
    Tilesets/MLC/TWBT - 24px
    Libs/Base
  Checking for Incompatible Addons from Meta Data...
  Setting Unset Config Variables to Their Defaults...
    Attempting to load rubble/userconfig.ini...
      population_cap
      strict_population_cap
      baby_cap_percent
      baby_cap_absolute
      visitor_cap
    Setting Remaining Variables from Meta-Data...
    (Any variables not listed above were explicitly set elsewhere.)
==============================================
Applying Tileset...
  Loading Rules Files...
    aaa_libs_base.tset.rules
  Loading Tileset Files...
    aaa_libs_base.tset.rbl
    tilesets_mlc_twbt.tset.rbl
  Applying Tileset Information...
    b_detail_plan_default.txt
    body_default.txt
    body_rcp.txt
    building_custom.txt
    c_variation_default.txt
    creature_amphibians.txt
    creature_annelids.txt
    creature_birds.txt
    creature_birds_new.txt
    creature_bug_slug_new.txt
    creature_desert_new.txt
    creature_domestic.txt
    creature_equipment.txt
    creature_fanciful.txt
    creature_insects.txt
    creature_large_mountain.txt
    creature_large_ocean.txt
    creature_large_riverlake.txt
    creature_large_temperate.txt
    creature_large_tropical.txt
    creature_large_tundra.txt
    creature_mountain_new.txt
    creature_next_underground.txt
    creature_ocean_new.txt
    creature_other.txt
    creature_reptiles.txt
    creature_riverlakepool_new.txt
    creature_small_mammal_new.txt
    creature_small_mammals.txt
    creature_small_ocean.txt
    creature_small_riverlake.txt
    creature_standard.txt
    creature_subterranean.txt
    creature_temperate_new.txt
    creature_tropical_new.txt
    creature_tundra_taiga_new.txt
    descriptor_color_standard.txt
    descriptor_pattern_iris_eye.txt
    descriptor_pattern_pupil_eye.txt
    descriptor_pattern_special.txt
    descriptor_shape_standard.txt
    entity_default.txt
    inorganic_metal.txt
    inorganic_other.txt
    inorganic_stone_gem.txt
    inorganic_stone_layer.txt
    inorganic_stone_mineral.txt
    inorganic_stone_soil.txt
    interaction_standard.txt
    item_ammo.txt
    item_armor.txt
    item_food.txt
    item_gloves.txt
    item_helm.txt
    item_pants.txt
    item_shield.txt
    item_shoes.txt
    item_siegeammo.txt
    item_tool.txt
    item_toy.txt
    item_trapcomp.txt
    item_weapon.txt
    language_DWARF.txt
    language_ELF.txt
    language_GOBLIN.txt
    language_HUMAN.txt
    language_SYM.txt
    language_words.txt
    material_template_default.txt
    plant_crops.txt
    plant_garden.txt
    plant_grasses.txt
    plant_new_trees.txt
    plant_standard.txt
    reaction_other.txt
    reaction_smelter.txt
    tissue_template_default.txt
  Applying init Patches...
  Running Tileset Scripts...
    tilesets_mlc_twbt.tset.lua
==============================================
Writing Files...
  Writing Raw Files...
    b_detail_plan_default.txt
    body_default.txt
    body_rcp.txt
    building_custom.txt
    c_variation_default.txt
    creature_amphibians.txt
    creature_annelids.txt
    creature_birds.txt
    creature_birds_new.txt
    creature_bug_slug_new.txt
    creature_desert_new.txt
    creature_domestic.txt
    creature_equipment.txt
    creature_fanciful.txt
    creature_insects.txt
    creature_large_mountain.txt
    creature_large_ocean.txt
    creature_large_riverlake.txt
    creature_large_temperate.txt
    creature_large_tropical.txt
    creature_large_tundra.txt
    creature_mountain_new.txt
    creature_next_underground.txt
    creature_ocean_new.txt
    creature_other.txt
    creature_reptiles.txt
    creature_riverlakepool_new.txt
    creature_small_mammal_new.txt
    creature_small_mammals.txt
    creature_small_ocean.txt
    creature_small_riverlake.txt
    creature_standard.txt
    creature_subterranean.txt
    creature_temperate_new.txt
    creature_tropical_new.txt
    creature_tundra_taiga_new.txt
    descriptor_color_standard.txt
    descriptor_pattern_iris_eye.txt
    descriptor_pattern_pupil_eye.txt
    descriptor_pattern_special.txt
    descriptor_shape_standard.txt
    entity_default.txt
    inorganic_metal.txt
    inorganic_other.txt
    inorganic_stone_gem.txt
    inorganic_stone_layer.txt
    inorganic_stone_mineral.txt
    inorganic_stone_soil.txt
    interaction_standard.txt
    item_ammo.txt
    item_armor.txt
    item_food.txt
    item_gloves.txt
    item_helm.txt
    item_pants.txt
    item_shield.txt
    item_shoes.txt
    item_siegeammo.txt
    item_tool.txt
    item_toy.txt
    item_trapcomp.txt
    item_weapon.txt
    language_DWARF.txt
    language_ELF.txt
    language_GOBLIN.txt
    language_HUMAN.txt
    language_SYM.txt
    language_words.txt
    material_template_default.txt
    plant_crops.txt
    plant_garden.txt
    plant_grasses.txt
    plant_new_trees.txt
    plant_standard.txt
    reaction_other.txt
    reaction_smelter.txt
    tissue_template_default.txt
  Writing Tileset Files...
    MLC 24x24 - Graphics.tset.png
    MLC 24x24 - Overrides.tset.png
    curses_1280x600.tset.png
  Writing Init Files...
    init.txt
    d_init.txt
    overrides.txt
      Assembling TWBT Overrides File...
        tilesets_mlc_twbt.tset.twbt
      Writing File.
Run time:  343.5461ms
Done.
UI Transition: "/log"
UI Transition: "/kill"

Anyone who wants to get a jump on the new version can add a file named "pack.meta" to the root of any addon packs that should allow auto-update. This file should be JSON encoded just like addon.meta. There is only one key that I feel comfortable revealing at this point: "Update" should be a URL to your packs download, just like you would use for a .webload file.

Speaking of .webload: Both it and the relatively recent "Downloads" addon.meta key will be going away, as they conflict with the new loader. "Downloads" will be replaced with a key in pack.meta, and .webload is replaced by the "Update" key.

Now is the time to suggest any major changes, as I want to do any breaking that is needed in the 7->8 transition rather than later...



Report of a bug: ANY zipped hierarchy of addons should actually be interpreted by the AXYS system in Rubble exactly the same, independently of if it is zipped or unzipped, when the path of access real (unzipped) or virtual (zipped) are the same by means of the tags->name of the root directory of the addonpack in the addon.meta file.

This is not actually true.
Please ask me more detailed info if you need it to clearly identify the bug described.

I am not quite sure what you mean here...

AXIS should treat a zip file more-or-less exactly like a read-only directory (with child directories and such), any other behavior is a bug that I want to know about. (BTW: I have never used AXIS to read from a zip outside of the addon loader, so if the addon loader doesn't use a feature it is untested).
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #100 on: April 18, 2016, 06:36:57 pm »

Now is the time to suggest any major changes, as I want to do any breaking that is needed in the 7->8 transition rather than later...
Things that many of my rubble converted mood needs:

1. Rubble support for the data/speech directory.
2. Rubble support for worldgen and embark files.
3. Enhancing the SO insert with sensible defaults, proposed algoritm:
Spoiler (click to show/hide)
4. I would like to be able to convert the directory addons/Libs in a zip able of autoupdating :o
5. The line numbers where Opening brace count does not match closing brace count for that line are important to be able of correcting the bad output, disabled raw tags, or simply literal braces in text.
6. Enhancing the rubble lexer to take into account the nested key tokens:
Spoiler (click to show/hide)

Milo: Please download: http://dffd.bay12games.com/file.php?id=11963 generate with rubble, and have a look, then you can have an idea of what I am trying to do, I would like some feedback from you about how the system could be done better.
I am not quite sure what you mean here...
The VANILLA library addon is central and their DF objects unmodifiable by the other mods, except by means of the delta semantic operators, inside the vanilla directory  3 visual addons with base mods and 1 hidden library are defined, if you try to convert this vanilla directory into a functionally equivalent zip, you will understand what I mean...

If you have a look, could you please say me what I have done wrong with local sodelta lua definition? it gives me a strange error message, this function concentrates the behavior of {SHARED_OBJECT_KILL_TAG;<ID>;<TAG>}, {SHARED_OBJECT_REPLACE_TAG;<ID>;<TAG>;<REPLACEMENT>} and {SHARED_OBJECT_ADD;<ID>;<PRERAWS>} and logs a detailed info about any unapplied delta to the modder, it is a little ambitious for my level, but probably not for yours, it uses contextual info (Lua globals) to be able to apply a serie of changes to the same object (for example an entity)
While not a perfect system I am working around the patch conflicts between mods like this:
1. I use the addon.meta "LoadPriority": value, to indicate the required order of application of mods.
Spoiler (click to show/hide)
2. Then I use delta operators (aka semantic diff) as defined in http://martinfowler.com/bliki/SemanticDiff.html
Code: [Select]
So, while textual diff works with a context of files and lines inside them to define where to apply the delta.
my DF aware delta operators define this context in terms of DF rubble objects and tokens inside them to define where to apply the deltas.
This makes the system more reliable, as you will modify the targetted object independently of in which file it is being defined
3. The temporal definition (local non committed version) of those operators is:
Code: [Select]
-------------------------------------------------------------------------------------------------------------------------------------------
-- Semantic Delta operators
-- {@CONTEXT;objectid;wheretoappend}
-- Fixes the objectid (for example ENTITY:MOUNTAIN) where the delta will be applied and also if the appended preraws go before or after (this is the default) the attachment token.
-- {@DELTA;token;value;preraws}
-- Modifies the end value of a token inside an object and  eventually attach before or after it the preraws, if the token id is given but not the value the token is killed.
-- if not token is given the preraws are appended at the end of the object. When no object is found a detailed DELTA warning is generated.
-------------------------------------------------------------------------------------------------------------------------------------------
___rubble_object_cursor = ""
___rubble_attach_after = true
local socontext = [[
local value, after = rubble.targs({...}, {"", "AFTER"})
___rubble_object_cursor = rubble.expandvars(value, "$", true)
if after == "BEFORE" then
___rubble_attach_after = false
else
___rubble_attach_after = true
end
]]
rubble.template("!CONTEXT", socontext)
rubble.template("@CONTEXT", socontext)
rubble.template("CONTEXT", socontext)
rubble.template("#CONTEXT", socontext)
rubble.template("#C", socontext)
local sodelta = [[
local token, value, preraws = rubble.targs({...}, {"", "", ""})
if token == "" then
local data = rubble.registry["Libs/Base:SHARED_OBJECT_ADD"].table
data[___rubble_object_cursor] = (data[___rubble_object_cursor] or "").."\n\t"..rubble.parse(preraws)
return
end
token = string.split(token, ":")
rubble.libs_base.sharedobject_walk(___rubble_object_cursor, function(tag)
if rubble.libs_base.matchtag(tag, token) then
if value == "" then
local preraws = "-"..tag.ID
for _, v in ipairs(tag.Params) do
preraws = preraws..":"..v
end
tag.Comments = preraws.."-"..tag.Comments
else
local fulltoken = "["..token..":"..value.."]"
if ___rubble_attach_after == true then
tag.Comments = preraws..fulltoken..tag.Comments
else
tag.Comments = fulltoken..preraws..tag.Comments
end
end
tag.CommentsOnly = true
else
rubble.warning("OBJECT: "..___rubble_object_cursor.." NOT found, {@DELTA;"..token..";"..value.."} mal formed or incorrect order of application of mods")
end
end)
]]
rubble.template("!DELTA", sodelta)
rubble.template("@DELTA", sodelta)
rubble.template("DELTA", sodelta)
rubble.template("#DELTA", sodelta)
rubble.template("@D", sodelta)
4. As they log with enough detail each DELTA failure, modders are guaranted in absence of these warning that ALL intended changes to the vanilla raws have been effectively applied in the order specified by the mods and their dependencies.
5. Some examples of use of the actual DELTA operator applying some changes in vanilla required by the modest mod maintained by Button:
Spoiler (click to show/hide)
Some mods depends on Modest mod that is almost complete but not finished, and modest  mod depends on working delta operators but less complete and functional that the one that I have asked your help to debug.
« Last Edit: April 21, 2016, 04:42:42 am by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

Dirst

  • Bay Watcher
  • [EASILY_DISTRA
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #101 on: April 18, 2016, 07:32:30 pm »

Anyone who wants to get a jump on the new version can add a file named "pack.meta" to the root of any addon packs that should allow auto-update. This file should be JSON encoded just like addon.meta. There is only one key that I feel comfortable revealing at this point: "Update" should be a URL to your packs download, just like you would use for a .webload file.

Speaking of .webload: Both it and the relatively recent "Downloads" addon.meta key will be going away, as they conflict with the new loader. "Downloads" will be replaced with a key in pack.meta, and .webload is replaced by the "Update" key.

Now is the time to suggest any major changes, as I want to do any breaking that is needed in the 7->8 transition rather than later...
That's almost exactly what I was envisioning, thanks!  Does it do a pre-pass thru all of the zips first, then load everything for real?

As for major changes, I would really like to find some way of supporting third (fourth?) party content like Stonesense assets, Armok Vision models, Soundsense packs, Therapist grids, etc.  As a starting point, these could be considered bricks that the engine more or less blindly unzips at the target location (just confirm to overwrite).  Something BAMMy might eventually make sense, but I wouldn't want you to spend a god-awful amount of time with special-case-hell just to get the initial version out the door.

Another suggestion would be to get Internet at your home :)
Logged
Just got back, updating:
(0.42 & 0.43) The Earth Strikes Back! v2.15 - Pay attention...  It's a mine!  It's-a not yours!
(0.42 & 0.43) Appearance Tweaks v1.03 - Tease those hippies about their pointy ears.
(0.42 & 0.43) Accessibility Utility v1.04 - Console tools to navigate the map

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #102 on: April 18, 2016, 07:40:59 pm »

Another suggestion would be to get Internet at your home :)
I am sure Milo would be happy to have a working Internet ;) If he could....

Dirst: Could you download that and have a look? http://dffd.bay12games.com/file.php?id=11963

It would be nice to have some feedback or opinions.

« Last Edit: April 19, 2016, 09:12:19 am by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #103 on: April 21, 2016, 01:42:08 pm »

Does it do a pre-pass thru all of the zips first, then load everything for real?

Nope, it loads the first pack and any others it requests first, then continues down the list recursively (skipping any it has already loaded). Addons in directories are loaded before packs, just like before. In general addons in directories should be reserved for WIP stuff and the addons that ship with Rubble.

As for major changes, I would really like to find some way of supporting third (fourth?) party content like Stonesense assets, Armok Vision models, Soundsense packs, Therapist grids, etc.

The trouble with "global" content like this is that it falls outside of the norm. It is a good idea, but I am not sure how best to handle it... Maybe a way to flag a directory and its children (including child directories) as a "copy file addon" that is then copied to a given AXIS path during the write stage?

1. Rubble support for the data/speech directory.

This is tricky. I don't want Rubble writing anything that cannot be moved by simply packing up the save, but unfortunately some mods need to write things elsewhere.

Currently it is possible to write files to this directory (via the AXIS path "df:data/speech" (or "df/data/speech" in v8)), but this is a minor pain. I suppose a trivial worry about not having self-contained output in one minor case is no good reason to not support it better. Look for a new first-part extension and file tag supporting writes to this directory in v8.

2. Rubble support for worldgen and embark files.

Same deal as data/speech, but with less reluctance on my part. I suppose something made with the raw parser that allows only modifying/adding named objects would be what is wanted?

3. Enhancing the SO insert with sensible defaults...

Good ideas, easy to do. Look for it in v8.

4. I would like to be able to convert the directory addons/Libs in a zip able of autoupdating :o

No. Too much stuff there depends on internal details of the current Rubble version. Library addons should be updated at the same time as the Rubble binary. When I add auto-update for Rubble itself this will be included as part of that.

5. The line numbers where Opening brace count does not match closing brace count for that line are important to be able of correcting the bad output, disabled raw tags, or simply literal braces in text.

This isn't quite clear. You want line numbers for where the brace counts don't match? I do too, but it is impossible for Rubble to calculate this with any accuracy (how can you tell which brace isn't paired? To put it simply, you can't.)

6. Enhancing the rubble lexer to take into account the nested key tokens:

Big problem here: Rubble has no concept of a DF "object", much less a sub-object. The Rubble template parser totally ignores raw tags, instead dealing with each file as a chunk of unstructured text. The only thing the template parser cares about is templates. Rubble does an OK job of faking a more sophisticated approach via the shared object system, but really even that looks at the object body simply as a block of text.

The main problem is that a Rubble raw file may or may not look like raws, depending on how extensive use of templates is. Every time I try to make something "smarter" it runs straight into the fact that, until after the final parse stage, a Rubble raw file is not really readable as raws. The body of a shared object is often fairly readable, but only because most of the time it has few templates. This is not always the case, leading to "smart" editing templates being unreliable.



@Abadrausar: I'll take a look at your project, it looks interesting...

The main problem I forsee is that it may be too complicated to maintain easily and/or too hard to convert existing mods for those unfamiliar with Rubble. The system itself looks very interesting, but I want to play with it some before I say much.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #104 on: April 21, 2016, 02:31:10 pm »

5. The line numbers where Opening brace count does not match closing brace count for that line are important to be able of correcting the bad output, disabled raw tags, or simply literal braces in text.
This isn't quite clear. You want line numbers for where the brace counts don't match? I do too, but it is impossible for Rubble to calculate this with any accuracy (how can you tell which brace isn't paired? To put it simply, you can't.)
The problem there is Masterwork, they delete one of the brackets of a token but not the other and that causes undefined behavior, they have some exceptional and unique content that I would like to reuse, but if I am not able of identifying which lines have the problem and correct it ... Reading the raws it can be seen that almost never the brackets in a line are unbalanced. It can be that if the utility is developed it could detect some false positives but inversely it is clear that the rest are the lines with the problem. I have tried to detect the unbalanced brackets myself reading the code but I lose soon my concentration
6. Enhancing the rubble lexer to take into account the nested key tokens:
Big problem here: Rubble has no concept of a DF "object", much less a sub-object. The Rubble template parser totally ignores raw tags, instead dealing with each file as a chunk of unstructured text. The only thing the template parser cares about is templates. Rubble does an OK job of faking a more sophisticated approach via the shared object system, but really even that looks at the object body simply as a block of text.
The main problem is that a Rubble raw file may or may not look like raws, depending on how extensive use of templates is. Every time I try to make something "smarter" it runs straight into the fact that, until after the final parse stage, a Rubble raw file is not really readable as raws. The body of a shared object is often fairly readable, but only because most of the time it has few templates. This is not always the case, leading to "smart" editing templates being unreliable.
Well at least we have compiled a list with the offensive tokens, maybe after some reflex ion someone could have a sound idea about the possible implementation:

Most of DF tokens are values (similar to files) but a few are keys (similar to directorys ) as BODY_DETAIL_PLAN, BODY, BUILDING_WORKSHOP, CREATURE_VARIATION, CREATURE, COLOR, COLOR_PATTERN, SHAPE, ENTITY, INORGANIC, ITEM_AMMO, ITEM_ARMOR, ITEM_FOOD, ITEM_GLOVES, ITEM_HELM, ITEM_PANTS, ITEM_SHIELD, ITEM_SHOES, ITEM_SIEGEAMMO, ITEM_TOOL, ITEM_TOY, ITEM_TRAPCOMP, ITEM_WEAPON, TRANSLATION, SYMBOL, WORD, MATERIAL_TEMPLATE, PLANT, REACTION or TISSUE_TEMPLATE; some key DF tokens have been forfotten like WORLD_GEN and PROFILE only because they occur in unusual places like the data/init directory; others are nested keys inside another root object and the value tokens after them are attached to them until the next apparition of another key token nested or not, you can clearly identify them looking at the indentation of the original vanilla files; an almost exhaustive list of those nested key tokens in vanilla: BP, CV_CONVERT_TAG, BODY_DETAIL_PLAN, ATTACK, CASTE, SELECT_CASTE, WEAPON, POSITION, TISSUE_STYLE, NOUN, ADJ, VERB, PREFIX, SYNDROME, USE_MATERIAL_TEMPLATE, GROWTH, REAGENT and PRODUCT; this is a list of nested tokens that should be flagged as they effectively create nested subobjects inside their root objects; rubble has been partially addressing the symptoms of that (castes and positions independent subsystems and ignoring the rest) instead of addresing the global problem: there exist key (as in directory) nested objects in DF raws and it would be more  simple to give then an uniform and global treatement instead of multiple more hard to maintain or incomplete subsystems...
@Abadrausar: I'll take a look at your project, it looks interesting...
The main problem I foresee is that it may be too complicated to maintain easily and/or too hard to convert existing mods for those unfamiliar with Rubble. The system itself looks very interesting, but I want to play with it some before I say much.
It is more easy that it seems when you follow the good set of conventions or rules, what is hard is finding those...Dark Ages II were working like a charm before I sabotaged myself the Delta operators trying to made them almost perfect and Modest mod is also 90% done. Complete list of Mods and addons of the unreleased  RC2:
Spoiler (click to show/hide)
I am very interested in exploring the Haiku_Vanilla concept:

Minor mods that may have a positive impact over FPS or give bigger relevance to the auto generated content of the simulation: Ruins, Tombs, Lairs, Titans, megabeast, semimegabeast, Bogeyman, vampires,mummies, clowns, ghost, zombies and necros, clowns and Mayor Demons, Gods, magic weather, night creatures… Dwarf Fortress File Depot definition of minor mod is relevant here.
How?
Haiku Vanilla

This is DF with the minimal number of active objects to have all the basics of a functional vanilla without modifying any of them only pruning all the non redundant object into a separate minimal directory nested inside the base of their type.

What we have until now is:

10 Creatures: 1 wagon,3 vermin,1 colonies (bees), 3 domestics (dog, cat and donkey) 2 civilized (dwarves and goblins) and nothing more. For the variety you have the titans, FB semiFB, night creatures and rest of random.
34 Inorganics: 5 to enable adventure coins, 6 for the bronze line, 5 for the steel line, 3 for the deep specials, and 15 for the rest: stones and ores.
21 Plants: 2 grass one subterranean (cave moss) another above ground (meadow-grass), 8 crops: 6 subterranean (Cave wheat, Dimple cup, Pig tail, Plump helmet, Quarry bush and Sweet pod) and 2 above ground (Sliver barb and Whip vine), 11 trees 7 subterranean (Black-cap, Blood thorn, Fungiwood, Goblin-cap, Nether-cap, Spore tree, Tower-cap and Tunnel tube) and 4 above ground (glumprong, feather tree, highwood and willow)
That makes a great total of 65 active objects to play with, nothing more (this is not strictly true) nothing less; AND HEY!, before you begin to protest, remember that this is a minimal mod done in the interest of SCIENCE, it could be FUN to play or not (it is up to you to see and decide), but this is not certainly the unique objective of this mod not even the primary.
The number of active objects have been already dramatically reduced (good news for your FPS) but I think that it could be reduced even further, some more SCIENCE required… When you experiment trying to remove even more objects you will find some of the hard limits of the game.
Vanilla_haiku is also a testbed of the possibilities of the library VanillaLibs to serve the requested content to other mods.  ;)

Some of those hard limits already more or less identified:

It seems that you must have at least two entities defined to elude world generation rejects.
A wagon creature must be define in order to be able to start the game in fort mode.
You need at least the vanilla base language files and then the specifics language files of the creature of your civilization if those creatures are able to speak.
BODY, BODY_DETAIL_PLAN, BOOK, BUILDING, DESCRIPTOR_COLOR, DESCRIPTOR_PATTERN, DESCRIPTOR_SHAPE, INTERACTION, ITEM, MATERIAL_TEMPLATE, REACTION, SECRET and TISSUE_TEMPLATE have not been investigated for pruning.
« Last Edit: April 21, 2016, 03:04:42 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::
Pages: 1 ... 5 6 [7] 8 9 ... 14