Topic: PyLNP 0.14e-pre1 - Cross-platform launcher with graphics pack patching (Read 326775 times)

Pidgeot · « **on:** July 17, 2014, 04:33:52 pm »

PyLNP is an enhanced port of LucasUP and tolyK's Lazy Newb Pack to Python, making it possible to run the program on all platforms DF runs on.

Features added include:

Support for managing multiple Dwarf Fortress folders.
Support for all Dwarf Fortress versions, with an user interface that adapts to match the capabilities of the version being configured.
Support for color scheme installation and preview.
Graphics packs are installed by patching init.txt and d_init.txt, preserving all unrelated settings.
System for installing mods (contributed by PeridexisErrant)

For a more complete description, please see the online documentation.

The source code is available at https://github.com/Pidgeot/python-lnp under the permissive ISC license (with permission from LucasUP) - this basically allows you to do whatever you want with the code. You can install a Python interpreter and run the code directly (Linux may need a few packages from their package repository; Windows and OS X only need standard Python), or you can download pre-built binaries (for Windows, Linux and OS X 10.6+). This only includes the launcher - Dwarf Fortress, graphics packs, etc. need to be downloaded separately and placed in the LNP folder structure (for details, see the documentation, or consult an existing starter pack).

Spoiler: Screenshots (Windows version) (click to show/hide)

Spoiler: Changelog (click to show/hide)

Binary downloads are available - files follow the naming pattern PyLNP_<version>-<platform>. (The first release omits the version number.) For Linux, use linux-i686 for 32-bit Linux and linux-x64 for 64-bit Linux.

Once again: The downloads links above only contain the launcher itself. For current packs, see these topics:
Windows: PeridexisErrant's Starter Pack, maintained by PeridexisErrant
Linux: Unofficial Linux Lazy Newb Pack, maintained by Enay
OS X: Lazy Mac Pack, maintained by jecowa

All thoughts, bug reports and suggestions are welcome.

fricy · « **Reply #1 on:** July 17, 2014, 05:37:48 pm »

Do you know how, good your timing is? PTW! You can expect some feature requests soon.

fricy · « **Reply #2 on:** July 18, 2014, 04:00:25 am »

Here is the first round of feature requests, sorry if it's a bit chaotic:

Dfhack shenanigans:
-Settings are stored in 3 separate files

DF/dfhack.init - settings loaded at game launch - global features like keybinds
DF/onLoad.init - settings loaded at game load - bugfixes mainly that need to be activated at load
/saves/regionx/raw/onLoad.init: save specific settings loaded at game load - additional settings like multilevel, rendermax

-Ability to edit dfhack keybinds (with tooltips loaded from the dfhack.init?)
-misc. settings like mousequery
-Additional automation settings:
-turn on autolabor, seedwatch, autonestbox, etc.
-load workflow settings from template (only needed at embark!)

Graphics tab
-Right now the size of the Macnewbie pack is bloated a bit, because I have to add the correct raws to every tileset in the list. It would save a some space if I could tell the launcher to use the raws from another set - for eg: Grimfortress, Taffer Vherid all use ASCII raws, CLA has one raw file difference.
-Add color scheme selector (with preview)
-Ability to select between different tileset resolution/version per graphics pack:
eg Taffer provides 16 different versions of his tileset with different walls/letters/dimension, it would be nice to if the user could select in the launcher which to install. Or to have the ability to chosse between Phoebus 16x16 and Phoebus 24x24 - this is more difficult, because the sprites will be different too, not just the tileset.
-Extra settings for new renderers:
TWBT and Rendermax need Standard Printmode, Stonesense needs 2D, so they are mutually exclusive, launcher needs to know it and provide a help/tooltip. Preview would be nice too.

TWBT: enable multilevel, perhaps allow editing of fog
TWBT: select secondary/tertiary tileset - should recommend from a list of text/map only tilesets, preferably same size as the main tileset
Rendermax enable
Stonesense enable (later overlay too, now it's not yet functional)

Powergoal: autodetect screen resolution and recommend a tileset dimension
Powergoal: preview tileset (screenshot)
Powergoal(bloat): generate preview based on selected tileset and colorscheme
Powergoal(bloat): add BRZ 2x or HQX 2x based converter to uprez tilesets (very low priority)

Autoupdate Pack - from github? dropbox? - whichever is easier to implement

Integrate Quickfort converter python script into a tab in the launcher: Powergoal
In the MacNewbie Pack I assembled a standalone drag-and-drop app with Platypus that writes .mak files into data/init/macros. It would make sense to add this functionality to the launcher instead. There are some simple edits needed for the qfconverter script: generate -mak filename and -title from the input .csv/.xls files and write it to the appropriate directory and some feedback messages when successful/unsuccessful. (The resulting macros then can be loaded in game with ctrl-L)
Powergoal(bloat): generate macro layout preview in the launcher like in the Quickfort app on windows.

Mod support: Powergoal(bloat):
-Complicates things quite a bit, because this will modify the raws, so if the user decides to change graphics pack hell will break loose. So it needs some brain storming, my initial idea is to save somewhere what edits were made to the raws in the save folder, and reapply the setting when changing graphics, maybe use .diff files to handle the edits. I'm open to ideas, and it's low priority anyway. I'm just putting here what could be integrated:

Accelerated/Modest/Essential DF mod support
Rubble support
Masterwork support
Warmist's mod manager (injects raws from dfhack)

mifki · « **Reply #3 on:** July 18, 2014, 04:16:37 am »

Somewhat offtopic here, but a thought came to my mind to implement an in-game control center (in lua,
probably). Most of the settings, including tileset and colomaps, could be applied without restarting the game, and also of course there should be an online repository with graphics and colormaps, so that they're not carried within newbpacks each time.

CLA · « **Reply #4 on:** July 18, 2014, 07:22:35 am »

Quote from: mifki on July 18, 2014, 04:16:37 am

tileset and colomaps, could be applied without restarting the game

Are you fucking shitting me? You mean I could finally edit tilesets and colors without having to restart df every goddamn time I want to see how something looks in game?

Warmist · « **Reply #5 on:** July 18, 2014, 07:37:04 am »

Quote from: mifki on July 18, 2014, 04:16:37 am

Somewhat offtopic here, but a thought came to my mind to implement an in-game control center (in lua,
probably). Most of the settings, including tileset and colomaps, could be applied without restarting the game, and also of course there should be an online repository with graphics and colormaps, so that they're not carried within newbpacks each time.

Hehehe... one of my goals was integration of most tool like LNP in df. Including querying online repositories (e.g. maybe dffd has an api). Mostly because i don't like having switch my attention between different programs/restart df/etc...

mifki · « **Reply #6 on:** July 18, 2014, 07:39:52 am »

Quote from: CLA on July 18, 2014, 07:22:35 am

Quote from: mifki on July 18, 2014, 04:16:37 am
tileset and colomaps, could be applied without restarting the game

Are you fucking shitting me? You mean I could finally edit tilesets and colors without having to restart df every goddamn time I want to see how something looks in game?

No problems at all. I was thinking more about an in-game configuration UI for players, but for modders I can just add commands to reload tilesets/colours.

mifki · « **Reply #7 on:** July 18, 2014, 07:44:00 am »

Quote from: Warmist on July 18, 2014, 07:37:04 am

Quote from: mifki on July 18, 2014, 04:16:37 am
Somewhat offtopic here, but a thought came to my mind to implement an in-game control center (in lua,
probably). Most of the settings, including tileset and colomaps, could be applied without restarting the game, and also of course there should be an online repository with graphics and colormaps, so that they're not carried within newbpacks each time.
Hehehe... one of my goals was integration of most tool like LNP in df. Including querying online repositories (e.g. maybe dffd has an api). Mostly because i don't like having switch my attention between different programs/restart df/etc...

Seems now I know what to do after I finish with twbt, if ever)

Zaxth · « **Reply #8 on:** July 18, 2014, 09:13:06 am »

I had to register to thank you for this. So thanks a lot this really helps

CLA · « **Reply #9 on:** July 18, 2014, 09:44:04 am »

Quote from: mifki on July 18, 2014, 07:39:52 am

Quote from: CLA on July 18, 2014, 07:22:35 am
Quote from: mifki on July 18, 2014, 04:16:37 am
tileset and colomaps, could be applied without restarting the game

Are you fucking shitting me? You mean I could finally edit tilesets and colors without having to restart df every goddamn time I want to see how something looks in game?

No problems at all. I was thinking more about an in-game configuration UI for players, but for modders I can just add commands to reload tilesets/colours.

That would be much appreciated.

Pidgeot · « **Reply #10 on:** July 18, 2014, 10:21:50 am »

Thanks for your suggestions, fricy!

Quote from: fricy on July 18, 2014, 04:00:25 am

Here is the first round of feature requests, sorry if it's a bit chaotic:

Dfhack shenanigans:
<snip>

I never use DFHack myself, so I admit it hasn't been a priority.

I'd like to get the current code a bit more polished and get all of the basic stuff in there first, then I can look at extensions and stuff.

Quote from: fricy on July 18, 2014, 04:00:25 am

Graphics tab
-Right now the size of the Macnewbie pack is bloated a bit, because I have to add the correct raws to every tileset in the list. It would save a some space if I could tell the launcher to use the raws from another set - for eg: Grimfortress, Taffer Vherid all use ASCII raws, CLA has one raw file difference.

Ideally, graphics packs would be provided in the form of a patch, rather than a full set of raws - and then we would "unpatch" before changing graphics. This would also make it useful for installing mods and stuff, and it would be possible to use graphics packs even if they aren't updated to the latest version.

That's a long-term goal though (as a minimum, my file parser needs to be rewritten to properly support hierarchical data), and it would be nice to have something simpler before that - I'll try to come up with some kind of hook for such things.

Quote from: fricy on July 18, 2014, 04:00:25 am

-Add color scheme selector (with preview)

Color schemes is definitely one of the things I want to add soon (after a bit more cleanup, though).

Quote from: fricy on July 18, 2014, 04:00:25 am

-Ability to select between different tileset resolution/version per graphics pack:
eg Taffer provides 16 different versions of his tileset with different walls/letters/dimension, it would be nice to if the user could select in the launcher which to install. Or to have the ability to chosse between Phoebus 16x16 and Phoebus 24x24 - this is more difficult, because the sprites will be different too, not just the tileset.

This would likely be integrated with the file reuse stuff.

Quote from: fricy on July 18, 2014, 04:00:25 am

-Extra settings for new renderers:
TWBT and Rendermax need Standard Printmode, Stonesense needs 2D, so they are mutually exclusive, launcher needs to know it and provide a help/tooltip. Preview would be nice too.
TWBT: enable multilevel, perhaps allow editing of fog
TWBT: select secondary/tertiary tileset - should recommend from a list of text/map only tilesets, preferably same size as the main tileset
Rendermax enable
Stonesense enable (later overlay too, now it's not yet functional)

I'll have to think about ways this might be doable (preferably without too much hardcoding).

Quote from: fricy on July 18, 2014, 04:00:25 am

Powergoal: autodetect screen resolution and recommend a tileset dimension
Powergoal: preview tileset (screenshot)

Should be doable. I'm not entirely sure what the criteria for recommending a tileset should be, but I could at least add some calculators or something.

Quote from: fricy on July 18, 2014, 04:00:25 am

Powergoal(bloat): generate preview based on selected tileset and colorscheme

Honestly, that would probably be my preferred way of doing previews.

Quote from: fricy on July 18, 2014, 04:00:25 am

Powergoal(bloat): add BRZ 2x or HQX 2x based converter to uprez tilesets (very low priority)

Doable, but definitely low priority.

Quote from: fricy on July 18, 2014, 04:00:25 am

Autoupdate Pack - from github? dropbox? - whichever is easier to implement

I don't intend to distribute a complete pack myself, but I could probably add some kind of hook for the individual pack creators? I'll think about a nice way to do it.

Quote from: fricy on July 18, 2014, 04:00:25 am

Integrate Quickfort converter python script into a tab in the launcher: Powergoal
In the MacNewbie Pack I assembled a standalone drag-and-drop app with Platypus that writes .mak files into data/init/macros. It would make sense to add this functionality to the launcher instead. There are some simple edits needed for the qfconverter script: generate -mak filename and -title from the input .csv/.xls files and write it to the appropriate directory and some feedback messages when successful/unsuccessful. (The resulting macros then can be loaded in game with ctrl-L)
Powergoal(bloat): generate macro layout preview in the launcher like in the Quickfort app on windows.

Don't use Quickfort either, but if nothing else, some kind of frontend is probably possible.

Quote from: fricy on July 18, 2014, 04:00:25 am

Mod support: Powergoal(bloat):
-Complicates things quite a bit, because this will modify the raws, so if the user decides to change graphics pack hell will break loose. So it needs some brain storming, my initial idea is to save somewhere what edits were made to the raws in the save folder, and reapply the setting when changing graphics, maybe use .diff files to handle the edits. I'm open to ideas, and it's low priority anyway. I'm just putting here what could be integrated:
Accelerated/Modest/Essential DF mod support
Rubble support
Masterwork support
Warmist's mod manager (injects raws from dfhack)

Definitely more of a long-term goal, but I would like to eventually have some support for these things.

fricy · « **Reply #11 on:** July 19, 2014, 02:48:37 am »

Quote from: Pidgeot on July 18, 2014, 10:21:50 am

Ideally, graphics packs would be provided in the form of a patch, rather than a full set of raws - and then we would "unpatch" before changing graphics. This would also make it useful for installing mods and stuff, and it would be possible to use graphics packs even if they aren't updated to the latest version.
That's a long-term goal though (as a minimum, my file parser needs to be rewritten to properly support hierarchical data), and it would be nice to have something simpler before that - I'll try to come up with some kind of hook for such things.

Sounds good, perhaps Rubble could be used for this instead of writing a new parser, but I might be mistaken. Take a look, and ignore if I'm talking nonsense.

Quote from: fricy on July 18, 2014, 04:00:25 am

-Extra settings for new renderers:
TWBT and Rendermax need Standard Printmode, Stonesense needs 2D, so they are mutually exclusive, launcher needs to know it and provide a help/tooltip. Preview would be nice too.

Quote

I'll have to think about ways this might be doable (preferably without too much hardcoding).

Preferably no hardcoding, in fact it belongs to the dfhack suggestions, just graphics related and mentioned it separately because I consider it higher priority than the keybinds and whatever. For starters provide a way to add strings to /saves/regionX/raw/onLoad.init, and the rest can wait.

Quote from: fricy on July 18, 2014, 04:00:25 am

Powergoal: autodetect screen resolution and recommend a tileset dimension
Powergoal: preview tileset (screenshot)

Quote

Should be doable. I'm not entirely sure what the criteria for recommending a tileset should be, but I could at least add some calculators or something.

Dead simple with "legacy" graphics: horizontal resolution/80 = ideal tileset. So if user has 1920x1080: recommend 24x24 tileset, or what's closest. TWBT will change it, but we need mifiki's input for that, and I need to get some information how retina mac displays work, because I have no idea.

Quote from: fricy on July 18, 2014, 04:00:25 am

Powergoal(bloat): generate preview based on selected tileset and colorscheme

Quote

Honestly, that would probably be my preferred way of doing previews.

Cool!

Quote from: fricy on July 18, 2014, 04:00:25 am

Autoupdate Pack - from github? dropbox? - whichever is easier to implement

Quote

I don't intend to distribute a complete pack myself, but I could probably add some kind of hook for the individual pack creators? I'll think about a nice way to do it.

I thought about it, and github (or another version control system) would be preferable. In fact the more I think about it the more I like the idea of not making users download 100 mega for a tileset fix or a simple script update. So let's say this is another priority feature.

Quote

Don't use Quickfort either, but if nothing else, some kind of frontend is probably possible.

Nothing fancy is needed, just a file-open dialog (+ drag-and-drop target), and a a minimal editing of the qfconverter to generate readable filenames, because right now it outputs gibersih. Slight correction: the filename needs to be: spreadsheet name + unique id because DF won't reload the same macro if the content changed but the name did not... Plus adding a way to delete macros would be useful too.
Target is linux/osx, windows already has quickfort.

Another problem I've been thinking about is handling the changes of graphics packs and (some dfhack features) on a per-save basis. What I mean is not to batch process the whole save directory, but let the user choose which save to update. It's not a problem yet, but will be if mod support comes into the picture.

splinterz · « **Reply #12 on:** July 19, 2014, 04:29:40 am »

really nice work here.

you can use another method to load the dfhack options as described here. it's a lot easier than worrying about updating an onload.init for every save.

for the tileset update to the d_init, you can split the file and replace the graphic section when you change tilesets/colors, as opposed to keeping track of every track/tree tag.

PeridexisErrant · « **Reply #13 on:** July 22, 2014, 01:30:30 am »

This looks great! I'd love to see this become the standard, both because having a properly cross-platform launcher is awesome and because I can actually do some coding in python. PTW with interest.

Loading graphics packs from a diff is an excellent idea. Besides saving space, it's a solid step toward a mod loader using the same mechanism. It would probably only support small to moderate raw-only mods, but this would include everything from Accelerated DF to Modest Mod and a host of minor tweaks. It should even be able to check compatibility, and only load non-conflicting mods.

For updates: I haven't done much with the idea of updating by a diff, as there's a lot that could go wrong with deleting something important if the parsing goes wrong. However the latest windows launcher does include an update notification tool; it just checks the DFFD version page (~70B) and if the version string is different notifies the user. You could potentially extend something similar to auto-download the new pack from a configurable address and then push all data to the new pack. To reduce the odds of losing user data, I offer the idea of a top-level folder "User Generated Content" - designed to copy to a new pack.

Regarding DFHack stuff: I would leave *any* development on this from until we have a stable version of 40_xx and DFHack for that version has matured, as it's bound to change significantly. This goes double for graphics stuff; recent TwbT screenshots include an isometric view, I've got stonesense for 34.11 running with printmode:standard, and so on.

Quickfort: I agree that improvements would be nice, but I think that should be a separate project (even if it's done by the same people and reuses code).

palu · « **Reply #14 on:** July 22, 2014, 01:26:13 pm »

For mod loading, you might want to check out this.

News:

Author Topic: PyLNP 0.14e-pre1 - Cross-platform launcher with graphics pack patching (Read 326775 times)

Pidgeot

PyLNP 0.14e-pre1 - Cross-platform launcher with graphics pack patching

fricy

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

fricy

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

mifki

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

CLA

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

Warmist

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

mifki

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

mifki

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

Zaxth

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

CLA

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

Pidgeot

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

fricy

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

splinterz

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

PeridexisErrant

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching

palu

Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching