Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Pidgeot]

After a lot of searching, it appears there are precisely 2 options to spawn a terminal with reasonable reliability:

What about a fallback to user specified function in pyLNP.json or user.json?

xdg-terminal *seems* reliable enough on its own, but if people have problems, PyLNP.user may need a field for this... we'll cross that bridge when we get to it.

What archive format support is required? I have zip and tar.(bz2|gz) as these are build into python

Personally I'd like to have 7z and/or xz in there eventually, but what you have now should be good enough for a first version. I see Python 3.3 has built-in support for LZMA, but that's no good for Python 2, so a stand-alone implementation would be needed for that.

If a graphics package is updated, is there a need to remove and reinstall the package itself?

I would leave this part out for now. If the user has mods installed, then we'd just end up throwing them out - once a patching system/mod installer is in place, we can consider automatic rebuilding.

[fricy]

• Suggestions how to handle the different flavors(?) of graphics packages? i.e. Phoebus comes with a TT and a noTT version (in the data/init folder). As TT is handled by the launcher itself, we can safely use anyone of these versions and ignore the other one, right? As I'm fairly new to DF: are there graphics packages that have different flavors beyond the TT and noTT? or can I just move the data/init/*tt files to data/init ?
• Mac (noob) question: if DwarfTherapist.dmg is updated, it is enough to copy the dmg file to the respective DT folder under utilities? I guess I have to extract the files, right?

1: I don't understand completely what you mean by TT and noTT, but the preferred way'd be to expose all tilesets in the art folder to the user to choose from, and the first one in the list should be what's in the init.txt. Basically this'd make the graphics list much simpler, by bundling together all ASCII graphics (without additional sprites, like Shizzle, Vherid, etc.) and just choose which tileset to use with which color scheme. Some extra config (file?) is needed to define the preferred default colors for the tilesets. Perhaps as simple as calling the color scheme the same as the tileset?
   And there'll be lot's of flavors with the TWBT plugin, separate tileset for text, for graphics, sprite overrides, and perhaps even a map tileset.
2: DMG is a mac specific archive file. You need to extract the .app from it.
This is coming along nicely.

[MagiX]

I'd recommend using this: http://bay12games.com/dwarves/dev_release.rss

Oh, how could I miss that Thank you.

Personally I'd like to have 7z and/or xz in there eventually, but what you have now should be good enough for a first version. I see Python 3.3 has built-in support for LZMA, but that's no good for Python 2, so a stand-alone implementation would be needed for that.

I agree with you at that point. I might implement LZMA for python3.3+ and leave it out for python 2, as I want to keep the number of dependencies as low as possible

I would leave this part out for now. If the user has mods installed, then we'd just end up throwing them out - once a patching system/mod installer is in place, we can consider automatic rebuilding.

Was just thinking ahead Maybe we should implement a "which mods are active" section in the pyLNP.user file or sth like that to keep track of what is running... at a later point in time.

1: I don't understand completely what you mean by TT and noTT

Basically, some (read: at least one) graphics packages (i.e. Phoebus, which happened to update while I was writing on my code) come with 2 different options. One option is with TRUETYPE:ON and one with TRUETYPE:OFF. If I want to update these packages automatically, I will end up with 2 different folders, namely data/init/phoebus/ and data/init/phoebus_nott/ which are identical beside the TRUETYPE flag. As I don't have that much experience with graphics packages (I play vanilla), I was asking if there are several packages which come with different data/init/* folders just to make it hasslefree to handle the TRUETYPE flag...?

2: DMG is a mac specific archive file. You need to extract the .app from it.

The same as above, python doesn't support DMG "out-of-the-box" so I guess I have to skip this one (for now), right?

[fricy]

Basically, some (read: at least one) graphics packages (i.e. Phoebus, which happened to update while I was writing on my code) come with 2 different options. One option is with TRUETYPE:ON and one with TRUETYPE:OFF. If I want to update these packages automatically, I will end up with 2 different folders, namely data/init/phoebus/ and data/init/phoebus_nott/ which are identical beside the TRUETYPE flag. As I don't have that much experience with graphics packages (I play vanilla), I was asking if there are several packages which come with different data/init/* folders just to make it hasslefree to handle the TRUETYPE flag...?

2: DMG is a mac specific archive file. You need to extract the .app from it.

The same as above, python doesn't support DMG "out-of-the-box" so I guess I have to skip this one (for now), right?

1. I see. No, I think only Phoebus does THAT. However: check Taffer if you want to see options. We should go in the direction that reduces the Graphics pack list, and provide these options instead as checkboxes/lists/whatever is better.
Back to the point: TTF should be a user option, and not a pack option.

2. http://stackoverflow.com/questions/6357914/how-do-i-install-a-dmg-file-from-the-command-line
But I think we can skip this, the preferred way for updating a pack'd be to pull from a git? repo - if that is even possible. (Is it??) Checking for updates to individual utilities is not necessary imho.

[MagiX]

1. I see. No, I think only Phoebus does THAT. However: check Taffer if you want to see options. We should go in the direction that reduces the Graphics pack list, and provide these options instead as checkboxes/lists/whatever is better.
Back to the point: TTF should be a user option, and not a pack option.

I see, so I just stumbled opon the only package that does that and thus there is no need for me to fix this in a general way, I guess
Can you elaborate a bit on the Graphics pack list modification with checkboxes etc?

2. http://stackoverflow.com/questions/6357914/how-do-i-install-a-dmg-file-from-the-command-line
But I think we can skip this, the preferred way for updating a pack'd be to pull from a git? repo - if that is even possible. (Is it??) Checking for updates to individual utilities is not necessary imho.

Found that link as well. Maybe later... I also have no way to test it, so someone using OSX would be helpful for this part at least.
I figured most addons, be it graphics, tilesets, utilities, whatever, are using dffd and thus focused on dffd. dffd also makes it really easy to check for new versions, as this is possible.
I looked at using a git repo for stuff like that. The only projects I know that uses git to update are good ol' sickbeard and related. I briefly looked at the code and dffd looked way easier

PS: I'm still learning a lot of stuff and syntax right now. All my programming knowledge is selftaught/googled and so far, most of my programming was in bash or modifying existing code to make it fit, or in completely other languages so it might take me a bit longer than other people to do it, but I'm working on it anyways

[Pidgeot]

But I think we can skip this, the preferred way for updating a pack'd be to pull from a git? repo - if that is even possible. (Is it??)

Github provides a nice "Download ZIP" link to essentially grab the complete repository, but it doesn't include contents of submodules. You'd need special parsing of that to download them individually (but then you might as well just manage them individually...), or a full-blown git client - both are certainly possible, but maybe going a bit too far...

Also, I'm not sure if Github *wants* to have X thousand people doing this on a regular basis... it's not a webhost, so using it as one might be problematic in the long run. That's part of the reason why I  think it's important to not overspecialize this - it's okay to do something special for a particular host, but there needs to be a generic solution in case that host is no longer available.

[Pidgeot]

After a lot of searching, it appears there are precisely 2 options to spawn a terminal with reasonable reliability:

• Bundle the xdg-terminal script and just run that on Linux, or
• Reimplement the xdg-terminal script in Python to avoid having another file in there

Remember how I said this? Yeah, turns out xdg-terminal doesn't work on MATE.

I could modify the script to add support for MATE - but there are a lot of desktop environments that aren't explicitly listed in the script, so any one of those could break it.

Maybe LXDE, GNOME, KDE, xfce, MATE and Cinnamon support is enough? Maybe I just need parity with xdg-open, which I'm already using (though not for anything as mission-critical)? I don't use Linux enough to have any idea about how widespread the others are, but without a way to spawn a new terminal, I can't launch dfhack on Linux at all - it crashes if there's no terminal, and freaks out if the terminal doesn't remain available throughout (meaning the launcher can't be closed).

So, four possibilities:
1) Require a specific terminal (xterm/rvxt/whatever) is installed
2) Take MagiX' idea of having the user set the terminal in the configuration file (I'm not sure how to make sure people will set it properly... maybe require one run to go through a terminal and have the user select the executable among the parent processes?)
3) Modify xdg-terminal and fix it everytime someone has a non-working desktop environment (THAT'LL be fun...)
4) Force the user to launch DF themselves if they want to use DFHack on Linux

I need to figure out the least terrible way of handling this, and I'm not sure which one that is...

[MagiX]

Github provides a nice "Download ZIP" link to essentially grab the complete repository, but it doesn't include contents of submodules. You'd need special parsing of that to download them individually (but then you might as well just manage them individually...), or a full-blown git client - both are certainly possible, but maybe going a bit too far...

As far as I can tell, the version check via github is a bit more involved...

Maybe LXDE, GNOME, KDE, xfce, MATE and Cinnamon support is enough? Maybe I just need parity with

What about number 5: pyLNP can't be closed until df is closed?
Disadvantage: df will need at least 3 windows (DF,DFhack,pyLNP)

Please check out my fork of pyLNP, after you made a backup!
I added a version check and a download feature.
Right now, it is still quite rough, i.e. works on python3 only, it updates everytime you start pyLNP (which might be a bit too much, but I can still change it later) and only uses commandline input/output. Please test it a bit (you can change the version number in the created packages.json in your LNP folder to test the download/extract) and let me know about feedback.

on my todo are:

• only update every x-hrs/days/whatever (specified by pyLNP.user ?)
• make it a more fault tolerant, i.e. no inet connection, extraction failed, no write access,...
• try to get the extract feature done for OSX... Need someone to test it afterwards
• implement LZMA support for python 3.3+
• Get it into the user interface
• Contact dffd admin and ask if it is ok that I have "fun" with their server and circumvent their limitations (python is not allowed)
• add python 2 support

Other suggestions?

[PeridexisErrant]

So, four possibilities:
1) Require a specific terminal (xterm/rvxt/whatever) is installed
2) Take MagiX' idea of having the user set the terminal in the configuration file (I'm not sure how to make sure people will set it properly... maybe require one run to go through a terminal and have the user select the executable among the parent processes?)
3) Modify xdg-terminal and fix it everytime someone has a non-working desktop environment (THAT'LL be fun...)
4) Force the user to launch DF themselves if they want to use DFHack on Linux

I need to figure out the least terrible way of handling this, and I'm not sure which one that is...

I tend to think that handling most cases is good enough, so my logic would be:
 - if someone wants to use DFHack on Linux, use xdg if that'll work.
 - otherwise make them configure it in the config, and don't spend time trying to make the launcher smart enough to help much.
 - if they can't do that, no dfhack through the launcher. 

I think this would cover most people under the first point, and those not covered are probably able to finish setting that up themselves.  And the worst cast is they have to start DF manually to play with DFHack - hardly a disaster. 

[lethosor]

Something that may work is using TkGui.withdraw() instead of sys.exit() when autoClose is enabled - this should hide the PyLNP window without exiting completely. I'm not sure how well this will work on all platforms, though - the launcher may remain open but invisible after exiting DF.

EDIT: exec* is not an option because utilities need to be launched after DF - so it's necessary to find a way to launch DF as an entirely stand-alone process, with its own terminal window. For Windows, I can use the start command; on OS X, I can use open. I haven't been able to figure out a useful possibility on Linux; I don't see a way to make Python do it directly, and I haven't been able to detect which terminal program the user is using to launch a new one. If anyone knows something I don't, I'm all ears.

Is there a reason for this? I think some utilities (DT?) check for DF on startup, but I don't know of any that exit immediately if DF isn't running (and either way, there's no guarantee which will finish launching first). Admittedly, exec() won't work with DFHack if PyLNP is launched without a terminal window, so it's not a complete solution.

Fake edit:

I tend to think that handling most cases is good enough, so my logic would be:
 - if someone wants to use DFHack on Linux, use xdg if that'll work.
 - otherwise make them configure it in the config, and don't spend time trying to make the launcher smart enough to help much.
 - if they can't do that, no dfhack through the launcher.

I think this would cover most people under the first point, and those not covered are probably able to finish setting that up themselves.  And the worst cast is they have to start DF manually to play with DFHack - hardly a disaster. 

It's worth mentioning that the current default is to not close the launcher when launching DF, which doesn't cause any problems with DFHack (until the launcher is closed, that is).

[shaver]

The same as above, python doesn't support DMG "out-of-the-box" so I guess I have to skip this one (for now), right?

I think you could run diskutil to mount it and get it out.

[PeridexisErrant]

If a graphics package is updated, is there a need to remove and reinstall the package itself?

I would leave this part out for now. If the user has mods installed, then we'd just end up throwing them out - once a patching system/mod installer is in place, we can consider automatic rebuilding.

Was just thinking ahead Maybe we should implement a "which mods are active" section in the pyLNP.user file or sth like that to keep track of what is running... at a later point in time.

A list of installed mods should be stored in a file the /raw/ folder in question, since it may be different for various saves.  This would also allow portability across tools, which is generally a good thing. 

And if we're looking at mods, there's a couple of us working on that at https://github.com/PeridexisErrant/Py-Mod-Loader


More generally on the idea of automatic updates to part of the pack, I would encourage extreme caution.  Many parts will have complicated dependencies, which may change without (machine-readable) notice between versions, and are very likely to be different between specific items.  For example, if you update graphics packs out of sync with the DF version they're based on, bad things happen.  Similar issues with things that update for an old version.  Forcing a canonical repo to pull updates from manages format, but otherwise you have to write an unpacking script for each thing and hope the author never changes how they pack it, or random things don't unpack how you expect them to. 

TL;DR - I would love to automate the work of maintaining an up-to-date pack, but I don't think it's possible. 

[Dwimenor]

Did you check $TERM environmental variable?

subprocess.Popen() in python - doesn't that launch external process? This should open terminal and keep it opened even if you close pyLNP

python 2 and 3 - is there specific reason to keep pyLNP compatible with py2?

[Dricus]

PyInstaller is used to create standalone executables and is not yet compatible with Python 3. I guess that's one of the main reasons.

[Pidgeot]

Please check out my fork of pyLNP, after you made a backup!
I added a version check and a download feature.
Right now, it is still quite rough, i.e. works on python3 only, it updates everytime you start pyLNP (which might be a bit too much, but I can still change it later) and only uses commandline input/output. Please test it a bit (you can change the version number in the created packages.json in your LNP folder to test the download/extract) and let me know about feedback.

I've only looked the code for now, but I still have comments:

Like Peridexis said, be careful about 100% automatic updates from "original" sources, because you might lose compatibility with the currently installed DF version. If you want to do component-wise updating, there needs to be a step where the user selects which parts to actually update - and you would probably want to leave old versions in case the new one doesn't work right.

You need to keep everything as generic as possible - that includes version checking! Take a look at the code that's already there for an idea about how to deal with that. (You'll need a regex per version checking URL.)

on my todo are:

• only update every x-hrs/days/whatever (specified by pyLNP.user ?)
• make it a more fault tolerant, i.e. no inet connection, extraction failed, no write access,...
• try to get the extract feature done for OSX... Need someone to test it afterwards
• implement LZMA support for python 3.3+
• Get it into the user interface
• Contact dffd admin and ask if it is ok that I have "fun" with their server and circumvent their limitations (python is not allowed)
• add python 2 support

Other suggestions?

I would talk to the DFFD admin sooner, not later. If he says no, then you won't be able to use the code as-is anyway.

I tend to think that handling most cases is good enough, so my logic would be:
 - if someone wants to use DFHack on Linux, use xdg if that'll work.
 - otherwise make them configure it in the config, and don't spend time trying to make the launcher smart enough to help much.
 - if they can't do that, no dfhack through the launcher. 

I think this would cover most people under the first point, and those not covered are probably able to finish setting that up themselves.  And the worst cast is they have to start DF manually to play with DFHack - hardly a disaster. 

To the best of my knowledge, Mint is the most popular distribution these days, and one of its core desktop environments is MATE - where xdg-terminal does not work out of the box, so already there, I would argue you're not covering "most people".

EDIT: exec* is not an option because utilities need to be launched after DF - so it's necessary to find a way to launch DF as an entirely stand-alone process, with its own terminal window. For Windows, I can use the start command; on OS X, I can use open. I haven't been able to figure out a useful possibility on Linux; I don't see a way to make Python do it directly, and I haven't been able to detect which terminal program the user is using to launch a new one. If anyone knows something I don't, I'm all ears.

Is there a reason for this? I think some utilities (DT?) check for DF on startup, but I don't know of any that exit immediately if DF isn't running (and either way, there's no guarantee which will finish launching first). Admittedly, exec() won't work with DFHack if PyLNP is launched without a terminal window, so it's not a complete solution.

I could probably get away with doing it the other way around, but as you mentioned, that only helps for the case where there is a terminal window to begin with.

Also, if it *becomes* necessary to delay launching utilities, I'd need to go back to starting DF first anyway, putting us back at square one.

It's worth mentioning that the current default is to not close the launcher when launching DF, which doesn't cause any problems with DFHack (until the launcher is closed, that is).

The fundamental issue is that DFHack needs access to a terminal while running, and there's (apparently) no way to guarantee that. While the specific problem of spamming the terminal only happens if you close the launcher while DFHack is running, it's only a symptom of the deeper problem. When there's no terminal (e.g. when using a pre-built executable), DFHack outright refuses to launch then.

Did you check $TERM environmental variable?

$TERM specifies terminal capabilities, it does not specify the terminal executable itself. For example, on my primary Linux VM (Mint 15/MATE), it gets set to xterm (which is not installed, and does not have an alias in path).

subprocess.Popen() in python - doesn't that launch external process? This should open terminal and keep it opened even if you close pyLNP

It spawns a process that will keep running when PyLNP is closed, yes, but it does not spawn a new terminal window, and DFHack breaks if it doesn't have a terminal window (or loses exclusive access to one).

python 2 and 3 - is there specific reason to keep pyLNP compatible with py2?

Like Dricus said, the ability to build executables is one of them, but it also increases the likelihood that the user can use any existing Python installation.