Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[fricy]

I see, so I just stumbled opon the only package that does that and thus there is no need for me to fix this in a general way, I guess
Can you elaborate a bit on the Graphics pack list modification with checkboxes etc?

The checkbox was just a method, it may not be the best...
For the tileset selection a drop-down list would be ideal I think, so in the case of Phoebus show all tilesets in the art folder, default to the tileset in the init.txt.
Then there's the TWBT plugin:
First: it's mutually exclusive with TTF.
Then: it needs extra modifications to the init.txt: set PRINT_MODE to STANDARD, set GRAPHICS to YES. And most importantly: it uses 2 tilesets in the init.txt: FONT and FULLFONT is set to the text-only tileset, while GRAPHICS_FONT and GRPAHICS_FULLFONT is set to the graphics tileset.
For TWBT I think we should load these text only tilesets from a separate directory.
(Then there'll also be an override.txt and an override tileset in the art folder for sprite overrides, and extra tilesets for map view may come along, we are not there yet.)

I figured most addons, be it graphics, tilesets, utilities, whatever, are using dffd and thus focused on dffd. dffd also makes it really easy to check for new versions, as this is possible.
I looked at using a git repo for stuff like that. The only projects I know that uses git to update are good ol' sickbeard and related. I briefly looked at the code and dffd looked way easier

If it's any help we could put a version.txt in the root of the github repo, that tells you the latest commit hash. That way you could easily determine if you have the latest or not.

Github provides a nice "Download ZIP" link to essentially grab the complete repository, but it doesn't include contents of submodules. You'd need special parsing of that to download them individually (but then you might as well just manage them individually...), or a full-blown git client - both are certainly possible, but maybe going a bit too far...

Also, I'm not sure if Github *wants* to have X thousand people doing this on a regular basis... it's not a webhost, so using it as one might be problematic in the long run. That's part of the reason why I  think it's important to not overspecialize this - it's okay to do something special for a particular host, but there needs to be a generic solution in case that host is no longer available.

No, a zip is not going to work for this, partly because the submodule problem, partly because grabbing the .zip won't have any benefit over grabbing them from dffd, and partly because we'd be banned from github eventually as you mentioned yourself.
However they don't forbid using the service for this, they just advise against distributing large files. Going with a full git client that only grabs the changes would be within their terms, and the bandwidth would be much lower than now - like 20-30 mega for a full pack (like 40.08 - » 40.09) update vs the 100 mega from dffd, and a couple of kb-s for most of the bugfix updates.
Take Dwarf Therapist for example: It's about 29 mb uncompressed, but 24 mb of that is QT and other Frameworks which won't change most of the time between versions. So an update is like 5mb. If I only push a new therapist.ini for a new version it's only 7kb this way. Maybe even lower, I don't know if they do any compression with git transfers.
I'm too lazy to type it agin, so here's a quote:

Spoiler: Mod manager flowchart (click to show/hide)

Here's the system I envisioned: Everything in this flowchart (except the bottom level) is organized via github submodule/subtree linking, so updates should propagate to the user with minimum maintainer interaction.
This model would simplify package maintenance, keep the mods/graphic pack in sync across platforms, keep the end user always up-to-date without needing to download the same old 100mb .zip from dffd, and then run upgrade scripts or copy save games. As a side effect it'd also lower the bandwidth usage of ddfd, which may result in lower cost for Toady as well.

The central mod/graphics repo should be modareted/trimmed/pruned so the new players are not overwhelmed by options, but there should be a way for the advanced users or the pack maintainers to add mods to their repo/local install if it's missing from the central repo. (see minimod 5 and 6 on the chart)
DFFD would still be needed, the github license agreement advises against using the service for general distribution. An update services like in the flowchart would be within service terms however.

And I agree with your thinking that we shouldn't depend on a single service, but going with a git client would mean we could choose from more than one git services, and I don't want to migrate away from dffd completely, the full pack would still sit there, only the update service would use git.

[PeridexisErrant]

I think going down the path of partial updates is a grave mistake.  Full pack updates are great, and I like the DF devlog integration in the Manilla launcher, but please don't break packs by asking users to update parts - or worse doing it automatically. 

Getting the files, unpacking them, dealing with dependencies, all are significant technical challenges.  It's a lot of code and a huge commitment to maintenance down the track (which I can't make).  It's also hostile to new users; there is simply no way that this is going to work consistently without the user knowing what's happening - and for the Starter Pack at least it's a core goal to just work without requiring anything of the user.  Every time a new player has to do anything besides learn to play DF, many just give up. 

Having the launcher do anything related to partial updates is an appallingly bad idea.  I won't ever use it.  I will recommend that other people don't use it.  It just doesn't fit the goal of a Newb Pack at all.

I'm not trying to be grumpy, just save us from our own enthusiasm.  Listen to the voice of experience before it's too late.  Please. 

[Pidgeot]

And I agree with your thinking that we shouldn't depend on a single service

That was the main point I was trying to make - I'm all for having this kind of functionality if it makes sense. I just think that it should not be *required*, that is, even if git is used as the primary update method, we should still have support for plain HTTP downloads, to handle add-ons/utilities that aren't in a git repository (for whatever reason).

I think going down the path of partial updates is a grave mistake.  Full pack updates are great, and I like the DF devlog integration in the Manilla launcher, but please don't break packs by asking users to update parts - or worse doing it automatically. 

Getting the files, unpacking them, dealing with dependencies, all are significant technical challenges.  It's a lot of code and a huge commitment to maintenance down the track (which I can't make).  It's also hostile to new users; there is simply no way that this is going to work consistently without the user knowing what's happening - and for the Starter Pack at least it's a core goal to just work without requiring anything of the user.  Every time a new player has to do anything besides learn to play DF, many just give up. 

Having the launcher do anything related to partial updates is an appallingly bad idea.  I won't ever use it.  I will recommend that other people don't use it.  It just doesn't fit the goal of a Newb Pack at all.

I'm not trying to be grumpy, just save us from our own enthusiasm.  Listen to the voice of experience before it's too late.  Please. 

Pack authors need final control - I completely agree there. However, final control is not mutually exclusive with component-wise download/updating - so long as the pack author controls this part too.

Take the example of your own starter pack: Sometimes you have multiple releases for a single DF version. Fetching only the updated parts could make sense, and for some (not all!) pack authors, the effort can be worth it, especially once we move out of the bugfix phase of DF development.

Another option is as a tool for the pack author: use it to gather the files for your next release, but leave out the extra information when distributing it.

Under no circumstance should this replace the simple option that's currently in place, and it should never be mandatory. Ever. Your Starter Pack should be able to keep it as simple as you'd like, and I will reject any addition that prevents this.

It is, however, my distinct impression that some pack authors (or users) would like to be able to deliver at least some updates more incrementally. Perhaps this becomes too complicated and bloated to include in the standard launcher, and if so, then it can simply live on as a distinct fork for the people who really want to do this, or it can be pulled out into its own tool. I do not yet know; I won't know until I see more code.

[Dwimenor]

To the best of my knowledge, Mint is the most popular distribution these days, and one of its core desktop environments is MATE - where xdg-terminal does not work out of the box, so already there, I would argue you're not covering "most people".

You probably based those assumptions on data available on distrowatch.com or some other source, that pulls data from distrowatch. There is one big problem with that. Distrowatch collects user-agent strings from people who go to this site. At the same time distrowatch is dedicated to a bit more tech savvy linux users. Hence Mint being first on this list or Debian being 3rd ("Vanilla" Debian has quite low desktop market coverage). But for a desktop app like pylnp don't think about distribution, think about freedesktop.org standards.

EDIT: exec* is not an option because utilities need to be launched after DF - so it's necessary to find a way to launch DF as an entirely stand-alone process, with its own terminal window. For Windows, I can use the start command; on OS X, I can use open. I haven't been able to figure out a useful possibility on Linux; I don't see a way to make Python do it directly, and I haven't been able to detect which terminal program the user is using to launch a new one. If anyone knows something I don't, I'm all ears.

Try this:
1. Create .desktop file

Code: [Select]

[Desktop Entry]
Name=dfhack
Comment=Run Dwarf Fortress with dfhack
Exec=path/to/dfhack
Terminal=true
Type=Application
Categories=Game;
3. subprocess.Popen(["xdg-open", "dfhack.desktop"])
4. Let desktop environment deal with it.

About exe thing:
py2exe supports python3
py2app works for mac (I don't know if it works with py3)
in linux you just distribute source code. Just move all pyLNP source files into one dir and keep one launcher file inside root dir (like current LNP for linux does).

[Pidgeot]

To the best of my knowledge, Mint is the most popular distribution these days, and one of its core desktop environments is MATE - where xdg-terminal does not work out of the box, so already there, I would argue you're not covering "most people".

You probably based those assumptions on data available on distrowatch.com or some other source, that pulls data from distrowatch. There is one big problem with that. Distrowatch collects user-agent strings from people who go to this site. At the same time distrowatch is dedicated to a bit more tech savvy linux users. Hence Mint being first on this list or Debian being 3rd ("Vanilla" Debian has quite low desktop market coverage). But for a desktop app like pylnp don't think about distribution, think about freedesktop.org standards.

Well, let me rephrase that then: it is A popular distribution, and the same issue applies to all MATE users anyway.

EDIT: exec* is not an option because utilities need to be launched after DF - so it's necessary to find a way to launch DF as an entirely stand-alone process, with its own terminal window. For Windows, I can use the start command; on OS X, I can use open. I haven't been able to figure out a useful possibility on Linux; I don't see a way to make Python do it directly, and I haven't been able to detect which terminal program the user is using to launch a new one. If anyone knows something I don't, I'm all ears.

Try this:
1. Create .desktop file

Code: [Select]

[Desktop Entry]
Name=dfhack
Comment=Run Dwarf Fortress with dfhack
Exec=path/to/dfhack
Terminal=true
Type=Application
Categories=Game;
3. subprocess.Popen(["xdg-open", "dfhack.desktop"])
4. Let desktop environment deal with it.

That's a very good idea, but I tried creating that file and running it with xdg-open:

Code: [Select]

$ xdg-open test.desktop
Warning: unknown mime-type for "test.desktop" -- using "application/octet-stream"
Error: No "view" mailcap rules found for type "application/octet-stream"
Opening "test.desktop" with Text Editor (application/x-desktop)

And yes, I did try making the file executable, just in case - didn't work. It looks like this is a long-standing bug from the GNOME days...

py2exe supports python3

Yes, but py2exe does not work for single-file distribution of Tkinter apps, and therefore I do not consider it a useful option. Pyinstaller was the only option for this.

Also, I want to keep dependencies to the absolute minimum, and since Linux does not include Tkinter support by default, I still need Linux binaries.

[Dricus]

About Tkinter: Why did you choose to use Tkinter? As far as I know it's based off of the ancient Tk toolkit. It's got quite a few nuts and bolts, but lacks useful things like for example a listbox with checkboxes, which is used in the starter pack for selecting the utilities you want to autostart. It also lacks (again, afaik) a good GUI editor, which makes it incredibly easy to create nice looking user interfaces.

About a week ago I started recreating a DF launcher in Python using PyQt4 (mainly for some Python and Qt practice) and ended up recreating a fully functional program with the first 2 tabs of the starter pack launcher. The GUI was created with Qt's Designer program, which makes creating UI's a breeze. Below I've included a screenshot. The window is fully resizable, everything scales nicely.

Spoiler (click to show/hide)

[PeridexisErrant]

About a week ago I started recreating a DF launcher in Python using PyQt4 (mainly for some Python and Qt practice) and ended up recreating a fully functional program with the first 2 tabs of the starter pack launcher. The GUI was created with Qt's Designer program, which makes creating UI's a breeze. Below I've included a screenshot. The window is fully resizable, everything scales nicely.

Spoiler (click to show/hide)

Well, I'm certainly not worried about running out of alternative launchers...

[lethosor]

Tkinter is included with Python by default on Windows and OS X, and from my experience, bundling PyQt tends to make extremely large executables.

[Dwimenor]

That's a very good idea, but I tried creating that file and running it with xdg-open:

Code: [Select]

$ xdg-open test.desktop
Warning: unknown mime-type for "test.desktop" -- using "application/octet-stream"
Error: No "view" mailcap rules found for type "application/octet-stream"
Opening "test.desktop" with Text Editor (application/x-desktop)


...I can't believe GNOME folks intentionally keep this as "it's not a bug, it's a feature" after all those years.

Can you try putting dfhack.desktop in $HOME/.local/share/applications and then just run "xdg-open dfhack" ?
I can sense some incoming problems with this approach, like GNOME asking user "run, cancel, open, run in terminal" every time. Can you test it?
Other approach: bound simple terminal with pyLNP. Dunno how to do it with tk, or if it's even possible.
Other approach: check for xterm->gnome-terminal->konsole->lxterminal->mate-terminal and run dfhack inside first found. Which is what shell script you linked earlier does.

[Dricus]

The executables I get with PyQt are 11.5 Mb on Windows, 14.4 Mb on MacOS and 22 Mb on Linux (which surprises me, actually). They are not small by any means, but IMO also not extremely large.

[lethosor]

The executables I get with PyQt are 11.5 Mb on Windows, 14.4 Mb on MacOS and 22 Mb on Linux (which surprises me, actually). They are not small by any means, but IMO also not extremely large.

Those sizes sound reasonable to me. I was probably doing something wrong, then.

[Pidgeot]

About Tkinter: Why did you choose to use Tkinter? As far as I know it's based off of the ancient Tk toolkit. It's got quite a few nuts and bolts, but lacks useful things like for example a listbox with checkboxes, which is used in the starter pack for selecting the utilities you want to autostart. It also lacks (again, afaik) a good GUI editor, which makes it incredibly easy to create nice looking user interfaces.

About a week ago I started recreating a DF launcher in Python using PyQt4 (mainly for some Python and Qt practice) and ended up recreating a fully functional program with the first 2 tabs of the starter pack launcher. The GUI was created with Qt's Designer program, which makes creating UI's a breeze. Below I've included a screenshot. The window is fully resizable, everything scales nicely.

Spoiler (click to show/hide)

Tkinter is part of a standard Python install (except on Linux because... reasons), so it minimizes dependencies, and it keeps the executables reasonably small. That was pretty much it.

I am perfectly aware that much better UIs are possible with other toolkits (wxPython, QT, etc.), and I am also aware that eventually, it would probably be a good idea to switch to one of those - which is why I keep all actual functionality in a separate class and just make the UI a dumb view which calls down to that. Tkinter simply seemed like the most "neutral" choice, but I'm not married to it or anything

That's a very good idea, but I tried creating that file and running it with xdg-open:

Code: [Select]

$ xdg-open test.desktop
Warning: unknown mime-type for "test.desktop" -- using "application/octet-stream"
Error: No "view" mailcap rules found for type "application/octet-stream"
Opening "test.desktop" with Text Editor (application/x-desktop)


...I can't believe GNOME folks intentionally keep this as "it's not a bug, it's a feature" after all those years.

Can you try putting dfhack.desktop in $HOME/.local/share/applications and then just run "xdg-open dfhack" ?
I can sense some incoming problems with this approach, like GNOME asking user "run, cancel, open, run in terminal" every time. Can you test it?

I moved the file to that directory, and xdg-open won't find it just by name. The same applies if I drop it directly into /usr/share/applications.

But it turns out that doesn't matter, because I tried double-clicking the file in the file manager, and got this lovely error message:

There was an error launching the application.

Details: Failed to execute child process "xterm" (No such file or directory)

Okay, maybe it's just an issue with that specific distribution (Mint 15/MATE). So I tried Fedora 20/MATE (which, you'll recall, also didn't supply xterm). Same issue.

Well, maybe it's been fixed. Let me go update my Arch/MATE VM and see what happens... nope, same problem.

...what is this i don't even (╯°□°）╯︵ ┻━┻

Other approach: check for xterm->gnome-terminal->konsole->lxterminal->mate-terminal and run dfhack inside first found. Which is what shell script you linked earlier does.

That basic approach is looking like the only remotely sane one (with the option to specify a fallback in PyLNP.user). And it's not very sane.

The executables I get with PyQt are 11.5 Mb on Windows, 14.4 Mb on MacOS and 22 Mb on Linux (which surprises me, actually). They are not small by any means, but IMO also not extremely large.

Is that with or without UPX compression? There might be something to gain there.

[Dricus]

Is that with or without UPX compression? There might be something to gain there.

It's without UPX, don't have that installed. Zipping the executables doesn't reduce the size significantly though.

[MagiX]

About a week ago I started recreating a DF launcher in Python using PyQt4 (mainly for some Python and Qt practice) and ended up recreating a fully functional program with the first 2 tabs of the starter pack launcher. The GUI was created with Qt's Designer program, which makes creating UI's a breeze. Below I've included a screenshot. The window is fully resizable, everything scales nicely.

Spoiler (click to show/hide)

Well, I'm certainly not worried about running out of alternative launchers...

Shouldn't we try to streamline our efforts to one launcher and make it "perfect" instead of x launchers that "just work"?
In this regard, what about we create a public TODO list like this one, where parts are already done/in the work, i.e. issues on Pidgeots bitbucket, and people that are willing to contribute give it a try?
I'd love to help out with the project, but as you saw, my coding skills have room for improvement and my DF experience is rather limited (less than 1 month!). Given the feedback regarding package version check and updating, I guess it's worthwhile to spend my time on more fruitful efforts .
Unless there are other suggestions, I guess I will take a look at a quickfort mak creation function and later "interface"

[PeridexisErrant]

Shouldn't we try to streamline our efforts to one launcher and make it "perfect" instead of x launchers that "just work"?
In this regard, what about we create a public TODO list like this one, where parts are already done/in the work, i.e. issues on Pidgeots bitbucket, and people that are willing to contribute give it a try?
I'd love to help out with the project, but as you saw, my coding skills have room for improvement and my DF experience is rather limited (less than 1 month!). Given the feedback regarding package version check and updating, I guess it's worthwhile to spend my time on more fruitful efforts .
Unless there are other suggestions, I guess I will take a look at a quickfort mak creation function and later "interface"

One of the biggest reasons for my excitement over this PyLNP in particular is that it has a decent chance to become a cross-platform standard, and it's got enough openness and interest to survive even when Real Life stops some people contributing much.  A big part of that standardisation and resilience is network effects, so having too many launchers would hurt all of them - but I don't think we're at that stage yet.  It might make snese eventually to fork an "expert launcher" from the "newb launcher"; what I was talking about is one that works perfectly for the latter use case.  Motto: "it just works (perfectly, without newbs having any knowledge of how)".  An expert's launcher would be quite diferent, and I think a single thing can only do one decently, let alone perfectly. 

Personally, I don't think the quickfort converter should be integrated with the launcher, to me it's a utility.  The tab would have to be hidden on Windows, since QF works differently there.  It could definitely use some work for *nix systems and some functionality would be similar, but it's a separate project.  Pursue if interested anyway!

DFHack:  it's complicated, and gets worse if we don't want to mess up a pre-existing configuration.  It's also tough to be flexible enough to cope with the format changes that keep happening for the more advanced stuff, which is most interesting to us.  In the Starter Pack I got around this (somewhat) by using an LNP-pnly init file, and modifying dfhack.init to run the commands in it when a map was loaded.  https://bitbucket.org/Pidgeot/python-lnp/issue/8/implement-a-dfhack-tab is my current suggestion for the PyLNP where it needs to be a little more general. 

Mods:  far far harder than it sounds, I made a thread to look at this last week and 260 posts later we have a lot of ideas that on further reflection wouldn't work.  And a lovely design plan for once it does, but...  http://www.bay12forums.com/smf/index.php?topic=142295