Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Dricus]

I think I agree with both PeridexisErrant and MagiX. Streamlining our efforts is a good idea, but I think we're still at a stage where all the implications of creating a generic cross-platform DF launcher are becoming apparent. So although we've got a few more or less working/finished alternatives, we're still discovering new stuff and still discussing about what a basic launcher should look like. Feedback and advise from pack authors like PeridexisErrant and fricy is invaluable in this process.

There is another aspect to this, which hasn't been discussed at all, which is the maintainability of the source code of a launcher. If we want to create a launcher that will have the potential to be the de facto standard for an extended period of time, the source code needs to be maintainable and be kept maintainable. For devs that want to contribute, it should be reasonably easy to figure out how the program works under the hood. That way, adding new functionality and modifying existing functionality should remain reasonably easy for a dev with a little bit of experience under his/her belt.

This is, in my humble opinion, where PyLNP already falls behind. With lnp.py (almost 800 lines of code) being the "brain" of the program, it's falling prey to the God Object antipattern. The code for the "dumb" view, tkgui.py, is already 1400 lines of code (which is huge for a single module), partly because the entire layout of the GUI has to be handcoded and partly because recurring patterns haven't been refactored into reusable functions. The way loading/changing/saving settings works is set up in a smart way (with all the dictionaries and stuff), but it's quite hard to figure out exactly how it works.

I don't mean to sound harsh or be offensive. It's just that in my 18-year-and-counting career of being a professional developer, I've seen stuff like this happen many times. The enthusiasm and motivation is great, but often stops us from slowing down and taking the time to do things "the right way (tm)".

To quote PE's slightly melodramatic call :

I'm not trying to be grumpy, just save us from our own enthusiasm.  Listen to the voice of experience before it's too late.  Please. 

I've published my recent efforts with PyQt4 to Github: https://github.com/Dricus/dwarf-fortress-launcher
Feel free to have a look, criticize, fork, file pull requests, etc. It's not finished, and I won't claim that my code is perfect, but it's reasonably well documented and I like the design of the code quite a bit (although, again, there's always room for improvement).

[Pidgeot]

There is another aspect to this, which hasn't been discussed at all, which is the maintainability of the source code of a launcher. If we want to create a launcher that will have the potential to be the de facto standard for an extended period of time, the source code needs to be maintainable and be kept maintainable. For devs that want to contribute, it should be reasonably easy to figure out how the program works under the hood. That way, adding new functionality and modifying existing functionality should remain reasonably easy for a dev with a little bit of experience under his/her belt.

This is, in my humble opinion, where PyLNP already falls behind. With lnp.py (almost 800 lines of code) being the "brain" of the program, it's falling prey to the God Object antipattern. The code for the "dumb" view, tkgui.py, is already 1400 lines of code (which is huge for a single module), partly because the entire layout of the GUI has to be handcoded and partly because recurring patterns haven't been refactored into reusable functions. The way loading/changing/saving settings works is set up in a smart way (with all the dictionaries and stuff), but it's quite hard to figure out exactly how it works.

In terms of design, I spend a little time contemplating the various options I can come up with, and pick the one that makes most sense (keeping KISS in mind). That, for example, means that unless I have a good reason to split something into its own file or function, I don't. File length (or method count) is not a good reason in and of itself.

It is certainly true that since the initial version, there are pieces of code that have become eligible for splitting (especially in tkgui.py). However, my priority has been getting the code to a state where someone wants to use it. Once we're past this initial push of feature requests (we're basically there once the DFHack stuff is done), there's plenty of opportunity to refactor and find a sensible design (both in terms of code and UI).

If you have a (concrete) idea about how to do something differently, tell me (or just do it and submit a pull request). I'm open to questions about how something works (or why it's done that way). I'm open to suggestions. I'm open to other UIs, too; if you want to make a nice GUI in Qt or whatever and hook it up, I'll take it.

I don't mean to sound harsh or be offensive. It's just that in my 18-year-and-counting career of being a professional developer, I've seen stuff like this happen many times. The enthusiasm and motivation is great, but often stops us from slowing down and taking the time to do things "the right way (tm)".

Sure, but sometimes, trying to do things "the right way" leads to not getting anything done.

I'm a professional developer too - I absolutely recognize the value of good design, and no, I would not consider PyLNP to be well-designed as a whole (parts of it, yes; all of it, no). It is good enough as a starting point, though (otherwise, I should not have been able to take a year-long break between the initial port and the past week of feature requests).

[Dricus]

Disclaimer: On second thought, I realize I'm oversimplifying things a bit here. So when reading this post, know that I know that there's a lot of nuances to all this ...

In terms of design, I spend a little time contemplating the various options I can come up with, and pick the one that makes most sense (keeping KISS in mind). That, for example, means that unless I have a good reason to split something into its own file or function, I don't. File length (or method count) is not a good reason in and of itself.

Scientific studies show a strong correlation between, amongst others, large units (methods, funcions, etc) and modules (files, classes) and low defect resolution efficiency. Having few modules in your app doesn't make it simple, when those modules have many responsibilities. Having many modules in your app doesn't make it complex, when those modules are small and have a clear responsibility.

It is certainly true that since the initial version, there are pieces of code that have become eligible for splitting (especially in tkgui.py). However, my priority has been getting the code to a state where someone wants to use it.

Sure, but sometimes, trying to do things "the right way" leads to not getting anything done.

You seem to be implying that there's a conflict of interests between making software usable in a reasonable amount of time and doing things "the right way". I'm not saying that there is only one way of doing things right in software development, hence the quotes. However, I'm strongly rejecting the notion that "doing things right" will ever prevent you from getting anything done. If you're not getting anything done, you're doing it wrong! In my experience, the opposite is true: Doing things right from the get go will make you go fast, no exceptions.

Once we're past this initial push of feature requests (we're basically there once the DFHack stuff is done), there's plenty of opportunity to refactor and find a sensible design (both in terms of code and UI).

There's always an opportunity. Nobody is imposing deadlines or anything like that.

If you have a (concrete) idea about how to do something differently, tell me (or just do it and submit a pull request). I'm open to questions about how something works (or why it's done that way). I'm open to suggestions. I'm open to other UIs, too; if you want to make a nice GUI in Qt or whatever and hook it up, I'll take it.

A few things come to mind:

• Split up lnp.py into separate modules according to it's responsibilities:
  • Loading utilities
  • Loading/installing keybinds
  • Loading/installing graphics packs
  • Starting processes (DF and utilities)
  • Detecting updates
  • Stuff I missed
• Replace the confusing dictionary stuff with a lightweight observer pattern to bind UI elements to their respective Dwarf Fortress settings.
• Split up tkui.py into separate modules for each window.
• Refactor the repeating patterns of creating controls to factory methods for each type of control you create.

I'm a professional developer too - I absolutely recognize the value of good design, and no, I would not consider PyLNP to be well-designed as a whole (parts of it, yes; all of it, no). It is good enough as a starting point, though (otherwise, I should not have been able to take a year-long break between the initial port and the past week of feature requests).

I'm well aware that I'm touching on a somewhat sensitive subject here, but I'm only trying to help by pointing out an opportunity for improvement. Thanks for not being all defensive about it :).

[Pidgeot]

In terms of design, I spend a little time contemplating the various options I can come up with, and pick the one that makes most sense (keeping KISS in mind). That, for example, means that unless I have a good reason to split something into its own file or function, I don't. File length (or method count) is not a good reason in and of itself.

Scientific studies show a strong correlation between, amongst others, large units (methods, funcions, etc) and modules (files, classes) and low defect resolution efficiency. Having few modules in your app doesn't make it simple, when those modules have many responsibilities. Having many modules in your app doesn't make it complex, when those modules are small and have a clear responsibility.

There are other factors in that problem, though, such as dependencies between objects. The vast majority of methods in lnp.py are really completely independent of each other (the object itself is little more than a data provider and common access point); they do not NEED to be part of the same object, but it is not obvious what the (actual, not theoretical) benefit would be if they were split.

There is no inherent difference (excluding syntax) between "self.lnp.install_graphics(...)" and e.g. "self.lnp.graphics.install(...)"; it takes the same amount of effort to use either one and it takes the same amount of effort to locate the method if you need to change it. There can be other reasons to prefer one over the other, of course, but I do not think it is obvious exactly where those reasons are significant enough.

That doesn't mean there aren't places where it makes sense to pull some functionality out to a new file/object (I'm sure there are), or that it shouldn't be done (I'm sure it should), or that the current design is the best possible design (I'm sure it isn't) - I'm just saying it's more complicated than "the file is too long, let's split it".

It is certainly true that since the initial version, there are pieces of code that have become eligible for splitting (especially in tkgui.py). However, my priority has been getting the code to a state where someone wants to use it.

Sure, but sometimes, trying to do things "the right way" leads to not getting anything done.

You seem to be implying that there's a conflict of interests between making software usable in a reasonable amount of time and doing things "the right way". I'm not saying that there is only one way of doing things right in software development, hence the quotes. However, I'm strongly rejecting the notion that "doing things right" will ever prevent you from getting anything done. If you're not getting anything done, you're doing it wrong! In my experience, the opposite is true: Doing things right from the get go will make you go fast, no exceptions.

That was not the point I was trying to make - doing things right from the get go is certainly both possible and preferable, and I definitely try to do so whenever possible. However, as you also say, there is no silver bullet - "the right way" is not always obvious, or unique... or objective.

Once we're past this initial push of feature requests (we're basically there once the DFHack stuff is done), there's plenty of opportunity to refactor and find a sensible design (both in terms of code and UI).

There's always an opportunity. Nobody is imposing deadlines or anything like that.

There is always an opportunity to re-design, but it is not always meaningful to do so. Getting the product out there gives more opportunity for feedback, and by extension, a better overview of what the program needs to deal with.

Refactor the repeating patterns of creating controls to factory methods for each type of control you create.

If you can think of a good way to do that, I'd be very interested to hear it. So far, I don't see much potential for abstraction beyond adding tooltip and binding directly at construction time, and that would currently contribute such a small difference that I have not considered it worthwhile to do (at least, not yet).

I'm a professional developer too - I absolutely recognize the value of good design, and no, I would not consider PyLNP to be well-designed as a whole (parts of it, yes; all of it, no). It is good enough as a starting point, though (otherwise, I should not have been able to take a year-long break between the initial port and the past week of feature requests).

I'm well aware that I'm touching on a somewhat sensitive subject here, but I'm only trying to help by pointing out an opportunity for improvement. Thanks for not being all defensive about it .

It's cool - I totally get what you're saying, and I appreciate it. I'm just viewing things a little differently

[fricy]

That was the main point I was trying to make - I'm all for having this kind of functionality if it makes sense. I just think that it should not be *required*, that is, even if git is used as the primary update method, we should still have support for plain HTTP downloads, to handle add-ons/utilities that aren't in a git repository (for whatever reason).
/snip
Pack authors need final control - I completely agree there. However, final control is not mutually exclusive with component-wise download/updating - so long as the pack author controls this part too.
Take the example of your own starter pack: Sometimes you have multiple releases for a single DF version. Fetching only the updated parts could make sense, and for some (not all!) pack authors, the effort can be worth it, especially once we move out of the bugfix phase of DF development.
Another option is as a tool for the pack author: use it to gather the files for your next release, but leave out the extra information when distributing it.
Under no circumstance should this replace the simple option that's currently in place, and it should never be mandatory. Ever. Your Starter Pack should be able to keep it as simple as you'd like, and I will reject any addition that prevents this.

Agreed.

One of the biggest reasons for my excitement over this PyLNP in particular is that it has a decent chance to become a cross-platform standard, and it's got enough openness and interest to survive even when Real Life stops some people contributing much.

+1

Personally, I don't think the quickfort converter should be integrated with the launcher, to me it's a utility.  The tab would have to be hidden on Windows, since QF works differently there.  It could definitely use some work for *nix systems and some functionality would be similar, but it's a separate project.  Pursue if interested anyway!

While I still think it's make sense for *nix where python comes default, let's just cross this off the list.

Mods:  far far harder than it sounds, I made a thread to look at this last week and 260 posts later we have a lot of ideas that on further reflection wouldn't work.  And a lovely design plan for once it does, but...  http://www.bay12forums.com/smf/index.php?topic=142295

I've been meaning to read that topic, but for a lack of time... Holy shit that's a lot of posts in a very little time.

The executables I get with PyQt are 11.5 Mb on Windows, 14.4 Mb on MacOS and 22 Mb on Linux (which surprises me, actually). They are not small by any means, but IMO also not extremely large.

What's more, QT (5.3?) is already in all the starter packs due to Therapist, and I can easily symlink those libraries further reducing the size. I'm not sure about the situation on windows and linux though, and DT just been rewritten to work with QT4 too (with linux in mind), so it might just be me, I need to look into it some more.
In the long run it wouldn't be bad to recommend using QT for all (new) utilities.
And (as a non-programmer) being able to make edits to the GUI appeals to me.

[Dricus]

I'm just saying it's more complicated than "the file is too long, let's split it".

You're making it more complicated by implying that splitting up large modules can somehow be a bad thing to do. While that's certainly a theoretical possibility, it's a very unlikely one. The benefit of splitting up is not in readability of single lines of code. It's in the maintainability of the system as a whole. By creating small modules, each with it's own responsibility, developers can get an overview of the parts that make up the program, just by looking at the names of the modules. Minimizing dependencies between separate modules is a lot easier than minimizing dependencies between different distinct parts of the same class. Small, independent modules are easier to change than large ones with many responsibilities. The notion of separating concerns is decades old and has proven itself invaluable.

There is always an opportunity to re-design, but it is not always meaningful to do so. Getting the product out there gives more opportunity for feedback, and by extension, a better overview of what the program needs to deal with.

Agreed.

[Dricus]

If you can think of a good way to do that, I'd be very interested to hear it. So far, I don't see much potential for abstraction beyond adding tooltip and binding directly at construction time, and that would currently contribute such a small difference that I have not considered it worthwhile to do (at least, not yet).

The general principle here is Don't Repeat Yourself (DRY). If you find yourself repeating small variations of the same pattern over and over again, you should consider making a function. The parts that vary will then become the parameters of that function. For a more detailed explanation: http://sourcemaking.com/refactoring/extract-method. IDE's like Eclipse make doing this a lot easier.

For example: In tkgui.py, many buttons are created in the same way: Create the button, assign a tooltip, configure the grid and register it with the controls dictionary. See the following snippet:

Code: [Select]

        backup_saves = Button(
            saverelated, text='Backup Saves',
            command=lambda: self.cycle_option('autoBackup'))
        create_tooltip(backup_saves, 'Makes a backup of every autosave')
        backup_saves.grid(column=0, row=2, sticky="nsew")
        self.controls['autoBackup'] = backup_saves
        compress_saves = Button(
            saverelated, text='Compress Saves',
            command=lambda: self.cycle_option('compressSaves'))
        create_tooltip(
            compress_saves, 'Whether to compress the savegames '
            '(keep this on unless you experience problems with your saves')
        compress_saves.grid(column=1, row=2, sticky="nsew")
        self.controls['compressSaves'] = compress_saves
This can easily be refactored into a function, eliminating the need of recoding the same button creation process over and over again:

Code: [Select]

    def create_option_button(self, parent, row, column, key, text, tooltip):
        button = Button(
            parent, text=text,
            command=lambda:self.cycle_option(key))
        create_tooltip(button, tooltip)
        button.grid(column=column, row=row, sticky="nsew")
        self.controls[key] = button

Code: [Select]

        self.create_option_button(saverelated, 2, 0, 'autoBackup', 'Backup Saves', 'Makes a backup of every autosave')
        self.create_option_button(saverelated, 2, 1, 'compressSaves', 'Compress Saves',
                                  'Whether to compress the savegames '
                                  '(keep this on unless you experience problems with your saves')Now, let's suppose you want to change the button to a checkbox. In stead of having to change a lot of your code, you can just change the create_option_button function (maybe rename it, which will be a simple search and replace) and be done with it.

[Pidgeot]

If you can think of a good way to do that, I'd be very interested to hear it. So far, I don't see much potential for abstraction beyond adding tooltip and binding directly at construction time, and that would currently contribute such a small difference that I have not considered it worthwhile to do (at least, not yet).

The general principle here is Don't Repeat Yourself (DRY). If you find yourself repeating small variations of the same pattern over and over again, you should consider making a function. The parts that vary will then become the parameters of that function. For a more detailed explanation: http://sourcemaking.com/refactoring/extract-method. IDE's like Eclipse make doing this a lot easier.

For example: In tkgui.py, many buttons are created in the same way: Create the button, assign a tooltip, configure the grid and register it with the controls dictionary. See the following snippet:

Code: [Select]

        backup_saves = Button(
            saverelated, text='Backup Saves',
            command=lambda: self.cycle_option('autoBackup'))
        create_tooltip(backup_saves, 'Makes a backup of every autosave')
        backup_saves.grid(column=0, row=2, sticky="nsew")
        self.controls['autoBackup'] = backup_saves
        compress_saves = Button(
            saverelated, text='Compress Saves',
            command=lambda: self.cycle_option('compressSaves'))
        create_tooltip(
            compress_saves, 'Whether to compress the savegames '
            '(keep this on unless you experience problems with your saves')
        compress_saves.grid(column=1, row=2, sticky="nsew")
        self.controls['compressSaves'] = compress_saves
This can easily be refactored into a function, eliminating the need of recoding the same button creation process over and over again:

Code: [Select]

    def create_option_button(self, parent, row, column, key, text, tooltip):
        button = Button(
            parent, text=text,
            command=lambda:self.cycle_option(key))
        create_tooltip(button, tooltip)
        button.grid(column=column, row=row, sticky="nsew")
        self.controls[key] = button

Code: [Select]

        self.create_option_button(saverelated, 2, 0, 'autoBackup', 'Backup Saves', 'Makes a backup of every autosave')
        self.create_option_button(saverelated, 2, 1, 'compressSaves', 'Compress Saves',
                                  'Whether to compress the savegames '
                                  '(keep this on unless you experience problems with your saves')Now, let's suppose you want to change the button to a checkbox. In stead of having to change a lot of your code, you can just change the create_option_button function (maybe rename it, which will be a simple search and replace) and be done with it.

Sure, but that's the easy part

The part that's difficult is the layout. Just a column/row isn't enough; sometimes I need to use .pack instead of .grid; sometimes I need the button to span two columns... I've been trying to come up with an elegant solution to *that*, without too much luck.

Having said that, some ideas have come to mind just a short while ago, and now that the issues list is cleared out and 0.3 is released, the next step will likely be to try some of those out (and other steps of refactoring, of course).

[PeridexisErrant]

As of v0.3, I'm preparing to move over - probably at v0.4.  I also keep thinking of new issues every time something gets released, so... sorry about that.  We're well into enhancement territory though!   

[Dricus]

Then refactor out the part that's making the easy stuff hard:

Code: [Select]

    def create_option_button(self, parent, key, text, tooltip):
        button = Button(
            parent, text=text,
            command=lambda:self.cycle_option(key))
        create_tooltip(button, tooltip)
        self.controls[key] = button
        return button

Code: [Select]

        autoBackup = self.create_option_button(saverelated, 'autoBackup', 'Backup Saves', 'Makes a backup of every autosave')
        compressSaves = self.create_option_button(saverelated, 'compressSaves', 'Compress Saves',
                                  'Whether to compress the savegames '
                                  '(keep this on unless you experience problems with your saves')

        autoBackup.grid(column=0, row=2, sticky="nsew")
        compressSaves.grid(column=1, row=2, sticky="nsew")

Or be fancy and do this:

Code: [Select]

    def create_option_button(self, parent, key, text, tooltip):
        button = Button(
            parent, text=text,
            command=lambda:self.cycle_option(key))
        create_tooltip(button, tooltip)
        self.controls[key] = button
        return button

    def create_option_button_grid(self, parent, key, text, tooltip, **grid_args):
        button = create_option_button(parent, key, text, tooltip)
        button.grid(**grid_args)

    def create_option_button_pack(self, parent, key, text, tooltip, **pack_args):
        button = create_option_button(parent, key, text, tooltip)
        button.pack(**grid_args)


Code: [Select]

        autoBackup = self.create_option_button_grid(saverelated, 'autoBackup', 'Backup Saves', 'Makes a backup of every autosave',
                                  column=0, row=2, sticky="nsew")
        compressSaves = self.create_option_button_grid(saverelated, 'compressSaves', 'Compress Saves',
                                  'Whether to compress the savegames '
                                  '(keep this on unless you experience problems with your saves',
                                  column=1, row=2, sticky="nsew")

        # Et cetera


[Beautato]

PeridexisErrant showed me this thread yesterday, I had not been paying attention to the efforts with PyLNP since its initial post, but much of the discussion here is echoing my thoughts over the past few days. (mostly about auto-updating and LNP launcher compiling the utilities for the *nix users)

As of v0.3, I'm preparing to move over - probably at v0.4. 

pending some improvements I am also very interested in using the PyLNP launcher.

[PeridexisErrant]

Spoiler: PyLNP.json (click to show/hide)

Code: [Select]

{
    "folders": [
        ["Main folder",""],
        ["Dwarf Fortress folder","<df>"],
        ["LNP folder","LNP"],
        ["User Content","User Generated Content"],
        ["-","-"],
        ["Savegame folder","<df>/data/save"],
        ["Utilities folder","LNP/Utilities"],
        ["Graphics folder","LNP/Graphics"],
        ["Init folder","<df>/data/init"]
    ],
    "links": [
        ["DF Homepage","http://www.bay12games.com/dwarves/"],
        ["DF Forums","http://www.bay12forums.com/smf/"],
        ["This Pack's homepage","http://www.bay12forums.com/smf/index.php?topic=126076"],
        ["-","-"],
        ["DF Wiki","http://dwarffortresswiki.org/index.php/DF2014:Quickstart_guide"],
        ["DF on Reddit",""],
        ["DF Map Archive","http://mkv25.net/dfma/index.php"],
        ["DFHack readme","https://github.com/DFHack/dfhack/blob/master/Readme.rst"],
        ["DF on TvTropes","http://tvtropes.org/pmwiki/pmwiki.php/VideoGame/DwarfFortress"],
        ["-","-"],
        ["CptnDuck's Tutorial vids","https://www.youtube.com/playlist?list=PL06686270DA5FF439"],
        ["Getting Started (book)","http://shop.oreilly.com/product/0636920022565.do"],
        ["Free basis of book","http://afteractionreporter.com/2009/02/09/the-complete-and-utter-newby-tutorial-for-dwarf-fortress-part-1-wtf/"],
        ["-","-"],
        ["DF Stories (1)","http://dfstories.com/all-stories/"],
        ["DF Stories (2)","http://dfst.org/"],
        ["Hall of Legends","http://www.bay12forums.com/smf/index.php?topic=89305"]
    ],
    "hideUtilityPath": true,
    "updates": {
        "checkURL": "",
        "versionRegex": "",
        "downloadURL": "",
        "packVersion": ""
    }
}


I've filled out the folders and links menus easily enough, but I don't quite know where to start with the update section.  Can it check this link?  How does the downloadURL work - if it's the download page that's fine, but the actual download link changes and I don't know it in advance.  Regex (help!!)?  Does or can the pack version autofill, or should I add that to my prep script?  I also remember seeing something about user agent issues and DFFD a while back; what's happening with that?

[Putnam]

DFFD doesn't allow downloads through Python, last I checked, but then I didn't try too hard.

[Pidgeot]

I've filled out the folders and links menus easily enough, but I don't quite know where to start with the update section.  Can it check this link? 

Currently, no, because DFFD blocks Python by default. It's possible to make Python disguise itself to slip through the block, but that isn't being done right now (the hope was that MagiX' work could be incorporated, but since he left it alone for the time being, it hasn't been).

I've added that now, though, so it'll be part of the next build, which will either go out very soon (today or tomorrow, to get a bug fix out of the pipeline), or after a round of refactoring (I don't know exactly when that will be ready, but I'm guessing near the end of the week, or some time next week).

An option in the meantime could be to do one of the following:

• After uploading a new version to DFFD, update a file (e.g. a copy of whatever DFFD gives you) in your GitHub, then use a direct link to the latest version of that file (click through to view it on GitHub, then click the Raw button for the link you need)
• Change your topic to include the full version number as a continuous string and link to that

There are other options, of course; the key is that as long as you have a URL to a page you have some amount of control over, it should be possible to get it working.

How does the downloadURL work - if it's the download page that's fine, but the actual download link changes and I don't know it in advance.

It is used as the URL that is opened if the user wants to update, so that's entirely up to you. This is only a check; the act of downloading and unpacking is left to the user (and because of that, you can use a DFFD link, even now).

Regex (help!!)? 

Since I don't want to force a specific format on the file (a DFFD-only solution is not good enough IMO, and I don't know what everyone will be using outside of that), it is unfortunately necessary for the pack author to provide a regular expression which will capture the version number (and ONLY the version number) in a so-called capturing group (marked by parentheses "()").

Fortunately, this is a one-time thing (as long as the format of your file doesn't change, you never need to touch it after it's been set once), and I'm perfectly willing to help.

For DFFD's check_version.php script (and anything else that contains a Version line like that), this field should be "Version: (.+)" (without the quotes).

If you e.g. changed the title of your forum post to "Dwarf Fortress Starter Pack 40_10 r1", you could use "<title>Dwarf Fortress Starter Pack (.+?)</title>"; if you added a line that said the same as DFFD's script, you should use "Version: ([^<]+)" (make sure there's an actual line break after it, though, so the HTML will contain a
 tag). That last one would still work if you started out using the topic, and then switched to DFFD when it's available.

If you're doing something completely different, feel free to ask me. Alternatively, if you want to do it yourself (and learn regular expressions in the process), copy the entire contents of whatever URL you're using into http://www.regexr.com/ and work on it until you get something that does the job (it should highlight precisely one thing, and if you hover your mouse over that thing, it should show just the version number as being matched by the group). If you're linking to an HTML page, make sure you're working on the source code of the page; not whatever your webbrowsre renders.

Does or can the pack version autofill, or should I add that to my prep script?

You need to add it to your prep script. There's no attempt to parse the version number; it only checks if there's a difference between the one it sees online and the one in PyLNP.json.

[Janus]

The DFFD site doesn't block Python requests. After a bit of investigation, it comes down to the fact that in Python there's no User Agent specified by default. I don't have Nginx on the server specifically configured to deny requests with no User Agent specified, so I'm not sure why they're getting blocked.

The solution is simple enough, though, just specify a User Agent in the request. Rough working example (my Python-specific knowledge is very limited):

Code: [Select]

import urllib2
from urllib2 import urlopen

TEST_URL = "http://dffd.wimbli.com/file_version.php?id=7622"

hdr = {'User-Agent': 'Python or whatever you want to put here, Blah Blah'}

req = urllib2.Request(TEST_URL, headers=hdr)
html = urlopen(req).read()
print html
Outputs for me in the Python shell:

>>> ================================ RESTART ================================
>>>
Updated: 1409005154
Date: Aug 25, 2014, 05:19:14 pm
Version: 40_10 r1
>>>