Topic: PyLNP 0.14e-pre1 - Cross-platform launcher with graphics pack patching (Read 325997 times)

Putnam · « **Reply #105 on:** August 28, 2014, 06:38:42 pm »

Attempting to download actual files through python gives me junk.

PeridexisErrant · « **Reply #106 on:** August 28, 2014, 07:20:01 pm »

Thanks! Translated for my 3.4 install, and I can now spit out the version string

Spoiler: code (click to show/hide)

import urllib.request
import re

TEST_URL = "http://dffd.wimbli.com/file_version.php?id=7622"
hdr = {'User-Agent': 'PyLNP testing'}

req = urllib.request.Request(TEST_URL, headers=hdr)
html = urllib.request.urlopen(req).read().decode('utf-8')

version = re.search('Version: (.+)', html).group(1)
print(version)

Quote from: Putnam on August 28, 2014, 06:38:42 pm

Attempting to download actual files through python gives me junk.

I don't think that would work through the launcher anyway, since the exact download link may not be predictable in advance - I follow a pattern, but if a user misses an update etc to cover many cases would take a list of things to try. You could potentially process the version string and constants somehow, but I can't see a way of doing that that would work for the compiled version.

Code: [Select]

'http://dffd.wimbli.com/download.php?id=7622&f=Dwarf+Fortress+40_10+Starter+Pack+r1.zip'
    ==
'http://dffd.wimbli.com/download.php?id=7622&f=Dwarf+Fortress+' + version.replace(' ','+Starter+Pack+') + '.zip'

I think just opening the main page and letting the user download whatever pack is a better way of doing it.

lethosor · « **Reply #107 on:** August 28, 2014, 08:29:10 pm »

I'm not entirely sure what you mean by "the exact download link may not be predictable in advance", but the filename in the URL doesn't matter - http://dffd.wimbli.com/download.php?id=7622&f=test.zip works just fine.

PeridexisErrant · « **Reply #108 on:** August 28, 2014, 09:07:12 pm »

Quote from: lethosor on August 28, 2014, 08:29:10 pm

I'm not entirely sure what you mean by "the exact download link may not be predictable in advance", but the filename in the URL doesn't matter - http://dffd.wimbli.com/download.php?id=7622&f=test.zip works just fine.

I take it all back, that's very useful - even if the feature only works on DFFD, that's most of what I care about. Eventually this could mean a pack that gives users the option to download updates, and then it's a small step to unzip and run an update script... which I already include

Pidgeot · « **Reply #109 on:** August 29, 2014, 05:59:42 am »

Quote from: Janus on August 28, 2014, 04:53:02 pm

The DFFD site doesn't block Python requests. After a bit of investigation, it comes down to the fact that in Python there's no User Agent specified by default. I don't have Nginx on the server specifically configured to deny requests with no User Agent specified, so I'm not sure why they're getting blocked.

It does send a default user agent string on Windows, which is what I've mostly been testing with. This is the HTTP request generated when I just use urlopen2 directly (the code currently in the repository), as shown by the Fiddler HTTP debugger:

Code: [Select]

GET http://dffd.wimbli.com/file_version.php?id=7622 HTTP/1.1
Accept-Encoding: identity
Host: dffd.wimbli.com
Connection: close
User-Agent: Python-urllib/2.7

The resulting download returns HTTP error 403 (Forbidden), and this HTML (irrelevant layout bits cut out):

Code: [Select]

<!DOCTYPE html>
<head>
<title>Access denied | dffd.wimbli.com used CloudFlare to restrict access</title>
</head>
<body>
<h2 class="cf-subheadline" data-translate="error_desc">Access denied</h2>
<h2 data-translate="what_happened">What happened?</h2>
<p>The owner of this website (dffd.wimbli.com) has banned your access based on your browser's signature (16183275254b0a5a-ua48).</p>  
</body>
</html>

The workaround I've implemented (and just pushed) is precisely to send a custom user-agent string ("PyLNP"):

Code: [Select]

# HG changeset patch
# User Michael Madsen<michael@birdiesoft.dk>
# Date 1409132432 -7200
#      Wed Aug 27 11:40:32 2014 +0200
# Node ID fecc2afc9d1c40d2fef2579e0e12857de451ec65
# Parent  f1ca127f6a819e78699340630e10d8f6e85fbf72
Change user agent of update checks to PyLNP

diff -r f1ca127f6a81 -r fecc2afc9d1c lnp.py
--- a/lnp.py	Wed Aug 27 02:13:12 2014 +0200
+++ b/lnp.py	Wed Aug 27 11:40:32 2014 +0200
@@ -23,10 +23,10 @@
 
 try:  # Python 2
     # pylint:disable=import-error
-    from urllib2 import urlopen, URLError
+    from urllib2 import urlopen, URLError, Request
 except ImportError:  # Python 3
     # pylint:disable=import-error, no-name-in-module
-    from urllib.request import urlopen
+    from urllib.request import urlopen, Request
     from urllib.error import URLError
 
 BASEDIR = '.'
@@ -712,8 +712,10 @@
             return
         if self.userconfig['nextUpdate'] < time.time():
             try:
-                version_text = urlopen(
-                    self.config['updates']['checkURL']).read()
+                req = Request(
+                    self.config['updates']['checkURL'],
+                    headers={'User-Agent':'PyLNP'})
+                version_text = urlopen(req).read()
                 # Note: versionRegex must capture the version number in a group
                 new_version = re.search(
                     self.config['updates']['versionRegex'],

MagiX · « **Reply #110 on:** August 29, 2014, 04:13:28 pm »

Quote from: Putnam on August 28, 2014, 06:38:42 pm

Attempting to download actual files through python gives me junk.

Quote from: PeridexisErrant on August 28, 2014, 07:20:01 pm

Thanks! Translated for my 3.4 install, and I can now spit out the version string

Feel free to check the last two commits on my pyLNP fork (it's otherwise behind, as I'm working on something else right now...hopefully will have some pictures and feature requests before my holidays)

lethosor · « **Reply #111 on:** August 30, 2014, 08:48:45 am »

The current commit (fecc2afc9d1c) is breaking settings (and causing the launcher to fail to open) - opening the launcher and toggling weather, for example, leaves behind [FPS_CAP:], [G_FPS_CAP:], [VOLUME:], and possibly others in init.txt.

Pidgeot · « **Reply #112 on:** August 30, 2014, 09:25:52 am »

Quote from: lethosor on August 30, 2014, 08:48:45 am

The current commit (fecc2afc9d1c) is breaking settings (and causing the launcher to fail to open) - opening the launcher and toggling weather, for example, leaves behind [FPS_CAP:], [G_FPS_CAP:], [VOLUME:], and possibly others in init.txt.

Should be fixed in the latest commit, c660577 (could have sworn I tested for this...)

trauma · « **Reply #113 on:** August 31, 2014, 12:49:09 pm »

If the terminal spawning thing is still relevant, I solved it like this in a different context recently:

Code: [Select]

${COLORTERM:-${TERM:-xterm}} -e foowhich would translate to python as

Code: [Select]

Popen([os.environ.get("COLORTERM", os.environ.get("TERM", "xterm")), "-e", "foo"])
The colorterm var changes if you set a different preferred terminal in MATE, so I assume it does for every DE.

Pidgeot · « **Reply #114 on:** August 31, 2014, 01:20:29 pm »

Quote from: trauma on August 31, 2014, 12:49:09 pm

Code: [Select]
${COLORTERM:-${TERM:-xterm}} -e foo

The problem is currently solved by means of a shell script (xdg-terminal) which tries to detect it based on the DE, but I'm very much intersted in a better solution. Unfortunately, the environment variable you propose is only set if the program is already running in a terminal, and I can't guarantee that - so this one won't do, I'm afraid.

It's a good idea, though, and I will strongly consider incorporating it in the shell script for the cases where it IS already in a terminal... but that's a little further down the line.

trauma · « **Reply #115 on:** August 31, 2014, 02:58:49 pm »

Huh, I never knew. Good thing I tend to start things via terminal, then

.

I plan on upgrading my df and then I'll test PyLNP, particularly the gfx patching. Thank you for your good taste in languages at any rate, I was afraid I'd have to start hacking the Java launcher

.

Pidgeot · « **Reply #116 on:** September 07, 2014, 09:21:25 am »

Version 0.4 is now out, containing new features like support for version checking on DFFD, support for TwbT overrides during graphics pack installation,and allows PyLNP.json to be stored in the LNP folder (the old placement will still work, though, and will be used for binary builds). It also adds better error handling in various places, and fixes various bugs.

Also, starting with this version, graphics pack installation will always patch inits - the old method of overwriting is gone.

Additionally, some refactoring has been done on the GUI side to make the code cleaner, and hopefully a little easier to work with. More work still needs to be done in this part (and the main LNP functionality hasn't been worked over yet), but this seemed like a good time to get a build out.

PeridexisErrant · « **Reply #117 on:** September 13, 2014, 07:54:05 am »

Hi all! With help from Pidgeot, I've got a fork with a Mods tab. It can merge and load mods already, but needs both more work and more content before it's ready to merge back in. If you're interested in playing with it, testing, or making mods - come on over.

lethosor · « **Reply #118 on:** September 17, 2014, 02:43:55 pm »

The automatic update check appears to be triggering certain antivirus programs: http://www.bay12forums.com/smf/index.php?topic=126076.msg5667036#msg5667036

Pidgeot · « **Reply #119 on:** September 17, 2014, 03:58:29 pm »

Quote from: lethosor on September 17, 2014, 02:43:55 pm

The automatic update check appears to be triggering certain antivirus programs: http://www.bay12forums.com/smf/index.php?topic=126076.msg5667036#msg5667036

Everything points to a false positive. Not much I can do about that, I'm afraid.

News:

Author Topic: PyLNP 0.14e-pre1 - Cross-platform launcher with graphics pack patching (Read 325997 times)

Putnam

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

PeridexisErrant

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

lethosor

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

PeridexisErrant

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

Pidgeot

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

MagiX

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

lethosor

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

Pidgeot

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

trauma

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

Pidgeot

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

trauma

Re: PyLNP 0.3 - Cross-platform Lazy Newb Pack port with graphics pack patching

Pidgeot

Re: PyLNP 0.4 - Cross-platform Lazy Newb Pack port with graphics pack patching

PeridexisErrant

Re: PyLNP 0.4 - Cross-platform Lazy Newb Pack port with graphics pack patching

lethosor

Re: PyLNP 0.4 - Cross-platform Lazy Newb Pack port with graphics pack patching

Pidgeot

Re: PyLNP 0.4 - Cross-platform Lazy Newb Pack port with graphics pack patching