Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Arx]

The entelect challenge, for South Africans who want R100K or others who just want a challenge.

[Telgin]

I spent most of today trying to set up a development mirror of some of our websites on my local PC so I could do development on something other than a live site, and I'm starting to come to the conclusion that this is not how you're supposed to do this.

So, the theory of it all is that I can pull source files from our Git repository to my local computer, do any development work there, commit and push the changes, then run a local script to copy the modified files to the live server.  Right now we use Git for source control, but all of the editing is still done on live servers directly.  Any changes we make locally are automatically uploaded when saved.

I have Apache, PHP and MySQL all set up locally.  I've modified the Windows hosts file to allow me to view those files in my browser as if I was visiting http://dev.whatever.  Using Apache's mod_substitute, I can even in place replace URLs to seamlessly point to dev.whatever URLs without having to modify anything directly in the source.  Perfect.

However, trouble really starts when it comes to resolving PHP includes.  See... the files were originally on a Linux server and I have a Windows workstation.  They're all absolute paths (laziness related to how PHP resolves nested includes, mostly).  I managed to do some really scary trickery with symbolic links to make /something/somewhere paths look like local paths on Windows, and that worked.  But... only as long as PHP's include_path setting is being checked.  That works for require / include, but not opendir, which I also need it to work for.

Ideally there would be some way to preprocess the PHP files before they're served to PHP's handler in Apache.  That way I again wouldn't have to actually modify the files when serving them locally: I could just have something in my local virtual host to translate the paths before PHP saw them.  Unfortunately, after searching for a while I'm all but certain this isn't possible with Apache.

This sucks.  Right now I can only think of three ways around this:

1. My script for pulling the files down and syncing them can scan every last file for an absolute Linux path and replace it with a relevant absolute Windows path.  The push script would do the opposite.  The directories are very specific and predictable, so it's doable, but slow and annoying.

2. Switch to a Linux workstation.

3. Scrap the entire idea of doing this on my PC and set up separate dev sites on the server for every developer so we can actually work independently with source control.  This feels like it's defeating half of the purpose of Git.

I really don't want to do any of those.    It would be amazing if there was a simple way to do this directory path translation on the fly so I could use the unmodified source files on this computer.

[MorleyDev]

A good starting place may be to make a Linux VM that completely recreates the live environment. You can test in that VM on Windows, or create Linux workstations from it as needed. A VM is easy to share around to other developers too.

I somehow wrote 3932 lines of end-to-end tested code which included 862 unit tests assertions (as-in, several times the same code is repeated with different test cases)...in the last week.

...On the downside, I have consumed so much coffee in the last week I can see through the pale facade of time and into the beating heart of this cosmic dance of bursting decadence and withheld permissions as it twists all our arms collectively. But if caffeination can win, and it can, then I'll still be here tomorrow to high-five you yesterday, my friend. Peace.

[Skyrunner]

Threading issues are annoying me :| I'm this close to being able to make my web app version 1 go live, but something about the Python threading class and file writing is wonky. I'm using the thread for queue to log writing.

[Mephisto]

Threading issues are annoying me :| I'm this close to being able to make my web app version 1 go live, but something about the Python threading class and file writing is wonky. I'm using the thread for queue to log writing.

Unless you are rolling your own logging just to try it yourself, why? Python has logging in the stdlib. The drop-down in the upper left allows you to change Python versions.

[Lightningfalcon]

I'm currently writing a program for RSA encryption in Java.  How would you go about changing the file?  Specifically I need to turn it into a BigInteger somehow.   Just a name for what that process is called would be useful.   

[Fenrir]

I'm currently writing a program for RSA encryption in Java.  How would you go about changing the file?  Specifically I need to turn it into a BigInteger somehow.   Just a name for what that process is called would be useful.

This looks close to what you’re trying to do: http://stackoverflow.com/a/5083666 . I would have to do some more searching to be completely sure you can convert Byte to BigInteger in Java (haven’t used it in a bit).

[Skyrunner]

Threading issues are annoying me :| I'm this close to being able to make my web app version 1 go live, but something about the Python threading class and file writing is wonky. I'm using the thread for queue to log writing.

Unless you are rolling your own logging just to try it yourself, why? Python has logging in the stdlib. The drop-down in the upper left allows you to change Python versions.

Looks too featureful for me o.o So fancy, with events and severity and propagnation and stuff.

[Fenrir]

Threading issues are annoying me :| I'm this close to being able to make my web app version 1 go live, but something about the Python threading class and file writing is wonky. I'm using the thread for queue to log writing.

Unless you are rolling your own logging just to try it yourself, why? Python has logging in the stdlib. The drop-down in the upper left allows you to change Python versions.

Looks too featureful for me o.o So fancy, with events and severity and propagnation and stuff.

https://docs.python.org/2/howto/logging.html#a-simple-example

Doesn’t look like you have to even know most of that stuff is there to use it effectively. Also,  it’s thread safe (https://docs.python.org/2/library/logging.html#thread-safety). I assume that means it doesn’t fuck up things when you do thready things with it. (Yes, I know almost nothing about multithreading.)

[Telgin]

A good starting place may be to make a Linux VM that completely recreates the live environment. You can test in that VM on Windows, or create Linux workstations from it as needed. A VM is easy to share around to other developers too.

This isn't a bad idea, and I did briefly consider it.  Lots of stuff I'll need to figure out and experiment with to really take advantage of it though.  For example, I actually have no idea how to use Git on Linux directly, which is probably a lot easier than trying to mount a shared volume between the host Windows OS and the Linux VM anyway.  It'll also be interesting to see how networking works in the VM, such as configuring Apache.

Hmm, I just realized I've still got to figure out how to get the code to use the local database rather than the production database.  I'd love to do that magically somehow without having to modify the database connection stuff in the code.  Even though it's only located in one place, it feels like something I'll forget or mangle when pushing.  Not sure it's really feasible to do it any other way though.  I would try doing something with the firewall to reroute outgoing connections to the production server's IP to localhost, but then I need to connect to it with Workbench sometimes...

Oh well, sounds like a fun learning experience.  And an annoying learning experience sure beats what looks like insurmountable impossibilities doing it through Windows.

[Skyrunner]

Threading issues are annoying me :| I'm this close to being able to make my web app version 1 go live, but something about the Python threading class and file writing is wonky. I'm using the thread for queue to log writing.

Unless you are rolling your own logging just to try it yourself, why? Python has logging in the stdlib. The drop-down in the upper left allows you to change Python versions.

Looks too featureful for me o.o So fancy, with events and severity and propagnation and stuff.

https://docs.python.org/2/howto/logging.html#a-simple-example

Doesn’t look like you have to even know most of that stuff is there to use it effectively. Also,  it’s thread safe (https://docs.python.org/2/library/logging.html#thread-safety). I assume that means it doesn’t fuck up things when you do thready things with it. (Yes, I know almost nothing about multithreading.)

Ok, I ended up using it. And I got it working as I wanted it to in less than 30 minutes
Also, I learned that my web server framework (CherryPy) uses the logger, too. Much fun was had redirecting the log outputs so that cherrypy stuff was logged separately from my web app stuff.

[MorleyDev]

For example, I actually have no idea how to use Git on Linux directly, which is probably a lot easier than trying to mount a shared volume between the host Windows OS and the Linux VM anyway.

Hmm, I just realized I've still got to figure out how to get the code to use the local database rather than the production database.  I'd love to do that magically somehow without having to modify the database connection stuff in the code.

Using Git on Linux is about the same as using it on Windows through the command line. I almost exclusively use git through the command line nowadays, faster and more flexible and git is simple enough to get to grips with pretty quickly.

As for the database...hmm, you may need to do some config management. Load the connection string (of if you're using PHP, the details) from a config file and your deploy script swaps the "dev" config file for the "live" config file.

[Fenrir]

I’ve been trying to get application layer multicast to work with Flash and an RTMFP server. It’s weird; all the documentation I’ve read suggests that it should be simple, but it’s not working. Either it’s doing multicast on the IP layer or it’s not successfully traversing the NAT.

[Telgin]

Using Git on Linux is about the same as using it on Windows through the command line. I almost exclusively use git through the command line nowadays, faster and more flexible and git is simple enough to get to grips with pretty quickly.

I figured as much, so far my experience with the command line and Git has been limited to fixing the cases where TortoiseGit gets confused and screws up my local repos.  That's happened a couple of times: sometimes it'll think I made changes someone else made, even if I pull their changes, or it'll overwrite my committed changes if I pull.  I probably did something wrong the couple of times that happened, but being able to actually reset the repo would be nice.  I have actually have no idea how you do that in TortoiseGit.

As for the database...hmm, you may need to do some config management. Load the connection string (of if you're using PHP, the details) from a config file and your deploy script swaps the "dev" config file for the "live" config file.

This is about what I figured I'd have to do.  I was hoping there was some sensible way to use the exact same code on the development machine and live server, but in the end it's probably going to be more trouble than it's worth to do that for the database.  A single file outside of the normal document root that controls that should be good enough.

Thanks for all of the input.  I may get this development studio modernized yet. 

[MorleyDev]

Config management is one of the most frequent development-test-deploy problems with the least elegant solutions where elegance is often relative to the size and complexity of the entire system.

At my last workplace there was originally a monolothic "company.config" file that had the configuration for the every part of the system in it and was stored in a predictable location in the file system. Doing something like that may be the best pragmatic starting point.

This meant we could configure machines by just leaving different config files on them and treating that config as it's own project with it's own deployment process for each machine. And for smaller systems, that works pretty well.

But when the system grows to a certain point (as ours had) you have to start splitting the config down to project-by-project configurations just because the overhead of managing a monolithic config file becomes too stressful compared to storing easily findab;e "dev-config", "live-config" etc for each project.

We had done that to all the "main" components that 90% of development work was focused in, but hadn't reached the point where those 10% could be moved.