Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[salithus]

MountainHome - DwarfFortress Package Manager

Overview

Concept
The reason I started working on this project is due to the frequent requests for various graphics/mods/utilities to be packaged into the various, and excellent, Starter Packs that are out there. Unfortunately for the maintainers of those packs, adding more functionality to accommodate specific user requests comes at an increased file size for the users who are uninterested in the additions. For example, with PeridexisErrant's Starter Pack, I personally find myself only using DF, DFHack, Phoebus's Graphics, a handful of scripts/plug-ins, DT and QuickFort. All of the other tools and graphics he packages are things I never bother with because they don't fit my interests and play style, yet I download them every time SP updates. This project is intended to provide a way to break that mould.

For Users
For Windows users, I'm creating a client that lets them manage what tools they use and automatically flags things that are ready to update. It is meant as the "where do I go next?" for people hooked on the game with the Starter Packs who want to expand their horizons without getting continually lost in the rabbit holes of forum threads.

For Developers
For developers, I'm trying to re-architect the dependencies between different versions of utilities, graphics, mods, and other 3rd party inventions. The diagram below shows the models I'm using and their relationships, but essentially a Component, such as DF itself, DwarfTherapist, DFHack, a graphics pack, or any packable item, will have meta information such as forum/wiki links, support information, and of course a title/description. Each specific version of that Component is a Release which has a download URL and a list of Mirrors and Dependencies. A Dependency is a specification of a Component and an optional Minimum and Maximum version of that Component that is required to run the given Release. For example, the DFHack Component's latest Release is version 0.40.13-r1. It would have a Dependency on the DwarfFortress Component with a minimum AND maximum version of 0.40.13.

What this does is let me flag both what can upgrade and what the impact is if you upgrade a Component. For example, when DwarfFortress has a 0.40.14 Release, clients will be notified of this. If they use the Windows client I'm writing to upgrade, they'll be warned that the DFHack Component will be disabled due to incompatibility. This lets the user decide whether to push forward without DFHack, or wait. Let's assume the user waits - once DFHack has a Release compatible with DwarfFortress 0.40.14, if they were to upgrade DwarfFortress, they would also be prompted to update DFHack as well, potentially with settings in the background to automatically upgrade compatible Components

What this does for you is provide an API with all of this information. http://mountainhome.azurewebsites.net/help lists (and will have better documentation in coming days) what calls can be made and provides some information on the data structures used. All responses are JSON, so it should be easy to incorporate into any client you want - I just ask that you let me know that you plan to leverage this so I can keep you in the loop if I make any breaking changes.

Spoiler: UML Diagram (click to show/hide)

MountainHome Software

Website
The project's website, while in development, is located at http://mountainhome.azurewebsites.net. It does not do anything flashy except provide a database of the Components and Releases discussed above. Anyone can view this information, but you need to be a SuperUser to edit it. Please PM me if you are interested in helping populate this database, as right now only DwarfFortress itself is listed.

Windows Client
The current version of the Windows Client is available here: http://dffd.wimbli.com/file.php?id=9775 I need to do some additional write-up on how to use it, but the interface itself is very barebones right now. In summary:
* On first run, it will require you to set a root directory for DF applications. This is similar to the root folder that the Starter Packs use. If you choose a folder that already has a DF install in it, the client will automatically detect the latest installed version.
* The main window shows a table of known components (pulled from the website's API above). White rows are up to date. Red rows are mandatory upgrades (e.g. you do not have DwarfFortress installed at all in the folder you chose). Purple rows are optional updates (e.g. you have DF 0.40.12 installed and 0.40.13 is available).

Spoiler: Component List on First Run (click to show/hide)

Project Assets

Project Repositories
ssg_core
SteppingStoneGames is something I worked on a long time ago but never took anyplace. I'm reviving it, starting with a core set of functionality I can use across all my future projects. This package has base classes for objects/entities, extension methods, and utility classes I frequently find myself using.

MountainHome
This is the repository for this particular project. It includes

• SSG.MountainHome.Core - Portable class library with the base models used across the solution.
• SSG.MountainHome.DAL - Class library (non-portable) that leverages Entity Framework for the DAL layer.
• SSG.MountainHome.Web - MVC5/WebAPI hybrid project for hosting the project's website, database, and WebAPI.
• SSG.MountainHome.Client - Reference implementation for consuming the WebAPI framework. Also leveraged in the Windows client for communication with the website.
• SSG.MountainHome.UI.WPF - Windows client mentioned above, built on Microsoft's WPF functionality. This is not cross-platform, but I chose it to broaden my skills for work. If anyone is interested in adding a cross-platform client to the project, you are more than welcome to come aboard.

Project Builds
I'm using AppVeyor to automatically manage builds out of GitHub.

https://ci.appveyor.com/project/salithus/ssg-core

https://ci.appveyor.com/project/salithus/mountainhome

Project	Branch	Status
ssg-core	master
ssg-core	develop
MountainHome	master
MountainHome	develop

From Here
Cutting this short - I'll spend some time this weekend writing up a new roadmap as I've already hit some of the major milestones I was looking to achieve.

[salithus]

Change Log
Version 1.0.1.0: http://dffd.wimbli.com/file.php?id=9775
* Required setting (DF Applications Path) is presented on First Run.
* Validating added to settings page - all valid settings required before Ok is clickable (valid directory selected).
* Installed DF versions detected (only latest shown on main screen) for .34.11 and .40.10 through .13 (I'll add the rest later, it's based on SHA1 of Dwarf Fortress.exe).
* If no version in installed of DF, it's considered a "Mandatory" upgrade and highlighted red.
* If an version less than the latest central db version is installed (e.g. you have .40.11 vs the site which lists through .40.13), AND all dependencies are already installed (none for DF), it's considered "Can Upgrade" and highlighted purple.
* Launch button works (requires a component selected that has a non-null version, meaning it's installed) for DF-only (more to come).
* Upgrade button works for DF-only (downloads latest DF zip to {DF Applications Path}\temp and unzips to {DF Applications Path}\{zipfoldername}, e.g. "df_40_13_win".
* Added Release Notes and Change Log data to the Release model - in future iterations that will be displayed on a confirmation dialog before upgrading a component.
* Lots of framework stuff to handle Settings viewmodel, error handling on forms, etc.

Version 1.0.0.0:
* Pulls the component list and identifies the latest version from the MountainHome site.
* Links menu (95% stolen from LNP :ninja:)
* Help menu has a fancy About page, borrowed from this guy: http://visualstudiogallery.msdn.microsoft.com/040c0484-8176-4b53-98d6-aaba136d00e4
* Beginnings of a Settings menu (File->Settings) with a "DF Applications Path" (this will be the root folder to either detect what you have installed or start dumping things you want to install or both).

[Nopenope]

I like the idea but I hope the desktop client, if any, will be in a multi-platform language, i.e. not C#.

[CLA]

I've been wanting to suggest something like this for a long time. What about integration with dffd? Any plans to integrate it with existing package managers on linux (i.e. as a user defined repository)?
Really looking forward to this.

[salithus]

I like the idea but I hope the desktop client, if any, will be in a multi-platform language, i.e. not C#.

First pass at it will definitely be Windows-only. A big part of why I'm doing this project in the first place is to expand my skills for work and we're heavily invested in C#/WPF there.

However once I get through that hurdle, I'd like to either take what I've learned and fold it into the PyLNP project, or refactor those libraries into something supportable on Mono or something. Also I'm keeping it open source in case anyone gets ambitious enough to make the client cross-platform.

I've been wanting to suggest something like this for a long time. What about integration with dffd? Any plans to integrate it with existing package managers on linux (i.e. as a user defined repository)?
Really looking forward to this.

The only thing I've considered about DFFD is having links to the file downloads there. If there's more that can be integrated I'm definitely open to the idea. As far as Linux package managers - I have no idea even how to get started doing that. If you or anyone has some reading material for me on it, I can add it to the wishlist.

[milo christiansen]

So... This is some kind of update server?

It looks like you are saying that items will be added to your database and then streamed to clients to allow automatic updates of starter packs and the like?

If so this could be useful for a bunch of things.

What about third-party clients? Could I connect a custom client to your server and get a list of all packages that have certain "tags" (or something like that)? Because that could be useful for streaming Rubble addons and/or updates.

[salithus]

So... This is some kind of update server?

It looks like you are saying that items will be added to your database and then streamed to clients to allow automatic updates of starter packs and the like?

If so this could be useful for a bunch of things.

What about third-party clients? Could I connect a custom client to your server and get a list of all packages that have certain "tags" (or something like that)? Because that could be useful for streaming Rubble addons and/or updates.

That's the idea, and yeah - I hadn't considered the tagging concept yet, but I'll add that to the list.

E: I'm also going to look at http://signalr.net/ for pushing out updates to clients that want push notifications....sometime in the future.

[salithus]

Version 1.0.0.0 of the Windows client is out: http://dffd.wimbli.com/file.php?id=9775

Key features:
* Pulls the component list and identifies the latest version from the MountainHome site.
* Links menu (95% stolen from LNP :ninja:)
* Help menu has a fancy About page, borrowed from this guy: http://visualstudiogallery.msdn.microsoft.com/040c0484-8176-4b53-98d6-aaba136d00e4
* Beginnings of a Settings menu (File->Settings) with a "DF Applications Path" (this will be the root folder to either detect what you have installed or start dumping things you want to install or both).

Also, in the MountainHome repo there is a new SSG.MountainHome.Client project that provides a C# Portable Class Library for consuming the Web API provided by the site.

[salithus]

Version 1.0.1.0 of the Windows client is out: http://dffd.wimbli.com/file.php?id=9775

Key features:
 * Required setting (DF Applications Path) is presented on First Run.
 * Validating added to settings page - all valid settings required before Ok is clickable (valid directory selected).
 * Installed DF versions detected (only latest shown on main screen) for .34.11 and .40.10 through .13 (I'll add the rest later, it's based on SHA1 of Dwarf Fortress.exe).
 * If no version in installed of DF, it's considered a "Mandatory" upgrade and highlighted red.
 * If an version less than the latest central db version is installed (e.g. you have .40.11 vs the site which lists through .40.13), AND all dependencies are already installed (none for DF), it's considered "Can Upgrade" and highlighted purple.
 * Launch button works (requires a component selected that has a non-null version, meaning it's installed) for DF-only (more to come).
 * Upgrade button works for DF-only (downloads latest DF zip to {DF Applications Path}\temp and unzips to {DF Applications Path}\{zipfoldername}, e.g. "df_40_13_win".
 * Added Release Notes and Change Log data to the Release model - in future iterations that will be displayed on a confirmation dialog before upgrading a component.
 * Lots of framework stuff to handle Settings viewmodel, error handling on forms, etc.

[salithus]

I'll be updating the OP tomorrow sometime (exhausted and about to crash), but wanted to highlight the changes I got through today:

* Enabled code-based database upgrades, so the database is now stable.
* Updated AppVeyor process to generate a build of the Windows client for me - this changes versioning to stay in sync with AppVeyor, but that's ok. (no changes to the client yet)
* Registration should work with basics (still need to add OAuth for Google/Facebook/whatever else anyone would like to register with, plus email verifications).
* Viewing data does not require login, creating/editing/deleting requires SuperUser role (PM me if you're interested in helping manage the database).
* Updated the default main, About, and Contact pages to be project-specific (mimics a lot of content in the OP).
* Updated the views related to Components to be more user-friendly.

Keeping up to date with DF itself from the client is as good as it gets for now - it would be nice to add some automation on the website backend to detect new DF versions as they're released and populate the db, but that's a task for another day.

I would really like to tackle Graphics packs next, but would like permission from the pack maintainers before I add their work to the database since that will mean additional bandwidth from their host while I test things, plus future bandwidth assuming this client or similar ever get widely adopted. If anyone wants to volunteer, it's appreciated, otherwise I'll be sending out PMs asking for permission tomorrow.

Failing that, I think I'll tackle Utilities next, starting with DFHack and maybe the Launchers, although that can get complicated.

Also I know from the DFFD page that at least someone besides me downloaded the client - so any feedback on it, or any feedback on the site or API would be greatly appreciated.

[Meph]

I think you need to explain a few things a lot simpler.

1 Website/database of "packages" (called Components)
2 API for reading from this database
3 Desktop client for streamlining updates to DF and 3rd party utilities.

1 What is the website going to do/be for? Whats in the databases, or whats inside the packages/components?
2 Whats does API stand for?
3 What is a desktop client? Do you mean streaming? Or streamlining?

Remember this:

Spoiler: How IT professionals see End Users (click to show/hide)

"We've tried nothing and we are all out of ideas!"

In short, everything you write here is perfectly understandable if your audience consists of developers/programmers, but no one else understands a word.

[salithus]

I think you need to explain a few things a lot simpler.

1 Website/database of "packages" (called Components)
2 API for reading from this database
3 Desktop client for streamlining updates to DF and 3rd party utilities.

1 What is the website going to do/be for? Whats in the databases, or whats inside the packages/components?
2 Whats does API stand for?
3 What is a desktop client? Do you mean streaming? Or streamlining?

Remember this:

Spoiler: How IT professionals see End Users (click to show/hide)

"We've tried nothing and we are all out of ideas!"

In short, everything you write here is perfectly understandable if your audience consists of developers/programmers, but no one else understands a word.

Doing a rewrite now BUT I'm not really ready for end-user testing yet. All the current Windows client does is install/launch DF and let you know when upgrades are available, which I don't see anyone bothering themselves with yet. I'll keep that in mind while I'm rewriting though - hopefully the next iteration is a bit better/clearer. Cheers!

E: OP updated - feedback appreciated!

[Meph]

Neat. Now I understand what you are talking about.

Since the Starter Packs (both PeridexisErrants and the older Lucas_NP one) are distributed in 'General Discussion' and on reddit, and not the modding section, maybe you can find more people that are interested and willing to provide testing and feedback there. Maybe posting a link in the Starter Pack thread would help.

[salithus]

Neat. Now I understand what you are talking about.

Since the Starter Packs (both PeridexisErrants and the older Lucas_NP one) are distributed in 'General Discussion' and on reddit, and not the modding section, maybe you can find more people that are interested and willing to provide testing and feedback there. Maybe posting a link in the Starter Pack thread would help.

Eventually yes, right now I'm focused on getting the interest of the people who frequent this forum - the package (Component) developers themselves. Ideally I'd like to get a handgful of graphics packs, DFHack, DT, and a mod or 3 set up on this system as a proof of concept before rolling it out to General Discussion.

[salithus]

Did some tweaking to the UI and added this screenshot to the OP:

Spoiler: Component List on First Run (click to show/hide)

Also did some major work on the website, so it's a lot prettier now.

Spoiler: Main Page (click to show/hide)

Spoiler: Components List (click to show/hide)

Spoiler: Components Detail (click to show/hide)

Spoiler: Components - Create Screen (click to show/hide)

Spoiler: Components - Edit Screen (click to show/hide)

Spoiler: Components - Delete Screen (click to show/hide)