Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Untelligent]

To encourage people to use the torrents, though, I would recommend a small fee per download.

That's hardly fair to people who can't torrent or don't know how to torrent or what a torrent is.

[BoredVirulence]

I agree that moving towards torrents, particularly for the large mods, would be very helpful as far as bandwidth is concerned, but we can hardly rely on it now, its an optimization for later, not a solution.

Getting payments involved is counter productive. It would only harm the community.

I think the best solution, for now, would be to downsize. As unfortunate as it may be, the larger mods could find separate mirrors. It wouldn't be ideal, but it could bring the price down to a more reasonable level. Perhaps then torrents could be introduced, and we could move the large mods back. With subtle design cue's, you should be able to find a way to make torrents more appealing, without removing the direct download option.

[breadman]

To encourage people to use the torrents, though, I would recommend a small fee per download.

Putting a paywall on, effectively, the least-viewed files would be a strong disincentive to download someone's save just to help them fix their minecart setup. The community would suffer, I think.

I'd hope that the server would be able to be the initial and permanent seed, to prevent the complete inability to get the file without paying for it.  Yes, this means the server still has outgoing bandwith costs, but they should be reduced by an order of magnitude.  Incidentally, that's why my proposals for the fee are so much higher than the actual bandwidth cost for the file in question.

That's hardly fair to people who can't torrent or don't know how to torrent or what a torrent is.

To those who can't torrent, true.  Then again, that's why the fees are so low.

To those who refuse to learn how, I have no sympathy.  It's not that much harder to download through a torrent program than to do so directly through the browser, though it can certainly be slower, and the DFFD site could provide some basic guidance near the download button.  Note that I wouldn't expect each uploader to learn how to host the torrent on their own.

I agree that moving towards torrents, particularly for the large mods, would be very helpful as far as bandwidth is concerned, but we can hardly rely on it now, its an optimization for later, not a solution.

True.  Though it could possibly be done within a week, it doesn't address the principal problem of the moment, which is that everything needs to be moved somewhere else.  It merely means that the bandwidth could potentially be dramatically reduced, which might be helpful in comparing options.

In addition, a payment incentive can wait until after the torrent modifications, and could even be skipped if costs drop to a reasonable level.

[mifki]

To encourage people to use the torrents, though, I would recommend a small fee per download.

Putting a paywall on, effectively, the least-viewed files would be a strong disincentive to download someone's save just to help them fix their minecart setup. The community would suffer, I think.

I'd hope that the server would be able to be the initial and permanent seed, to prevent the complete inability to get the file without paying for it.  Yes, this means the server still has outgoing bandwith costs, but they should be reduced by an order of magnitude.  Incidentally, that's why my proposals for the fee are so much higher than the actual bandwidth cost for the file in question.

That's hardly fair to people who can't torrent or don't know how to torrent or what a torrent is.

To those who can't torrent, true.  Then again, that's why the fees are so low.

You can't be serious about the fees, huh? I hardly know any paid file hosting service, but paid downloads for a free game is a really strange idea.

[Footkerchief]

You can't be serious about the fees, huh? I hardly know any paid file hosting service, but paid downloads for a free game is a really strange idea.

This part is all that matters:

I agree that moving towards torrents, particularly for the large mods, would be very helpful as far as bandwidth is concerned, but we can hardly rely on it now, its an optimization for later, not a solution.

True.  Though it could possibly be done within a week, it doesn't address the principal problem of the moment, which is that everything needs to be moved somewhere else.  It merely means that the bandwidth could potentially be dramatically reduced, which might be helpful in comparing options.

In addition, a payment incentive can wait until after the torrent modifications, and could even be skipped if costs drop to a reasonable level.

Let's move on so the thread can focus on the immediate problem.

[Nokao]

As Meph wrote, problems are "popular" packages,
like Masterwork DF, Lazy Newb / Starter Pack, etc.

They can smoke up 20-40 GB of traffic every day, that for my hosting company is: 1-2 $ every day

Hosting companies are changing philosophy,
providers with no limit in upload bandwidth doesn't exist anymore,
and the few that exist will write an email like "WTF you consume 40 GB a day? I'm raising your costs".

I'm searching for a few options to get slower but cheaper services,
but I think that file sharing option is really the best.

If this game was open source, sourceforge could have been a great partner.
But maybe we can ask for help to some Indie games community ?
Like ... making a partnership and asking for co-hosting with them ?

p.s. as I promised before, I can still offer my help if the price is < 1000$/year.

[lethosor]

SOLUTION 1:
Who'se managing the domain http://dwarffortresswiki.org ? Is it Toady One owned+mantained ?

Everything DF-related have "an echo" inside http://dwarffortresswiki.org
So why not attaching files directly there ? It's easy, we could restrict big file upload for Mod-Creators and certain kind of users.

If it's owned+mantained by Toady One, I could offer my help to him as a web developer to make this transformation,
and make a yearly donation to him (paying the hosting) to support his increase of expenses because he'll upload many new files.

The wiki's maintained by lethosor, I believe.

The only sites owned/maintained by Toady are www.bay12games.com and www.bay12forums.com.

Locriani (Briess on the wiki) is the one who manages the servers, actually.
As far as I know, most (or all) of the web servers are currently on a 1 GB plan, while the database server (and possibly the auxiliary server) are on higher-tier plans. I doubt we have 250 GB and terabytes of bandwidth available, or that our current ad setup would be sufficient to offset the costs, unfortunately.

[mifki]

p.s. as I promised before, I can still offer my help if the price is < 1000$/year.

Maybe I missed it, but what does this mean in terms of monthly traffic?

Oh, silly me! There's a page with download statistics http://dffd.wimbli.com/rankings.php?sort=file_downloads_per_day&limit=100 (daily counts are approximation, but at least something).

For the most popular 100 packages it gives 107GB/day or 3.3TB/month. Not that much, actually.

[Nokao]

For the most popular 100 packages it gives 107GB/day or 3.3TB/month. Not that much, actually.

It's 2500 $ / year for me, and I'm in Europe not USA.
This is why I'm trying to figure out another way.

For example, this: http://virpus.com/linux-vps.php

[Janus]

A request to people hosting files, please don't change your file hosting behavior at the moment if possible; for the next few days I'd like to get a view of what kind of traffic the site is getting now.

Also, just so it's clear, I have no plans on my part of ever charging for access to upload or download, or really charging for anything else for that matter.

Maybe I missed it, but what does this mean in terms of monthly traffic?

Oh, silly me! There's a page with download statistics http://dffd.wimbli.com/rankings.php?sort=file_downloads_per_day&limit=100 (daily counts are approximation, but at least something).

For the most popular 100 packages it gives 107GB/day or 3.3TB/month. Not that much, actually.

That's a nice thought. That's valuable knowledge, though it's good to keep in mind that it's an average over the life of the file listings, while the monthly total fluctuates a great deal on months of major releases or greater publicity. Still, a Linode plan might possibly end up being viable. I checked through their terms of service and they don't appear to have anything prohibiting file hosting sites. The concern would still be for that one potential month where you get a huge influx of bandwidth usage, and that month's bill ends up being astronomical due to the overage charges, making the plan more costly in the long run.

[mifki]

The concern would still be for that one potential month where you get a huge influx of bandwidth usage, and that month's bill ends up being astronomical due to the overage charges, making the plan more costly in the long run.

On the other hand, hopefully peopel will use torrents or other file hosting services more often. So I'd suggest starting with 8TB Linode plan. We don't have a lot of other options and time left, and neither better  traffic usa data for peak months. Of course if my calculations are correct.

What was the (possible) deal with Toady again - did he have a server or was just going to pay?

[Reelya]

I agree that moving towards torrents, particularly for the large mods, would be very helpful as far as bandwidth is concerned, but we can hardly rely on it now, its an optimization for later, not a solution.

Getting payments involved is counter productive. It would only harm the community.

I think the best solution, for now, would be to downsize. As unfortunate as it may be, the larger mods could find separate mirrors. It wouldn't be ideal, but it could bring the price down to a more reasonable level. Perhaps then torrents could be introduced, and we could move the large mods back. With subtle design cue's, you should be able to find a way to make torrents more appealing, without removing the direct download option.

Just put a timer or queue on larger downloads, or ration the amount of data a single IP address can download in a 24 hour period.

[Meph]

A request to people hosting files, please don't change your file hosting behavior at the moment if possible;

Ok.

[mndfreeze]

What about moving just the file side of things to MEGA?  Its Kim dotcom's new version of megaupload, due to their past with the old megaupload they might have better options or prices then standard providers.


Here is a quick comparison I found on wiki abotu file hosting providers.  I'm sure not all info is accurate and up to date, but it might help narrow down an affordable option.


http://en.wikipedia.org/wiki/Comparison_of_file_hosting_services

[Thief^]

What's the pricing like on Amazon S3? It even serves torrents of all the files you store on it, to save on bandwidth

There's also google cloud storage now as a competitor.

EDIT: Looking back I see someone ran the numbers, but it was assuming http bandwidth, not people using torrents. I guess we need a hard number about how much bandwidth DFFD uses! You could also consider splitting the file storage between a high-storage low-bandwidth service and a low-storage high-bandwidth service (like a VPS) based on how much the file is accessed, which might get you the best of both worlds if only a few files are responsible for the majority of the bandwidth.

EDIT2: Is there a javascript bittorrent client? Then people wouldn't even need torrent software.