Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Janus]

Hey everyone, I'm the one who created and still administrate the DFFD site. The site has been down for several days, and I apologize for not posting sooner, but I've been looking into recovery options and determining the path forward, and what would be possible before making any sort of announcement. And so here we are. Unfortunately not the best news. But it could be worse.

In short:

• The site's server ran out of storage space. I'd thought we had a monitor in place to alert us on low disk space, but apparently not.
• The site database was damaged by this, and due to a further error on my part (which I'm continuing to kick myself for), was effectively lost. It might well have been recoverable initially, but it isn't now.
• I similarly thought we had automated backups in place, and I haven't been maintaining manual backups. It turns out we didn't.
• After talking with the server host and attempting several file recovery options for the database files, these efforts were ultimately, unfortunately, but unsurprisingly, not successful.

I take full responsibility. I absolutely should have made sure that backups were being maintained, and that there was a disk space usage alert in place.

So... all of the site files and user uploads are intact, but the rather important database component was lost. And the last backup I have of the database is from 2015, at the time of the transition to the new host. Very old. But perhaps a little surprisingly, it includes file IDs up through 10460. The files beyond that, up to file ID 16147, came later.

Spoiler: The plan, now implemented (click to show/hide)

All that being said, here is what I'll be working towards. I plan to get the site back up and running with the up-to-date site files and user uploads, and restore from the 2015 database (I know). Any user accounts created since then will need to be created again. And any passwords changed since then will be back to what they were at that time.
For file listings which existed then but had been updated since, the up-to-date user uploaded file will be available, but a fair bit of the info in the file listing will contain the info from back then, such as the description and file title. Also homepage links, comments, votes, view and download counts.
For all file listings which were created and files uploaded since then, restored file listings will be made available with only some info recovered from the JSON data for each file stored as static files outside the database. The download itself, the filename, last update time, and version info will be correct, as should be the author name and association to that user if their account existed back then. Other info remains lost.

The above is my plan, I have a fair bit of work to do to get there, but it should be entirely doable. It might take several days, particularly since I unfortunately can't devote all of my time to it at the moment. And since I'll be doing a full OS reinstall and going from there. I'm doing what I can.

All file listings which had their information lost (listing IDs 10461 through 16147) have been recreated as best as possible from available information which was outside the database. All of the user files themselves (for all file listings, not just those that needed to be recreated) are up to date and still available for download, and the last update time, file version, and DF version info are recovered. Likewise the name of the author. For user accounts that existed around the start of 2015, their orphaned file listings have been re-attributed back to those users; the rest of the orphaned file listings will only have the author's name indicated and won't be associated with an actual user account.
User accounts created since early 2015 were lost, and will need to be recreated by those users.

For help with recovering your orphaned file listings on a new account, see the Q&A in the post below.

Going forward, several automated full backups will be maintained for the server, and a monitor put in place for if/when disk space gets low at some point in the future.

I know all of this is far from ideal. Anyone who feels the need to vent, by all means do so. This was my responsibility.

[Janus]

If you want more information on what happened, scroll up to the post above.

For questions about recovering your account or file listings, see below.

Q: I think I created my account before early 2015, but I can't log in. Is my account lost?
A: You may have changed your account password since 2015, and an old password is on your account. If you can't remember what your password might be and can't get logged back in, you can always try to reset your password. If it indicates it can't find an account based on the username or email address you provided, your account was probably newer and was lost.

Q: My account no longer exists, but my orphaned file listings do. Can I regain control of them?
A: You have two options. The preferable option first.

• Create your new account, then use the contact form over on the DFFD site to point me to one of the file listings associated with your old account that you want to reclaim. Then provide a link to a page somewhere else on the internet (this forum, or the DF wiki, or elsewhere) where you have previously posted a link to one of those files on the DFFD. I may then contact you through that platform just for confirmation. If it all checks out, I'll re-attribute all file listings associated with your old account to your new account.
• You could simply upload your file to a new listing, and preferably notify me via a PM here on the forum or through the site's contact form that the old file listing is no longer needed. I can make the old file listing automatically redirect to your new one.

Q: What about the missing/outdated info on file listings? The title, description, homepage, ratings, and download and view counts?
A: For the title, description, and homepage link, it may be worth visiting the Wayback Machine to see if there's a backup of the file listing to copy info from. As for lost/inaccurate ratings, and download and view counts, those aren't recoverable.

Q: What about all of the missing comments and ratings? And can I get back my lost account?
A: Unfortunately, all of those newer than the start of 2015 are lost. They would need to be recreated / resubmitted by you.

[voliol]

This is of course a shame, but I suppose it by happenstance was not horribly timed. Since raw-related mods will need to be updated anyways for the new version, the descriptions of old ones breaking may not be the end of the world.

[Mr Crabman]

I suppose this mean a new user ID is going to be different probably? That's honestly the main part of this that I find bothersome (because my github repos link to the old one), since re-uploading the mods themselves will be simple enough for me.

[Immortal-D]

I had no idea that DFFD was a one-man operation, and for this long.  I thought for sure there was a team involved.  To be doing this as long as you have.. props.

[clinodev]

Oof! That's rough!

Thank you for all your efforts!

[brewer bob]

Thanks for the update on the situation and all the effort put into trying to solve it!

[TheLifeOfRyanB]

Me lying in bed on holiday wondering how many file ids can be identified by web scraping for dffd links+targeted Google search while not having the tools to hand to actually attempt this...

[Some Grue]

What kind of database was that? Do you have raw files from the corrupted base?
Once I had recovered almost 90% of Postgres db which I and my fellow admin had screw up (I made unintended SQLinjection with update query, he forgot to include this database in backups). Maybe not all is lost.

[Janus]

I suppose this mean a new user ID is going to be different probably? That's honestly the main part of this that I find bothersome (because my github repos link to the old one), since re-uploading the mods themselves will be simple enough for me.

If it's an account that didn't exist back in 2015, that will be the case.

What kind of database was that? Do you have raw files from the corrupted base?
Once I had recovered almost 90% of Postgres db which I and my fellow admin had screw up (I made unintended SQLinjection with update query, he forgot to include this database in backups). Maybe not all is lost.

It's MySQL. Believe me, I've exhausted every avenue of possible recovery for the database over the past several days. It's unfortunately toast.

[Some Grue]

It's MySQL. Believe me, I've exhausted every avenue of possible recovery for the database over the past several days. It's unfortunately toast.

Even carving data from raw files? If so, then all hope is lost

[Salmeuk]

please don't worry too much about this. it may have been a mistake, even a preventable one, but the service was freely provided for over a decade without hiccup. honestly it couldn't have come at a better time considering the new version is right around the corner!

if anything, I would put out a message to all account owners, with the above information. If the data remains but isn't accessible, it might be worthwhile simply starting from scratch while also dumping all that previous data in some kind of mega torrent, so people who have genuine interest in recovering their saves might dig through it.*


*I have no idea what I am talking about

do you have a patreon / donation link we could contribute to? I really appreciate DFFD as a service to the community.

[A_Curious_Cat]

Well… that explains everything.  Guess I’ll have to recreate my account, then.  Thanks for explaining what’s going on.



And remember,
Faith, Hope, and Backups,
But the Greatest of these is Backups.

[Janus]

do you have a patreon / donation link we could contribute to? I really appreciate DFFD as a service to the community.

Nah, no need, really. The site is (normally) extremely low maintenance for me, and Toady has been graciously handling the hosting since 2015, so he's shouldered all of the costs. If you want to support the site, just continue to support Toady.

[Gruust]

Wow. MySQL is that bad? Or is this a Virtuozzo VServer or similar/file system/underlying issue?

btw: it would be nice to have an optional "license" declaration on file uploads such that distribution/use rules are clear.