Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Cryxis, Prince of Doom]

Ptw

[My Name is Immaterial]

I'm working on a new prediction algorithm to figure out how many posts will be submitted in a day.

Based on historical data and current trends, I'd give 60/40 odds that today's (May 12) post count will be between 2089 and 2489.

The way my algorithm works is by taking the differences in post count day by day from the last 6 years, turning them into a percentage of that day's posts, then averaging 4 days worth of those percentages over 6 years. This is intended to help accommodate for seasonal shifts. Then that average is multiplied by the previous day's post count, and used to predict the coming day's post count.

I can give a +-500 range of posts with 97.6% certainty, and a +-200 range of posts with a 58.3% certainty.
This translates to a 98.4% likelihood of predicting it within +-25%, and a 62.99% likelihood of predicting it within +-10%
It should be mentioned that these percentages are based off a 127 test sample, which is pretty small. I also screwed up some numbers, and after rerunning them in hopes of making it better, my certainty dropped. Hmm. I might need to peel off the 2014 data; it's the wrench in the works.

TL:DR and in plain English: I can guess how many posts are going to be made, but I can't tell you with much certainty. I can give you a 1000 post range for (almost) certain, but so can anyone else who takes 5 minutes to look at the data. It's the fact that I can predict a ~500 post range correctly 60% of the time has me really excited. It may not sound like much, but it's a start!

It's still being refined, but I have high hopes! I don't see getting much more than 75% certainty with my more precise criteria, but that alone would amaze me.

There's no reason this couldn't be applied to the other data sets, but I started with Posts/Day, since it's a much more tangible metric.

[Cryxis, Prince of Doom]

This is almost encouraging me to post more or less by the end of the day or convincing others to do the same :p
Nah I'm not that mean

[My Name is Immaterial]

I'm not worried. It would take considerable effort to influence the numbers enough. You could do it with 10 reasonably active posters doubling their average count for a day, but that's not an easy feat.

[Zrk2]

-snip-

Huh. That's pretty neat. Is the data more or less exponential?

I'm not worried. It would take considerable effort to influence the numbers enough. You could do it with 10 reasonably active posters doubling their average count for a day, but that's not an easy feat.

Challenge accepted?

[My Name is Immaterial]

-snip-

Huh. That's pretty neat. Is the data more or less exponential?

Not sure what you mean. There are no exponents involved, if that's what you're asking.

[My Name is Immaterial]

My algorithm predicted that we would make ~2289, give or take 200, posts on May 12th, and there were...

*Drumroll*

2204! 85 posts off is 3.7% off! That's good!

Tomorrow's estimate is 2179 posts. I'm expecting a range of +-218 posts, since that's 10%.

[Orange Wizard]

Wow, cool.

[My Name is Immaterial]

1930 posts. Damn. That's 11% off, or 249 posts, but I was only 63% certain. I'll need to refine this, and compensate for different days of the week and months.

[mastahcheese]

These calculations excite me.

I demand more predictions.

[Pancakes]

*The following is a demented view on how I think stats work

  I suppose one could calculate the average growth or decay rate of posts of an individual, particular board, make a prediction based on previous trends of that board, and sum them all to get a more precise prediction?

  Also, it might make it more accurate to claim (for example) that the announcements board will have 1 new post per month, with 100% accuracy, but other boards, like...DF Adventure Mode Discussion, might not be as reliable for post count records. To get rid of that problem, I assume, day-by-day variance should be analyzed to see which boards are most faithfully being posted in.

And, uh.... STANDARD DEVIATION! Right? Am I statistics now?

[My Name is Immaterial]

These calculations excite me.

I demand more predictions.

I'm glad that they excite you! They excite me too!
May 14th's prediction is 1865. With 4 hours left in the day by the forum's count, and only 1662 posts made today, the prediction is probably going to a little low, but who knows?

  I suppose one could calculate the average growth or decay rate of posts of an individual, particular board, make a prediction based on previous trends of that board, and sum them all to get a more precise prediction?

Indeed you could, if the forum displayed stats like that. Sadly, it simply cumulates the data from all of the boards to give us blanket stats. If it gave individual board stats, then I would jump on that in an instant. I've looked for a way to make that work, but I've come back empty handed. This is what the stats look like, for reference.

  Also, it might make it more accurate to claim (for example) that the announcements board will have 1 new post per month, with 100% accuracy, but other boards, like...DF Adventure Mode Discussion, might not be as reliable for post count records. To get rid of that problem, I assume, day-by-day variance should be analyzed to see which boards are most faithfully being posted in.

I already track day-by-day variance to make these predictions, but again, being able to access the individual board stats would be amazing.

And, uh.... STANDARD DEVIATION! Right? Am I statistics now?

You're pretty close! I'm not using Standard Deviations in this, because I don't find it really useful. It might work better with that, though. I'll have to try it out. Thanks for the idea!

[TheDarkStar]

You might not be able to get stats for all the threads, but the postcounts for the ten most posted-in boards are available.

[My Name is Immaterial]

You might not be able to get stats for all the threads, but the postcounts for the ten most posted-in boards are available.

Yes, their total post count. However, I'm looking at daily posts, not total. THEORETICALLY, I could check them every 24 hours and figure the difference to get the daily posts in a board, but that's more maintenance work than I want to handle. Alternatively, it could be automated, but I don't have the skills to make that happen.

[penguinofhonor]

.