Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Zathras]

Just keep a running tally of quotes- have a counter that increments on every {quote} and decrements on every {/quote} (except square brackets, obviously).  If the counter isn't 0, ignore the red text.

It doesn't work like that: the script reads the html, not the bbccode, which means it's not nicely ordered {quote}{/quote} pairs, but deeply nested <divs> and <spans> with endless crap in between, including different "bbc_quote" styles for nested quotes (they have a different background colour, which is reflected in the html), and for things like quotes with links to profiles, links to posts, or no links, and so on, all in a single line (the html for the contents of a post is, internally, stored in a single, very long line).

Also, I don't walk the text, perl walks the text; for me to keep a counter, I'd have to walk the text myself, which defeats the purpose of using perl for it... perl is really good and smart about reading text the same way you and I do: top-down and letft-right. Matching quote pairs requires it to read inside-out instead, for which I think the answer is recursion -- I have a couple of ideas to try that, but won't have time to tinker until the weekend or thereabouts; we'll see how it pans out.

I do know that my current approach for the redtext is not the right one, just because of how convoluted it is. For comparison, it takes one line of code to find the post author, process replacements, and store it; it takes one line to find the reply number and date, and one more line for the url, but finding the redtext takes five lines at the moment, and it doesn't quite work. No, the answer is simpler, and more elegant, I just haven't found it yet. Or maybe I have, just need time to give the latest hare-brained ideas a whirl.

On top of that, there are also other inexplicable bits in how the forum creates the html. For example, in Salad, this post was missed by the finder; I noticed it and put it in manually, but the reason it missed it is because the colour is not "red", as all other red text is, but is instead "#ff0000"; I don't know why that is, but maybe the edit had something to do with it. I'm OK with missing edited posts (they are illegal, after all), or with putting an exception for "#ff0000", but it's just an example of how nutty the unexpected bits of the problem sometimes are.

Oh well, it's not like it's the launch codes for the nuclear warheads or anything; if it works, it works. I'll have something new, and hopefully better, some time next week. But thank you for the discussion, it helps.

[Toaster]

Do you have an updated version of the source?  I could eyeball it to see if I notice anything, though my Perl isn't the best.

[Zathras]

I think I found it. It was simple really:

Code: [Select]

while ( /(.*)<blockquote.*?<\/blockquote>(.*)/ ) { $_ = $1 . $2 ; };...should strip off quotes correctly in pairs from the inside-out, and give the remainder (original text only) to the finder. I just made a couple of tests, and I can't find anything it missed. I'll posted a new one to Salad today.

Yes, I'll update the source in the OP unless I can find any troubles remaining in the finder. I would like to add your sorting suggestion first, though, but I can't do that for the next couple of days. In the mean time, the text finding part of the source is below (which is the important bit, really, the rest is just juggling what it found and calculating lurkiness).

Spoiler: Finding section (click to show/hide)

Code: [Select]

#  I - Find the stuff

while (<STDIN>) {

# find post author, and increment index
/<h4><a href=.*>(.*)<\/a><\/h4>/ && ( $posts[++$i][0] =
$replace{$1} ? $replace{$1} : $1 ); # Change to replacement if any.

# find post url, as fixed by preprocessing script posturlfix.pl
/<h5.*url=(.*?)>/ && ( $posts[$i][1] = $1 );

# find post number, date and time.
/<strong>Reply #(\d*) on:<\/strong> (.*?) (\d{2}):(\d{2}):\d{2} (am|pm).*&#187/ &&
( ($posts[$i][2], $posts[$i][3], $posts[$i][4], $posts[$i][5], $posts[$i][6]) =
($1, $2, $3, $4, $5) );

# prevent signatures and quotes with messing the red text finder
next if /<div class="signature".*\/div>/;
while ( /(.*)<blockquote.*?<\/blockquote>(.*)/ ) { $_ = $1 . $2 ; };

# find red text, record it and the url in $votes{dude}
@reds = /<span style="color: (?:red|\#ff0000);" class="bbc_color">(.*?)<\/span>/g;
( ($votes{$posts[$i][0]}[0], $votes{$posts[$i][0]}[1]) =
  ($reds[$#reds], $posts[$i][1]) ) if @reds;
};


[Zathras]

I've added the sorting suggestion. Alive first, then dead, then nonplayers/mods. I like that better than the colour coding, and should be readable enough, I think. There may be an improvement in sorting the dead by alignment, but we'll see about that later. I've also eliminated the last post datestamp, as it really didn't add much information: you only care how long ago it was, and if you really want to look, the URL is right there at the end of the line.

Since I've not found problems with the redtext finder, I'm updating the source of the OP to the latest version, but still let me know if you see red text incorrectly pegged.

This is what it looks like right now (data from Paranormal17, urls are empty as it's just for show):

LurkerTracker

Argembarger has 30 posts; the last 3 hours ago. Red text: Dariush. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30]
Dariush has 30 posts; the last 3 hours ago. Red text: Kamina:. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30]
JanusTwoface has 52 posts; the last 1 hours ago. Red text: Ottofar. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52]
Jim Groovester has 116 posts; the last 0 hours ago. Red text: Ottofar. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116]
KaminaSquirtle has 87 posts; the last 1 hours ago. Red text: Ottofar. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75][76][77][78][79][80][81][82][83][84][85][86][87]
Leafsnail has 46 posts; the last 0 hours ago. Red text: unvote. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46]
Mysteriousbluepuppet has 32 posts; the last 0 hours ago. Red text: kamina. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32]
Ottofar has 21 posts; the last 2 hours ago. Red text: Kamina,. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21]
Toaster has 42 posts; the last 1 hours ago. Red text: Ottofar. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42]
Zathras has 58 posts; the last 0 hours ago. Red text: Vote Dariush.. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57][58]
webadict has 37 posts; the last 0 hours ago. Red text: Mish. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37]

Criptfeind: dead, town medium has 28 posts; the last 336 hours ago. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28]
Mr.Person: dead, town agent has 19 posts; the last 215 hours ago. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19]
Org: dead, town medium has 18 posts; the last 454 hours ago. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18]
Pandarsenic: dead, tough doppelganger has 29 posts; the last 215 hours ago. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29]
SHAD0Wdump: dead, town telepath has 48 posts; the last 336 hours ago. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48]

Moderator has 51 posts; the last 0 hours ago. Posts: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51]
Nonplayer has 2 posts; the last 499 hours ago. Posts: [1][2]
Replacements considered: Eduren => Argembarger; Mish => Dariush; Solifuge => Dariush.
The Morgue: Criptfeind: dead, town medium; Org: dead, town medium; Mr.Person: dead, town agent; Pandarsenic: dead, tough doppelganger; SHAD0Wdump: dead, town telepath.

I think it's in a pretty good state as of now, can't think of much more to tinker with. Suggestions and ideas are always welcome.

[Vector]

I think it might be better to sort the dead players by death order, and within that by alignment, then name.

Just a thought.

[Argembarger]

The LurkerTracker is getting more and more organized and elegant. Huzzah!

[qwertyuiopas]

I have a few scripts to share: A perl script that parses a folderfull of HTML files and gathers a list of post IDs, and a javascript that runs in IE6 and, using that list, uses the forum's own inline quote system to get the original BBCode(in quote format, so it has the time/date in it already). The perl search could be combined with the javascript, by converting the javascript portion to perl, to get the raw code for each post.

Spoiler: perl (click to show/hide)

Code: [Select]

use strict;
use warnings;

undef $/;
my $dir;
opendir $dir, "./in";
my $filename;
while($filename=readdir($dir))
{
    if($filename ne "." and $filename ne "..")
    {
        print "./in/$filename\n";
        my $in;
        open($in,"<","./in/$filename");
        my $contents = <$in>;
        while($contents =~ /\<div class\=\"inner\" id\=\"msg_(\d+)\"\>/)
        {
            print "$1\n";
            $contents = $';
        }
        print "~~~~~\n\n"
    }
}
closedir $dir;

./in/1.htm
231089
231167
231180
231189
231191
231194
231206
231209
231259
231266
231267
231290
231291
231299
231315
231318
231347
231349
231564
231633
231689
231710
231720
231734
231741
231752
231867
231870
231879
231891
231892
231897
231942
231993
232012
232049
232073
232221
232268
232272
232466
232468
232485
232495
232542
232560
232574
232643
232879
232913
~~~~~

Spoiler: javascript (click to show/hide)

Code: [Select]

<script language = "javascript">
function con()
{
    try{return new ActiveXObject("Microsoft.XMLHTTP");}catch(e){}
    return new XMLHttpRequest();
}

function quote(num)
{
    r=con();
    r.onreadystatechange=quoteReady;
    r.open("GET","http://www.bay12forums.com/smf/index.php?action=quotefast;quote=" + num + ";xml;pb=message;mode=0",false);
    r.send(null);
}

function quoteReady()
{
    if(r.readyState == 4 && r.status == 200)
      output.value+=r.responseText;
}
var quotes;
var locked=0;
function parse()
{
    if(locked)
      return;
    quotes = input.value.split("\n");
    parseline(0,quotes);
}
function parseline(i)
{
    remainder.value="" + i + "/" + quotes.length;
    if(i>=quotes.length)
    {
        locked=0;
        return;
    }
    var q=("" + quotes[i]);
    if(q.charAt(0)==".")
    {
        output.value+="\n\n~~~~~{"+q.substring(0,q.length-1)+"}~~~~~\n\n";
    }
    else
    {
        var str=q.replace(/\D+/,"");
        if(str.length > 0)
        {
            quote(str);
            output.value += "\n~~~~~~~~~~\n";
        }
    }
    setTimeout("parseline("+(i+1)+")",200);
}
</script>

<textarea rows="10" cols="60" name="input"></textarea><br>
<input type="button" value="quote" onClick=parse()>
<input type="button" value="clear" onClick="input.value='';output.value=''">
<textarea rows="1" cols="40" name="remainder"></textarea><br>
<textarea rows="20" cols="120" name="output"></textarea>

Code: (sample output (only one post, because the set of post numbers I already had is mainly spoilered, and that interferes terribly with the spoiler in my post. Accursed forum software and it's lack of nested spoilers!)) [Select]

~~~~~~~~~~
<?xml version="1.0" encoding="ISO-8859-1"?>
<smf>
<quote>[quote author=Sir Edmund link=topic=21545.msg232560#msg232560 date=1217067818]
My char is just going to sleep in the metal doored room for a couple of hours trying to make as little noise as possible.

goals: erm...waking up refreshed in a couple of hours (sleep for a turn though i have no alarm)

[/quote]</quote>
</smf>


Edit: Why the javascript runs in IE6: Because, IMO, IE8 < IE7 < IE6, and firefox unconditionally prevents cross-site scripting, or at least it did when I first wrote the scripts. Plus, I was using it to get an archive of some locked topics, where there is no quote link, because that would lead to the post reply page, but quoting is fine if you imitate the inline quoting procedure.
Further edit: The locked topics were in a semi-hidden section, so it needed my login cookie to access it, mandating a browser-based script, thus javascript, thus an IE browser, thus IE6... Would be best to translate the javascript into something else.

[Zathras]

Interesting, thanks for sharing. What is the purpose of the script, though? Is it to produce quotes for posts in locked topics? Is it to recover BBCode from the posts' HTML? Does it handle BBCode for nested quotes, colours, text formatting and so on? Under what circumstances would you use it?

The IE6 dependency makes it somewhat limited for those of us of the Linux persuasion, though, or for those that let IE6 die a well deserved (and way overdue) death...

[Argembarger]

Thanks are due to Zathras, for stepping in to help me help him find the gremlin in his code,

LurkerTracker is now confirmed stable under Windows Vista 64 Bit Home Edition using Strawberry Perl 5.12.2.0

Hahaha.

Time to idly tinker with this script in my spare time between frantically dashing to classes and writing essays

[Zathras]

Yep. Arg will now be able to spam trackers everywhere, not just me. Lurkers beware!

The source in the Original Post has been updated with that small regexp fix we found.


There are a few things that could be improved. Some easy and some hard. I'll list the ones I know here as reference, but please add if you think of ways it could be better. The easy ones will be done when I have time to tinker (not soon); the hard ones would probably require a major rewrite, so I wouldn't hold my breath... they may never come.



Easy

* The Red text field should be only specified if there's actual red text to show; bits like the underlined should be avoided:
The Captain has 3 posts; the last 10 hours ago. Red text: . Posts: [1][2][3]

* Learn about plurals. Be smart about "has 1 post" vs. "has 5 posts" in the player list, and "Post" vs. "Posts" in the url list as needed. Maybe in the autoprod as well "prod Dude. They haven't posted in so many hours"... although "they" has the benefit of being gender neutral; I dislike both "he/she" or assuming we're all dudes using "he". Maybe it should use "please prod Dude and Bloke. They haven't posted..." when plural and "please prod Dude, who hasn't posted..." when singular.

* Add an exception similar to Moderator for ICs in Beginner Mafia games. IC posts should not track red text, not be prodded, be marked as "IC" instead of player name(s), and be sorted with the Mod rather than with the players.

* Currently things like replacements/deaths, time to trigger a prod and timezone are controlled by directly editing the source. I'd like to find a smarter way, like a configuration file that you could use as parameter or something, so you could have a number of games to track with just a parameter change instead of tinkering with the source every time. However, configuration files are a chore, and may have platform-dependant considerations or other complications. I want to do this, but I'm lazy.




Hard

* Getting the pages in real time instead of having to save them to disk first.
* Doing away with the preprocessing step.
* Learning about day/night cycles. Know when days end so the morgue could be ordered chronologically, votes be reset at day start, and so on. Getting ambitious, this could also track lynched vs. night-killed vs. other fate/status, or give a list of who was voting a lynched player at day end. But this one is really hard since day/night, votecounts, flavour and so on are unpredictable and change wildly from game to game or mod to mod.
* Learn to deal with resurrections and/or role changes.
* Learn how to deal with a dead player replacing back in for someone else, keeping the past self posts separate from the new identity, and an entry in the morgue for the roleflip that did happen.
* Learn a bit about alignments so town/scum could be automatically coloured green/red or something. Difficult for third parties, or nonstandard games.
* Install the telepathic user interface, and enable it to whiten your teeth while you sleep.



Anything else? Wishing is free....

[Argembarger]

There could be a few ways some of those Hard goals could be implemented, but all the ones I can think of involve direct and purposeful moderator involvement (such as a key word or phrase that the lurkertracker recognizes but which will not come up naturally anywhere else) or the LurkerTracker's end-user dialing in that information directly.

The last easy goal, and hard goals #1 and #2 would be absolute blessings for the end-user

I would say focus on those first, and then once the LurkerTracker is easy, fun and safe to use, then work on the other goals (mostly because the other goals, especially

* Learning about day/night cycles. Know when days end so the morgue could be ordered chronologically, votes be reset at day start, and so on. Getting ambitious, this could also track lynched vs. night-killed vs. other fate/status, or give a list of who was voting a lynched player at day end. But this one is really hard since day/night, votecounts, flavour and so on are unpredictable and change wildly from game to game or mod to mod.
* Learn to deal with resurrections and/or role changes.
* Learn how to deal with a dead player replacing back in for someone else, keeping the past self posts separate from the new identity, and an entry in the morgue for the roleflip that did happen.
* Learn a bit about alignments so town/scum could be automatically coloured green/red or something. Difficult for third parties, or nonstandard games.

seem directed towards giving players slightly less homework to do )

[Mr.Person]

I must be missing something about that second-to-last hard goal. Why is that hard? You're talking about dead players having their name and role colored, right? Perl has if-then statements, correct? You've already got checks if a player is dead and what role they are, put 2-and-2 together. If they're town, find every time their name is in a line and color it. Then check that line for their role-name and color that. Might get buggy if there's a long line containing many players with a mafia and town player sharing the same role, so consider splitting the morgue into multiple lines with a townie line, mafia line, and a third-party line.

Third-parties are usually in yellow. If a player isn't town or mafia, color their name yellow and say they're a third-party. You could also check for certain keywords (cult or cultist, sk or serial killer, survivor, mafia-ally or mafia ally, jester, and lyncher) and put that as alignment, but if the script can't find anything at all, just say the alignment is third-party.

This isn't a big deal since most mods keep track of this in the OP.

[Argembarger]

Actually the code doesn't check to see if someone is dead and what role they are;

the person using the lurkertracker actually feeds that information in a section of the script that replaces certain names with others, in order for the tracker to handle replacements and who is a mod and such.

I suppose a quick bridge that the user could do for this is to type it in the script as,

Code: [Select]

Dead_Scum => "[color=red]Dead_Scum[/color]: dead, scum"
in the Replacements Considered section, and then it would naturally come out as

"Dead_Scum: dead, scum"

when the lurkertracker report is posted.

It's just that that particular solution is just more work for the person using the lurkertracker... It's probly viable, but I think Zath wants the lurkertracker to determine for itself who is dead and what role they were and such, which is a much harder task unless of course all moderators agree to use special keywords to indicate this stuff.

And yeah it's not all that vital, just something to get the lurkertracker to look prettier and more organized and easy to read, I expect.

I think it's already pretty damn organized and easy to read considering what it's doing

[Mr.Person]

What I meant was that if Zathras is already feeding the information of who's dead and what their role was, the script can just look for its own output and color it automatically.

I know why you listed it as hard, but that goal itself is actually really easy.

[Leafsnail]

* Learn a bit about alignments so town/scum could be automatically coloured green/red or something. Difficult for third parties, or nonstandard games.

If you ever get this one to work, we might as well stop playing.