Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[freeformschooler]

I will just say the name can't really be improved much further, I think.

[Aklyon]

I cant find le move button D:
Someone do a screencap of the move button,

Spoiler: Ze Button (click to show/hide)

[DrPoo]

I cant find le move button D:
Someone do a screencap of the move button,

Spoiler: Ze Button (click to show/hide)

Le thx, now tell me your email's so i can add yer as commiters..

[counting]

Mine already in my profile.

[Angel Of Death]

When will this game be ready?

[DrPoo]

When will this game be ready?

Way too early to say that, we are still awaiting response from soulwynd, so he can upload his DB and let us code on it.

Mine already in my profile.

Yes, but ya know, you cant really see the adress, only message it

[counting]

Mine already in my profile.

Yes, but ya know, you cant really see the adress, only message it

I didn't know that before @@... I thought everyone could see that
countingtls@ g m a i l .com

[Stargrasper]

Just because I didn't see anyone else mention it during my very quick skim over the thread...robots.txt.  There's also some meta tags.  And one or two other protocols designed to restrict robots.  Mind these don't do anything on their own.  These are just the rules set out by the webmaster and it's up to the robot to follow those rules.  Don't and you'll get blacklisted.  For example, the B12 robots.txt completely blocks most bots from touching the forums.  The Wikipedia robots.txt blocks robots from touching dynamically generated pages and also includes a spider trap.  Spider traps are devious little designs used to f**** robots that don't parse the robots.txt file.

I didn't pay that close attention, but it looks like you guys were at least partially serious about making this game.  If so, pay attention to restriction protocol or you'll find the game end badly when your bots get blacklisted.

http://www.bay12games.com/robots.txt
http://www.wikipedia.org/robots.txt
http://www.robotstxt.org/

[DrPoo]

Just because I didn't see anyone else mention it during my very quick skim over the thread...robots.txt.  There's also some meta tags.  And one or two other protocols designed to restrict robots.  Mind these don't do anything on their own.  These are just the rules set out by the webmaster and it's up to the robot to follow those rules.  Don't and you'll get blacklisted.  For example, the B12 robots.txt completely blocks most bots from touching the forums.  The Wikipedia robots.txt blocks robots from touching dynamically generated pages and also includes a spider trap.  Spider traps are devious little designs used to f**** robots that don't parse the robots.txt file.

I didn't pay that close attention, but it looks like you guys were at least partially serious about making this game.  If so, pay attention to restriction protocol or you'll find the game end badly when your bots get blacklisted.

http://www.bay12games.com/robots.txt
http://www.wikipedia.org/robots.txt
http://www.robotstxt.org/

Fuck.. didnt think about this one? But how does wikipedia quest THEN work?
Maybe we could make a bot browser, so it looks like a human browsing..

[FunctionZero]

Just because I didn't see anyone else mention it during my very quick skim over the thread...robots.txt.  There's also some meta tags.  And one or two other protocols designed to restrict robots.  Mind these don't do anything on their own.  These are just the rules set out by the webmaster and it's up to the robot to follow those rules.  Don't and you'll get blacklisted.  For example, the B12 robots.txt completely blocks most bots from touching the forums.  The Wikipedia robots.txt blocks robots from touching dynamically generated pages and also includes a spider trap.  Spider traps are devious little designs used to f**** robots that don't parse the robots.txt file.

I didn't pay that close attention, but it looks like you guys were at least partially serious about making this game.  If so, pay attention to restriction protocol or you'll find the game end badly when your bots get blacklisted.

http://www.bay12games.com/robots.txt
http://www.wikipedia.org/robots.txt
http://www.robotstxt.org/

Fuck.. didnt think about this one? But how does wikipedia quest THEN work?
Maybe we could make a bot browser, so it looks like a human browsing..

Wikipedia only disallows crawling of the special pages. All the articles are fair game.
So Wikipedia Quest could still crawl the regular pages.

So basically it just prevents the bots from looking at pages like Special:Administrators and such.
There's an easibly readable list in every robots.txt, so you can see what each site disallows.

[freeformschooler]

Yeah, should be easy enough. Looks like bay 12 bots can still parse all of the threads and stuff, just not special pages.

[FunctionZero]

When this gets done, I'm taking over the Poopcrawler's GoogleCode page.

[Aklyon]

Yeah, should be easy enough. Looks like bay 12 bots can still parse all of the threads and stuff, just not special pages.

As long as they don't act like the wayback machine.

[Stargrasper]

Wikipedia only disallows crawling of the special pages. All the articles are fair game.
So Wikipedia Quest could still crawl the regular pages.

So basically it just prevents the bots from looking at pages like Special:Administrators and such.
There's an easibly readable list in every robots.txt, so you can see what each site disallows.

Which is precisely the reason I linked to two example robots.txt files.

Fuck.. didnt think about this one? But how does wikipedia quest THEN work?
Maybe we could make a bot browser, so it looks like a human browsing..

Besides being unethical and quite possibly illegal...it also wouldn't work.  Especially if the webadmin/sysop/etc is monitoring with anything resembling diligence.

Do yourself a favor and just follow the rules.  I had to make a webcrawler find, parse, and listen to robots.txt a couple of years ago and it's not terribly difficult.  Don't forget about the metatags and anything else that might restrict a robot.  Sysops get pissed when unauthorized robots start constantly crawling their pages (hence spidertraps).  It costs them time, money, bandwidth, and other resources every time a bot touches their network.  Even if you follow the rules, sites that don't outright block everything will still block you specifically if you're enough of a nuisance.  Toady will block you in his robots.txt file if you start sucking up too much bandwidth and money. 

[Aklyon]

So, what I believe is a summary so far:
1: Make sure it understands everything it needs to and restricts itself thusly. (robots.txt, metatags, etc.)
2: Do not make it go too fast. Or do too much at once.
3: Don't use it all the time.