Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Avis-Mergulus]

Grr...

LXML won't install. This thing's been sitting here saying "Installing lxml" for the past 4 days.

You can try using BeautifulSoup with html.parser or html5lib instead. Here, check out the docs. There are a bunch of instructions there.

[Arx]

This is not what I expected to happen. Wow.

[penguinofhonor]

This should assuage any worries that robots will take jobs from hardworking shitposters. There will still be a need for people to maintain the bots and optimize their inputs.

[SirQuiamus]

The scrapers are nice, but I think more carefully crafted files are the way to go. Here are a few generated from only LW sentences containing the word "remove":

Yes, that's an excellent idea. Since the data is already sitting on the B12 servers, it's not necessary to dump all of it to your hard drive: just parse the html on the fly and pick the posts you want.

HNNNNNGGGGGGGGG AVAST BEST WAIFU SAVE ALL MIDGET DWARF AND CALL PUTIN BLUFF BEST DAY OF LIFE REMOVE CRYPTMAN FROM PREMISES GIVE BACK YOG MERKEL CLAY

GLORY GLORY HALLELUJAH

Grr...
LXML won't install. This thing's been sitting here saying "Installing lxml" for the past 4 days.

How are you trying to install it? You need to run pip with superuser rights, if that's what you're using. If you're on Linux, you can find popular Python packages in your distro's repositories.

[penguinofhonor]

I have so far compiled about twelve pages of my non-FG posts to feed to robots. I think I'm getting somewhere.

Here's the borscht I was trying to create a Kurdish state and conditions get shitty enough, tensions will rise and conflict can break out across some other recipes I saw a book for Mouse Guard RPG, which seems to come from clunkiness.

Each country that signs the bill will allow other signing countries to apply for protections on their influence in the next big curling game.

[WealthyRadish]

One advantage of making a file centered on a keyword or phrase is that you can get consistently coherent results with a relatively small sample size (since most sentences will have a guaranteed point of possible splitting/divergence). That LW_remove file used only 70 relatively short sentences, much less than was needed to get decent results from BourbonGiraffe. It did have some quirks however, like there was one post that had REMOVE ELF PEOPLE FOR THE HEARTHLAND repeated 6 times consecutively. For a real LW post of course this would be perfectly normal, but it may be less ideal imitating others.

I think a good route would be to make a file centered around sentences only containing a keyword or phrase, and then pad it with some particularly generic sentences that use common grammar and language (that can be reused for other artisan files, like the one I want to make for "cheeki breeki"). That should help the algorithm splice things up cleanly and keep variation high without the problems of a massive file.

[chaotic skies]

I'm using pip on Windows with superuser access. I might have broken something, so I'll check again, but I still find it funny I typed in the command and then the prompt wouldn't close until three days later when my computer restarted

[Andres]

hi evryone

[chaotic skies]

Hey.

Mind if I steal all your posts?

[exdeath]

It's the code in the first post.

Code: (This) [Select]

import random

class Markov(object):

def __init__(self, open_file, chain_size=3):
self.chain_size = chain_size
self.cache = {}
self.open_file = open_file
self.words = self.file_to_words()
self.word_size = len(self.words)
self.database()

def file_to_words(self):
self.open_file.seek(0)
data = self.open_file.read()
words = data.split()
return words

def words_at_position(self, i):
"""Uses the chain size to find a list of the words at an index."""
chain = []
for chain_index in range(0, self.chain_size):
chain.append(self.words[i + chain_index])
return chain

def chains(self):
"""Generates chains from the given data string based on passed chain size.

So if our string were:

"What a lovely day"

With a chain size of 3, we'd generate:

(What, a, lovely)

and

(a, lovely, day)
"""

if len(self.words) < self.chain_size:
return

for i in range(len(self.words) - self.chain_size - 1):
yield tuple(self.words_at_position(i))

def database(self):
for chain_set in self.chains():
key = chain_set[:self.chain_size - 1]
next_word = chain_set[-1]
if key in self.cache:
self.cache[key].append(next_word)
else:
self.cache[key] = [next_word]

def generate_markov_text(self, size=25):
seed = random.randint(0, self.word_size - 3)
gen_words = []
seed_words = self.words_at_position(seed)[:-1]
gen_words.extend(seed_words)
for i in xrange(size):
last_word_len = self.chain_size - 1
last_words = gen_words[-1 * last_word_len:]
next_word = random.choice(self.cache[tuple(last_words)])
gen_words.append(next_word)
return ' '.join(gen_words)

So the program end after X words with X being a number?

Why not add a fake word at the end of each text called text end. So if you genrate this "word" as next word, the text ends?
Another thing program should do (didnt checked it enought to se if it has this, I am at work), to have a "text start" fake word, so first word of generated text will only a word that started a text on the seed text.

[Zanzetkuken The Great]

What happens if you feed it a dictionary and a list of famous quotes, then have it run in such a way in order to add what it comes up with to the end of the source?

[chaotic skies]

This sounds like a good idea. Someone do this. I'm on my phone and busy right now.

[Andres]

Hey.

Mind if I steal all your posts?

i mind so dont steal my posts

[chaotic skies]

*pouting* Alright. I was jost going to use them to generate shitposts. I would have given them back as they were, I swear!

[Sergarr]

I've done some research into new perspective shitpost simulation methods and look what I've found.

This could revolutionize the shitpost industry!