Not everything is duplicate, in the first case you mentioned the language_word file shows:
WORD:ARMOUR
NOUN:armour:armours
VERB:armour:armours:armoured:armoured:armouring
WORD:ARMOUR_2
NOUN:armourer:armourers
WORD:ARMOUR_3
NOUN:armoury:armourys
Basically all 4 words were probably in 1 diction entry to begin with (my script that processes the dictionary file, just chops them up when they have too many nouns in them, they aren't always just duplicates) , because I didn't combine anything below 7 letters in length....
also only the US armor:armors and armory:armories exist so great catch, the correction i made:
<W:ARMOR><N:0111110001:armor:armors><A:110:armored><V:armor:armors:armored:armored:armoring> <S:ARTIFICE:PROTECT><T:DWARF:tosid:ELF:datome:GOBLIN:nuklat:HUMAN:stalcon:ORC:bugburuk:SUCCUB:guxice:KOBOLD:chins:NAGA:una:NECRO:der>
<W:ARMORY><N:0111110001:armory:armories> <S:ARTIFICE:PROTECT><T:DWARF:kumil:ELF:darame:GOBLIN:lumab:HUMAN:namar:ORC:uduut:SUCCUB:gicavice:KOBOLD:ibanins:NAGA:unemi:NECRO:tersan>
<W:ARMOUR><N:0111110001:armorer:armorers><D:MAKER OF ARMS OR ARMOR OFFICIAL IN CHARGE OF ARMS ARMORY ARSENAL> <S:ARTIFICE:PROTECT><T:DWARF:ocigber:ELF:obedi:GOBLIN:testra:HUMAN:oquobmi:ORC:nuurbagrinkhakh:SUCCUB:dirosse:KOBOLD:hantabente:NAGA:szena:NECRO:wsera>
I added the verb armor to armor and kept ARMOUR to hold the word armorer:armorers a person who makes armor or is in charge of an armory.... may deprecate later as I'm considering removing -er words for a suffix entry (its a rear compund adj) of er and ers instead. that may end up being the solution to seriously reduce file size, I'm already doing it with -ness and -ship words.
fixed apall, didn't notice it was the british one in there.
heh a lot of the words I censored were derogatory words used in racism and sexism. I was parsing originally a 32,000 entry dictionary, a circa 1990s oxford dictionary. so basically somewords just slipped through such as anal, anus, and areola (its in there lol) , but I've had both sides weighing in on it. I'd hate to go full G rated but triple xxx is over the top. I'm working backwards from one dictionary, and forward from the original set in masterwork, which did include some PG-13 words, primarily for certain races such as necromancers and succubus. I may include them in a separate symbol group (say ADULT_WORDS) so that people that want to cull them from their civ names can. I'll definitely will make a note of them now though so that if I don't get down to my target count those will be some of the last...
I agree there is many too obscure or not fitting to the setting.
ABATTOIR - Slaughterhouse in french I believe - out
ABNEGATE - I had to look it up, to give up or renounce esp religion doctrine faith or belief - out - who knew this word..
what ABLUTION the word before that, its ceremonial hand washing.
ABSINTH - WORMWOOD ANISEED-FLAVOURED LIQUEUR BASED ON THIS -out - I was tossing all the liquor out of the file anyway missed it.
ACERBIC - HARSH AND SHARP ESP IN SPEECH OR MANNER ACERBITY - never heard it before, rather obscure - out
ADIPOSE - out didn't know the technical term for fatty tissue or of fatty tissue was in there
ADULTERATE - out another technical term about adding things to dilute a substance
ADVENTITIOUS - out obscure
AMELIORATE - obscure
removing AMINE - basically its of or from ammonia lol
AMITY - obscure means of friends, friendship
ANGOSTURA - a flavoring
ANTHRACITE - keeping - HARD TYPE OF COAL BURNING WITH LITTLE FLAME AND SMOKE - its an actual item in game in Masterwork
ANTHROPOMORPHISM - out
ARRAS- HIST RICH TAPESTRY OR WALL-HANGING - hmmm... i didn't know that, out
ARREARS - behind in payment? out
ARRIVISTE - ambitious ruthless person - eh. I'm going to put that in the wait pile
ARTY -out
ASBESTOS out
ASEPSIS out
ASIATCIC out
ASSIDUOUS out
ASTERICK out - you didn't mention it, but I didn't want language symbols included
what about ASTRINGENT?
ATAXIA - imperfect body control? out
ATALIER - artist workshop? out
ATROPINE out
ARGON - you didn't say it, inert gas element, seems useless as its not an in game thing removed
ARGEY_BARGEY - you din't mention it, but WTF? a rather odd word for DISPUTE WRANGLE
ATTAR - perfume from roses.... out
AUDIOTAPE out - missed it technical word
AUDIOVISUAL out -missed it technical word
AUGEAN - FILTHY... they can say FILTHY lol out
AURICULA - technical flower part term out
AUSCULTATION - technical term for listening to the heart out
AUSSIE- completely missed it out
AUSTRAL - completely missed it out
AUTARCHY - despot - out
AUTARKY - self sufficient economy - out
AX - contained <V:axe:axes:axed:axed:axing> moved to axe then next word. otherwise the entry is out.
BAA - but its the sound of a sheep! I like that one lol
BAILIFF, BAILIWICK - D:LAW DISTRICT OF A BAILIFF JOC PERSON'S PARTICULAR INTEREST - I was considering the term before of BAILIFF, we already have so many terms for law officer, and I think I even tossed BAIL and BAIL OUT... I'll remove both and if anyone begs I'll put them back in.
BAIRN - archaic term for child... there is a lot of terms for children, i can toss this one for now.
BALLCOCK (not what one might think it is)<D:FLOATING BALL ON A HINGED ARM CONTROLLING THE WATER LEVEL IN A CISTERN>(it really isn't what most people think it is) out.
BALLBOY also the word before that, I can retrieve my own balls thank you.
BANDBOX - out missed it, its hat box lol
BANDMASTER - out missed it also
BANDWIDTH,BANDWAGON,BANDAEU what about BANDSMAN and BAND_GROUP?
BAPTIZE (debatable, but the word is first and foremost a Christian term)
BARIUM - right I was looking for these before, there is so many materials that aren't in the game yet, out. I also need to review ingame materials and verify I have a word for each... but I want the file down to around 12 to 14k entries before I do that.
BAT_WEAPON (as in baseball bats?) - as in baton, stick, etc, lol the game term came about in the 1800s, before that any short stick was a bat, source for the term batter(to beat usu with a weapon) and battery(crime of beating someone usu with a weapon)... I remember identifying it separately as I went through duplicates. It has a verb form in it to bat:bats:batted:batted:batting lol.... I'll toss that until someone requests its return. also tossing
BATS,BATTY for crazy,
BATHOS for lapse in mood lols.oh and
BATWING (were not playing the latest batman game...)
BIDET but its such an awesome word!
BILL_WEAPON yeah when parsing duplicates, I think there was originally 5 or 6 BILLs so I just added quick identifiers. <D:HIST WEAPON WITH A HOOKED BLADE> its also above that a bit as
BILLHOOK, I don't think either weapon is in the game. I never got back to them. I was keeping weapon names because so many mods have additional weapons not in the original. but your right if they want the word in they can either add it in their mod pack or request its return.
Good work otherwise! (It's worth noting that the forum refused to let me PM or post this until I removed the square brackets from everything).
yeah I noticed that it doesn't like brackets at all... that's why my actual dictionary operates with <> instead so I can post them to the forums and PMs without issue. I uploaded a file containing the entire dictionary its linked at the bottom of the list above. Its better to edit it, as you can see how it propagates across all the files in 1 place easier. The one oddity, my script takes the first lowercase english word as the word for the ENGLISH file... so in the case of ARMOUR above the next version will say T_WORD:ARMOUR:armorer in the language_ENGLISH.txt file, because the first lowercase english word is armorer in that entry line.
thanks for the assistance, and I've made these changes to my files, they will show up in the next version, dictionary now at 15850 entries. I mentioned a few others you might not of noticed, that I'm still on the fence on.