Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Skyrunner]

I'm thinking of making a "encryption" program (of course, very superficial compared to the actual encryption ciphers in use). For fun

Would this method be sufficient to be a cipher? o.O

For an example, I'll use the phrase "APPLE", and a 4x4 grid, though the grid should be much larger when actually made.

1. Split data by random characters.

Code: [Select]

APPLE -> zAvP1PoLmE2. Pad data.

Code: [Select]

zAvP1PoLmE (10 characters) -> zAvP1PoLmEo3z02v (16 characters)3. Twist data; that is:
The character "A" consists of the data 01000001. Split it like this: 01|00|00|01, then mix it up so that the segments are arranged as 4-1-3-2 instead of 1-2-3-4, which would mean "A" would turn into 01010000 "P".

Code: [Select]

(not actual twisted data)
lpdzctzdthpstvsg4. Arrange data as a matrix of equal lengths, and spin it.

Code: [Select]

lpdz
ctzd
thps
tvsg

turns into

clpd
tzpz
tthd
vsgsNote that each "square" is spun in a different direction. The first 4x4 is spun clockwise, the second 2x2 counter-clockwise. And so on for multiple layers.
5. Print as hex.

Code: [Select]

63 6C 70 64 20 74 70 73 7A 20 74 68 76 64 20 76 73 67 73
6. Insert information.

The format would be in characters. An example of information, using two characters:

• 4 bits: Length. Provides for 15 possibilities, starting from 16 and going up in multiples of 2 all the way to 1024, for now.
• 4 bits: Misc. method information; there may be multiple ways to twist the data.
• 1 bit: Direction—start from counter-clockwise, or clockwise?
• 7 bits: Version, since I might change the encryption algorithm.

In this case, the length is encoded as 0000, the misc method as 0000, the direction as 1, and the version as 0000001. This turns into the characters [null][DEL], which is represented as 007F in hex. The final cipher is represented as:

Code: [Select]

00 7F 63 6C 70 64 20 74 70 73 7A 20 74 68 76 64 20 76 73 67 73
I'm wondering if this is sound as a cipher, or if I did silly stuff that anyone could see through, or useless actions that turn out to be the same in the end (like keeping substituting between four equations when it has five variables in hopes of finding the values for each).

EDIT: One method for encoding in the 4bits could be rules for twisting data, such as twist one CCW, two CW, three CCW, four CW, then back to one, or so on.

[Virex]

I don't know about the strength of this encryption, but the question is how you would decrypt it? You're currently providing the key with the string, but that relies on no one knowing your encryption method, which is a poor defense.

[alway]

You're currently providing the key with the string, but that relies on no one knowing your encryption method, which is a poor defense.

This. The point of encryption is to allow anyone to decrypt it IFF they possess the key to do so. It must be made in such a way that knowing the decryption algorithm gives no benefits in figuring out what is encrypted; modern encryption/decryption algorithms are shared publicly, but this is of no use in actually cracking the encryption unless the encryption itself is fundamentally flawed (as has been the case for several encryption schemes).
A really simple, secure scheme would be a simple XOR with a random key string, though this can fall prey if your random string isn't sufficiently random, as it becomes the only possible crackable part of this algorithm. XOR your message string with the key string, and you get an output string which is completely devoid of information without access to the key string. The downside of the method is the key must be equal in length to the message sent; which for your hobbyist aims may or may not be a problem (you seem to already be fine with sending 16 bytes for a 5 byte message).

Add padding to the end obscure the length of the message (as the length itself is information about the message, particularly with short messages; which is bad to leave unencrypted). The amount of padding could be determined through a number of ways, so long as it can't give away information. Either a random number of padding characters (as post-encryption, padding characters look the same as the rest of the message; that is, totally random) or some similar completely non-reversible generated number. So long as the padding characters used by the algorithm are understood to be padding characters by the decryption, the number of padding characters at the end doesn't need to be figured out through a reverse of the system that generated them, simply because the decryption will see them as padding characters and throw them out.

Assuming your random number algorithm hasn't been cracked, that algorithm is completely secure. Now as to the reason it isn't used to encrypt things by the pros: the key length. The key must be sent to the person using the decryption, and must be done so securely. Which means for each encrypted message, you must send information which is at least the length of the message securely; and if you can do that, there isn't a real reason not to just send the message itself using that secure method. Which is why real encryption is more tricky than a simple XOR. Real encryption has to figure out how to get pretty much the same properties as described above, but be able to encrypt gigabytes or terabytes of information with a handful of bytes.

When it comes to security, it isn't a question of 'can I figure out the original, 1:1, using the message,' it's a question of 'can I reduce the time my brute force takes from 2^64 tries to 2^32 tries.' In the case of yours, even a variation on that theme to make it key based would probably still give way too much information about the message. Remember, modern computers run billions of instructions per second; hundreds of billions if done on the GPU (upwards of ~2^40 FLOPS on high-end GPUs). The trick to encryption is making it strong enough to take longer than the lifetime of the universe to brute force the solution. If any information, whether positive (this letter is 'A') or negative (this letter is not 'G', 'R', or 'H') can be gleaned from the results, your security decreases by the amount of information which can be derived.

[Skyrunner]

I see, much food for thought o.O

[dreadmullet]

I've need of some C++ file reading advice. I am reading .blend files to extract the mesh data. It involves reading through all of a 500+ kb file and getting a few hundred kilobytes from it. I wish to make it as efficient as possible. A few questions:

1. Would it be faster to read the entire file and store it in memory, and then extract the useful data from memory? I'm currently reading one character at a time in some places, as it searches for a string. I've also read something about memory-mapped files.

2. Multithreading? Can I read the same file using multiple threads at the same time? What about multiple files at the same time?


Thanks for any advice, you guys are always awesome.

[Mego]

I've need of some C++ file reading advice. I am reading .blend files to extract the mesh data. It involves reading through all of a 500+ kb file and getting a few hundred kilobytes from it. I wish to make it as efficient as possible. A few questions:

1. Would it be faster to read the entire file and store it in memory, and then extract the useful data from memory? I'm currently reading one character at a time in some places, as it searches for a string. I've also read something about memory-mapped files.

2. Multithreading? Can I read the same file using multiple threads at the same time? What about multiple files at the same time?


Thanks for any advice, you guys are always awesome.

1. Depends on how much memory you have available for use at any given time, and how many files you are going to have open at once (see #2).

2. Same file: only if you have each thread start at a different place in the file. My brain is too fried right now to think if that's possible/practical. Multiple files: Sure.

Suggestions: Use regex (specifically boost::regex for C++) to make the searching more efficient.

[MorleyDev]

2. Multithreading? Can I read the same file using multiple threads at the same time? What about multiple files at the same time?

Yes but (except maybe on an SSD) benefits would be limited, if any at all. The Hard Disk itself can only read from one location at any given time. It's why having 3 SATA Hard Disks, one for the the OS, one for data you primarily intend to read from (executables for example), and one you primarily write too would give a huge performance boost to a PC compared to all on one SATA.

[Nadaka]

I've need of some C++ file reading advice. I am reading .blend files to extract the mesh data. It involves reading through all of a 500+ kb file and getting a few hundred kilobytes from it. I wish to make it as efficient as possible. A few questions:

1. Would it be faster to read the entire file and store it in memory, and then extract the useful data from memory? I'm currently reading one character at a time in some places, as it searches for a string. I've also read something about memory-mapped files.

2. Multithreading? Can I read the same file using multiple threads at the same time? What about multiple files at the same time?


Thanks for any advice, you guys are always awesome.

1: reading 1 byte at a time is almost always the slowest option. File access is thousands of times slower than memory access, memory access is dozens of times slower than cache access and cache access is several times slower than register access (all of this will vary by computer). Ideally, you know how large the cache is on your cpu and you read into a an array small enough to fit in it. The simple solution is to use a buffering file reader of some sort.

2: do not use multi threading to read from a file. Feel free to use it to read from different files though.

[Nadaka]

2. Multithreading? Can I read the same file using multiple threads at the same time? What about multiple files at the same time?

Yes but (except maybe on an SSD) benefits would be limited, if any at all. The Hard Disk itself can only read from one location at any given time. It's why having 3 SATA Hard Disks, one for the the OS, one for data you primarily intend to read from (executables for example), and one you primarily write too would give a huge performance boost to a PC compared to all on one SATA.

In my experience, you get benefits up to around 4 or 5 threads reading/writing different files off of one disk. You usually can not saturate a SATA controllers bandwidth by reading/writing a single file. A previous project I was on involved with copying, verifying, deduplicating and processing hundreds of terabytes of data.

[Pnx]

2. Multithreading? Can I read the same file using multiple threads at the same time? What about multiple files at the same time?

Yes but (except maybe on an SSD) benefits would be limited, if any at all. The Hard Disk itself can only read from one location at any given time. It's why having 3 SATA Hard Disks, one for the the OS, one for data you primarily intend to read from (executables for example), and one you primarily write too would give a huge performance boost to a PC compared to all on one SATA.

In my experience, you get benefits up to around 4 or 5 threads reading/writing different files off of one disk. You usually can not saturate a SATA controllers bandwidth by reading/writing a single file. A previous project I was on involved with copying, verifying, deduplicating and processing hundreds of terabytes of data.

Was this with a hard disk or a solid state drive?

[Shinotsa]

I, with absolutely no knowledge of anything, have come to you computer magicians with a quick question. How long would it take to make a DOS program that could multiply each in a list of numbers by 1000, outputting the new file to a specific folder? I ask because I need to convert a few hundred files, each containing hundreds of data points, from seconds to milliseconds and I am wondering whether or not it would take less time to have someone whip up a program than to spend 6 hours doing it by hand in excel.

If I'm asking something stupid feel free to ignore me and continue with your binary wizardry or whatever it is that you do in here.

[RulerOfNothing]

It shouldn't take very long to write such a program. What format is the list in?

[Shinotsa]

MS DOS format text file, saved from a single vertical column in excel.

[Skyrunner]

Oh, that's a perfect job for a macro. Use Excel's Macro ability, and record it as "=(first number's cell) *1000 [enter]" then run it many many times. It shouldn't take more than an hour even if you open each file.

Unless you already tried it and it takes too long...

Another method is write the first cell next to the vertical column as what I just did, then click the small box in the corner, drag to the side, ctrl-down, drag back so you only highlighted a single column.

[Mego]

I, with absolutely no knowledge of anything, have come to you computer magicians with a quick question. How long would it take to make a DOS program that could multiply each in a list of numbers by 1000, outputting the new file to a specific folder? I ask because I need to convert a few hundred files, each containing hundreds of data points, from seconds to milliseconds and I am wondering whether or not it would take less time to have someone whip up a program than to spend 6 hours doing it by hand in excel.

If I'm asking something stupid feel free to ignore me and continue with your binary wizardry or whatever it is that you do in here.

Does it *have* to be a DOS program, or can it be a command-line (aka no GUI) program? There's a significant difference. DOS programs would be more difficult, with library issues and all.

In all honesty, Excel would probably be easier. Import the text file, make a quick formula, drag it down, copy, paste, save.