Sapping tha sentry human was already going to happen, the cryswarm isn't (entierly) stupid.
Anyway, Would that system work for converting things into crystals, or would we end up as a giant thousand-part swarm too easily?
Honestly, without adding artificial constraints (or some super-dangerous enemys), I don't see how you can stop us from growing super-massive since we can grow nearly exponentially, Step 1) fly into forest Step 2) Kill everything that moves that not really dangerous. At that point we should have tons of crystals, and we could split into 5 or so and fly to 5 different forests and clear them and grow until we kill everything.
On the different types of crystal front:
I like the idea of having a multi-crystal system (as opposed to just two crystals) better.
Some examples:
Small crystal: Our regular crystals, weak in everything, size 1, worth 1
Normal crystal: Our major crystals, larger and better then small crystals at everything, size 4, worth 4
Large crystal: Giant crystals that are strong and can take the place of multiple normal crystals, but only as good as a normal crystal in utilizing magic and intelligence/memory, size 16, worth 16.
Flawless crystal: Normal size crystals excellent at utilizing magic and good at intelligence/memory, however are extremely fragile and using them for physical tasks is dangerous due to their fragility, size 4 worth 16
And maybe have different things give different types of crystals (eg. rabbits/birds small/normal, humans give normal/large, wizards/magical creatures give flawless, mythical creatures give large) and the same with different type of absorption.
Not quite sure how well it would fit what you already have in mind for how crystals work, but personally i think it would be nice to have different crystal types.