It's arguable. For a defiler, either of them... probably a good idea. You can spare a cat point and it's not like you won't be able to spare the generics. If you're cornac and not higher, the cost is even smaller (relatively).
Now, personally, I pretty much always take the tree if I'm not building something incredibly strapped for category points. A cat point and six generic talent points for an additional 10% resall (to say nothing of the lifebinding emerald or wozzname's rock) is incredibly desirable. That it can be +30 (5*6) stats or +10% damage or whatever else sort of gem you feel like shoving into your armor doesn't hurt. It's a quite versatile passive effect.
But, the cost is hefty. It's a category point and at-minimum six generics if you're beelining straight to imbue item 5/5 and ignoring everything else (Gem portal's alright, but useless for non-alchs iirc -- unless you can get some alchy gems from murdering rand-class monsters -- and stone touch is pretty iffy. Extract gems is a massive money generator if you're after that.), which I'd suggest. That's the other part of the downside, though. The only directly (and unconditionally) useful thing in the tree is imbue item. Extract gems is a bit more useful nowadays, though, what with that particular late-game money sink of incredible power. The only other thing to consider is actually investing in any of the talents will lock you out of anti magic, but that's a non-issue for the defiler metaclass anyway.
Overall, you probably didn't make a mistake, especially playing one of the defiler classes. They've got generics and cat points to burn, really, so the "downside" of taking the stone alchemy tree is heavily mitigated.