I'd expect the growth of a chair etc. to take about a year, and each tree to be capable of growing only one furniture size item at a time, and that would be at the expense of fruit. Basically, the resources put into producing fruit would be diverted into slave labor instead.
Not only do you get more than one fruit per tree, but dorfs take less than a year to make a piece of furniture. Or a pair of linen socks. I would expect it to take less time than a year?
As far as I know trees move considerably slower than dwarven carpenters, and the latter don't actually grow the logs, they just process logs that grown over several prior years, so I don't think that's a fair comparison.
There are few trees that produce fruit the size of cabinets, and I'd expect the volume of the fruit "converted" into furniture to be at least equal to the one of the furniture, and probably double that (squishy fruit vs lignine/cellulose dense furniture). Some especially bountiful trees might be pushed to produce two cabinets per fruiting season (i.e. once per year), and you could probably get several wooden swords out of a normal tree, or quite a few wooden mugs.
Flax isn't a tree, and a sock typically requires the thread harvested from a lot of flax plants, so I don't have much of an idea of how you'd grow "shrub" based items. Maybe the equivalent of a farming tile (which I assume elves don't use) might somehow grow together to form one sock, or even a pair of them. The growing time for grown "shrub" based items would probably the the normal growth time until harvest for that plant, and ought to follow the seasonal growth pattern of surface crops that DF doesn't implement currently (with current DF surface shrub growth times you'd probably get about two harvests per season). I've got trouble seeing how to grown "shrub" based items without harming the plant, but maybe annual plants flower and disperse their seeds while tangled up as a sock, dies, and the sock can then be picked up from the dead tangle of the plants. If so, it would require a full growth cycle, not shortened somewhat by harvest.