If you used them up to get the labor, you mean? It'd be totally ridiculous if you didn't. Two turns from now, you'd effectively have an extra 5,000 population that didn't eat or contribute to overcrowding; four turns later you'd have ten thousand more people than you should. Granted, they wouldn't pay taxes and you'd lose out on the tiles giving extra food, but the fact that they don't eat means they pay for themselves within 2-3 turns, and none of this impacts your natural population negatively. They also couldn't construct items or buildings, but by the time you've run out of city tiles you want worked or improved, I think it's pretty clear that uselessness is not the problem.
If they're used up, on the other hand, they provide a slight bonus to efficiency without becoming mandatory for anyone who wants more labor (meaning everyone). Consider:
Irrigation gives +1 Food, so each horse (from a horse farm, pastures are less efficient) essentially costs 1 Food.
Hence, the "free" labor they give costs 2 Food apiece.
If you work a tile that gives 2 Food, you break even (minus the extra costs of building horse farms rather than farms).
If you work a tile that gives 3 Food, you gain 1 Food every 2 turns you work a horse farm; in other words, horse farms effectively give +0.5 food above normal farms. Of course, you're free to use this labor getting wood or the like instead, so it also gives a bit of flexibility.
Given how low-tech and relatively cheap they are, this is a good investment overall; it's like getting half an extra farm for 1 labor and 1 material. The only catch is that you have to actually have something else worth working on: Either you can use it to periodically work tiles you otherwise wouldn't, or you can replace population workers on essential tiles to let them do something in the city. In other words, if you have nothing you want done in the city, you need "extra" improved tiles elsewhere.