Not exactly a "frustrated with dwarf X" post, but perhaps designation or zone jobs (gathering plants can be both) override civilian alerts. Therefore, next time a dwarf happens to be outside, see if you can cancel the job they're currently doing in some way to get them to go inside per your civilian alert. If the job in question was caused by a designation or zone:
1. Remove/deactivate any garbage dump zones, plant gathering zones, pit/pond zones, pen zones, etc. that are generating the job. ("Pen/Pasture X
1 (X
2)" means you need to deactivate a pasture zone, if the job has a dwarf doing it. The
j - Jobs - menu is helpful here. X
1 is name of animal, if it has one; X
2 is taming status, ranging from Semi-Wild to ☼Trained☼ if not domesticated by your civ; Tame is reserved for domesticated animals pretty much)
2. If the job is question isn't caused by a zone, hit
x when selected on either the job (if in jobs menu) or the dwarf doing it (if in unit list menu); either way, the worker doing the job will be set to "Inactive", meaning none. If
x is unavailable (meaning you can't remove a worker from the job; designation jobs often have this, as do "jobs" that fulfill basic needs, I.E. eating/drinking/sleeping) you can try to
cancel it, but if that fails too (non-workshop-task jobs usually can't be cancelled), you may not be able to save your dwarf. Activities are a different thing entirely, and can't be cancelled by you or have workers removed from them. (Text in green or in magenta with "!"? You have yourself an activity. Some examples below.)