You can definitely do that. You'll just need to tell each of your squads individually, plus your civilians, to use that alert level, rather than being able to tell everyone at once. And since you tell them all from the same screen, it's not that big of a deal.
You do have to use the 'alert' screen rather than the 'schedule' screen, though, to set which alert the groups are using.
Breaking it down a bit:
First, make the burrow you want to have your civilians retreat to when there's danger. Name it something useful.
To make the 'danger' alert level, press 'm', then 'a', to get to the alert screen. Then press 'c' to add an alert and 'N' to name it. Once you've named your alert, press the left-arrow twice to get to the column with the list of burrows, select the burrow you want your civvies to stay in, and press enter. To set what your military will do on that alert, press 's' to go to the squads screen, then press '*' until the heading says 'Squad Schedules:' and the name of your new alert. From there you can set up the orders for each of your squads for that alert, as usual.
When you want to activate that alert, press 'm' and then 'a' to get to the alert screen, select the alert, press enter to set your civilians to that alert, then press right once and go down the list of squads, pressing enter on each one to set the squad to that alert. When the danger is over, use the same steps with the 'inactive' or 'active/training' alerts to set your dwarves back to their regular schedules.
It's also worth noting that you can mix and match: Say you have two entrances, one of which is building-destroyer-resistant and the other of which isn't. Your main danger alert might send three squads to each entrance. But then if if turns out that there are no building destroyers, you might want to let the dwarves guarding that entrance go back to training. You can do that without disturbing the dwarves at the other entrance by just setting the three squads for the entrance in question back to 'active/training'.