I think a bit part of that question boils down to exactly what you mean by heirarchy. What anarchists mean when they're against heirarchy is that they're against a system in which one person (or people) can dictate orders to another person, on the basis of being a more important person, which the other person must carry out or face punishment.
But that's not exactly the same thing as a structure. An structure can exist which has layers - e.g. local democratic collectives send delegates to a regional forum, that forum debates events and policies, then each collective agrees to carry out different actions to achieve unified aims. Sure, the central forum could be called a "heirarchy" but it's not like the US federal government or anything: the apparatus through which it enacts action is the collectives themselves, so if the people in the collectives don't want to do what the forum says, they just don't do it, and probably change the people they're sending to be delegates.
e.g. a collective can have a president, as long as that president is fairly elected (and in most anarchist systems, can be recalled by popular vote at any time, so no set terms, which reduces the power of incumbency). The president can direct how the collective functions, because they've been delegated the authority to do so by the collective, but they don't have coercive authority: they only have personal authority. When leaders can be replaced at the snap of a finger for getting too bossy, the whole game changes. Think about a business comparison. Imagine you're in a work team, one that has a manager and workers, and the manager at any time can say "you're fired!". Compare that to another type of team, in which the team chooses a manager to follow, and can pick someone else at any time. The dynamic is completely different, even though someone could argue that superficially, they have the same roles and duties to perform, and that if you look at a business chart, you see the same "roles" as before. Sure, the tree structure of the company might appear the same, but it's going to be a completely different place to work, if at each level your subordinates can vote "no confidence" in your management.
So I think there are two main difference between an anarchist structure vs a typical one. In an authoritarian heirarchy, the higher-ups can order or replace the lower-downs. In an anarchist hierarchy, coercive authority flows from the bottom up: lower-level workers can order the replacement of managers because that's their right in the system. Job descriptions may be the same in each system, but you should be able to see that the outcomes would be different, despite having the same "heirarchy" of positions.
The other difference is a consequence of that: Authoritarian heirarchies usually have an enforcer arm that's separate from the main hierarchy, who's job is to discipline the main heirarchy and only answers directly to the leader. The need for such a separate enforcer team should be clear: if you only have a single heirarchy under your command then you're too beholden to them for your power, and can't expect them to just obey any order you give them.