I simply meant that most libertarians would prefer a government with limited and clear duties. This would obviously be more efficient at whatever duties it set out to accomplish than a government with vague responsibilities and significant creep, but it more a property of clearly defined governments than small governments. The other efficiency bonuses would come in the areas the government is no longer involved in (except to insure the market remains free).
One of the fundamental principles one should keep in mind when thinking about government is that, in many ways, we literally have no fucking clue what we're doing, what will work, and what won't. Free market forces, with numerous small and innovative companies competing for the right solution, are often an effective way to accomplish without risk to the general population or the focused structure of the government - they can take avenues governmental inertia would rarely allow. This is, again, why many libertarians are similarly opposed to large companies that dominate the landscape, as you lose most of the benefits of the free market when this occurs.
The libertarian view heavily favours governments not doing things simply because they are able, and not doing things that restrict freedom unnecessarily, and not interfering with functioning free markets. Most agree with large scale infrastructure being a valid government area of control, since it's outside the reach of a free market - the distortion would be to heavy to make the market effective. They also usually believe this should be handled by the smallest level of government capable of handling - that towns should handle their own town infrastructure without the help of the state or federal government, except in areas where only the state of federal government can handle the task. (We wouldn't build a highway piecemeal, funded by every area it runs through, for example)
This part isn't so much about efficiency as another important libertarian belief about small government:
The people actually involved in a situation are those best suited to deciding it. Women seeking or having experienced abortions, for example, would be the goto people when considering regulations and rules regarding the procedure, and it would be better handled at a state level as the expectations, beliefs, and acceptable risks may vary depending on the group of people you ask. They believe that despite our similarities, people are different, and one-size-fits-all solutions will be actively harmful to the groups the solution does not fit.
Obviously, this can not be completely avoided, but it should be minimized with an emphasis on local government - each level should only handle the tasks the level below can not handle efficiently, to insure solutions are tailored as closely to the relevant problems as possible. So they also believe in small government in that respect.