In various books and articles, I've defended the Competence Principle, which I argue is a principle of authority and legitimacy which must be added to any full theory of authority and legitimacy.
The Competence Principle says something like this:
A political decision which results has authority and legitimacy only if it is produced by a competent decision-making process. The agents within that process must be competent as individuals, or at least collectively, and must act in good faith.
It's more complicated than that, but that's the basic idea.
As an example to motivate this, consider a jury trying a capital murder case. Suppose the jury finds the defendant guilty out of ignorance, malice, idiocy, selfishness, or spite. We'd think it illegitimate to enforce their decision. The jurors should understand the case and decide it competently and in good faith.
Now, suppose you're on board with this. Consider the behavior of the American government at the federal, state, and local levels over the past few months. Consider how poorly the vaccine roll outs are going, including instances of evil (the CDC recommending we let more old people die because they are disproportionately white) or mere incompetence (the inability to distribute them quickly, taking weekends and nights off, etc).
How does the incompetence of the American government impact what we think it should do? For instance, there may be a strong case for having Denmark's competent government do many things, but our government's incompetence is a strong presumption in favor of it being charged with doing far less. A stupid, malicious, or incompetent government ought to be "small".