A 20-year bull market has convinced us all that CEOs are geniuses, so we watch with astonishment the troubles of Donald Rumsfeld and Paul O’Neill. Here are two highly regarded businessmen, obviously intelligent and well-informed, foundering in their jobs.

Actually, we shouldn’t be surprised. Rumsfeld and O’Neill are not doing badly despite having been successful CEOs but because of it. The record of senior businessmen in government is one of almost unrelieved disappointment, from Herbert Hoover, thought to be the smartest executive in America; to General Motors president Charles Wilson, who was utterly ineffective as Eisenhower’s secretary of Defense; to Robert McNamara, the wunderkind of the business establishment, to Donald Regan, former head of Merrill Lynch. McNamara’s attempt to use a tangible measure–the famous “body count”–to quantify progress in the Vietnam War is almost a caricature of misapplying managerial methods to the complex and qualitative realm of politics. In fact, with the exception of Robert Rubin, it is difficult to think of a CEO who had a successful career in government.

Why is this? Well, first the CEO has to recognize that he is no longer the CEO. He is at best an adviser to the CEO, the president. But even the president is not really the CEO. No one is. Power in a corporation is concentrated and vertically structured. Power in Washington is diffuse and horizontally spread out. The secretary might think he’s in charge of his agency. But the chairman of the congressional committee funding that agency feels the same. And White House political operatives think they have ultimate authority on all big decisions. In his famous study “Presidential Power and the Modern Presidents,” Richard Neustadt explains how little power the president actually has and concludes that the only lasting presidential power is “the power to persuade.”

Take Rumsfeld’s attempt to transform the cold-war military into one geared for the future. It’s innovative but (and?) deeply threatening to almost everyone in Washington. The Defense secretary did not try to sell it to the Joint Chiefs of Staff, Congress, the budget office or the White House. As a result, the idea is collapsing. In fact, as the surplus shrinks and the White House confronts choices between spending on education, health care and defense, Rumsfeld is likely to end up with little money and less reform.

Second, what power you have, you must use carefully. For example, O’Neill’s position as Treasury secretary is one with little formal authority. Unlike finance ministers around the world, Treasury does not control the budget. But it has symbolic power. The secretary is seen as the chief economic spokesman for the administration and, if he plays it right, the chief economic adviser to the president. “The podium is your power, so you use it very carefully,” says Robert Hormats, who has spent years in both government and the private sector. “Many of O’Neill’s comments about the dollar, corporate taxation, IMF bailouts and pensions are interesting. But his role is not to toss out nifty ideas. It sets markets on edge.”

O’Neill has been publicly critical of the IMF’s bailout packages for developing countries while at the same time approving such packages for Turkey, Argentina and Brazil. As a result, he has gotten the worst of both worlds. The bailouts continue, but their effect in bolstering investor confidence is limited because the markets are rattled by his skepticism.

Perhaps the government doesn’t do bailouts well. But that leads to a third rule: you can’t just quit. Jack Welch’s famous law for re-engineering General Electric was to be first or second in any given product category, or else get out of that business. But if the government isn’t doing a particular job at peak level, it doesn’t always have the option of divesting itself of that function. The Pentagon probably wastes a lot of money. But it can’t get out of the national-security business.

The key to former Treasury secretary Rubin’s success may have been that he fully understood that business and government are, in his words, “necessarily and properly very different.” In a recent speech he explained: “Business functions around one predominant organizing principle, profitability… Government, on the other hand, deals with a vast number of equally legitimate and often potentially competing objectives–for example, energy production versus environmental protection, or safety regulations versus productivity.”

Rubin’s example shows that talented people can do well in government if they are willing to treat it as its own separate, serious endeavor. But having been bathed in a culture of adoration and sycophancy–what one senior executive called “the Sun Kingsyndrome”–it’s difficult for a CEO to believe he needs to listen and learn, particularly from those despised and poorly paid specimens, politicians, bureaucrats and the media. And even if he knows it intellectually, he just can’t live it. After all, Rumsfeld simply needed to read one of his own “rules,” which says, “In our system leadership is by consent not command.” Yes sir, Mr. Secretary!