At six o'clock on a freezing November morning, Hillary Clinton's famously rumpled and gruff top strategist, Mark Penn, was hailed by another giant of her campaign for the Democratic nomination at the departure gate of the shuttle to Washington DC.Terry McAuliffe, a notorious blowhard of the party, wanted to know what Mr Penn had made of the event they had both attended the previous night, when the candidates (there were nine Democrats in the race at that stage) each addressed a raucous crowd of sign-waving, feverishly excited supporters inside an echoing basketball arena.

After a series of earnest and dull addresses by Hillary Clinton and lesser lights of the presidential campaign, Barack Obama had sprinted on to the stage to deliver a mesmerising speech that brought the house down. It would prove to be a turning point in what until that point had been a faltering campaign that had shown more promise than substance.

The core of Mr Obama's message was a promise that if elected he not only intended to unite Republicans and Democrats, "Red States and Blue States" but that he would also clean out the Augean stables in Washington DC. He would end forever the excessive influence of lobbyists and venal politicians on the way America does business. To that end his campaign would have nothing to with lobbyists and accept none of their money. It was to be a people's movement, rather than a campaign waged under the old rules. The speech could have been written with Mark Penn in mind. As "Worldwide chief executive" of the British-owned lobbying firm Burson Marsteller and chief strategist to Mrs Clinton, he saw no conflict of interest in his two roles.

One minute he was representing the controversial mercenary company Blackwater, whose guards had killed numerous unarmed Iraqi civilians, and the next he was advising Hillary Clinton not to apologise for her Senate vote to go to war in Iraq, but to say instead that "the vote turned out to be a terrible decision for everyone". Another controversy flared over Mr Penn's role in representing Wal-Mart and its efforts to defeat union organising campaigns.

To many Democrats, Mr Penn is seen as a figure who represents for Bill and Hillary Clinton what Karl Rove was to George Bush: a cynical manipulator of numbers, who can magically steer a candidate through controversies and moral hazards by looking at the polling figures and advising them where to position themselves.

On that early morning flight from Iowa to DC, Mr Penn, a pollster by trade, expressed himself more than happy with Hillary's dry-as-a-stick address to the Democratic audience in Des Moines. What mattered, he confided, was the polling data which pointed to an easy win in Iowa. Reports that dyed-in-the-wool Iowa Republicans were openly expressing interest in Mr Obama's inspirational message of unity was just anecdotal fluff, he said. Hillary Clinton was going to win the first and most important contest in the primary race. The answer was in the polling numbers, Mr Penn said as he headed for the first-class section.

So confident was he in the inevitability of Mrs Clinton blowing away the precocious upstart, and wrapping the nomination up by 5 February, or Super Tuesday, that he predicted he would not have to do much travelling during the campaign. Instead he would hole up at Burson Marsteller's sleek DC headquarters. "I'll stay in Washington and monitor all the polling remotely," he said, changing the subject to one of his proudest achievements, his role advising Tony Blair and helping to steer him to his third electoral victory by getting him to focus on the small-bore issues that appeal to voters. But Mrs Clinton is increasingly struggling to secure the nomination with the numbers stacked up against her.

As one of the most influential political advisors of his generation, Mark Penn finally fell on his sword on Sunday. He resigned from the Clinton campaign after he was caught by the conflict of interest between his work as a Washington DC lobbyist and his role as her chief strategist.

In an act of extraordinary hubris, Mr Penn rolled up at the Colombian embassy in Washington last Monday to advise the ambassador on the best way to get a controversial free trade agreement through Congress. The compant, which is a wholly owned subsidiary of the advertising company WPP, was billing $300,000 for a year's worth of advice on a trade deal that the Clinton campaign is specifically and publicly opposed to.

American trades unions oppose the trade deal because they fear the loss of jobs. Human rights groups oppose it because of the actions of death squads linked to the government of ÃÂlvaro Uribe. At first the Colombian embassy described the meeting as one of a series with Barack Obama's team and that of the Republican candidate, John McCain.

But when Mr Penn publicly apologised for what he said was "an error of judgement" in attending the meeting the Colombians were furious at the "insult" and promptly sacked him and his firm on Saturday. By Sunday, his position in the Clinton campaign was also untenable with Hillary Clinton reported to be "furious" with him for his indiscretion.

Even though Mr Penn has been among their most loyal backers since the darkest days of the Monica Lewinsky scandal, both Bill and Hillary Clinton agreed he should be cut adrift from the campaign on Sunday. Mr Penn will, however, continue to give discreet advice and his polling firm Penn, Schoen and Berland Associates will still be used to guide the campaign in the difficult weeks ahead.

Despite the terse statement from the campaign that "after the events of the last few days, Mark Penn has asked to give up his role as chief strategist of the Clinton campaign", the impression lingered that he will continue to whisper advice, but this time from more of a distance.

With a blizzard of polling data to back up his position, Mr Penn advised Mrs Clinton to go on the attack against Mr Obama. Other advisers cautioned that this was a fatal error and that the voters had responded better when they saw her human side. It was glimpses of her vulnerability after her loss in Iowa that enabled her to rebound they said.

For months top advisers to Hillary Clinton have been telling her to get rid of Mr Penn, whom they blamed for some of the most spectacular miscalculations of her once "inevitable" campaign for the nomination. One of Mr Penn's mistake, it appears, was to assume that Mrs Clinton could win the nomination by focusing solely on the big states in the primary season, all but ignoring the smaller states that typically vote Republican in a general election.

This gave an opening to Mr Obama who tapped into his cross-over appeal among Republicans and independents in these smaller states most of which he won handsomely ending up with more delegates than Mrs Clinton despite all her big state wins. By the time Super Tuesday rolled around, Mr Obama had built up enough momentum to halt the advance of the Clinton machine.

But the greatest error was Mr Penn's belief that he could win the election by running Hillary Clinton's strategy to take the White House in the way he advises his PR firm's clients: drilling down into the polling data to discover what his book, Microtrends, describes as "the small forces behind tomorrow's big changes".

Mr Penn's philosophy boils down to the belief that "Americans overwhelmingly favour small, reasonable ideas over big, grandiose schemes". He calls it "niching", writing that "there is no one America anymore" but "hundreds of Americas".

Mr Penn is Bill and Hillary Clinton's God of small things and his poll-driven approach to politics has worked wonders in the past. He gave her the confidence to launch her first Senate run, telling her that New Yorkers were open to her running. And despite the deep undercurrent of anti-Clinton sentiment in the country, Mr Penn's polling persuaded her that she could win the nomination and the White House by running as a hawkish no-nonsense candidate who was at ease with picking up the baton in George Bush's war on terror. Mr Penn's polling numbers had worked their magic for Hillary and Bill Clinton many times before.

But inside the Clinton campaign there was fury that Mr Penn's obsession with polls allowed Mr Obama to capture the public's imagination by declaring himself to be the agent of change, a theme that has come to define the 2008 election.

The tensions between Mr Penn and advisors occasionally burst into the open as when Clinton campaign operatives gathered in her headquarters in Arlington, Virginia to preview a TV commercial. "Your ad doesn't work," Mr Penn yelled at Mandy Grunwald, one of the Clinton's closest friends in the advertising business. "Oh, it's always the ad, never the message," Ms Grunwald fired back, in a clash that got so heated that the campaign's political director Guy Cecil left the room, saying, "I'm out of here."

The latest upheaval in the Clinton campaign comes at a time when she only has a narrow path to winning the nomination and that depends on her winning the Pennsylvania primary on 22 April in two weeks' time. When the race for the Democratic nomination is finally wrapped up, it will all come down to a contest between Mr Obama's inspiration and Mr Penn's divination.

Another Clinton adviser whispered this weekend as Mr Penn was being hung out to dry: "Nothing can stop Obama now."

--Leonard Doyle

© 2008 independent.co.uk