Scholars working on the contemporary Middle East are living through something of a golden age where evidence is uniquely plentiful for three reasons.
First, the disastrous failures of US policy in Afghanistan and Iraq have led to an unprecedented programme of declassification of documents (some with significant redactions) as part of the cathartic process of trying to understand how so many mistakes were made before and after 9/11.
Second, the cache of cables dumped by WikiLeaks, coupled with further revelations from material leaked by Edward Snowden, has provided an exceptional level of insight into the workings of the intelligence agencies over the past three decades, together with priceless new information about the decision-making processes and about operational activities.
And third, there has been a cache of materials found locally following the military interventions of the past 12 years - such as audio tapes recovered from the presidential palace in Baghdad in 2003 that recorded thousands of hours of meetings, discussions and even phone calls made by Saddam Hussein and his inner circle, or boxes of cassettes that belonged to Osama bin Laden that were retrieved from a compound in Kandahar two year earlier.
This treasure trove allows us to understand the failures, incompetence and poor planning that accompanied the invasions of Afghanistan and Iraq in astonishing detail, but also to frame these within the context of a wider region - and a wider period. These two countries form part of a belt that stretches from the Mediterranean to the Himalayas, linking East and West, and that for millennia has served as the world's central nervous system. Trade, commodities, people, even disease, spread through the webs of networks that connect these locations to each other and ultimately connect the Atlantic coasts of Europe and North Africa to the Pacific coast of China and South-east Asia.
Dislocation in one part of these networks causes problems in others; equilibrium is easy to disturb, but not easy to restore. What the cache of documents relating to the past decade and a half shows is that the mistakes made were predictable and obvious. They show that the West has never fully understood the Silk Roads.
It is still voguish to argue that George W Bush took on Saddam Hussein as a means to avenge his father, President George HW Bush, who was deemed not to have finished what he started with the first Gulf War in 1990-91. The fact that the younger Bush asked Colin Powell, the new Secretary of State, for clarification about US "regime-change policy in Iraq" within 72 hours of taking office in 2001 certainly suggests that the Bush family had kept a close eye on the situation in Baghdad over the previous decade. But then so too had the Clinton administration during the intervening years.
The US championed the imposition of sanctions on Iraq in the 1990s, strictly enforcing UN Security Council Resolution 687, which forbade "the sale or supply... of commodities or products other than medicine and health supplies" - including "foodstuffs". These measures were intended to force disarmament, including the termination of biological and chemical weapons programmes, and to force agreement on recognition of the sovereignty of Kuwait.
The impact of the blanket restrictions on Iraqi exports and financial transactions was devastating - especially on the poor. Initial estimates in The Lancet in 1995 suggested 500,000 children alone died from malnutrition and disease as a direct result of the sanctions over the course of five years. When Madeleine Albright, the Secretary of State, was interviewed on the television programme 60 Minutes and asked about the fact that more children had died in Iraq as a result of sanctions than in Hiroshima in 1945, she replied nervelessly: "I think it is a very hard choice." Nevertheless, she went on, "We think the price is worth it."
US insistence to reshape Iraq was made law in 1998, when President Bill Clinton signed the Iraq Liberation Act, making it the formal "policy of the United States to support efforts to remove the regime headed by Saddam Hussein from power in Iraq and to promote the emergence of a democratic government to replace that regime". Clinton also announced that millions of dollars were being made available for "the Iraqi democratic opposition", with the express aim of enabling the dissonant voices opposed to Saddam to "unify [and to] work together more effectively". So when George W Bush asked for clarification about Iraq when he became President, he was assessing the long-standing commitment of the US to intervene in the internal affairs of another sovereign state - whose leader was not as co-operative or as malleable as Washington would have liked.
This pattern had its parallels elsewhere across the Middle East in the decades after the Second World War, where those who were thought to be unsupportive of US national interests were removed (such as Mohammad Mossadegh in 1953, or King Farouk of Egypt, who was the target of "Project Fat F*****" and removed four years earlier). Conversely, those who led regimes in countries such as Iran and Pakistan that were flawed, corrupt and incompetent were lavished with money, support and arms - and, in the case of the former, advanced nuclear technologies, supplied by Washington.
Control of the region lying between East and West was seen in the later 20th century through the prism of the Cold War and of competition with the Soviet Union. Nevertheless, the mineral wealth of the heart of Asia, particularly its oil and gas but also its pipeline infrastructure, made the belt commercially important as well as strategically vital.
So perhaps it was not surprising that two days after the 9/11 terrorist attacks took place, an action plan was issued that set out the importance of engaging Iran and of contacting the authorities in Turkmenistan, Uzbekistan, Kyrgyzstan, Kazakhstan and China - Afghanistan's neighbours and near-neighbours. A plan was set out to "[re-]energise" them urgently, with a view to preparing them for forthcoming military action against the Taliban. The first step of the response to 9/11 was to line up the countries of the Silk Roads.
In fact, ambitions were soon going well beyond ensuring their co-operation. By 30 September 2001, the Secretary of Defence, Donald Rumsfeld, was offering President Bush his "strategic thoughts" about what the US could and should seek to achieve as part of its imminent "war aim". "Some air strikes against al-Qaeda and Taliban targets are planned to begin soon," he noted, marking the start of military action. It was important, he wrote, to "persuade or compel states to stop supporting terrorism".
What he proposed next, however, was dramatic and astonishingly ambitious. "If the war does not significantly change the world's political map, the US will not achieve its aim." What this meant was then spelt out clearly. "The [United States government] should envision a goal along these lines: new regimes in Afghanistan and another key state (or two)." He did not need to specify which states he was talking about: Iran and Iraq.
While Iraq had long been an American bugbear, the case of Iran was more complicated. The revolution of 1979 had produced a spectacular failure in relations between the two countries; but by the mid-1980s, there were attempts to rebuild ties - partly because of mutual interests in opposing the Soviets in Afghanistan, but also because the US concluded that keeping the Iran-Iraq war going was in their interests. The result was the shipping of armaments to the Khomeini regime, in collaboration with Israel, who saw Saddam Hussein as a direct and serious threat: ties with Tehran became so close between the two that Prime Minister Yitzhak Rabin was able to declare: "Israel is Iran's best friend, and we do not intend to change our position."
There were signs of another thaw in relations with Iran after the terrorist attack on the Dhahran air base in Saudi Arabia in 1996 that killed 19 servicemen. An angry rebuttal was issued to President Clinton's missive that Tehran was complicit in the bombing; but it was followed by a clear opening. The President should rest assured, the reply stated, that Iran had "no hostile intentions towards Americans". On the contrary, the "Iranian people not only harbour no enmity but [also] have respect for the great American people". Including Iran within an "axis of evil" a few years later, amid signs of an improving relationship, was opening a dangerous can of worms.
Then again, opening cans of worms seemed not to worry many of those taking decisions in the weeks and months after 9/11. Despite there being no evidence to link the hijackers to Iraq, attention was focused on a major invasion and on regime change. The question was simple, as planning notes for a meeting between Donald Rumsfeld and General Tommy Franks, the chief of Central Command, make clear: "How [to] start?"
Three possible triggers were envisaged - all of which could justify military action. Perhaps Saddam "moves against the Kurds in [the] north?" wondered Rumsfeld in November 2001; maybe a "connection to Sept 11 attack or to anthrax attacks" (following mailings to several media outlets and to two US senators in September 2001); or what if there were a "dispute over WMD inspections?" This seemed a promising line - as revealed by the comment that follows: "Start now thinking about inspection demands."
Over the course of 2002 and at the start of 2003, pressure was ramped up on Iraq, with the issue of chemical and biological weapons and that of weapons of mass destruction taking centre stage. The US pursued this with an almost evangelical zeal. In the absence of "incontrovertible evidence" of a link between 9/11 and Baghdad, one report noted, only Tony Blair could be relied on to support war, while another underlined the fact that "many, if not most, countries allied with or friendly towards the United States - especially in Europe - harbour grave doubts about... an all-out attack on Iraq". Work therefore went into establishing a legal framework for full-scale war in anticipation of the likelihood that the United Nations would not give a clear mandate for action.
The head of the International Atomic Energy Agency, Jose Bustani, was ousted in a special closed session, pushed out for being uncooperative and unhelpful. Statements issued by weapons inspectors, meanwhile, were ignored. In January 2003, it was declared that "we have to date found no evidence that Iraq has revived its nuclear weapons programme since the 1990s" - which chimed with an update the same day by the United Nations Monitoring, Verification and Inspection Commission that although inspectors occasionally faced incidents of harassment, "Iraq has on the whole co-operated rather well so far" with the demands of inspectors.
This was rubbished by Colin Powell when he addressed the UN on 5 February 2003, and claimed that "every statement I make today... is backed up by sources, solid sources. These are not assertions. What we're giving you are facts and conclusions based on solid intelligence." They were nothing of the sort: trailers that were described as mobile biological weapons facilities "hidden in large groves of palm trees and... moved every one to four weeks to avoid detection" turned out to be weather balloons - just as the Iraqis had said they were. There was no nuclear weapons programme, just as the Iraqis had said. No support had been given to al-Qaeda or terrorists, either - as documents and audio tapes from Baghdad reveal: in fact, Saddam Hussein had reined in all those suspected or implicated in terrorism, in order to avoid punitive action.
It was not just the decision to invade Iraq that was spectacular for its idiocy; so too was the execution of the invasion plan. It was naively assumed that removing Saddam would turn Iraq into a land of milk and honey. There was no need to worry, insisted Paul Wolfowitz, the former president of the World Bank, who was then serving as Deputy Secretary of Defence, eight days after the invasion began in 2003. "We're dealing with a country that can really finance its own reconstruction, and relatively soon." Oil revenues, he breezily predicted, would bring in $50bn to $100bn over the next "two or three years".
Expectations for the involvement in Iraq were as foolish as they had been in Afghanistan, where it was assumed there would be "no military involvement after the Taliban were defeated". In Iraq, 270,000 troops would be needed to start with, according to plans drawn up by US Central Command; but three and a half years later, there would be no need for more than 5,000 ground troops. This all looked plausible when presented on PowerPoint slides to those who saw what they wanted to see. These were to be light wars, quick strikes that would enable a new balance to be established across a pivotal region of Asia - all to the advantage of the West.
Few today believe we did the right thing in supporting the attack on Iraq. Even Jeb Bush recently declared that he would not have supported it had he known then what he knows now. The cost and consequences of the military intervention have been catastrophic. The disruption in the Middle East has caused a fracture of Iraq and the rise of Isis if not in its place (yet), then as a new power to be reckoned with; the Taliban has regrouped and eats into the weak structures left behind by coalition forces on a daily basis; reputational damage to the West in the eyes not only of the Arabic-speaking world but beyond has been substantial too; then there is the credibility of the intelligence agencies who allowed reports to be "sexed up" by political masters to suit their own ends.
And there is the cost: not only the lives lost by servicemen, the value of which cannot even be estimated, not the tens of billions spent on the war. The biggest cost of the war, as new research from Harvard suggests, is the cost of looking after the 170,000 veterans who are 70 per cent or more disabled as a result of their injuries. The long-term cost to the US economy is estimated to be $6 trillion (PS3.9trn) - or $75,000 for every single household in the United States.
If there is a silver lining, it is - perhaps - the shuffling of decks that brings Iran back to the table as a mainstream player within the region. The West finds itself short of friends in a region it has interfered in for too long with disastrous effects. The same mistakes have been made in the same region for too long.