The US and Iran engage in a tantalising dance

President Hassan Rouhani's speech to the UN General Assembly promises a sea change after 33 years of troubled relations between the Iran and the US - and perhaps a new start over issues ranging from stalled nuclear negotiations to the Syrian crisis and the sad state of human rights and individual freedoms for Iranians.

Both sides show positive signs, but poisonous vibes are not rare either, be it editorials of hardline dailies in Tehran or the alarming statements of Israeli Premier Benjamin Netanyahu and his supporters in the US Congress. Symbolic gestures such as the release of prisoners of conscience, though mostly insiders who had gone awry, and Jewish New Year greetings are part of Rouhani's "charm offensive". New appointments such as Mohammad Javad Zarif as minister of foreign affairs and other level-headed officials in non-ministerial positions are also promising even though the new cabinet is not completely free of figures with dark political histories.

Rouhani not only enjoys the mandate of the majority of Iranians, but also the conditional blessing, albeit grudgingly, of Supreme Leader Ali Khamenei.

The Supreme Leader - constitutionally accountable to no earthly authority and certainly not the will of his own people - went so far as unambiguously backing the new president in his demand for the Revolutionary Guard to stay out of politics. This may be seen as a signal for a freer press, more individual freedoms, and less intervention and favouritism in the economy. Khamenei has made no similar concession during his 24 years as the Guardian Jurist.

This change of heart - driven by a pragmatism long practiced by the revolutionary clergy - emerged because of the realities of today's Iran: social disillusionment, political isolation and an economy in serious trouble, thanks to the state's mismanagement and debilitating international sanctions. Moreover Iran's new middle class, with its own youth culture, is no longer content with hollow revolutionary slogans of the past. Political demands for a fair election were clear in June 2009's widespread protest movement against rigging ballot boxes, which kept previous president Mahmoud Ahmadinejad in office.

The 2013 election results confirmed public repulsion for rampant corruption and mismanagement, favouritism and skullduggery under Ahmadinejad. If any more catalyst was needed, it came with western-imposed sanctions that crippled the Iranian economy and disrupted the lives of ordinary people. Soring prices combined with banking restrictions, volatile rates of exchange and shipping insurance bans reduced Iran's oil exports well below 50 per cent, compared to 2010 figures. In effect, the sanctions deprived a rentier state of its addictive oil revenue. For decades, the free flow of oil had allowed the clerical elite to project an image of power toward its own citizens and gestures of defiance, often futile, toward the outside world.

Facing an array of domestic and international challenges, the new president has a relatively narrow window, perhaps no more than six months to a year. But he has some leverage, no doubt, in his negotiations with the US and its allies. Iran could be a key player in reining in the Syrian regime. Flexibility on the nuclear issue may also persuade the West to ease sanctions and offer economic incentives.

President Barack Obama's warning in his speech to the UN General Assembly that the US does not tolerate the production of weapons of mass destruction, should be seen as an attempt to address these concerns. Rouhani's references in his speech to tolerance, mutual respect, peace and non-violence, on the other hand, indicated a desire for fruitful negotiation and settlement of old differences.

Judging by the experience of three earlier administrations in post-revolutionary Iran, Rouhani faces a difficult course. Mahdi Bazargan's provisional government was brought down after eight months in office at the outset of the 1979 hostage crisis, and he and his cohorts, labelled as "compromising liberals", were banished by Ayatollah Khomeini to a political wilderness from which they never returned.

The presidency of Abolhasan Banisadr was doomed from the start, even after receiving Khomeini's momentary blessing in February 1980. Thanks to the militant clergy and lay allies, his political end came 15 months later with his dramatic flight to exile in Paris. And 16 years later, Mohammad Khatami's moderate, albeit feeble, presidency was contained to the point of inaction by a radical trends almost a year after he assumed presidency. Undermining Khatami's government eventually brought Ahmadinejad to power and made possible his farcical style and messianic message.

The window of opportunity may close even sooner for Rouhani once the hardliners around the Supreme Leader have licked their wounds and regrouped. If Rouhani cannot deliver even a mild version of what he promised, he will be vulnerable to attacks from opponents. Making headway with nuclear negotiations, relaxing sanctions and improving the domestic economy all depends on a large extent on the international community, and in particular the US, and their appreciation of Rouhani's quandary. His failure may very well result in greater militancy and deeper resentment and may even trigger a military strike that would be disastrous to Iran, to the security of the region and to the US's standing in the Middle East.

Beyond the good will of the international community, Rouhani's success or failure may hinge on a systemic problem in Iran's ancient political culture. Since the emergence of the office of the minister, or vazir, in the early Islamic era, there has been a fundamental tension between rulers - be it sultans or shahs - and the minsters that headed the state administration. Claiming divine mandate, the rulers were unaccountable to their subjects even though they were expected to preserve the social order, and maintain peace and security. The ministers, on the other hand, were not immune from the ruler's rebuke, dismissal and even loss of life. The tension between the two offices led in Iranian history to many ministerial casualties, prompting the notion of "vaziricide", or "killing of the chief minister".

Rouhani's political future may well fall victim to the same systemic tension. Or, if domestic and international circumstances give him sufficient time and a winnable chance, we may yet see, after decades of post-revolutionary exhausted rhetoric, a new political maturity that could overcome the demons of a Manichean political culture.

Abbas Amanat is professor of history and international studies at Yale University and director of the Programme in Iranian Studies at Yale MacMillan Centre for International and Area Studies.

The Whitney and Betty MacMillan Centre for International and Area Studies at Yale University.