Maybe it was inevitable – some evil destiny – that Oakland would be the place where an honorable protest against national inequality and injustice would go horribly wrong.

Ever since it lost the jobs brought by the shipyards and other heavy industry during and after World War II, the city has been a place with great gaps of income between the affluent in the hills and the poor residents, mostly black, in the flats; of high crime, of endemic public mismanagement and the residue of a racist police culture.

The racism came with the white Southerners who, like many blacks, were drawn by the jobs in the West during the war. Because of it, the city seems to teeter indefinitely between left-wing political correctness and Bull Connor-style police repression.

Is it possible, as Wallace Stegner used to say about California, that Oakland is like the rest of the country, only more so?

Still, you have to ask why. What is it about this city particularly – with its unmatched location and climate, its great port and waterfront, its rich history – that keeps bringing misery down upon it like some Olympian curse? Why do tens of thousands of commuters to San Francisco sit for hours each day in bottlenecks at the Caldecott Tunnel and the Bay Bridge when they could ride a ferry from Jack London Square and be downtown in 30 minutes?

Or did this struggling city with its many poor people just happen to be a convenient landing place for the dedicated reformers, street people and crazies, some of them from other cities and even other states, who call themselves Occupy Oakland? In what way was poor Oakland a surrogate for Wall Street?

A decade ago, Oakland, then with 400,000 inhabitants, seemed at last to be struggling out from under its curse. Then in 2003, after four rogue cops had been found brutalizing and robbing suspects and planting evidence, it paid $10 million in damages in a civil suit and agreed to a broad settlement in federal court to clean up and monitor its police force. Although they stood trial twice on criminal charges, the cops themselves were never convicted.

At about the same time, Mayor Jerry Brown, trading on a booming national economy, made deals with real estate interests to develop 6,000 housing units – enough to bring another 10,000 middle-class residents to the city's lifeless downtown. Some community activists, complaining that it would force poor people out, called it Jerryfication. He pushed to bring more art and cultural enterprise downtown and to Jack London Square. He put a couple of Venetian gondolas on Lake Merritt. He tried to reform the schools.

That his deals involved tens of millions of dollars in the kind of redevelopment funds he now disdains is another story. And it was probably inevitable that the 2008 recession would leave nearly half of the planned housing undone. There's now a gleaming new office and retail building at the waterfront that's been vacant since its completion.

More telling, perhaps, is that the Oakland Police Department still hasn't complied with many of the provisions of the redundantly named Negotiated Settlement Agreement that it signed in 2003. Given the cops' record, every complaint of brutality, no matter how far-fetched, has some resonance.

Thelton Henderson, the federal judge who took over medical care in state prisons and who has monitored the Oakland settlement, threatens to take over the Police Department if it doesn't shape up by January. It's that threat as much as Police Chief Anthony Batts' well-publicized frustration with his bosses at City Hall that appeared to lie behind his resignation last month. Oakland now has its third police chief in just over two years.

Worse, the city's severe fiscal problems, along with its liberal budget priorities, have driven the police force down from just under 800 officers five years ago to 637 today and maybe as few as 570 next year. In the meantime, the city is still paying $10 million a year for the deal it made in 1995 to bring Al Davis and his Raiders back to Oakland.

Most of the rap for those messes belongs to the bosses. Jerry Brown is just a bright aberration in a lackluster parade. Ron Dellums, mayor for four years after Brown – from 2006 to 2010 – was famously absent as often as he was present. Jean Quan, mayor for the 10 months since, had collaborated with two of her opponents and shrewdly worked Oakland's ranked voting system to beat former state Senate leader Don Perata, the political heavyweight who was favored to win. Their refrain, "Anybody but Perata," worked.

But Quan, long a liberal activist, seems to have been paralyzed by the hard realities of the job, first allowing the cops to evict demonstrators from their encampment outside City Hall, then allowing them back, then pleading with them to leave voluntarily, then declaring that they must go. She now faces a possible recall launched even before protesters camped outside City Hall. She's been attacked relentlessly by councilmember and labor union officer Ignacio De La Fuente, a client and close ally of Perata, who may see the recall as another shot at the mayor's job.

But beyond Quan's vacillation lies a political establishment that has rarely been able to decide which way to turn. The mayor and council resisted the entreaties of Batts and City Attorney John Russo for curfews and broader injunctions against the gangs that terrorize parts of Oakland. Russo, maybe the ablest public official Oakland had, has since quit. They've been divided about priorities between the understaffed police department and maintenance of a generous package of youth programs and other social services. In part because they can't agree on those priorities, voters last week trounced their proposal for an additional $80 in parcel taxes.

As to the political correctness, some prominent East Bay politicians, among them Dellums and Rep. Barbara Lee, lent formal support to a Black Muslim Bakery whose leaders had long been implicated in shootings, rapes, kidnapping, torture and all manner of corruption. That came to an end in 2007 with the murder of journalist Chauncey Bailey by a bakery janitor. In June bakery leader Yusuf Bey IV, who had first sweet-talked the shooter into accepting all responsibility, was convicted of ordering the killing and sentenced to life.

Arguably, the most hopeful part of the Oakland story at this moment is in the schools, which two years ago were finally freed from the state oversight that was imposed after flagrant fiscal mismanagement in the 1990s. In the past five years they have made significant gains in scores on the California Standards Test.

There also was a hopeful sign this month in the response of five out of seven council members – in the past often fiercely divided – to the demonstrators who shouted them down at a press briefing and to the killing of a young man at the periphery of the encampment at City Hall. For the first time they seemed to agree that the encampment had to go. If that resolve extends to other matters, it could make a big difference.

What makes it all unpredictable is demographics. In the decade since the 2000 Census, Oakland lost about 10,000 inhabitants. In the same period, however, it lost 33,000 African Americans – roughly one fourth of its black population. Replacing them were some 12,000 Latinos, 5,000 Asians and 7,000 whites. As a result, the city is now 25 percent Latino, 26 percent white and 27 percent black.

Most immediately, that means greater social diversity and, most likely, a broader range of skills and economic circumstances. It could also mean that before the next census – long after the marches and melees of the last month are forgotten – Oakland's politics and its fortunes, like the nation's with its great influx of new Latino voters, could change significantly. It's also possible that the national wave of Occupy protests, notwithstanding their lack of a political agenda and the anarchic noise at their fringes, could have a profound national impact. In any case, Oakland has too much going for it to remain forever a synonym for defeat and despair.

© Copyright The Sacramento Bee. All rights reserved.

Peter Schrag, a retired editorial page editor of The Bee, writes frequently on California issues and has lived in Oakland for the past 14 years.

• Read more articles by Peter Schrag