First the Harkness Mansion on East 75th Street sold last year for $53 million; then, this year, two spreads at the redone Plaza have sold for even more. But recently, without officially putting anything on the market, Seagram liquor heir and Warner Music Chairman and C.E.O. Edgar Bronfman Jr. has been quietly asking over $61 million for his 31-foot-wide townhouse at 15 East 64th Street.

The house is now in contract to be sold—almost five years since it was temporarily listed for $40 million. Though the sales price falls below the Harkness record of $53 million, according to a source, this will be only the second townhouse in New York City history to break the $50 million mark.

Deliciously, Mr. Bronfman reportedly paid only $4.375 million for the building in 1994, though the four-year gut job to re-combine its nine apartments into one townhouse probably wasn’t cheap.

Perhaps as extraordinary, according to a second source, Mr. Bronfman has downsized. He’ll be replacing that steel-paneled, two-courtyard townhouse with a matronly apartment at 1040 Fifth Avenue, co-listed by the brokerages Edward Lee Cave and Sotheby’s International for $19.5 million.

He didn’t employ any brokerages to sell his East 64th Street house this time around, that source said, although at least one agent was given less than two weeks to bring in a potential buyer for the 1916 mansion, at a price tag of more than $61 million.

Reached through a publicist, Mr. Bronfman declined to comment. But what was his old place like? According to a New York Times profile from 1999, there was a “house manager” and a private guard; a vestibule sheathed in bronze-shaded waxed-steel; a 2.5-story indoor piazza; a Nigerian fertility statue; and a secret doorway leading to the upstairs bedrooms.

The house was “pushing upward of 15,000 square feet,” so naturally there was a terraced children’s playroom, a master bedroom with in-facing balcony (and a library with a glass-floored balcony, too). Then came broom closets on every floor: wife Clarissa Bronfman had said she didn’t “want to see any mop buckets in the elevator.”

One can only hope the family will schlep their Mies van der Rohe furniture and Roberto Matta art to 1040 Fifth, a five-bedroom apartment. Its children’s playroom and Central Park-fronting living room are both over 27-feet-long, but on the downside, monthly maintenance is $8,614.

Cave broker Armin Allen, who declined to discuss the deal, said the 1040 Fifth spread isn’t mansion-like. “They’re just not vast,” he said about the rooms, “the way you have [with] some apartments, for instance at 740 Park, which has extremely large living rooms and entertainment space.”

Mr. Bronfman will have to make do with the wood-burning fireplace in his new library, though there’s no glass-floored terrace.