The pugnacious, golf-loving chairman of mega-landlord Vornado Realty Trust, Mr. Roth won the bid to build a tower from the Port Authority in 1999. But his initial plans fell apart in 2001, when a tenant backed out; a lawsuit followed, and over time he gradually revived the deal.

Now, once again, the idea of a tower atop the bus station is on track… to getting shelved. At least for the most part.

After crafting a design for a 40-story tower with British starchitect Richard Rogers in 2008, Vornado’s status as the designated developer is set to expire in August—that is, unless Mr. Roth is ready to start paying worth $500 million over the 99-year lease.

He’s not. He never wooed a tenant for the 1.3 million-square-foot building (law firm Paul Weiss was kicking the site’s tires last year before the economy imploded) making it unfeasible in this market.

But all hope is not lost for Mr. Roth, who declined comment.

According to an official familiar with talks, Vornado and the Port Authority are discussing a possible two-phased plan in which the real estate firm would renovate and expand retail in the dreary bus hub.

ebrown@observer.com