The Pritzker Prize-winning Mr. Rogers was brought on for the project by Pataki development chief Charles Gargano in part as a means to draw public support for the project. Mr. Rogers designed, among other projects, Paris’ Centre Pompidou museum with Renzo Piano.

Left with just a renovation (to the tune of at least $800 million) and possibly a modest expansion, the architectural fees grew to a point where the oversight board of the Javits expansion was concerned about costs, according to multiple people familiar with discussions. Last month, the board members were presented with a preliminary yearlong contract of architectural fees of more than $20 million, according to those familiar with the discussions, an amount the board wanted reduced substantially.

Mr. Rogers is not without work in the city—he is the architect for Tower 3 at the World Trade Center, where early construction work has begun.

The New York-based FXFowle, the ESDC spokesman said, is still on the architectural team.

Update: 5:00 p.m.

ESDC spokesman Warner Johnston wrote to say that given the change of scope in the project, "Rogers Stirk Harbour + Partners – in full agreement with the ESDC and its joint venture partners, FXFOWLE ARCHITECTS and A. Epstein and Sons International, Inc – believes that the present objectives are best served by the client working with a local team and has withdrawn from its role as lead designer."