When Abraham Lincoln arrived in New York in February 1860, he must have felt a long way from Springfield. He was in town to speak at Cooper Union, in what amounted to his only major campaign trip in securing the Republican nomination for president. During his short stay, the gangly country lawyer was confronted with sights he would never forget: a swarm of carriages, horses and pedestrians on Broadway below 10th Street; Henry Ward Beecher’s packed church in Brooklyn; and the crippling poverty and homelessness of the notorious Five Points slum.

“Dorothy, you are not in Kansas anymore,” said Barry Lewis, the architectural historian known for his walking tours of New York on PBS. “I can’t imagine what he would have thought because he was a small-town guy. I’m not sure he could have handled New York; it was just too complicated, too many people, too many contrasts between rich and poor.”

Mr. Lewis had just joined Lincoln historian Harold Holzer at the New York Historical Society on Wednesday, Oct. 14, to discuss the 16th president’s complex relationship with America’s largest city. In front of a large, appreciative audience, the two discussed each of Lincoln’s three trips to New York.

Lincoln’s first trip is generally remembered for his defining address at Cooper Union. (Barack Obama, another rangy politician from Illinois, would give another highly touted speech there during his run for the White House in 2008.) Yet Mr. Holzer and Mr. Lewis suggest that it was Lincoln’s visit to Matthew Brady’s photography studio that may have best served his lofty ambitions. Not long after Lincoln sat for his portrait, there were more than 40,000 copies of the photo spread throughout the country.

Though he is remembered as stoic and retiring, Lincoln was a savvy manipulator of public perception, Mr. Holzer further noted. After his Cooper Union speech, Lincoln didn’t return to his hotel, but instead hurried to Horace Greeley’s New York Tribune to further tweak the version of the speech that would be sent to smaller papers across the country.

“He knew that this was the way to get his name up in lights,” Mr. Holzer said, after signing books for audience members. “Stars want to appear on Broadway; he had to appear in New York.”

When Lincoln returned a year later as president, his visit stopped traffic. Walt Whitman recalled being trapped on a side street as Lincoln moved through the city. (Anyone stranded when the Obamas shut down Sixth Avenue can surely sympathize.) But that short visit would be his last. New York was always a thorn in Lincoln’s side. The city twice voted against him, and the mayor even threatened to make New York a “free port” by seceding from the Union. Then came the Draft Riots of 1863, in which angry mobs perpetrated some of the ugliest racial violence in the nation’s history.

Following Lincoln’s assassination, however, as his cross-country funeral moved through the streets of New York, more than a hundred thousand people clamored to honor him.

“There was a sorrow that he had been so reduced physically,” Mr. Holzer said, standing in the shadow of the massive cast of Lincoln’s head that occupied the center of the foyer. “It seemed he had carried the suffering of the country in his face.”

One young New Yorker was particularly moved.

“Teddy Roosevelt was leaning out of his window to see it as a child, and he never forgot it.”