The Historic Florence Foundation withdrew its appeal Tuesday to stop the demolition of the old Trust building on West Evans Street in Florence.
The group says it didn't have legal standing to appeal and didn't want to waste money on attorney fees and court costs.
A hearing on the appeal was scheduled for Wednesday.
The Florence Design Review Board granted city council a certificate of appropriateness to tear down the 97-year-old building.
The city wants the land so Francis Marion University can build its medical school in that location.
The Florence Historic Foundation says it's not giving up and will still try to persuade city leaders to choose another site for the medical school.