Officials said a total of 38 cars derailed, which sparked a small fire. The fire involved "hazardous materials," spokesperson Justin Wilson said.
The agency shared photos on social media of smoky air over buildings in the city northeast of downtown Atlanta.
Officials went door to door evacuating homes for starting at 2 a.m. as the pinkish-orange plume of smoke rose into the air.
"We were just in panic mode and wasn't sure what was going on," resident Jaquelyn Bustamante told FOX5.
Evacuation orders were eventually lifted around 6 a.m., but crews were still continuing "atmospheric monitoring" in the area.
The derailment appears to have been caused by a washed-out track from intense rain and flash flood conditions in the area, CSX said in a statement.
The railroad told FOX5 that of the 37 cars that derailed, at least two cars carried ethanol which leaked from the accident.
Two employees involved in the derailment were able to get to safety and were being evaluated for possible injuries, according to local rescue officials. No injuries were immediately reported.
State and federal water management personnel told FOX5 they plan on conducting tests to make sure none of the chemicals got into a nearby creek or the runoff.
Additional flooding along the same creek led to additional rescues in Gwinnett County.
Firefighters in Gwinnett County shared a video of first responders rescuing a pair from inside a car in the middle of the flood along Hillcrest Road.
The fire service described the two rescued individuals as being uninjured.
The remnants of Hurricane Delta also brought severe weather to parts of the South as the system moved up the East Coast.
Horry County Fire Rescue said Sunday that a possible tornado was reported around 4:34 p.m. in Conway, with "multiple structures" that were damaged.
"One person suffered minor injuries and was treated on scene," the agency tweeted.
Two tornadoes were confirmed to have touched down on Saturday as the remnants of Delta passed through Georgia which caused minor damage, FOX5 reported.
What's left of Delta is bringing heavy rain to parts of the mid-Atlantic and East Coast throughout the day on Monday.
There's also an area of low pressure moving across the upper Midwest and Great Lakes that will keep the wet weather in the forecast through Tuesday.
The heaviest of rain will fall over parts of Northern New England, which will help with the ongoing drought in the area.
Fox News' Janice Dean and the Associated Press contributed to this report.