Most of the fatalities were reported among the laborers sleeping inside the building, said the local news agency Asia News International, citing police sources.
According to police officials at the scene, dozens of workers were inside the factory when the fire broke out in Sadar Bazar area.
The factory was operating in a residential area. Firefighters rushed to the scene after the fire was reported early morning.
The fatalities are feared to rise as some sustained grave injuries.
Delhi fire services chief Atul Garg said: “We have saved almost 50 people so far from the site.”
“It was really difficult to conduct rescue operation because of narrow lanes. Fire brigade and ambulance could not enter the area because it has been densely populated,” Garg told media.
He also said that factory was filled with packing material. Those injured were rushed to Hindu Rao Hospital and RML Hospital.
The officials are yet to find out the cause of the fire.
Prime Minister Narendra Modi expressed condolences for the victims of the tragedy.
“The fire in Delhi’s Anaj Mandi on Rani Jhansi Road is extremely horrific. My thoughts are with those who lost their loved ones. Wishing the injured a quick recovery. Authorities are providing all possible assistance at the site of the tragedy,” he said on Twitter.