Known in the military as “ghost soldiers”, they either did not exist or no longer reported for duty, however their salaries were still paid.
A statement from the PM’s office said the payments have been stopped.
Correspondents say rampant corruption in the Iraqi army is seen as one of the reasons why it has struggled to contain Islamic State militants.
A spokesman for Prime Minister Haidar al-Abadi, quoted by AFP news agency, said the investigation began when the latest salary payments were made.
“Over the past few weeks, the prime minister has been cracking down to expose the ghost soldiers and get to the root of the problem,” said Rafid Jaboori.