Colombian senators have been calling for the men, who were based in the country, to be extradited from the US.
But US ambassador William Wood said the soldiers are immune from prosecution.
More than 200 Colombian citizens have been extradited to the US to face trial for drug trafficking, under a bilateral deal between the two countries.
Colombian politicians have asked the government to push for the US to hand over the men, arguing that the extradition agreement works both ways.
"In practical terms, these military personnel committed the alleged crime in Colombia, and according to the extradition treaty, which is bilateral, they should be tried here," legislator Gustavo Petro said.
President Alvaro Uribe, who is visiting China, has said he will review the issue "very carefully".
The US ambassador in Colombia has said the men cannot be extradited because of an agreement between the two countries signed in 1974.
He also argued that the men were working for US embassy staff in Colombia and therefore qualify for diplomatic immunity.
He sought to assure Bogota that the soldiers, who are thought to be in military custody somewhere in the US, would not escape justice.
"We do not tolerate corruption," he said.
The whole affair has been extremely embarrassing for the US, which supplies Colombia with hundreds of millions of dollars in aid to fight drug trafficking, says the BBC's Jeremy McDermott.
However, he adds that the Colombian authorities are unlikely to insist upon the extradition because they depend on the US aid.
Five US soldiers were arrested at a US military base in Texas after they stepped off a flight from Colombia on 29 March.
They are suspected of attempting to smuggle 16kg (35 lb) of cocaine on a US military aircraft.
According to the Associated Press news agency, one of the men was later released.
The agency also reports that three of the five suspects were initially detained on Colombian soil - a point Colombian senators say supports their demands for extradition.
The US has more than 1,000 soldiers and civilian contractors working in Colombia as part of a plan to combat a 40-year-old Marxist insurgency and one of its major revenue sources - the cocaine trade.
Colombia is the third biggest recipient of US aid, after Israel and Egypt.
© 2005 BBC News