The Obama administration has quietly repatriated a dozen detainees from a small U.S. military prison in Afghanistan, moving a modest step closer toward winding down the United States’ controversial post-9/11 detainee system.

President Barack Obama, in a letter to Congress released on Thursday, informed U.S. lawmakers that about 38 non-Afghan prisoners remained at the Parwan detention center outside of Kabul, down from around 50 a few months ago.

A U.S. defense official, speaking on condition of anonymity, said that a Frenchman, a Kuwaiti and 10 Pakistani prisoners were sent back to their respective home countries at the end of May.

The remaining detainees include Yemeni, Tunisian and more Pakistani nationals, and a Russian who the United States is also considering trying in a military or civilian court.

The transfers, which are not publicly disclosed, underscore the challenges the Obama administration faces in shutting down Parwan and the larger U.S. military prison at Guantanamo Bay, Cuba, which has been widely criticized by human rights groups since being populated in the wake of the Sept. 11, 2001 attacks.

Many of the detainees have not been charged with a crime, but the release of any military detainees has the potential to intensify the political backlash the Obama administration is facing over its handling of suspected militants captured in Afghanistan and elsewhere since 2001.