White House physician Ronny Jackson will not return to his role as the president’s personal physician, according to two senior administration officials, after a string of allegations caused the Navy rear admiral to withdraw his nomination last week to lead the Department of Veterans Affairs.

Sean Conley, a Navy officer who took over Jackson’s role as the president’s personal doctor last month, will continue in the role, the officials said.

Jackson bowed out last week after Montana Sen. Jon Tester, the top Democrat on the Veterans’ Affairs Committee, released a document summarizing allegations by current and former colleagues that Jackson overprescribed pills, drank on the job and created a hostile work environment. Jackson has denied the allegations and has returned to work in the White House Medical Unit.