CHARLESTON, S.C. -- Attorney General Eric Holder is "cautiously optimistic" about how things are going in Washington state and Colorado following the legalization and state regulation of marijuana.

But the nation's top law enforcement official, who spoke to The Huffington Post in an interview on Friday, also said it was tough to predict where marijuana legalization will be in 10 years.

"I'm not just saying that, I think it's hard to tell," Holder said in a jury room at the federal courthouse in Charleston, which he visited as part of the Justice Department's Smart on Crime initiative . "I think there might have been a burst of feeling that what happened in Washington and Colorado was going to be soon replicated across the country. I'm not sure that is necessarily the case. I think a lot of states are going to be looking to see what happens in Washington, what happens in Colorado before those decisions are made in substantial parts of the country."

Under Holder, the Justice Department has allowed marijuana legalization to move forward in Washington and Colorado and has issued guidance to federal prosecutors that is intended to open up banking access for pot shops that are legal on the state level .

Based on the reports he has received out of Washington and Colorado, Holder also said he thinks things are going about how he'd expected them to go.

"I think what people have to understand is that when we have those eight priorities that we have set out, it essentially means that the federal government is not going to be involved in the prosecution of small-time, possessory drug cases, but we never were," Holder said. "So I'm not sure that I see a huge change yet, we've tried to adapt to the situation in Colorado with regard to how money is kept and transacted and all that stuff, and try to open up the banking system."

"But I think, so far, I'm cautiously optimistic," Holder continued. "But as I indicated to both governors, we will be monitoring the progress of those efforts and if we conclude that they are not being done in an appropriate way, we reserve our rights to file lawsuits."

Holder's positive outlook on how legalization is going in Washington and Colorado stands in contrast to the views expressed by Drug Enforcement Administration head Michele Leonhart, who reportedly criticized President Barack Obama for comparing marijuana to alcohol . Leonhart claimed earlier this month that voters were mislead when they voted to legalize and regulate marijuana on the state level, that Mexican drug cartels are "setting up shop" in Washington and Colorado and that this country should have "never gone forward" with legalization. Another DEA official recently claimed that "every single parent out there" opposed marijuana legalization.

Washington and Colorado, of course, aren't the only places in the U.S. reforming their approach to marijuana. In March, Washington, D.C., decriminalized the possession of small amounts of marijuana . Asked about D.C.'s move, Holder said it didn't make sense to send people to jail on possession charges.

"Well, I'll tell you, as a former judge, I had to put in jail substantial numbers of young people for possessory drug offenses, and it was not from the perspective I had as a judge necessarily a good use of law enforcement resources," Holder said. "When I became U.S. attorney we put in place certain guidelines so that people would not end up, especially young people, with criminal records and all that then implies for them."

"So again, we'll see how it works in Washington, D.C.," Holder said.

Asked about his own personal history with marijuana, Holder told HuffPost he used pot in college and had characterized it as "youthful experimentation" in background checks for various federal nominations.

"Yeah, I certainly have said in my four, five, whatever number confirmation hearings I've had that you fill out the forms, that I had 'youthful experimentation' -- I think was the phrase that we were told to use -- when I was in college," Holder said.

Holder also acknowledged the Obama administration has made the political decision not to unilaterally "reschedule" marijuana by taking it off the list of what the federal government considers the most dangerous drugs, though that is something the attorney general has the authority to do. Instead, Holder has said DOJ would be willing to work with Congress if they want to reschedule marijuana, which doesn't seem likely to happen in the near future.

"I think that given what we have done in dealing with the whole Smart on Crime initiative and the executive actions that we have taken, that when it comes to rescheduling, I think this is something that should come from Congress," Holder said. "We'd be willing to work with Congress if there is a desire on the part of Congress to think about rescheduling. But I think I'd want to hear, get a sense from them about where they'd like to be."