Tax-free savings accounts are a great way to save, right? That's not always the case, and some investors are facing a whack of unexpected charges.

Like many Canadians, David Doole opened a tax-free savings account last year -- a few months later he had to dip into it.

"For a down payment on a car," he said. "And I then started to re-contribute to it thinking it was tax-free."

But after he topped up the account, the Canada Revenue Agency sent him a bill for $315.

Doole is not alone. Seventy-thousand Canadians are facing similar penalties. Apparently, the federal government only counts money going in to tax-free savings accounts. It doesn't count all the money coming out.

Here's how the math worked for Doole: his original $5,000 deposit was the maximum allowable contribution. When he withdrew $4,500 the government doesn't count it until next year. So while his actual account balance dropped to $500. His contribution level remained unchanged.

When he put the $4,500 back in his contribution was now counted as $9,500 and he had to pay a penalty on the "over contribution."

CTV Consumer reporter Chris Olsen contacted the CRA for answers. Spokeswoman Caitlin Workman said the CRA looks at every return and assesses each one based on the facts of that specific case.

The government urges people to fill out a form online detailing every deposit and withdrawal for a whole year.

"We want to use the information that Canadians have provided. So if Canadians provide us with information that they haven't gone over the $5,000 limit and they are under their contribution room, then we'll take that into account," Workman said.

That sounds hopeful. But Doole has already filled out the form and sent the government a letter of apology.

"[The letter] asked for forgiveness and apparently I didn't receive it," he said.

Instead, he got the bill for $315, which he's paid.

The CRA website says you can generally "make a withdrawal at any time for any reason with no tax consequences." What it doesn't say as clearly is don't put it back in until next year.

"The government I just don't think made it clear enough to people like us about how it works and they are penalizing us for their poor communication," Doole said.

With a report from CTV British Columbia's Chris Olsen