Finance Minister Joe Oliver lost his Toronto riding of Eglinton-Lawrence by 3,490 votes, but he has bigger numbers on his mind.

The Liberals, led by prime minister-designate Justin Trudeau, have pledged to run a deficit of up to $10 billion annually for the next three years to fund infrastructure projects. Oliver says that without a fiscal cushion, it will be easy for that number to balloon if Canada experiences more economic headwinds.

“It will require discipline to get back into balance, particularly before the next election,” he says.

The former CEO of the Investment Dealers Association of Canada (now IIROC), Oliver, 75, looked back on the Conservatives’ fiscal record and spoke of the challenges that await his Liberal successor.

Advisor.ca: Do you see your government’s defeat as a repudiation of your economic policy?

Oliver: I don’t. I think the overwhelming political impetus was the desire for change, and I think the change was more related to style and tone.

When I was at the door campaigning, I talked about the contrast between our plan to keep taxes low and balance the budget as opposed to the Liberal promise to raise taxes and incur massive deficits. And occasionally, but very rarely, did I encounter someone who said to me, “No, we’ve got to raise taxes because we’ve got to spend more money.”

When the Liberal government says they’re going to incur a deficit of $10 billion a year for three years – and if you look at their platform, it’s clear they’re going to be well above that – it will require discipline to get back into balance, particularly before the next election.

Read: Economy grows by a sliver in August, says StatsCan

Advisor.ca: There could be unforeseen circumstances, such as when your government provided a $9-billion bailout to the auto industry in 2009.

Oliver: That was right in the depths of the recession. If we hadn’t done that when the Americans were doing it, I don’t think we would have an automotive industry today.

When you have a surplus, you have a bit of a contingency built in. But if you’re in a deficit already and you don’t have a contingency, things can go worse. And the international economic situation, we’ve said repeatedly, is fragile.

China, the second-largest economy in the world, has had a deep decline in growth from 10% to well under 7%. Europe is continuing to languish with relatively modest growth. The Greek situation isn’t entirely resolved – and Ukraine is doing what it can, but Russian political and military pressure is having an economic impact.

If we hadn’t been in a solid fiscal situation, we wouldn’t have emerged from the Great Recession the way we did, with 1.3 million net new jobs and a stronger job creation record than the other G7 countries.

Advisor.ca: Some 400,000 manufacturing jobs were also lost since your party came to power in 2006.

Oliver: There were clearly sectors that were impacted negatively. Manufacturing in southwestern Ontario was definitely hit. But with the lower dollar and the resurgence of the U.S. economy, there are signs that things are reversing. Companies that sat on money are starting to see export potential, and we remain hopeful that will produce tangible improvements in employment, particularly in manufacturing.

Read: Snapshot: Canadian economic data

Advisor.ca: What did you make of federal NDP Leader Tom Mulcair’s campaign promise of balanced budgets?

Oliver: I think Thomas Mulcair was obviously savouring the prospect of forming the next government and realized that to get there, he had to be somewhat more centrist. But as he moved to the centre and the Liberals moved a bit to the left, he lost some of his supporters and wasn’t able to get more centrist supporters.

Advisor.ca: During the election campaign, the Liberals characterized TFSAs as only benefiting the rich, and promised to roll back the contribution limit to $5,500.

Oliver: TFSAs are not for the financial elite: 60% of people who maxed out on their contributions were earning less than $60,000. The average of the 11 million contributors earn less than $75,000.

Advisor.ca: What poses the greatest threat to the Canadian economy?

Oliver: The price of oil.

Advisor.ca: And oil prices aren’t expected to significantly increase any time soon.

Oliver: We knew demand was going down because global growth had declined, and we knew that supply was going up because the United States has found vast amounts of shale oil and gas. And then the final trigger was the Saudis’ decision to protect their market share rather than price.

I think most economists are suggesting prices are going up. But no one’s talking about $100 a barrel.

Read: U.S. Rate Hike Could Weaken Commodities

Advisor.ca: What are you most proud of during your time as Finance Minister?

Oliver: This year’s budget, which I believe was balanced not only fiscally, but also socially. We invested $42 million to help establish the world-class Canadian Centre for Aging and Brain Health Innovation at Baycrest in Toronto that will deal with Alzheimer’s and other cognitive-impairment issues.

I’m also proud that I pushed the removal of capital gains tax for donations of real estate and private company shares that I think is going to generate some transformative charitable giving in the country.

(This interview was edited for style and length.)