The ousted boss of the City watchdog has taken a coded swipe at the chancellor, saying that he had “unfinished business” and that the job of tidying up investment banking was far from complete.Martin Wheatley yesterday made clear his unhappiness after George Osborne refused to renew his contract as chief executive of the Financial Conduct Authority, despite levying billions of pounds in fines and making top-level attempts to reform the culture of trading floors.

“I am disappointed to be moving on and I do so with a sense of unfinished business,” Mr Wheatley said in his first public comments since his eviction.

He made clear that the culture of investment banking remained largely unreformed despite several years of scandals. “The top of organisations have understood the environment . . . but I still think it is quite hard to deliver. It’s tough and it takes time,” he said.

“As we saw with FX [the foreign exchange-rigging scandal], there are some parts, the wholesale parts of financial services, where it is actually not yet seen as relevant. There’s still a view that: ‘Well, that’s a caveat emptor market’ and big financial players trading with one another. That’s the area that has much further to go.” Mr Wheatley’s damning appraisal of the City will pile further pressure on the FCA and Mr Osborne as the search gets under way to recruit a successor for the handover in September.

The chancellor will have the final say on any hiring decision but will want to avoid the perception that he has replaced a chief executive known for taking a hard line on wrongdoing with one more sympathetic to the big banks.

John Griffith-Jones, the chairman of the FCA, declined to say whether he had supported the Treasury’s decision. “The appointment, or the question of renewal of Martin’s contract in March, is a decision for the chancellor. It is the job of the board, in particular the chairman, to run the organisation,” he said.

Mr Wheatley left little doubt that if he had remained in charge of the FCA the banking industry would have faced no respite in his tough approach to regulation. He had considered banning some firms from trading as an alternative to fines, he said. “The use of trading bans, or, in our terms, ‘variations of permission’, is something we actively look at each time we feel that we’ve got to take an action. Those variations of permission can be removing somebody from part of the market, or a significant part of the market, for a period of time,” he said.

Mr Wheatley’s remarks came after the FCA’s annual meeting, where members of the public questioned senior executives on the regulator’s actions. At a fiery session in central London, Mr Wheatley and his team faced repeated criticism for their handling of the compensation process for victims of the mis-selling of interest rate hedging products.

Mr Wheatley defended the FCA-led redress scheme, pointing out that nearly £2 billion had been paid back to small and medium-sized businessess.

Several small businessmen argued that no bank had been fined despite overwhelming evidence of the systematic mis-selling of tens of billions of pounds of interest rate swaps to customers who did not understand what they were buying.

Mr Wheatley later said that banks could still face action from the regulator for their behaviour. “We haven’t ruled out that there will be actions taken at some point,” he said.