Speaking at a fringe event at Liberal Democrat conference entitled, 'Can the market deliver financial inclusion for older people', Kramer called for an alternative to traditional banking structures, suggesting credit unions be made more prevalent.
The event, jointly hosted by Age UK, RBS and Paypoint, was hosted by the presenter of Radio 4 Money Box, Paul Lewis.
Pressed on her views by Lewis, Kramer explained her belief that the underlying problems with the banks are cultural, and that if they didn't want to change they should become facilitators for others to provide alternative services, possibly by way of a levy.
Managing director of retail products at the Royal Bank of Scotland (RBS), Les Matheson, described a number of ways in which he believes RBS is supporting older people's access to finance, including offering post office facilities, providing a 'money manager' service and creating a customer charter.
He pointed to the fact that RBS was the first bank to say it would not remove cheques until an alternative that customers were comfortable with was agreed upon.
Although, on the phasing-out of cheques, he did establish that banks were merely responding to a recent decline in the usage of cheques, which had seen an "eleven per cent" decrease in usage in the previous year.
Peter Brooker, head of corporate affairs at Paypoint, said it had to be accepted that "bank accounts do not suit everyone", which is why financial services need to recognise that cash is vital. He called on the government to look carefully at the regulatory barriers to increasing access points.
Tom Wright, chief executive of Age UK, told the audience how in his view banking is vital as a utility, like water. He emphasised the importance of ensuring payment systems are designed to facilitate the elderly.
Drawing on his own experience of NatWest's "fiddly" online banking, he stressed how difficult this would be found by older people with impairments. Wright called on the banks to no longer consider elderly customers a “niche”.
Defending the banks, Matheson argued that work was being done to ensure older people are accommodated, pointing to the introduction of £5 notes to ATMs.
However, he urged the audience to recognise that "cash comes at a cost".