There's a painfully wide innovation gap between the heavily regulated, slow-moving banking industry and the nimble tech companies concentrated in Silicon Valley. Now Hans Morris, a longtime banker and the former president of Visa (NYSE:V), is trying to bridge the distance.

Morris, who spent the past four years at the private-equity giant General Atlantic, has started a venture capital firm that will finance and advise financial technology startups. Nyca Partners, so named for the New York and California experiences it hopes to marry, plans to invest between $8 million and $12 million a year in companies that can "transform the way financial services are provided," Morris said in an interview this week.

Technology companies tend to be "more open-minded" and can "figure out how to integrate things in ways that really are breathtaking," but "if you work in financial services, you care more about details, and problems," he says. "Having exposure to both and being able to talk to both worlds is big part of why I think there's a good opportunity here."

It's a pointed mission for a man who once was in the running to be the next chief executive of Visa, the world's largest payments network. Morris left the San Francisco company in 2009, after two years spent helping it go public with then-CEO Joseph Saunders.

Saunders retired in late 2012, ceding Visa's management to former JPMorgan Chase (JPM) executive Charlie Scharf. Like most executives at large, established financial services companies, Scharf and deputy Ryan McInerney now must figure out how Visa will maintain its place between the traditional banking system and the outside tech companies increasingly threatening its business. (McInerney, Visa's president, presumably will be discussing some of these topics on Thursday, at a Financial Services Roundtable panel discussion on innovation.)

For Morris, a former longtime Citigroup (NYSE:C) investment banker, the big financial companies where he spent most of his career are the least able to incubate new ideas. But the tech companies that excel at innovation don't have the experience or the practical knowledge to make their ideas work in the more regulated banking industry.

"It's hard for any big company to build software when things are changing rapidly" or to move quickly in the current banking regulatory cycle, but "to get to real scale is much, much harder than most entrepreneurs believe," he says.

Morris is particularly interested in investing in and advising startups that are working on new ways to help improve how merchants accept payments, how financial companies use data and how banks and other lenders offer credit to people and small businesses who might not qualify for traditional loans. Nyca has already invested in and advised the workplace lender workpays.me, which this week agreed to be sold to Global Analytics for undisclosed terms.

Trying to develop such "alternative credit solutions" has become a big goal for banks and nonbanks alike, especially as regulators clamp down on some more traditional sources of short-term credit, like payday loans and banks' so-called deposit advances. One of the most high-profile efforts is coming from Raj Date, the former No. 2 of the Consumer Financial Protection Bureau, who in February announced plans to offer a subprime credit card.

"I'm a big fan of his and I share many of Raj's observations about consumer credit," Morris says. "To me the criteria is that you have to be offering a very fair, transparent product that is in the interests of the borrower … and you have to have some sort of competitive advantage in your underwriting."

Nyca Partners will be funded by the partners' own investments and does not currently plan to seek outside financing. Morris would not discuss who else he plans to bring on board, other than to say that they will have "backgrounds similar to mine" as an experienced operating executive.

"You can invest time, capital or some of both," he says.

Morris joined General Atlantic as a managing director in 2010, and spent four years leading the firm's investments in financial services. But in January, he stepped back from General Atlantic, where he maintains a special advisor role. He remains on the boards of the companies he joined for General Atlantic, including peer-to-peer startup Lending Club.

His time at the "brilliant" General Atlantic helped hone Morris' appreciation for "exposure to innovation," while encouraging him to strike out on his own, he says. General Atlantic "tends to focus on later-stage companies, and a lot of the stuff that I found most personally interesting was earlier-stage companies," he says.