日本語
한국어
Tiếng Việt
简体中文
Artificial intelligence is rapidly changing how financial institutions detect fraud, serve customers, evaluate risk and make operational decisions. Yet an AI system that performs as designed can still produce an outcome a bank, regulator or customer would find unacceptable.
For banks, trust in AI must extend beyond reliability, security and data accuracy. It must also include the rules, limits and oversight that determine how the technology is allowed to operate.
Rathi Murthy, chief technology officer at Varo Bank, argues that responsible AI depends on the architecture surrounding the system. Financial institutions must define what AI knows, what it is optimizing, how much authority it receives, when employees can intervene and who remains accountable for the outcome.
AI Performance Alone Does Not Establish Trust
Consider an AI model that identifies a transaction as potentially fraudulent. Trusting the model’s ability to detect suspicious activity does not automatically determine the correct response. The bank could flag the transaction, decline it, freeze the account or contact the customer.
Each option gives the system a different level of authority and creates different consequences if the model is wrong. A false decline could prevent a customer from accessing money needed for rent, food or other essential expenses.
Deloitte’s 2026 Global Human Capital Trends research found that 60% of executives regularly use AI to support decisions. As AI becomes more influential, banks must ask what safeguards are necessary before the technology can shape or execute decisions affecting customers, employees and the institution itself.
Reliable AI Requires Authoritative and Current Data
Trust begins with the information used to train and operate an AI system. Bank leaders need confidence that the underlying data is accurate, current, properly governed and relevant to the decision being made.
Fragmented data and inconsistent systems can create fragmented outcomes. AI does not automatically solve those problems. Instead, it can amplify errors, outdated information and conflicting records at a speed that makes them more difficult to identify.
Even high-quality data may not provide a complete picture. Generative AI is probabilistic, and a confident response is not necessarily a correct one. Financial institutions should understand not only how frequently a model produces accurate results, but also where its knowledge is incomplete and how it communicates uncertainty.
AI Objectives Must Reflect the Bank’s Values
Every AI system is designed to optimize something, such as reducing fraud losses, improving conversion rates, increasing efficiency, accelerating service or limiting credit risk. However, optimization is not the same as judgment.
A model can meet its assigned performance target while producing results that customers, employees, business leaders or regulators would reject. The objective given to an AI system therefore represents a values-based decision, whether the organization formally recognizes it or not.
Bank executives should evaluate AI initiatives through three human-centered questions:
- Does the system expand human potential?
- Does it reduce or prevent harm?
- Does it preserve meaningful human choice?
These questions help leaders examine the tradeoffs behind automated decisions. Stronger fraud controls may reduce losses but also block legitimate customers. A credit model focused narrowly on minimizing risk may exclude applicants whose circumstances are not fully reflected in conventional data. An automated service interaction may be faster while offering less empathy, explanation or opportunity for appeal.
Technology cannot determine which tradeoffs an institution is willing to accept. Leaders must define the intended outcome, identify who should benefit, specify unacceptable harms and decide which choices should remain with the person affected.
AI Authority Should Increase Gradually
There is a significant difference between an AI system that provides information, recommends an action, makes a decision and independently executes that decision. These functions represent progressively higher levels of authority.
A system’s technical capability should not automatically determine what it is permitted to do. The level of authority should depend on factors such as potential harm, reversibility, explainability and the system’s performance within established safeguards.
A staged approach can help banks introduce AI more safely:
- Begin in shadow mode: Allow the system to generate recommendations while employees continue making the final decisions and customer outcomes remain unchanged.
- Run a limited pilot: Apply the technology to lower-risk, reversible decisions with narrowly defined thresholds.
- Expand authority carefully: Increase the system’s responsibilities only after reviewing accuracy, consistency, bias, customer impact and operational performance.
AI may be capable of drafting code, answering customer questions, changing workflows or triggering other systems. Moving from recommendation to independent action should be treated as a leadership decision, not as an automatic consequence of technological progress.
A Human-in-the-Loop Model Must Provide Real Authority
The phrase “human in the loop” does not necessarily mean that people retain meaningful control. If an AI system presents one highly confident recommendation and an employee is expected to approve it quickly, the employee may have formal authority but little practical ability to exercise judgment.
Effective human oversight requires more than a final approval button. AI systems should explain the evidence behind their recommendations, identify missing information and communicate uncertainty. Employees must also have enough time, training and organizational support to challenge the system, request additional information or choose another course of action.
Organizations should not automatically treat an employee override as a failure. A disagreement with the model may reveal a limitation in the data, workflow or operating context. Tracking overrides can help banks identify blind spots and improve both the technology and the surrounding process.
As AI becomes faster and more persuasive, the ability to pause and assess a recommendation becomes increasingly important. Technology can expand the amount of information people process, but it cannot replace human awareness, judgment or responsibility.
Human Accountability Cannot Be Delegated to AI
AI may influence a decision or execute an action, but it cannot accept responsibility for the result. It cannot answer to a customer who was harmed, explain its reasoning to a regulator or appear before a bank’s board.
“The model decided” is not an acceptable explanation for an adverse outcome. Before AI becomes part of a consequential financial process, responsibility must be clearly assigned.
Bank leaders should identify:
- Who owns the business outcome?
- Who is responsible for the data and model?
- Who oversees the controls?
- Who has the authority to intervene when the system behaves unexpectedly?
Accountability also requires traceability. Institutions should be able to reconstruct the information used by the system, the recommendation it produced, any uncertainty it communicated, employee overrides and the final result.
This record enables banks to investigate failures, improve controls and demonstrate responsible governance. A negative outcome does not always mean the process was poorly designed, just as a positive result may sometimes be the product of chance. Effective reviews should focus on evidence, learning and improvement rather than simply assigning blame.
AI Governance Must Reflect the Organization’s Starting Point
There is no single AI architecture that will work for every financial institution. Established banks may have deep industry expertise, mature controls and extensive institutional knowledge, but they may also face fragmented systems and legacy technology.
Before adding AI to those environments, leaders need to determine which data, processes and controls can be trusted and which require modernization.
AI-native companies can design their data and workflows around artificial intelligence from the beginning. Their challenge may be moving too quickly without enough institutional experience. They must build traceability, escalation procedures and accountability into their operating models rather than attempting to add them later.
Regardless of an organization’s age or technology strategy, the essential questions remain consistent:
- What does the AI system know?
- What is it being asked to optimize?
- What authority is it allowed to exercise?
- Can employees meaningfully intervene?
- Who will stand behind the result?
Trust cannot be added as a final compliance step. It must influence data governance, system objectives, delegated authority, human oversight and accountability from the start.
Eight Questions for Responsible Banking AI Implementation
- Where is the model’s data incomplete, outdated or not authoritative, and how does the system communicate uncertainty?
- What tradeoffs are being made to achieve the model’s performance targets?
- Does the AI objective preserve human agency, support human potential and reduce harm?
- Was the system’s level of authority explicitly approved by leadership, or is it acting simply because it is technically capable?
- Do employees have the evidence, time and authority needed to challenge AI recommendations?
- How are human overrides recorded and analyzed to uncover model or workflow weaknesses?
- If the system harms a customer, which human leader is ultimately responsible?
- Can the institution reconstruct the data, reasoning, actions and decisions that produced the outcome?
AI can help banks improve efficiency, detect risk and expand human capabilities. But greater technological power also increases the importance of deliberate governance. Financial institutions must decide not only what AI can do, but what it should do, when it should act and where human judgment must remain in control.
Rathi Murthy is chief technology officer at Varo Bank. She previously served as chief technology officer and president of Expedia Services and held technology leadership roles at Verizon Media, Gap Inc., American Express, Yahoo, Sun Microsystems and WebMD.
Source: TheFinancialBrand.com