Streamlining Financial Services with Event-Driven Architecture

150 million - That’s the total number of credit card transactions processed by the banking industry each day in the US(1). When expanded over a year, this volume escalates to approximately 54.8 billion transactions(1). This immense flow of funds requires financial institutions to operate at an extraordinary pace, where delays are not just undesirable, they are unacceptable. Capital serves as the lifeblood of the finance sector, and the underlying software systems act as the vital channels ensuring each transaction is executed safely, securely, and in real time. While theoretically this seems like simple service flow in the software industry, in reality there are more variables at play than seen on the surface. 

Financial software systems must navigate a labyrinth of variables to maintain the pace and integrity of transactions. These variables include the transaction type (stocks, bonds, funds, etc.), the nature of the transaction (inter-bank vs. intra-bank, B2B vs. B2C), among others. To manage this complexity, banks develop a suite of software services tailored to handle specific transaction categories based on their unique requirements.

However, these services do not operate in isolation; they are interdependent. Consider a fund transfer from account A to account B: this seemingly simple operation activates a series of background checks by multiple software services. These checks verify account statuses, authenticate user identities, enforce transaction limits, and trigger user notifications for transactions exceeding predefined thresholds.

This interconnectedness poses a significant challenge: in a system processing millions of transactions per hour, what happens if one service's delay impedes others? Should subsequent operations be put on hold until the bottleneck is resolved? This is where Event-Driven Architecture (EDA) becomes pivotal.

Understanding Event-Driven Architecture (EDA) 

EDAs are like the nervous system of the finance world. Just as your body reacts to stimuli—like your hand pulling away from a hot stove—EDAs enable financial systems to react to events, such as a stock price change or a transaction. This setup allows for real-time responses. EDA ensures that no single service becomes a choke point, enabling a seamless flow of transactions. By adopting an EDA framework, banks can orchestrate their software services to operate concurrently and with minimal inter-service dependency, ensuring efficiency and reliability in handling transactions. This architecture facilitates asynchronous communication within the software system and allows financial institutions to adapt quickly to changes in the market or regulatory environment.

EDA Components

The core components of EDA include event producers, event channels, and event consumers. Event producers are the sources of events; they generate events based on certain actions or occurrences within the system. These could be anything from a user action, like clicking a button, to a system alert, such as reaching a threshold value. Once an event is produced, it is published to an event channel, which acts as a pipeline transmitting the event from the producer to interested software services. Event channels ensure that events are delivered efficiently and securely, often implementing patterns like publish/subscribe to facilitate communication. 

Event consumers, on the other hand, subscribe to these channels to listen for specific events. Upon receiving an event, consumers process it and execute predefined actions, which may involve updating a database, triggering another process, or even generating new events, thereby creating a responsive and dynamic system architecture. This decoupling of producers and consumers allows for greater scalability, flexibility, and resilience, making EDA an attractive choice for systems where real-time responsiveness and adaptability are crucial.

This structure supports a modular approach, where each part of the system can operate independently, enhancing fault tolerance and flexibility. Now that we have a foundational understanding of EDA, let’s try to understand a few common use cases of EDA in Finance. 

Use Case A: Enabling Real-Time Fraud Detection using EDA

In the banking sector, fraud can have dire consequences, affecting customer trust and financial stability. Traditional fraud detection systems often rely on batch processing of transactions, leading to delays in fraud detection and intervention. This lag time between transaction occurrence and fraud identification allows fraudulent activities to proliferate, increasing the risk of significant financial loss and damage to the bank's reputation.

Solution with Event-Driven Architecture:

Implementing an Event-Driven Architecture (EDA) can revolutionize the way banks detect and respond to fraud. By leveraging EDA, banks can process transactions in real-time, enabling immediate fraud detection and rapid response. The steps outlined earlier describe how each transaction generates an event that is immediately analyzed by a fraud detection system. This system then triggers appropriate actions, such as alerting the customer and freezing the account if necessary, thereby minimizing the potential for fraud and enhancing customer trust and security.

Step 1: Transaction Event Whenever a customer makes a transaction (e.g., a withdrawal, transfer, or payment), the transaction system generates an event. This event contains all relevant information about the transaction, such as the amount, the account from which it was made, the destination account (if applicable), the timestamp, and the transaction type.

Step 2: Publish Event The transaction system publishes the event to a central event broker. The event broker acts as a mediator that handles events from publishers (in this case, the transaction system) and distributes them to interested subscribers.

Step 3: Subscription to Transaction Events The fraud detection system subscribes to the event broker to receive all transaction events in real-time. Upon subscribing, it specifies the types of events it's interested in, ensuring it only receives transaction-related events.

Step 4: Fraud Detection System Once the fraud detection system receives a transaction event, it immediately analyzes the transaction details against various fraud detection algorithms. These algorithms might include checking for unusual transaction locations, abnormally large amounts, rapid succession of transactions, and so on.

Step 5: Publish Fraud Alert Event If the fraud detection system identifies a transaction as suspicious, it generates an alert event and publishes it back to the event broker.

Step 6: Fraud Alert Event Subscription Systems subscribed to receive fraud alerts, such as the customer notification system and the account locking system, receive the alert event. The customer notification system can then alert the customer via email or SMS about the suspicious activity, asking for confirmation. Simultaneously, the account locking system can temporarily freeze the account to prevent further unauthorized transactions until the activity is verified.

Use Case B: Real-Time Loan Approval Process using EDA

The traditional loan approval process in banks is often cumbersome and time-consuming, involving multiple steps such as document submission, credit checks, and manual reviews. This lengthy process can lead to customer dissatisfaction, as applicants wait for days or even weeks to receive a decision. Additionally, the manual handling of applications increases the risk of errors and inconsistencies, further delaying the approval process.

Solution with Event-Driven Architecture:

Adopting an Event-Driven Architecture streamlines the loan approval process, making it faster, more efficient, and transparent. As described in the following section, once a loan application is submitted, various systems work in parallel to perform credit scoring, document verification, and ultimately, make a loan approval decision. This parallel processing significantly reduces the time from application to decision, improving the customer experience. Moreover, the automated, event-driven workflow reduces manual intervention, decreasing the likelihood of errors and ensuring consistent application handling.

Step 1: Loan Application System A customer applies for a loan through the bank's digital platform. Upon submission, the loan application system generates an event containing the customer's application details, including personal information, loan amount, and purpose.

Step 2: Loan Application Event The loan application system publishes the event to the event broker, making the application details available for various subscribing systems.

Step 3: Event Broker - Event Subscription by Multiple Systems Several systems subscribe to loan application events, including the credit scoring system, the document verification system, and the loan approval system.

Step 4: Parallel Processing As soon as the event is received, the credit scoring system assesses the customer's creditworthiness by analyzing their credit history, income, and debts. Simultaneously, the document verification system verifies the authenticity of the documents provided. Both systems process these tasks in parallel, significantly speeding up the approval process.

Step 5: Loan Approval Event (Event-Driven Decision Making) Once the credit scoring and document verification systems complete their tasks, they publish events indicating the outcomes of their processes. The loan approval system subscribes to these outcome events.

Step 6: Final Approval and Notification - Customer Notification & Loan Disbursement System Based on the outcomes received from the credit scoring and document verification systems, the loan approval system makes a decision. If all checks are passed, it approves the loan and publishes a loan approval event. The customer notification system, subscribed to these events, then informs the customer of the approval status via their preferred communication channel.

By addressing these problems with EDA-based solutions, banks can significantly enhance their operational efficiency and customer service. Banks across the world have started adopting EDAs and Citi is no exception to that. 

Scaling Citi's Commercial Cards Platform with Event-Driven Architecture

Citi faced the daunting task of scaling its commercial cards API platform to meet a surge in B2B transaction demands(2). The bank needed a system that could not only handle a massive increase in transaction volumes but also integrate seamlessly with financial systems and provide real-time product access. The challenge was creating a scalable architecture that could process millions of monthly requests while ensuring operational excellence and adaptability for future growth.

Citi's solution was to adopt an event-driven microservices architecture (EDA), a strategic move that significantly streamlined its transaction processing(2). EDA enabled Citi to break down complex transaction workflows into smaller, manageable events, allowing for the independent operation and scaling of specific functions based on real-time needs.

Streamlining Transaction Processing: A Closer Look

• Transaction Validation System : EDA transformed how transactions were validated by Citi’s transaction system. Instead of processing these transactions through a bulky, monolithic system, transactions now trigger specific events that are handled by a dedicated validation microservice. This setup allows Citi to dynamically allocate resources to this service during peak times, improving processing speed and efficiency.

• Payment Processing System: After validation, the transaction triggers another event for payment processing. This step is managed by its own microservice, which can be scaled up to handle increased volumes, ensuring transactions are processed swiftly and securely.

• Account Updates: The final step – updating account information, is also event-driven. Once payment is processed, an event activates the account update microservice. This service can be independently adjusted to manage high volumes, especially during peak periods like month-end, keeping account information accurate and current.

• Enhancing Connectivity with Spring Cloud Stream:  At the heart of Citi’s EDA implementation is Spring Cloud Stream, which simplifies the messaging infrastructure connecting the microservices. It abstracts the common challenges associated with the event publishing and subscription services, allowing developers to focus on the business logic of each microservice. This not only makes development more efficient but also improves the system's scalability and maintainability.

By embracing an event-driven microservices architecture, Citi has successfully scaled its commercial cards API platform to efficiently manage a significant increase in transaction volumes.  It successfully scaled to handle millions of requests per month, a significant leap from its initial capacity. This approach has ensured Citi’s ability to maintain operational reliability, introduce new features swiftly, and remain competitive in the commercial card market, ready to meet future demands with strategic flexibility.

Challenges & Considerations

In the finance sector, adopting event-driven architecture (EDA) offers significant benefits, including enhanced system flexibility and improved response times. However, for executives steering their organizations through this technological shift, understanding the corresponding challenges related to EDA when planning such a shift is crucial. Below is an overview the top 4 challenges that development teams should keep in mind when contemplating a shift to EDA styled architecture in the finance industry:

• Architectural Complexity: The shift towards EDA can lead to an overly complex system architecture, especially in environments processing a high volume of transactions. For example, a payment processing system divided into numerous microservices for handling different aspects of a transaction might experience inefficiencies due to excessive communication between services. The principle here is to avoid unnecessary complexity by starting with a simpler design and evolving only in response to proven needs, ensuring the system remains efficient and manageable.

• Transactional Integrity:  In the finance sector, the sequence and accuracy of transactions are critical. The asynchronous nature of EDA introduces challenges in preserving this sequence, which is crucial for operations like calculating account balances or executing stock trades. Techniques such as implementing sequence numbers or employing stateful processing are necessary to ensure events are processed in the correct order, thus maintaining the integrity of financial transactions.

• Governance and Discoverability: As EDA systems scale, effectively managing and discovering events in the software system becomes increasingly challenging. This issue is compounded in the finance industry, where regulatory compliance and auditability are essential. Without proper governance, introducing new events or modifying existing ones could lead to inconsistencies and compliance issues. Establishing clear standards for event documentation, schema management, and version control is vital for maintaining system integrity and regulatory compliance.

• Data Integrity: An API call or operation is idempotent if it provides consistent outcome no matter how many times it's triggered. Idempotency is crucial in financial systems to prevent duplicate processing of the same event, which could lead to inaccuracies in financial records. Achieving idempotency in a distributed EDA system requires careful design, including the use of unique event identifiers and logic to prevent duplicate event processing. This ensures that even if the same event is received multiple times, it will only be processed once, preserving the accuracy of financial data.

In summary, while EDA offers significant advantages for the finance industry, addressing these challenges is essential for its successful implementation. By focusing on simplifying complexity, ensuring transaction integrity, enhancing governance and discoverability, and maintaining data integrity through idempotency, developers can create robust, efficient, and compliant financial systems that fully leverage the benefits of event-driven architecture.

At Creospan, we specialize in assisting banking and financial services clients with modernizing their software systems and overcoming challenges, including those related to EDAs. Whether you're seeking to update your existing systems, facing issues with scalability or reliability, or aiming to introduce new features that appear complex and time-intensive, our team is ready to provide the support you need.

References 

1. Capital One Research, Number of Credit Card Transactions per Second, Day & Year, June 09, 2023

2. EDA Summit, Citi Commercial Cards API Platform: Our Event-Driven Microservices Adoption Journey, June 2023.