Introduction

The analysis of Internet traffic distribution is a cornerstone of modern network management, underpinning resource allocation, quality of service provisioning, and security monitoring. Traditional approaches to traffic modeling often rely on static statistical distributions, such as the Zipf-like distribution observed in Web file access patterns [1], or on self-similarity models that capture long-range dependence in network flows [2]. While these models have provided valuable insights, they fundamentally assume that the underlying traffic generation processes are stationary. This assumption is increasingly untenable in the presence of sophisticated cyber threats, which introduce abrupt, non-stationary perturbations into user behavior and network traffic patterns [3]. For instance, a distributed denial-of-service (DDoS) attack can dramatically alter flow length distributions, while a stealthy data exfiltration campaign may cause subtle, yet persistent, shifts in traffic profiles [4]. Consequently, there is a pressing need for analytical frameworks that can adapt to these dynamic environments in real time.

Markov chain models have been widely employed to model sequential user behavior and traffic state transitions [5]. In the context of cyber security, these models have been used to detect anomalies by comparing observed transition probabilities against a learned “norm profile” [6]. However, a critical limitation of these classical Markov models is their assumption of time-homogeneous transition probabilities. When a cyber attack occurs, the behavioral patterns of both legitimate users and malicious actors change, rendering the pre-computed transition matrix obsolete. Some works have attempted to address this by using hidden Markov models (HMMs) to infer latent attack states from observed traffic [7], or by employing variable-length Markov models to capture complex attack sequences [8]. Nevertheless, these methods typically require batch retraining on new data, which is computationally expensive and introduces latency that is unacceptable for real-time defense.

To overcome these challenges, we propose a dynamic Markov chain framework that integrates online stochastic approximation algorithms to recursively estimate time-varying transition tensors. Our approach treats observed network anomalies as perturbation signals that drive the evolution of user behavior. Specifically, we employ a Robbins-Monro gradient descent procedure [9] to continuously update state-to-state transition probabilities without requiring batch retraining. This recursive estimation mechanism incorporates an adaptive damping factor that attenuates noise from false positive anomaly detections while preserving sensitivity to genuine cybercrime-induced traffic distortions. The updated transition tensors are then projected onto a normalized probability simplex to ensure mathematical consistency, enabling real-time computation of equilibrium traffic distributions. Finally, a utility function maps these dynamic distributions to quantifiable operator benefits, such as bandwidth utilization efficiency and latency reduction metrics.

The primary contribution of this work is a principled, mathematically grounded method for recursively estimating non-stationary Markov dynamics under cyber perturbations. Unlike existing approaches that rely on static models or batch retraining, our framework operates in a streaming fashion, making it suitable for real-time network operations. Furthermore, by explicitly modeling the influence of anomaly signals on transition probabilities, we provide a direct link between cyber threat detection and traffic distribution optimization. This work therefore offers a novel analytical tool for network operators to dynamically adjust their strategies in response to evolving cyber threats.

The remainder of this paper is organized as follows: Section 2 reviews related work on Markov chain models for traffic analysis, online stochastic approximation, and cyber threat detection. Section 3 presents the necessary preliminaries and formulates the problem of non-stationary traffic distribution estimation. Section 4 details the proposed Recursive Stochastic Tensor Estimation (RSTE) framework, including the online update procedure, the adaptive damping mechanism, and the projection step. Section 5 describes our experimental evaluation using real-world network traffic traces and simulated cyber attack scenarios. Section 6 discusses the implications of our findings, limitations of the current approach, and directions for future research. Section 7 concludes the paper.

Related Work

The intersection of Markov chain modeling, Internet traffic analysis, and cyber security has produced a rich body of literature. We organize this review into three thematic areas: static Markov models for traffic characterization, dynamic and hidden Markov models for cyber threat detection, and online stochastic approximation methods for non-stationary systems.

Static Markov models for Internet Traffic analysis

Early applications of Markov chains to Internet traffic focused on modeling the statistical properties of packet arrivals, flow sizes, and user session dynamics. For instance, the Zipf-like distribution observed in Web file access patterns was modeled using a Markov chain to capture the sequential dependencies in user requests [1]. Similarly, self-similarity models, which describe the long-range dependence in network traffic, were often approximated using Markovian arrival processes [2]. In the context of cyber crime, a Markov chain model was proposed to analyze user behavior during traffic sharing, where the transition probabilities between states (e.g., idle, benign downloading, malicious activity) were estimated from historical data [5]. This model provided a baseline for computing the probability of a user being a cyber criminal based on their observed state sequence. However, a fundamental limitation of these static models is their assumption of time-homogeneous transition probabilities. When a cyber attack occurs, the behavioral patterns of both legitimate users and malicious actors change, rendering the pre-computed transition matrix obsolete. For example, during a DDoS attack, a user who was previously in a benign downloading state may suddenly transition to a state associated with participating in the attack, a change that a static model cannot capture.

Dynamic and Hidden Markov models for Cyber Threat detection

To address the non-stationarity introduced by cyber threats, researchers have turned to hidden Markov models (HMMs) and variable-length Markov models (VLMMs). HMMs have been used to infer latent attack states from observed network traffic, where the hidden states represent different phases of an attack (e.g., reconnaissance, exploitation, command-and-control) [7]. The emission probabilities of the HMM are learned from labeled attack data, and the Viterbi algorithm is used to decode the most likely sequence of hidden states given a sequence of observations. This approach has been applied to detect botnet traffic, where the HMM is trained on the network behavior of known botnets [10]. Similarly, VLMMs have been employed to model the sequential ordering of events in cyber attacks, capturing dependencies that extend beyond a single time step [8]. These models can predict the next attack step with higher accuracy than fixed-order Markov chains. However, both HMMs and VLMMs typically require batch training on a fixed dataset, and they do not naturally accommodate online updates as new data arrives. Retraining these models from scratch after each new observation is computationally prohibitive for real-time applications. Furthermore, the state space in these models is often fixed and does not adapt to the evolving threat landscape.

Online Stochastic Approximation for Non-Stationary systems

Online stochastic approximation algorithms, such as the Robbins-Monro procedure [9], provide a principled framework for recursively estimating parameters from streaming data. These methods have been applied to a wide range of problems, including adaptive filtering, reinforcement learning, and system identification. In the context of Markov chains, stochastic approximation has been used to estimate the transition probabilities of a stationary chain from sequentially observed transitions [11]. The key idea is to update the estimate of the transition matrix using a gradient descent step on the negative log-likelihood of the observed transition, with a decreasing step size that ensures convergence. More recently, this approach has been extended to non-stationary Markov chains, where the transition probabilities are allowed to drift over time [12]. In this setting, the step size is kept constant or adaptively tuned to track the changing dynamics. However, these methods typically assume that the non-stationarity is smooth and gradual, and they do not explicitly model the influence of external perturbations, such as cyber attack signals, on the transition probabilities. The proposed RSTE framework addresses this gap by incorporating a cyber-risk context vector, derived from real-time anomaly signals, into the update rule. This allows the model to react quickly to abrupt changes caused by cyber events while remaining robust to noise.

Comparison with the Proposed approach

The proposed Recursive Stochastic Tensor Estimation (RSTE) framework differs from existing works in several key aspects. First, unlike static Markov models [5] that assume time-homogeneous transition probabilities, RSTE explicitly models the transition dynamics as a function of both time and a latent cyber-risk context. Second, while HMMs [7] and VLMMs [8] require batch training and do not naturally support online updates, RSTE employs a Robbins-Monro stochastic approximation procedure that updates the transition tensor incrementally upon each observed user transition. Third, unlike existing online stochastic approximation methods for non-stationary Markov chains [12], RSTE explicitly models the influence of external perturbation signals (anomaly vectors) on the transition probabilities through a context-dependent update rule and an adaptive damping factor. This enables the model to distinguish between genuine cyber events and noise, a capability that is absent in prior work. The primary novelty of RSTE is therefore its integration of online stochastic approximation with a context-aware, tensor-based representation of non-stationary Markov dynamics, specifically tailored for cyber-affected traffic analysis.

Recent studies have demonstrated the growing impact of artificial intelligence (AI), deep learning, blockchain, and cloud computing across diverse application domains. Tiwari et al. [23] highlighted the ethical challenges of AI-driven healthcare systems, emphasizing transparency, fairness, privacy, and the importance of Explainable AI (XAI). Tiwari et al. [24] proposed a neural network-based energy-efficient job scheduling approach that improves resource utilization and reduces energy consumption in green cloud computing environments. Furthermore, Lenka et al. [25] developed a blockchain-enabled deep learning framework for secure IoT data analytics, enhancing data integrity and cybersecurity through decentralized architecture. Collectively, these studies demonstrate that integrating AI with advanced computing technologies improves decision-making, energy efficiency, and security while addressing critical ethical and operational challenges.

Preliminaries and Problem Formulation

This section establishes the foundational mathematical concepts required to understand the proposed framework and formally defines the problem of estimating non-stationary traffic distributions under cyber perturbations. We begin by reviewing the classical theory of time-homogeneous Markov chains, then introduce the principles of online stochastic approximation, and finally discuss constrained optimization on probability simplices.

Foundations of Non-Stationary Markov processes

A discrete-time Markov chain is a stochastic process $\{X_t{\}}_{t=0}^{\mathrm{\infty }}$ defined over a finite state space $\mathcal{S}\mathcal{=}\{1,2,\dots ,K\}$, where the probability of transitioning to a future state depends only on the current state. This Markov property is formally expressed as $P(X_{t+1}=j\mid X_t=i,X_{t-1},\dots ,X_0)=P(X_{t+1}=j\mid X_t=i)$. For a time-homogeneous chain, these transition probabilities are constant over time and are collected into a transition matrix $\mathbf{P}\in {\mathbb{R}}^{K\times K}$, where each element is defined as:

The matrix $\mathbf{P}$ is row-stochastic, meaning that for each row $i$, we have $\sum _{j=1}^K{P}_{ij}=1$ and $P_{ij}\ge 0$ for all $j$. The $n$-step transition probabilities, which describe the probability of moving from state $i$ to state $j$ in exactly $n$ steps, are given by the Chapman-Kolmogorov equation:

for any integer $m$ such that . This equation implies that the $n$-step transition matrix is simply ${\mathbf{P}}^n$, the $n$-th power of the one-step transition matrix. A central object of interest in traffic analysis is the stationary distribution $\pi \in {\mathbb{R}}^K$, which satisfies:

subject to $\sum _{i=1}^K{\pi }_i=1$ and ${\pi }_i\ge 0$. The stationary distribution represents the long-run proportion of time the chain spends in each state, and it is unique if the chain is irreducible and aperiodic. In the context of Internet traffic, the states might represent different user activities (e.g., idle, browsing, streaming, malicious), and the stationary distribution provides the expected traffic distribution across these activities.

However, the assumption of time-homogeneity is violated in cyber-affected environments. When a cyber attack occurs, the transition probabilities change abruptly. For example, during a DDoS attack, the probability of transitioning from an idle state to a state associated with sending attack traffic may increase dramatically. To model this non-stationarity, we introduce a time-varying transition tensor $\mathcal{P}\mathcal{(}t)\in {\mathbb{R}}^{K\times K\times C}$, where $C$ is the dimension of a latent cyber-risk context vector ${\mathbf{c}}_t\in {\mathbb{R}}^C$. The element ${\mathcal{P}}_{ijc}(t)$ represents the probability of transitioning from state $i$ to state $j$ at time $t$, given that the context vector has its $c$-th component most activated. The context vector ${\mathbf{c}}_t$ is derived from real-time network anomaly signals, which we discuss in Section 4. The marginal transition probability, which is the quantity we ultimately seek to estimate, is obtained by averaging over the context dimensions:

where $w_c(t)$ is a normalized weight indicating the relevance of context dimension $c$ at time $t$. This formulation allows the transition dynamics to evolve as a function of both time and the external cyber threat environment.

Principles of Online Stochastic Approximation

Online stochastic approximation provides a computationally efficient method for recursively estimating parameters from streaming data. The canonical Robbins-Monro algorithm [9] is designed to find the root ${\theta }^{\ast }$ of a function $f(\theta \mathbb{)}\mathbb{=}\mathbb{E}[H(\theta ,\xi )]$, where $\xi$ is a random variable and $H(\theta ,\xi )$ is an unbiased estimate of $f(\theta )$. The update rule is:

where is a step size (learning rate) and ${\xi }_t$ is the observation at time $t$. For the algorithm to converge almost surely to ${\theta }^{\ast }$, the step sizes must satisfy the following classical conditions:

The first condition ensures that the algorithm can travel arbitrarily far from the initial estimate, while the second condition ensures that the cumulative noise from the stochastic updates remains bounded. A common choice that satisfies these conditions is ${\eta }_t=1/t$.

In the context of estimating transition probabilities, we can view the problem as minimizing the negative log-likelihood of the observed state transitions. Let ${\mathbf{p}}_i$ denote the $i$-th row of the transition matrix $\mathbf{P}$, which is a probability vector over the $K$ possible next states. Given an observed transition from state $i$ to state $j$ at time $t$, the gradient of the negative log-likelihood with respect to ${\mathbf{p}}_i$ is:

where ${\mathbf{e}}_j$ is the $j$-th standard basis vector. The Robbins-Monro update for the transition probability vector then becomes:

This update is intuitive: it moves the estimated probability vector towards the observed outcome ${\mathbf{e}}_j$, with the step size controlling the rate of adaptation. For stationary chains, a decreasing step size ensures convergence to the true probabilities. For non-stationary chains, a constant or adaptively tuned step size is often used to allow the estimate to track the changing dynamics [12].

Constrained optimization on Probability Simplices

A fundamental constraint in Markov chain modeling is that each row of the transition matrix must lie on a probability simplex. The standard $(K-1)$-dimensional probability simplex is defined as:

After each stochastic update, the estimated probability vector ${\mathbf{p}}_i^{(t+1)}$ may violate these constraints due to numerical errors or the stochastic nature of the update. Therefore, a projection step is required to map the updated vector back onto the simplex. The Euclidean projection onto the simplex is defined as:

This projection can be computed efficiently using a sorting-based algorithm [13]. The projection ensures that the estimated transition probabilities remain valid probability distributions.

In the context of the proposed RSTE framework, the projection step is applied to each slice of the transition tensor $\mathcal{P}\mathcal{(}t)$ along the context dimension. Specifically, for each state $i$ and each context dimension $c$, the vector ${\mathcal{P}}_{i\cdot c}(t)$ must lie on the simplex ${\mathrm{\Delta }}^K$. The projection is performed after each stochastic update to maintain mathematical consistency. Furthermore, the stationary distribution $\pi (t)$ computed from the marginal transition matrix $\mathbf{P}(t)$ also lies on the simplex ${\mathrm{\Delta }}^K$, and it is used to inform traffic distribution optimization. The utility function that maps the stationary distribution to operator benefits is discussed in Section 4.

Having established these preliminaries, we now formally define the problem. Given a sequence of observed user state transitions $\{(i_t,j_t){\}}_{t=1}^T$ and a corresponding sequence of anomaly context vectors $\{{\mathbf{c}}_t{\}}_{t=1}^T$, the goal is to recursively estimate the time-varying transition tensor $\mathcal{P}\mathcal{(}t)$ such that the resulting marginal transition matrix $\mathbf{P}(t)$ accurately reflects the current cyber-affected traffic dynamics. The estimated tensor must satisfy the simplex constraints at all times, and the update procedure must be computationally efficient enough for real-time operation. The next section presents the proposed Recursive Stochastic Tensor Estimation framework that addresses this problem.

Dynamic Markov framework with Online Stochastic Approximation

This section presents the technical details of the proposed Recursive Stochastic Tensor Estimation (RSTE) framework. We first describe the overall system architecture and data flow, then detail the core algorithmic components: the context-aware stochastic gradient update, the adaptive damping factor, the softmax projection with temperature parameter, and the computation of the time-varying stationary distribution.

System architecture and data Flow

The RSTE framework is designed to operate as a streaming module within a larger Internet traffic analysis system. As illustrated in Figure 1, the framework receives two primary inputs: a stream of observed user state transitions $\{(i_t,j_t){\}}_{t=1}^T$ from a user classification module, and a stream of anomaly signal vectors $\{{\mathbf{a}}_t{\}}_{t=1}^T$ from a Transformer-encoder-based anomaly detector. The RSTE module processes these inputs to produce a time-varying transition tensor $\mathcal{P}\mathcal{(}t)$, from which a marginal transition matrix $\mathbf{P}(t)$ and a stationary distribution $\pi (t)$ are computed. These outputs are then fed into downstream decision-making components, such as a utility function that optimizes bandwidth allocation and latency management.

Figure 1. System Context Diagram for RSTE Integration

The anomaly signal vector ${\mathbf{a}}_t\in {\mathbb{R}}^{16}$ is generated by a Transformer encoder pre-trained on the CICIDS2017 dataset using a masked reconstruction objective and fine-tuned with a supervised contrastive loss. This encoder processes aggregated NetFlow records over a sliding window of length $L$, tokenizing the traffic features into a sequence. The [CLS] token’s final hidden state from the encoder is projected through a linear layer to produce ${\mathbf{a}}_t$. This vector captures contextualized information about the current network traffic pattern, with high values in certain dimensions indicating the presence of specific types of cyber attacks.

The cyber-risk context vector ${\mathbf{c}}_t\in {\mathbb{R}}^D$ is then derived from the anomaly signal through a linear transformation:

where $\mathbf{W}\in {\mathbb{R}}^{D\times 16}$ and $\mathbf{b}\in {\mathbb{R}}^D$ are learned parameters. The context vector ${\mathbf{c}}_t$ serves as a latent representation of the current cyber threat environment, with each dimension $d$ corresponding to a distinct behavioral mode. The dimension $D$ is a hyperparameter that controls the granularity of the context representation; in our implementation, we set $D=8$. The context vector is normalized to have unit norm, ensuring that the relative activation levels across dimensions are comparable over time.

Context-Aware Stochastic Gradient Update

The core of the RSTE framework is a recursive update procedure that adjusts the transition tensor $\mathcal{P}\mathcal{(}t)$ upon each observed user transition. Unlike standard stochastic approximation methods that update all parameters equally, our approach selectively updates only the tensor slice corresponding to the most activated context dimension. This selective update mechanism prevents irrelevant threat contexts from corrupting the estimation of other behavioral modes.

Let ${\mathcal{P}}_t\in {\mathbb{R}}^{K\times K\times D}$ denote the estimated transition tensor at time $t$, where $K$ is the number of user behavioral states. The element ${\hat{p}}_{ij}^{(d)}(t)$ represents the estimated probability of transitioning from state $i$ to state $j$ under the $d$-th context dimension. Upon observing a transition from state $i_t$ to state $j_t$ at time $t$, we first identify the dominant context dimension:

where $c_{t,d^{\prime }}$ is the $d^{\prime }$-th component of the context vector ${\mathbf{c}}_t$. This selection focuses the update on the context dimension that is most relevant to the current cyber threat environment. For example, if the context vector indicates that a “DDoS” dimension is most active, only the tensor slice corresponding to that dimension is updated.

The gradient of the negative log-likelihood with respect to the selected tensor slice is then computed. For the observed transition $(i_t,j_t)$ under context dimension $d_t$, the gradient is:

where ${\mathbf{e}}_{j_t}$ is the $j_t$-th standard basis vector. The Robbins-Monro update for the selected row of the tensor slice is:

for all $j\in \{1,\dots ,K\}$, where $\mathbb{I}[\cdot ]$ is the indicator function and ${\eta }_t$ is the adaptive damping factor described in the next subsection. This update moves the estimated probability vector for state $i_t$ under context $d_t$ towards the observed outcome, with the step size controlling the rate of adaptation. The tensor slices for all other context dimensions $d\ne d_t$ remain unchanged at this step.

Adaptive Damping Factor for False Positive Attenuation

A critical challenge in cyber threat detection is the high noise-to-signal ratio, where false positive anomaly signals can trigger unnecessary updates to the transition tensor. To address this, we introduce an adaptive damping factor ${\eta }_t$ that dynamically adjusts based on the deviation of the current anomaly signal from its recent running mean.

Let ${\overline{\mathbf{a}}}_t$ denote the running mean of the anomaly signal over a sliding window of length $W$:

The deviation of the current anomaly signal from this mean is measured by the squared Euclidean norm:

The adaptive damping factor is then computed as:

where is a base learning rate and is a sensitivity parameter. The intuition behind this formulation is as follows. When the anomaly signal ${\mathbf{a}}_t$ deviates significantly from its recent average (i.e., ${\delta }_t$ is large), indicating a potential false positive spike, the damping factor ${\eta }_t$ decreases. This prevents the model from overreacting to transient noise. Conversely, when the signal is stable and close to its mean (i.e., ${\delta }_t$ is small), the damping factor approaches $\alpha$, allowing faster adaptation to genuine, persistent shifts in the threat environment.

The parameters $\alpha$ and $\beta$ control the behavior of the damping factor. The base learning rate $\alpha$ determines the maximum rate of adaptation, while $\beta$ determines how strongly the damping factor responds to deviations. In our implementation, we set $\alpha =0.1$ and $\beta =5.0$, which we found to provide a good balance between sensitivity and robustness across a range of cyber attack scenarios. The sliding window length $W$ is set to 100 time steps, which corresponds to approximately 10 seconds of network traffic at a typical sampling rate.

Softmax Projection with temperature Parameter

After the gradient update, the unnormalized tensor values ${\tilde{p}}_{ij}^{(d)}(t+1)$ may violate the simplex constraints. Specifically, for each state $i$ and each context dimension $d$, the vector ${\tilde{p}}_{i\cdot }^{(d)}(t+1)$ may not sum to one or may contain negative values. To ensure mathematical consistency, we project these values onto the probability simplex using a softmax normalization with a temperature parameter $\tau$.

The softmax projection is applied to each row of each tensor slice:

The temperature parameter controls the “sharpness” of the resulting probability distribution. When $\tau$ is small (e.g., $\tau =0.1$), the softmax function approximates a hard max, making the distribution more deterministic and favoring the most likely transition. When $\tau$ is large (e.g., $\tau =10.0$), the softmax function produces a more uniform distribution, allowing the model to explore alternative transitions. This temperature parameter provides a mechanism to balance between exploitation (in well-characterized threat environments) and exploration (in uncertain environments).

The choice of $\tau$ is context-dependent. In our framework, we set $\tau$ as a function of the entropy of the context vector ${\mathbf{c}}_t$:

where $H({\mathbf{c}}_t)=-\sum _{d=1}^D{\tilde{c}}_{t,d}\log {\tilde{c}}_{t,d}$ is the entropy of the normalized context vector ${\tilde{\mathbf{c}}}_t={\mathbf{c}}_t/\parallel {\mathbf{c}}_t{\parallel }_1$, ${\tau }_0$ is a base temperature, and $\gamma$ is a scaling factor. When the context vector has high entropy (indicating uncertainty about the current threat environment), the temperature increases, promoting exploration. When the context vector is concentrated on a single dimension (indicating a well-characterized threat), the temperature decreases, promoting exploitation. In our implementation, we set ${\tau }_0=0.5$ and $\gamma =2.0$.

Computation of the Time-Varying Stationary distribution

The final step in the RSTE framework is to compute the time-varying stationary distribution $\pi (t)$ from the estimated transition tensor. This distribution represents the long-run proportion of time users are expected to spend in each behavioral state, given the current cyber threat environment.

First, we compute the marginal transition matrix $\mathbf{P}(t)$ by averaging the tensor slices weighted by the context vector:

where the weights $w_d(t)$ are obtained by normalizing the context vector:

This softmax normalization ensures that the weights are positive and sum to one, and it amplifies the contribution of the most activated context dimensions.

The stationary distribution $\pi (t)$ is then computed as the left eigenvector of $\mathbf{P}(t)$ corresponding to the eigenvalue 1:

subject to $\sum _{i=1}^K{\pi }_i(t)=1$ and ${\pi }_i(t)\ge 0$. This eigenvector can be computed efficiently using the power iteration method, which converges rapidly for ergodic Markov chains. The stationary distribution is then passed to a utility function that maps it to quantifiable operator benefits, such as bandwidth utilization efficiency and latency reduction metrics.

The internal architecture of the RSTE framework, showing the data flow from raw inputs to the final stationary distribution, is depicted in Figure 2.

Figure 2. Internal Architecture of the RSTE Framework

The RSTE framework thus provides a complete, mathematically grounded pipeline for recursively estimating non-stationary Markov dynamics under cyber perturbations. The combination of context-aware updates, adaptive damping, and temperature-controlled projection enables the model to adapt to evolving threats while remaining robust to noise. The next section presents an experimental evaluation of this framework using real-world network traffic traces.

Experimental evaluation

To validate the efficacy of the proposed Recursive Stochastic Tensor Estimation (RSTE) framework, we conducted a comprehensive set of experiments using real-world network traffic data augmented with simulated cyber-attack scenarios. The evaluation focuses on three key aspects: the accuracy of transition probability estimation under non-stationary conditions, the robustness of the adaptive damping mechanism against false positives, and the impact of dynamic equilibrium distributions on network utility metrics. We compare RSTE against several baseline methods, including a static Markov Chain (SMC) model [5], a standard Hidden Markov Model (HMM) trained in batches [7], and an online stochastic approximation method without context awareness (OSA) [12].

Experimental Setup and Datasets

Dataset and Preprocessing: We utilized the CICIDS2017 dataset [14] as the primary source of network traffic traces. This dataset contains labeled benign and malicious traffic flows, including attacks such as DDoS, Brute Force, Web Attacks, and Botnets. NetFlow records were aggregated into 1-second windows, and features such as packet count, byte volume, flow duration, and inter-arrival times were extracted. A Transformer-based encoder, pre-trained on the benign portion of CICIDS2017 and fine-tuned with supervised contrastive loss, generated 16-dimensional anomaly vectors ${\mathbf{a}}_t$ for each time window. These vectors served as the input to the RSTE framework’s context module.

State Space Definition: User behavioral states were defined based on clustering of traffic features using K-Means ($K=8$). The resulting states included: Idle, Web Browsing, Video Streaming, File Download, P2P Sharing, Email Communication, Database Query, and Malicious Activity. The ground truth for state transitions was derived from the labeled attack timestamps in CICIDS2017, allowing us to compute the true time-varying transition probabilities for evaluation purposes.

Baseline Configurations:

• Static Markov Chain (SMC): Transition probabilities were estimated from the first 10% of the data and kept constant throughout the test period. This represents the traditional stationary assumption [5].

• Hidden Markov Model (HMM): A Gaussian HMM with 8 hidden states was trained on sliding windows of 1000 samples. The model was retrained every 5 minutes to simulate batch updates, reflecting common practices in anomaly detection [7].

• Online Stochastic Approximation (OSA): This method employs the Robbins-Monro update rule [9] but assumes a fixed transition matrix structure without context-dependent tensor slicing or adaptive damping [12]. The step size was set to ${\eta }_t=0.01$.

Evaluation Metrics: We employed the following metrics to assess performance:

1. Kullback-Leibler (KL) Divergence: Measures the difference between the estimated transition distribution ${\hat{\mathbf{P}}}_t$ and the ground truth ${\mathbf{P}}_t^{\ast }$. Lower values indicate better estimation accuracy.

2. Mean Squared Error (MSE) of Stationary Distribution: Quantifies the error in the computed equilibrium distribution ${\pi }_t$ compared to the empirical stationary distribution derived from ground truth labels.

3. Utility Score: A composite metric combining bandwidth utilization efficiency and latency reduction, calculated as $U_t=w_1\cdot {\text{Efficiency}}_t-w_2\cdot {\text{Latency}}_t$, where weights $w_1=0.6$ and $w_2=0.4$ were chosen to reflect typical operator priorities.

Transition Probability Estimation accuracy

The primary objective of RSTE is to accurately track time-varying transition probabilities in the presence of cyber threats. Figure 3 illustrates the evolution of the transition probability from the Web Browsing state to the Malicious Activity state during a simulated DDoS attack sequence.

Figure 3. Dynamic deformation of the transition probability landscape under varying cyber-risk context conditions during a simulated attack sequence

As shown in Figure 3, the RSTE framework rapidly adapts to the sudden increase in malicious transitions triggered by the attack. The contour plot reveals how the probability mass shifts towards the malicious state as the cyber-risk context vector activates the corresponding dimensions. In contrast, the SMC model fails to capture this shift, maintaining a constant low probability, while the OSA method exhibits significant oscillations due to its inability to distinguish between noise and genuine threat signals. The HMM shows a delayed response, attributable to its batch retraining interval.

Table 1 summarizes the average KL divergence and MSE of the stationary distribution across all test scenarios.

Table 1. Performance Comparison of Transition Probability Estimation Methods Method Avg. KL Divergence MSE of Stationary Dist. Computational Time (ms/sample) :— :—: :—: :—: Static Markov Chain (SMC) [5] 0.452 0.089 0.02 Hidden Markov Model (HMM) [7] 0.187 0.041 12.5 Online Stochastic Approx. (OSA) [12] 0.124 0.033 0.15 RSTE (Proposed) 0.068 0.018 0.18

The results in Table 1 demonstrate that RSTE achieves the lowest KL divergence and MSE, indicating superior accuracy in capturing non-stationary dynamics. Although RSTE has a slightly higher computational cost than OSA due to the tensor operations and projection steps, it remains significantly faster than the batch-trained HMM, making it suitable for real-time applications. The high error rates of SMC confirm the inadequacy of static models in cyber-affected environments.

Robustness to False Positives via Adaptive Damping

A critical feature of RSTE is its adaptive damping factor, designed to attenuate noise from false positive anomaly signals. To evaluate this, we injected synthetic noise into the anomaly vector stream, simulating false alarms at varying frequencies. Figure 4 compares the stability of the estimated transition probabilities for RSTE and OSA under these noisy conditions.

Figure 4. Temporal migration of equilibrium probability mass across behavioral states illustrating rapid adaptation to emerging threat patterns

Figure 4 highlights the trajectory of the equilibrium probability mass for the Malicious Activity state. During periods of high false positive noise (indicated by the shaded regions), the OSA method exhibits erratic fluctuations, incorrectly shifting probability mass towards the malicious state. In contrast, RSTE maintains a stable estimate, thanks to the adaptive damping factor ${\eta }_t$ which reduces the learning rate when the anomaly signal deviates sharply from its running mean. This demonstrates the effectiveness of the damping mechanism in preserving sensitivity to genuine threats while ignoring transient noise.

Impact on Network Utility Metrics

Finally, we assessed the practical benefit of using RSTE-derived equilibrium distributions for network resource allocation. We simulated a traffic engineering scenario where bandwidth is allocated proportionally to the predicted stationary distribution of user states. The utility score, combining bandwidth efficiency and latency, was computed for each method.

Table 2 presents the average utility scores achieved by each method under mixed traffic conditions (benign and attack phases).

Table 2. Network Utility Metrics Under Dynamic Traffic Conditions Method Bandwidth Efficiency (%) Avg. Latency (ms) Utility Score :— :—: :—: :—: Static Markov Chain (SMC) [5] 72.4 45.2 0.58 Hidden Markov Model (HMM) [7] 81.5 38.7 0.71 Online Stochastic Approx. (OSA) [12] 84.2 36.1 0.76 RSTE (Proposed) 88.9 32.4 0.84

As shown in Table 2, RSTE yields the highest utility score, driven by both higher bandwidth efficiency and lower latency. The improved efficiency stems from the accurate prediction of traffic shifts, allowing the network operator to proactively allocate resources to states likely to experience increased load (e.g., shifting capacity from Web Browsing to Malicious Activity mitigation during an attack). The lower latency is a direct consequence of reduced congestion due to better load balancing. The SMC model performs poorly because its static allocations become suboptimal during attack phases, leading to congestion and increased latency.

Ablation study

To further understand the contribution of individual components of the RSTE framework, we conducted an ablation study. We evaluated three variants of the proposed method:

1. RSTE-NoContext: Removes the context-aware tensor slicing, updating all dimensions equally.

2. RSTE-FixedDamping: Replaces the adaptive damping factor with a constant step size $\eta =0.1$.

3. RSTE-NoProjection: Removes the softmax projection step, relying only on normalization.

Table 3 reports the performance of these variants in terms of KL Divergence and Utility Score.

Table 3. Ablation Study Results for RSTE Components Variant KL Divergence Utility Score :— :—: :—: RSTE-NoContext 0.095 0.79 RSTE-FixedDamping 0.082 0.81 RSTE-NoProjection 0.074 0.82 Full RSTE 0.068 0.84

The results in Table 3 indicate that each component contributes to the overall performance. The removal of context awareness (RSTE-NoContext) leads to the largest degradation in accuracy, highlighting the importance of focusing updates on relevant threat dimensions. The fixed damping variant (RSTE-FixedDamping) suffers from increased sensitivity to noise, reducing the utility score. The absence of projection (RSTE-NoProjection) introduces numerical instability, though its impact is less severe than the other components. The full RSTE framework combines these elements to achieve the best overall performance.

Discussion and Future Directions

Building upon the experimental validation presented in the previous section, we now discuss the broader implications of the Recursive Stochastic Tensor Estimation (RSTE) framework, acknowledge its current limitations, and outline promising avenues for future research. The discussion is organized around three central themes: the scalability and computational efficiency of the tensor-based approach, the robustness of the framework against adversarial manipulation of the anomaly signal, and the ethical considerations surrounding the behavioral profiling inherent in the model.

Scalability Challenges and Computational efficiency in High-Dimensional Tensor Updates

Despite the promising results demonstrated in Section 5, the current implementation of the RSTE framework faces scalability challenges when applied to very large state spaces or high-dimensional context vectors. The transition tensor $\mathcal{P}\mathcal{(}t)\in {\mathbb{R}}^{K\times K\times D}$ grows cubically with the number of states $K$ and linearly with the context dimension $D$. For a network with hundreds of distinct user behavioral states (e.g., fine-grained application-level profiling), the memory footprint of storing and updating the full tensor becomes prohibitive for real-time operation on commodity hardware. The computational cost of the softmax projection, which is applied to each of the $K\times D$ rows of the tensor, also scales unfavorably.

To address this limitation, future work could explore low-rank tensor factorization techniques. Instead of storing the full tensor explicitly, one could approximate it as a sum of $R$ rank-one tensors using a CANDECOMP/PARAFAC (CP) decomposition {[}15{]}. This would reduce the storage complexity from $O(K^{2}D)$ to $O(R(K+D))$, where $R$ is the rank of the approximation. The stochastic gradient update could then be performed directly on the factor matrices, maintaining the online learning capability of the framework. Furthermore, the softmax projection could be replaced with a simpler normalization step, such as L1 normalization, which is computationally cheaper and still ensures the simplex constraints are satisfied, albeit with a potential loss of the temperature-controlled exploration-exploitation trade-off.

Another avenue for improving scalability is to employ a sparse tensor representation. In many practical scenarios, only a small fraction of the possible state transitions are observed with significant probability. For example, a user in the \emph{Idle} state is unlikely to transition directly to \emph{Database Query} without passing through intermediate states. By storing only the non-zero or high-probability entries of the tensor, the memory and computational requirements can be drastically reduced. This sparse representation would require modifications to the update rule to handle missing entries, but it aligns well with the natural sparsity of real-world traffic patterns.

\subsection{Adversarial Robustness Against Signal Manipulation and model Poisoning}

The RSTE framework's reliance on the anomaly signal vector (${\mathbf{a}}_t$`$ as a primary input introduces a potential vulnerability to adversarial manipulation. A sophisticated attacker who gains knowledge of the Transformer-based anomaly detector could craft adversarial perturbations to the network traffic that cause the anomaly vector to produce misleading context signals. For example, an attacker could inject noise that mimics the signature of a DDoS attack, causing the RSTE framework to incorrectly shift probability mass towards the Malicious Activity state and trigger unnecessary defensive measures, thereby wasting network resources. Conversely, an attacker could suppress the anomaly signal during an actual attack, causing the framework to underestimate the threat and fail to adapt.

This adversarial vulnerability is a well-known challenge in machine learning-based security systems [16]. To mitigate this risk, future work should investigate the integration of adversarial training into the anomaly detector’s fine-tuning process. By augmenting the training data with adversarial examples generated using techniques such as the Fast Gradient Sign Method (FGSM) or Projected Gradient Descent (PGD), the Transformer encoder could learn to produce anomaly vectors that are robust to small, malicious perturbations [17]. Furthermore, the RSTE framework itself could be made more robust by incorporating a consensus mechanism that aggregates anomaly signals from multiple, independently trained detectors. If an attacker manages to fool one detector, the consensus would still reflect the true threat environment, reducing the impact of the adversarial manipulation.

Another direction for enhancing robustness is to introduce a validation step before applying the context-aware update. Instead of immediately using the dominant context dimension $d_t$ from the current anomaly vector, the framework could compare the current vector to a short-term history of context vectors. If the current vector represents an abrupt and isolated deviation from the recent trend, the update could be deferred or applied with a significantly reduced learning rate. This would provide an additional layer of defense against transient adversarial perturbations, complementing the existing adaptive damping mechanism which is designed to handle false positive noise rather than targeted attacks.

Ethical Implications of Behavioral Profiling and Strategies for Bias Mitigation

The RSTE framework, by its very nature, constructs a detailed behavioral profile of network users, categorizing them into states such as Malicious Activity and tracking their transition probabilities over time. While this profiling is intended for the benign purpose of network security and resource optimization, it raises significant ethical concerns regarding privacy, surveillance, and potential for misuse. The framework’s ability to detect shifts in user behavior in response to cyber threats could be repurposed for mass surveillance, where the “threat” is defined broadly to include political dissent or other non-malicious activities. Furthermore, the state definitions derived from clustering traffic features may inadvertently encode biases present in the training data. For example, if the CICIDS2017 dataset contains a disproportionate number of attack samples from certain geographic regions or user demographics, the model might learn to associate those groups with a higher probability of transitioning to the Malicious Activity state, leading to discriminatory outcomes.

To address these ethical concerns, future research should focus on developing fairness-aware and privacy-preserving variants of the RSTE framework. One approach is to incorporate differential privacy into the stochastic gradient update [18]. By adding calibrated noise to the gradient before updating the tensor, the framework can prevent an adversary from inferring whether a specific user’s transition was used in the training process. This would protect individual user privacy while still allowing the model to learn aggregate behavioral patterns. The trade-off between privacy and utility would need to be carefully managed, as excessive noise could degrade the accuracy of the transition probability estimates.

Another important direction is the development of bias mitigation techniques for the state clustering and context vector generation steps. The K-Means clustering used to define the behavioral states could be replaced with a fair clustering algorithm that ensures balanced representation across different demographic groups [19]. Similarly, the Transformer encoder’s fine-tuning process could incorporate a fairness constraint, such as an adversarial debiasing network, that prevents the anomaly vector from encoding information about protected attributes [20]. These measures would help ensure that the RSTE framework’s outputs are not systematically biased against any particular group of users.

Finally, it is crucial to establish clear governance and transparency mechanisms for the deployment of such behavioral profiling systems. Network operators should be required to disclose the use of the RSTE framework, the types of data it collects, and the decisions it informs. An independent audit process could verify that the model’s outputs are not being used for discriminatory purposes. By proactively addressing these ethical considerations, the research community can help ensure that the benefits of dynamic traffic analysis are realized without compromising fundamental rights and values.

Conclusion

This paper introduced the Recursive Stochastic Tensor Estimation (RSTE) framework, a novel approach for modeling non-stationary user behavior in cyber-affected traffic distribution systems. By representing transition probabilities as a time-varying third-order tensor and updating it via a Robbins-Monro stochastic approximation procedure, RSTE overcomes the fundamental limitation of static Markov chain models that assume time-homogeneous dynamics. The framework integrates real-time anomaly signals from a Transformer-based encoder into a context-aware update mechanism, selectively modifying only the tensor slice corresponding to the most activated threat dimension. An adaptive damping factor attenuates noise from false positive detections while preserving sensitivity to genuine cyber events, and a temperature-controlled softmax projection ensures mathematical consistency of the probability estimates.

Experimental evaluation on the CICIDS2017 dataset demonstrated that RSTE achieves superior accuracy in tracking time-varying transition probabilities compared to static Markov chains, batch-trained hidden Markov models, and context-unaware online stochastic approximation methods. The framework exhibited robust performance under noisy anomaly signals, maintaining stable estimates during periods of high false positive rates. Furthermore, the dynamic equilibrium distributions produced by RSTE translated into tangible improvements in network utility, with higher bandwidth efficiency and lower latency compared to baseline approaches. An ablation study confirmed the contribution of each component—context-aware updates, adaptive damping, and softmax projection—to the overall performance.

The primary contribution of this work is a principled, mathematically grounded method for recursively estimating non-stationary Markov dynamics under cyber perturbations, without requiring explicit labeling of attack types. The RSTE framework provides network operators with a real-time analytical tool to dynamically adjust traffic distribution strategies in response to evolving cyber threats, bridging the gap between anomaly detection and resource optimization. Future work should address scalability through tensor factorization, enhance adversarial robustness, and incorporate ethical safeguards against bias and privacy violations.