Customer choice behavior can be broadly categorized as the state of choices and acts that affect a customer's purchasing behavior. A comprehensive statistical model for consumer switching from one brand to another is provided. Employing the Hidden Markov Model and Dynamic Programming techniques, various purchasing possibilities of customers are ascertained based on their brand purchase or non-purchase decisions.

Keywords: Consumer, Dynamic Programming, Hidden Markov Model, Purchase.

Throughout the past few decades, consumers have become increasingly important to commercial markets. In the business environment, one must create a model that examines operations in order to evaluate customer behavior for marketing decisions. In order to better understand the lifetime worth of consumer-firm interactions, marketing scientists have begun to construct models in this domain. Thus far, there has been insufficient focus on understanding customer dynamics and how interactions between the organization and its clients impact relationships and behavioral decisions made by both parties.

According to scientific study, customer choice behaviors can be broadly categorized as the state of choices and acts that affect a customer's purchasing behavior. When making a purchase, consumers weigh both logic and emotion when selecting the most prominent drives. The study illustrates how shifts in consumer views affect their purchasing behavior. These shifts can be interpreted as probabilities when evaluated in relation to other circumstances, and they can be organized into arrays to represent a Markov chain.

Numerous studies examine how consumers behave when making purchases of goods that are related to the probabilistic behavior of making repeat purchases and switching brands. The random decisions made by repeated purchases, such as negative-binomial, logarithmic series, beta-binomial, condensed negative binomial, beta, lognormal, beta-geometric, Pareto-negative binomial, Weibull, Lomax, Poisson – Weibull distributions, etc,^1,2 have been the focus of Ehrenberg,³ Chatfield et al.,⁴ Chatfield & Goodhardt,^5,6 Kumaraswamy & Bhatracharyulu,^7,8 etc. Lipstein⁹ created a statistical analytical model for consumer behavior regarding the effects of advertising in the marketplace. A stochastic matrix is created in order to evaluate the effect of consumer attitudes on advertising. A brand transferring Markov model was developed by Whitaker¹⁰ to extract a loyalty measure from the changes in brand shares and purchase pressure. Kumaraswamy  Bhatracharyulu^11,12 develop a statistical linear structure to investigate recurring purchase behavior based on performance indicators of different brands and further captive and quantify the relationships between the customer attitudes, an HMM is created.

Let there are $k$ ‘’ brands, $B_1,B_{2\dots }{B}_k$ . Each brand has only two states, no-purchase and Purchase indicated as 0 or 1. Let us suppose that evidence indicates that the probability of a purchase followed by another purchase is made with $\alpha$ and the probability that a no-purchase is followed by a no-purchase with $\beta$ . This information can be summarized as follows

Consider into account just two state processes: one with a purchase and the other without. Let us assume that the data points to two states: purchase and no-purchase, respectively, and that the probability of a purchase followed by another purchase is represented by $\alpha$  and the chance of a no-purchase followed by a no-purchase by $\beta$ . This data can be summed up as follows:

 $A=\left[\begin{array}{cc}\alpha & 1-\alpha \\ 1-\beta & \beta \end{array}\right]$   (1)

Assume furthermore that there are favorable connections between the brand purchasing scenario and purchase behavior. Envision of three distinct possibilities for the instant:$L,$ $M,$ and $H$ stand for least purchase, moderate buy, and hefty purchase, respectively. Using the data at hand, the probability connections are provided by

 $B=\left[\begin{array}{ccc}{p}_{PL}& p_{PM}& p_{PH}\\ p_{NPL}& p_{NPM}& p_{NPH}\end{array}\right]$   (2)

The average purchasing behavior, either $P$ or $NP$ , is the system's state. A Markov process of order one is the change from one state to another. Since, the present state and the fixed probability in (2.1) are the only things that immediately determine the next state. But since we are unable to see the causes linked with the transaction, the true states remain hidden.

Assume that the distribution of the initial state, represented by $\prod$ , is assessed using the information at hand

 $\prod =\{{\pi }_1,{\pi }_2\}$   (3)

The matrices $(\prod ,A,B)$  are row stochastic i.e. each row is a probability distribution, each element is a probability and row sum is unity.

A hidden markov model (HMM) is a statistical model that is used to explain a system that is believed to build observable occurrences that rely upon internal variables related to the markov process (i.e., hidden states that are not visible). Double embedded random processes, which include Markov processes with unknown parameters, are modeled in by an HMM. The hidden parameters must be retrieved from the observable parameters, that is, A HMM has two distinct processes. The first procedure dealt with a markov chain that has states and transition probabilities attached to it; the states are concealed and hence invisible. In the second procedure, emissions are displayed based on a state-dependent probability distribution at every time epoch.

Let's make a glance at a specific "$n$ " period of purchases, where we observe that if $n=4$ , then $(L,M.H,M)$ or $\left(L,M,L,H\right)$ or,$\left(M,L,M,L\right)$ etc. Given the evidence, we might be interested in finding the Markov process's most probable average purchase state sequence. The dynamics of consumers during the switching process are not fully captured by this method. We adopt into consideration a comprehensive overview of the variables that influence switching and non-switching during the purchasing process. All of these elements were included in our model.

Assume, based on evidence, that there is a 0.8 chance that a purchase of a specific brand will be followed by another purchase, and a 0.7 chance that a no-purchase will be followed by another no-purchase and assume that these probabilities are held in the distant past. This can be summarized as $\left[\begin{array}{cc}0.8& 0.2\\ 0.3& 0.7\end{array}\right]$ . Let's additionally consider into account a relationship between purchase behavior and the purchase scenario. Three distinct scenarios—light purchase, moderate purchase, and heavy purchase, or $L,M,$ and $H$ , respectively—are taken into consideration in order to overcome complexity. Lastly, given the information at hand, the probability relationship between the scenario and the purchase behavior is given by $\left[\begin{array}{ccc}0.1& 0.3& 0.6\\ 0.8& 0.1& 0.1\end{array}\right]$  and additionally, let us assume that $\left[0.70.3\right]$  is the initial state distribution of purchase and no-purchase. The purchase behavior—either a purchase or no purchase—is the system's state. A Markov process with order one is involved in the change from one state to the next. The actual states are concealed, though. Since, we haven't personally witnessed the behavior in the past. Nonetheless, we ought to pay attention to the brand-purchase scenarios. These scenarios provide us with probabilistic information regarding the purchase behavior.

Therefore the states are hidden and the system is said to be hidden markov model (HMM). The main aim is to gain insight into different aspects of the markov process with use of available information. The state transition matrix $A=\left[\begin{array}{cc}0.8& 0.2\\ 0.3& 0.7\end{array}\right]$ , the observation matrix $B=\left[\begin{array}{ccc}0.1& 0.3& 0.6\\ 0.8& 0.1& 0.1\end{array}\right]$ and the initial distribution $\pi =\left[0.70.3\right]$ also these matrices are row stochastic.

Consider a particular 4 – period of interest from the distant past. For which we observe the series of purchase scenarios L M L H, letting 0 – represents L, 1 – represents M and 2 – represents H. Therefore the observed sequence

$O=\left(0,1,0,2\right)$  (*)

We have to determine the most likely state sequence of the markov process for the given observation sequence (*)

For this example, we have $T=4$ , $N=2$ ,$M=3$ , $Q=\left\{P,NP\right\}$ , $V=\left\{0,1,2\right\}$  and the matrices of order $A={\left(a_{ij}\right)}_{NxN}$ , $B={\left(b_{jk}\right)}_{NxM}$ $=P$  (Observation $k$ at time step $t/$  state $q_j$ at $t$ )

Consider a state generic sequence of length four $X=\left(x_0,x_1,x_2,x_3\right)$ and the corresponding observational sequence $O=\left(O_0,O_1,O_2,O_3\right)$ and also ${\pi }_{x0}$ is the probability of starting state $x_0,b_{x0}\left(O_0\right)$ be the probability of initial observation $O_0$ and $a_{x0,x1}$ be the transit probability from $x_0$  to $x_1$ . The probability for the whole state sequence $X$ is

 $P(X)={\pi }_{x_0}.b_{x_0}(O_0).a_{x_0,x_1}.b_{x_1}(O_1).a_{x_1,x_2}.b_{x_2}(O_2).a_{x_2,x_3}.b_{x_3}(O_3)$   (**)

We are computed the each possible state sequence probabilities of length four for the given observation sequence (*). There is $N^T$ possible state sequences are available for purchase or no-purchase on particular observed sequence.

We are interested to estimate the probability for purchase or absorbing state over the period of time, the following central issues are encountered.

1. Evaluation Problem: For the given model $\lambda$ , we estimate that the probability of sequence of visible states generated by model $\lambda$ .
2. Decoding Problem: We are able to determine the most likely hidden state sequences that led to the creation of the visible sequence.
3. Learning Problem: For the given training sequence, estimate the transition and emission probabilities when the hidden and visible states are well defined.
4. Illustration: Consider again the previous example with the observed sequence as given in (*). Using (**) we can evaluate, say,

$P\left(HCHC\right)=0.7\left(0.1\right)\left(0.2\right)\left(0.1\right)\left(0.3\right)\left(0.1\right)\left(0.2\right)\left(0.1\right)=0.00000084$

We calculated the odds of every potential four-length state sequence to the sequence that was observed in (*). Table 1 lists all of these outcomes, and the other column's normalized probabilities are all added up to one. We list the ideal sequence in the HMM sense, which is NPNP, from Table 2.

We briefly discuss the connection between dynamic programming (DP) and HMMs. Dynamic programming is comparable to a - pass in which "max" is used instead of "sum". The dynamic programming (DP) approach is also utilized to compute the probabilities of the state sequence. Every moment we have to complete pick NNNP, the maximum probability sequence. The dynamic programming algorithm stated as given below.

${\text{η}}_{\text{t}}\text{(i)}=\underset{\text{j}\in \text{{0,1,}\mathrm{...}\text{,N}-\text{1}}}{\text{max}}\left[{\text{η}}_{\text{t}-\text{1}}{\text{(j)a}}_{\text{ij}}{\text{b}}_{\text{i}}{\text{(O}}_{\text{t}}\text{)}\right]\text{For}\text{t}=\text{1,2,}\mathrm{..}\text{,T}-\text{1 and i}=\text{1,2,}\text{.}\text{.}\text{.}\text{,}\text{N}-\text{1}$  

${\text{η}}_{\text{0}}\text{(i)}={\text{π}}_{\text{i}}{\text{b}}_{\text{i}}{\text{(O}}_{\text{0}}\text{), For i}=\text{1,2,}\mathrm{...}\text{,N}-\text{1}\text{.}$  

At each successive time stamp t, the DP determines the best path at each of the states I = 0, 1, …, N-1. The best overall path with maximum probability is$\underset{\text{j}\in \text{{0,1,}\mathrm{...}\text{,N}-\text{1}}}{\text{Max}}\left[{\text{η}}_{\text{T}-\text{1}}\text{(j)}\right].$ the computation procedure of DP can be augmented to retrieve the optimal path by choosing the highest probability score in final state.

Consider the above example, the initial period path of length one values are $P\left(P\right)=0.7*\left(0.1\right)=0.07$ and $P\left(N\right)=0.3*\left(0.8\right)=0.24$  hence the best path of length on ending with state is (No-Purchase).

The probabilities for second period path of length of two values are

$\begin{array}{l}P\left(PP\right)=0.07\left(0.8\right)\left(0.3\right)=0.0168\hfill \\ P\left(PN\right)=0.07\left(0.2\right)\left(0.1\right)=0.0014\hfill \\ P\left(NP\right)=0.24\left(0.3\right)\left(0.3\right)=0.0216\hfill \\ P\left(NN\right)=0.24\left(0.7\right)\left(0.1\right)=0.0168\hfill \end{array}$

The optimal path of length two ending with N is NN, whereas the most likely state sequence of length two ending with is. Continue in consideration that the dynamic programming algorithm only needs to keep track of the highest-scoring paths at each possible state at each stage, instead of a list of every path that could possibly exist. This is the secret to the DP algorithm's effectiveness.

The probabilities for third period path of length, three values are$P\left(PPP\right)=0.001344$ ,$P\left(PNP\right)=0.000042$ ,$P\left(NPP\right)=0.001728$ ,$P\left(NNP\right)=0.000504$ ,$P\left(PPN\right)=0.002688$ ,$P\left(PNN\right)=0.000784$ , $P\left(NPN\right)=0.003456$ and $P\left(NNN\right)=0.009408$ .(Figure 1)

Figure 1 Dynamic programming.

The article discusses the connections between HMMs and dynamic programming (DP). HMMs can make it possible to analyze huge amounts of sequence data very effectively. Dynamic programming offers a methodical process for figuring out the best set of choices. For optimality, these can be used to shortest path problems. Observing four steps, the maximum probability of 0.00169344 occurs at the final state where P is NNNP and the arrows from P to the best path, NNNP, can be used to trace this out with Dynamic Programming. The optimal state sequence when utilizing the HMM technique is NPNP. Consequently, the state and sequence of the Hidden Markov Model and the optimal Dynamic Programming sequence are different.

None.

The author declares that there are no conflicts of interest.

None.

1. Weibull W. A Statistical Distribution Function of Wide Applicability. Journal of Applied Mechanics. 1951; 293–297.
2. Kumaraswamy K, Bhatracharyulu NCh. Compounding life distribution – Poisson Weibull, Journal of Computer and Mathematical Sciences. 2018;9(12):1882–1889.
3. Ehrenberg ASC. The pattern of consumer purchases Journal of The Royal Statistical Society, Series-C (Applied Statistics). 1959;8(1):26–41.
4. Chatfield C, Ehrenberg ASC, Goodhardt G.J. Progress on a simplified model of stationary purchasing behavior. Journal of the Royal Statistical Society, Series C (Applied Statistics). 1966;129(3):317–367.
5. Chatfield C, Goodhardt GJ. The beta–binomial model for consumer purchasing behaviour, Journal of the Royal Statistical Society, Series-C (Applied Statistics). 1970;19(3):240–250.
6. Chatfield C, Goodhardt GJ. A consumer purchasing model with Erlang inter-purchase time. Journal of The American Statistical Association. 1973;68:828–835.
7. Kumaraswamy K, Bhatracharyulu NCh. On stochastic distribution of repeated purchase consumers. Int. J. Agricult. Stat. Sci. 2019;15(2):841–846.
8. Kumaraswamy K, Bhatracharyulu NCh. Compound distribution of Poisson-Weibull. International Journal of Mathematics and Statistics Invention. 2021;9(1):29–34.
9. Lipstein B. A mathematical model of consumer behavior. Journal of Marketing Research. 1965;2(3):259–265.
10. Whitaker D. The derivation of a measure of brand loyalty using a Markov brand switching model. Journal of the Operational Research Society. 1978;29(10):959–970.
11. Kumaraswamy K, Bhatracharyulu NCh. Statistical brand switching model: An Hidden Markov approach. Opsearch. 2023;60(2):942–950.
12. Kumaraswamy K, Bhatracharyulu NCh. Statistical model for brand loyalty and switching. Statistics and Application. 2023;21(1):1–9.