This paper presents and discuses the use of dummy variable multiple regression techniques in the analysis of samples drawn from several related or dependent populations ordinarily appropriate for random effects and mixed effects derived from the two factor analysis of variances model with one observation per-cell or treatment combinations. Using the extra sum of squares principle the method develops necessary sums of squares, degrees of freedom and the F-ratios required to test the significance of factor level effects thereby helping to resolve the problem of one observation per treatment combination, encountered in the usual two factor analysis of variance models with one observation per cell. The method provides estimates of the overall and factor mean effects comparable to those obtained with the two factor analysis of variance methods. In addition, the method also provides estimates of the total or absolute effects as well as the direct and indirect effects of the independent variables or factors on the dependent or criterion variable which are not ordinarily obtainable with the usual analysis of variance techniques. The proposed method compares favorably with the usual Friedman’s two-way analysis of variance test by ranks using some sample data.

Keywords: friedman’s two-way ANOVA, mixed–effects ANOVA, dummy variable, regression, extra sum of square, treatment

Dummy variable analysis of variance technique is an alternative approach to the non-parametric Friedman’s two-way analysis of variance test by ranks used to analyze sample data appropriate for use in parametric statistics for two factor random and mixed effects or analysis of variance models with one replication or observation per treatment combinations.^1,2

To develop a non-parametric alternative method for the analysis of matched samples that are appropriate for use with two factor random and mixed-effects analysis of variance models with only one observation per cell or treatment combination, we may suppose that a researcher has collected a random sample of size ’a’ observations randomly drawn from a population ‘A’ of subjects or blocks of subjects exposed to or observed at some ‘c’ time periods, points in space, experimental conditions, tests, or treatments that are either fixed or randomly drawn from population B experimental conditions, points in time, tests or experiments comprising numerical measurements.

The proposed method

Let $y_{ij}$  be the $i^{th}$  observation drawn from population A, that is the observation on the $i^{th}$  subject or block of subjects exposed to or observed at the $j^{th}$  level of factor B that is $j^{th}$  treatment or time period for i=1,2,…,a; j=1,2,…,c.

Now to set up a dummy variable multiple regression model for use with a two factor analysis of variance problem, we as usual present each factor or the so called parent independent variable with one dummy variable of 1s and 0s less than the number of its categories or levels.² Thus factor A, namely subject or block of subjects with ‘a’ levels is represented with a-1 dummy variables of 1s and 0s, while factor B with c levels is represented by c-1 dummy variables of 1s and 0s.

Hence we may let

$\begin{array}{l}{x}_{i;A}=\{\begin{array}{c}\begin{array}{l}1,if{y}_{ij}isanobservationontheithsubjectorblockof\\ subjectsandjthleveloffactorB(treatment)\end{array}\\ 0,otherwise\end{array}\\ fori=1,2,\mathrm{...},a-1;andallj=1,2,\mathrm{...},c\end{array}$ (1)

$\begin{array}{l}Alsolet\\ x_{j;B}=\{\begin{array}{c}\begin{array}{l}1,if{y}_{ij}isanobservationorresponseatthejthlevel\\ offactorB(treatment)andithleveloffactor\\ A(subject,orblockofsubjects)\end{array}\\ 0,otherwise\end{array}\\ forj=1,2,\mathrm{...}c-1;andalli=1,2,\mathrm{...},a.\end{array}$ (2)

Then the resulting dummy variable multiple regression model fitting or regressing the dependent or criterion variable $y_{ij}$ on the dummy variables representing factors A (subject or block of subjects) and B (treatment) is

$y_l={\beta }_0+{\beta }_{1;A}{x}_{l1;A}+{\beta }_{2;A}{x}_{l2;A}+\mathrm{......}+{\beta }_{\alpha -1;A}{x}_{l\alpha -1;A}+{\beta }_{1;B}{x}_{l1;B}+{\beta }_{2;B}{x}_{l2;B}+\mathrm{....}+{\beta }_{c-1;B}{x}_{lc-1;B}+e_i$ (3)

For $l=\text{1},\text{2},\dots ,\text{n}=\text{a}.\text{c }$  sample observations where $y_l$  is the $l^{th}$  response or observation on the criterion or dependent variable; $x_{ls}$ are dummy variables of 1s and 0s representing levels of factors A and B; ${\beta }_{ls}$  are partial regression coefficients and $e_{ls}$  are error terms, with $E\left(e_i\right)=0$ ,for $l=\text{1},\text{2},\dots ,\text{n}=\text{a}.\text{c }$ . Note that since there are only one observation per row by column, that is factor A (subject or block of subjects) by factor B (treatment) combination; for one to be able to have an estimate for the error sum of squares for the regression model, and hence be able to test desired hypotheses, it is necessary to assume that there are no factors A by B interactions or that such interactions have been removed by an appropriate data transformation. Also note that an advantage of the present method over the extended median test for dependent or matched samples and also over the Friedmans two –way analysis of variance test by ranks is that the problem of tied observations within subjects or blocks of subjects does not arise, and hence unlike in the other two non-parametric methods under reference there is no need to find ways to adjust for or break ties between scores within blocks of subjects.³ The expected or mean value of the criterion variable is from equation 3.

$E\left(y_l\right)={\beta }_0+{\beta }_{1;A}{x}_{l1;A}+{\beta }_{2;A}{x}_{l2;A}+\mathrm{....}+{\beta }_{\alpha -1;A}{x}_{l\alpha -1;A}+{\beta }_{1;B}{x}_{l1;B}+{\beta }_{2;B}{x}_{l2;B}+\mathrm{....}+{\beta }_{c-1;B}{x}_{lc-1;B}$  (4)

To find the expected or mean effect of any of the factors or parent independent variables, we set all the dummy variables representing that factor equal to 1 and all the other dummy variables found in equation 4 equal to 0.Thus for example the expected or mean effect or value of factor A (subject or block of subjects) on the dependent variable is obtained by setting $x_{l;A}=1and{x}_{j;B}=0$  in equation 4 for $l=\text{1},\text{2},\dots ,a-\text{1};\text{j}=\text{1},\text{2},\dots ,c-\text{1}$ .

Similarly the expected or mean value of factor B (treatment) is obtained by setting $x_{l;B}=1and{x}_{j;A}=0$  in equation 4 for $l=\text{1},\text{2},\dots ,c-\text{1};\text{j}=\text{1},\text{2},\dots ,a-\text{1}$  thereby obtaining

$E\left(y_{l;A}\right)={\beta }_0+{\displaystyle \sum _{l=1}^{a-1}{\beta }_{l;A}andE(y_{l;B})={\beta }_0+{\displaystyle \sum _{l=1}^{c-1}{\beta }_{l;B}}}$  (5)

Now the dummy variable multiple regression model of equation 3 can equivalently be expressed in matrix form as

$\underset{\_}{y}=X\underset{\_}{\beta }+\underset{\_}{e}$  (6)

Where $\underset{\_}{y}$  is an nx1 column vector of observations or scores on the dependent or criterion variables; X is an nxr design matrix of ‘r’ dummy variables of 1s and 0s; $\underset{\_}{\beta }$  is an rx1 column vector of partial regression coefficients; and $\underset{\_}{e}$  is on nx1 column vector of error terms, with $E(\underset{\_}{e})=\underset{\_}{0}$  where ‘n’=a.c observations and ‘n’=(a-1)+(c-1)=a+c-2 dummy variables of 1s and 0s included in the regression model.

Similarly the expected value of $\underset{\_}{y}$  is from equation 4.

$E(\underset{\_}{y})=X.\underset{\_}{\beta }$  (7)

Application of the usual methods of least squares to either equation 3 or 6 yields an unbiased estimate of the regression parameter $\underset{\_}{\beta }$  as

$\underset{\_}{\widehat{\beta }}=\underset{\_}{b}={\left(X^{\prime }X\right)}^{-1}{X}^{\prime }\underset{\_}{y}$ (8)

Where ${\left(X^{\prime }X\right)}^{-1}$  is the inverse matrix of the non-singular variance-covariance matrix $X^{\prime }X$ . A hypothesis that is usually of research interest is that the regression model of either equation 3 or 6 fits, or equivalently that the independent variables or factors have no effects on the dependent or criterion variable, meaning that the partial regression coefficient is equal to zero stated symbolically that we have the null hypothesis.

$H_0:\underset{\_}{\beta }=\underset{\_}{0}versus{H}_1:\underset{\_}{\beta }\ne \underset{\_}{0}$ (9)

As in equation 3 this null hypothesis is tested using the usual F-test presented in an analysis of variance Table where the total sum of squares is calculated in the usual way as

$SSTotal=\underset{\_}{y^{\prime }}\underset{\_}{y}-n.{\overline{y}}^2$ (10)

With n-1=a.c-1 degrees of freedom where $\overline{y}$  is the mean value of the dependent variables.

Similarly the treatment sum of squares in analysis of variance parlance which is the same as the regression sum of squares in regression models is calculated as

$SSTreatment=SSR=\underset{\_}{b^{\prime }}.X^{\prime }.\underset{\_}{y}-n.{\overline{y}}^2$ (11)

With (a-1)+(c-1) =a+c-2 degrees of freedom. The error sum of squares SSE indicates the difference between the total sum of squares, SST and the sum of squares regression SSR; thus,

$SSE=SST-SSR=\underset{\_}{y^{\prime }}\underset{\_}{y}-\underset{\_}{b^{\prime }}{X}^{\prime }.\underset{\_}{y}$ (12)

With $(a.c-1)-\left((a-1)+(c-1)\right)=(a-1)(c-1)$  degrees of freedom.

These results are summarized in an analysis of variance Table (Table 1)

The null hypotheses H₀ of Equation 13 is tested using the F-ratio of Table 1. The null hypothesis is rejected at the if the calculated F-ratio is greater than the tabulated or critical F-ratio at a specified $\alpha$ -level of significance, otherwise the null hypothesis H₀ is accepted.

If the model fits, that if not all the elements of $\beta$  are equal to zero, that is if the null hypothesis H0 of equation 9 is rejected, then one may proceed to test further hypothesis concerning factor level effects, that is one may proceed to test the null hypothesis that factors A (subject or block of subjects) and B (treatment) separately have no effects on the dependent or criterion variable. In other words, the null hypotheses

$\begin{array}{l}{H}_0:{\underset{\_}{\beta }}_A=\underset{\_}{0}versus{H}_1:{\underset{\_}{\beta }}_A\ne \underset{\_}{0}\\ and\\ H_0:{\underset{\_}{\beta }}_B=\underset{\_}{0}versus{H}_1:{\underset{\_}{\beta }}_B\ne \underset{\_}{0}\end{array}$ (13, 14)

Where ${\underset{\_}{\beta }}_{A}and{\underset{\_}{\beta }}_B$  are respectively the (a-1) and (c-1) vectors of partial regression coefficients or effects of factor A (subject or block of subjects) and B (treatment) on the criterion or dependent variable. However a null hypothesis that is usually of greater interest here is that of equation 14, that is that treatments, points in time or space of tests or experiments do not have differential effects on subjects.

Now to obtain appropriate test statistics for use in testing these null hypothesis we apply the extra sum of squares principle to partition the treatment or regression sum of squares SSR into its two component parts namely, the sum of squares due to factor A (subject or block of subjects), SSA and the sum of squares due to factor B (treatment), SSB, to enable the calculation of the appropriate F-ratios.

Now the nxr matrix X for the full model of equation 6 can be partitioned into its two component sub-matrices namely $X_A$ , an nx(a-1) design matrix of a-1 dummy variables of 1s and 0s representing the included a-1 levels of factor A (subject or block of subjects) and $X_B$ , an nx(c-1) matrix of the c-1 dummy variables of 1s and 0s representing the included c-1 levels of factor B (treatment). The partial regression coefficient $\underset{\_}{b}$ , estimated being an rx1 column vector of regression effects of equation 8 can also be partitioned into the corresponding partial regression coefficients estimated such as, ${\underset{\_}{b}}_A$ ,which is an (a-1)x1 column vector of partial regression coefficients or effects of factor A and ${\underset{\_}{b}}_B$ which is a (c-1)x1 column vector of the effects of factor B on the dependent variable. Hence the treatment sum of squares SST, that is the sum of squares regression SSR of equation 11 can be equivalently expressed as

$\begin{array}{l}SSTreatment=SSR=\underset{\_}{b^{\prime }}{X}^{\prime }\underset{\_}{y}-n.{\overline{y}}^2=(X\underset{\_}{b}{)}^{\prime }.\underset{\_}{y}-n.{\overline{y}}^2;\\ equivalentlyas\\ SSR={\left((X_A{X}_B)\left|\begin{array}{c}{\underset{\_}{b}}_A\\ {\underset{\_}{b}}_B\end{array}\right|\right)}^{\prime }\underset{\_}{y}-n.{\overline{y}}^2=\left({\underset{\_}{b^{\prime }}}_A.{X^{\prime }}_A.\underset{\_}{y}+{\underset{\_}{b^{\prime }}}_B.{X^{\prime }}_B.\underset{\_}{y}\right)-n.{\overline{y}}^2\end{array}$ (15)

or equivalently

$SSR=\underset{\_}{b^{\prime }}{X}^{\prime }\underset{\_}{y}-n.{\overline{y}}^2=\left({\underset{\_}{b^{\prime }}}_A.{X^{\prime }}_A.\underset{\_}{y}-n.{\overline{y}}^2\right)+\left({\underset{\_}{b^{\prime }}}_B.{X^{\prime }}_B.\underset{\_}{y}-n.{\overline{y}}^2\right)+n.{\overline{y}}^2$ (16)

Which when interpreted is the same as the statement
$SSTreatment=SSR=SSA+SSB+SS(\overline{y}=\widehat{\mu })$ (17)

Where SSR is the sum of squares of regression for the full model with r=a+c-2 degrees of freedom; SSA is the sum of squares due to factor A (subject or block of subject); with a-1 degrees of freedom; SSB is the sum of squares due to factor B (treatment) with c-1 degrees of freedom; and $SS(\overline{y}=\widehat{\mu })$  is an additive correction factor due to mean effect. These sums of squares namely SSR, SSA and SSB are obtained by separately fitting the full model of equations 6 with X, and the reduced regression models of $X_{A}and{X}_B$ again separately on the criterion or dependent variable $\underset{\_}{y}$ .

Now if the full model of equation 6 fits, that is if the null hypothesis of equation 9 is rejected, then the additional null hypotheses of equations 13 and 14 may be tested using the extra sum of squares principle.^4,5 If we denote the sums of squares due to the full model of equation 6 and the reduced models due to the fitting of the criterion variables $\underset{\_}{y}$  to any of the reduced design matrices $X_{A}and{X}_B$  by SS(F) and SS(R) respectively then following the extra sum of squares principle^4,5 the extra sum of squares due to a given factor is calculated as

$ESS=SS\left(F\right)-SS\left(R\right)$ (18)

With degrees of freedom obtained as the difference between the degrees of freedom of SS(F) and SS(R); that is as Edf=df(F)-df(R). Thus the extra sums of squares for factors A (subject or block of subjects) and B (treatment) are obtained as follows respectively

$ESSA=SSR-SSA;ESSB=SSR-SSB$ (19)

With $(a-1)+(c-1)-(a-1)=b-1$  degrees of freedom and $(a-1)+(b-1)-(b-1)=a-1$  degrees of freedom.

Note that since each of the reduced models and the full model have the same total sum of squares SST, the extra sum of squares may alternatively be obtained as the difference between the error sum of squares of each reduced model and the error sum of squares of the full model. In other words, the extra sum of squares is equivalently calculated as $ESS=SS(F)-SS(R)=SST-SS(F)-SST-SS(R)=SSE(R)-SSE(F)$ (20)

With degrees of freedom similarly obtained. Thus the extra sum of squares due to factors A (subject or block of subjects) and B (treatment) are alternatively obtained as follows respectively. $ESSA=SSEA-SSE$ (21)

With c-1 and a-1 degrees of freedom. Where SSR and SSE are respectively the regression sum of squares and the error sum of squares for the full model and SSEA and SSEB are respectively the error sums of squares for the reduced models for factors A and B. The null hypotheses of equations 13 and 14 are tested using the F-ratios

$F_A=\frac{MESA}{MSE}$ (22)

With a-1 and (a-1)(c-1) degrees of freedom where

$MESA=\frac{ESSA}{c-1}$  (23)

Is the mean extra sum of squares due to factor A (subject or block of subjects) and

$F_B=\frac{MESB}{MSE}$ (24)

With a-1 and (a-1)(c-1) degrees of freedom where

$MESB=\frac{ESSB}{a-1}$  (25)

Is the mean extra sum of squares due to factor B (treatment).These results are summarized in Table 2a which for ease of presentation also includes the sum of squares and other values of Table 1 for the full models.

If the various F–ratios and in particular the F-ratios based on the extra sums of squares of Table 2b indicate that the independent variables or factor levels have differential effects on the response, dependent, or criterion variable, that is if the null hypotheses of either equation 13 or 14 or both are rejected, then one may proceed further to estimate desired factor level effects and test hypotheses concerning them.

In fact an additional advantage of using dummy variable regression models in two factor or multiple factor analysis of variance type problems is that the method also more easily enables the estimation of factor level effects separately of several factors on a specified dependent or criterion variable. For example it enables the estimation of the total or absolute effect, the partial regression coefficient or the so called direct effect of a given independent variable here referred to as the parent independent variable on the dependent variable through the effect of its representative dummy variables as well as the indirect effect of that parent independent variable through the mediation of other independent variables in the model.⁶ The total or absolute effect of a parent independent variable on a dependent variable is estimated as the simple regression coefficient of that independent variable represented by codes assigned to its various categories when regressed on the dependent variable. The direct effect of a parent independent variable on a dependent variable is the weighted sum of the partial regression coefficients or effects of the dummy variables representing that parent independent variable on the dependent variable where the weights are the simple regression coefficients of each representative dummy variable regressing on the specified parent independent variable represented by codes. The indirect effect of a given parent independent variable on a dependent variable is then simply the difference between its total and direct effects.⁶

Now the direct effect or partial regression coefficient of a given parent independent variable on a dependent variable is obtained by taking the partial derivative of the expected value of the corresponding regression model with respect to that parent independent variable. For example the direct effect of the parent independent variable ‘A’ say on the dependent variable Y is obtained from equation 5 as

$\begin{array}{l}{\beta }_{A}dir=\frac{dE(y_i)}{d_A}={\displaystyle \sum _{l=1}^{a-1}{\beta }_{l;A}.\frac{dE(x_{l;A})}{d_A}+{\displaystyle \sum _l{\beta }_{l;Z}.\frac{dE(x_{l;Z})}{d_A}}}\\ or\\ {\beta }_{A}dir={\displaystyle \sum _{l=1}^{a-1}{\beta }_{l;A}.\frac{dE(x_{l;A})}{d_A}}\\ \sin ce{\displaystyle \sum _l{\beta }_{l;Z}}.\frac{dE(x_{l;Z})}{d_A}=0\end{array}$ (26)

For all other independent variable ‘z’ in the model different from ‘A’.

The weight ${\alpha }_{l;A}=\frac{dE(x_{l;A})}{d_A}$  is estimated by fitting a simple regression line of dummy variable. $x_{l;A}$ regressing on its parent independent variable, A represented by codes and taking the derivative of its expected value with respect to ‘A’. Thus, if the expected value of the dummy variable $x_{l;A}$ regressing on its parent independent variable ‘A’ is expressed as $E\left(x_{l;A}\right)={\alpha }_0+{\alpha }_{l;A}.A$

Then the derivative of this expected value with respect to A is

$\frac{dE(x_{l;A})}{d_A}={\alpha }_{l;A}$ (27)

Hence using Equation 27 in Equation 26 gives the direct effect of the parent independent variable A on the dependent variable Y as

${\beta }_{A}dir={\displaystyle \sum _{l=1}^{a-1}{\alpha }_{l;A}.{\beta }_{l;A}}$ (28)

Whose sample estimate is from Equation 8

${\widehat{\beta }}_{A}dir=b_{A}dir={\displaystyle \sum _{l=1}^{a-1}{\alpha }_{l;A}.b_{l;A}}$ (29)

The total or absolute effect of ‘A’ on ‘Y’ is estimated as the simple regression coefficient or effect of the parent independent variable ‘A’ represented by codes on the dependent variable ‘Y’ as

${\widehat{\beta }}_A=b_A$ (30)

Where $b_A$  is the estimated simple regression coefficient or effect of ‘A’ on ‘Y’. The indirect effect of ‘A’ on ‘Y’ is then estimated as the difference between $b_A$ and $b_{A}dir$ , that is as

${\widehat{\beta }}_{A}indir=b_{A}indir=b_A-b_{A}dir$ (31)

The total, direct and indirect effects of factor B are similarly estimated.

This paper has proposed the use of dummy variable multiple regression methods for the analysis of several related or dependent samples appropriate for random effects and mixed effects two factor analysis of variance with one observation per cell or treatment combination.

Using the extra sum of squares principle, the method developed necessary sums of squares, degrees of freedom and the F-ratios required in testing for the significance of factor level effects.

The method provided estimates of the overall and factor mean effects comparable to those obtained with the two factor analysis of variance method. In addition the method also provided estimates of the total or absolute effects as well as the direct and indirect effects of the independent variables or factors on the dependent or criterion variable which are not ordinarily obtainable with the usual analysis of variance techniques. The proposed method is illustrated with some sample data and shown to compare favorably with the usual Friedmans two-way analysis of variance test by ranks often used for the same purpose.

None.

Author declares that there are no conflicts of interest.

1. Oyeka ICA. Ties Adjusted Two way Analysis of Variance tests with unequal observations per cell. Science Journal of Mathematics & Statistics. 2013;(2013):1–6.
2. Boyle, Richard P. Path Analysis and Ordinal Data. American Journal of Sociology. 1970;75(4):461–480.
3. Oyeka ICA, Afuecheta EO, Ebuh GU, et al.  Partitioning the total chi-square for matched Dichotomous Data. International Journal of Mathematics and Computations. 2012;16(3):41–50.
4. Draper NR, Smith H. Applied Regression Analysis. John Wiley & sons, New York, USA. 1966.
5. Neter J, Wasserman W, Christopher N. Applied Linear Statistical Models. New York, USA. 1974.
6. Wright, Sewall. The Methods of Path Coefficients. The Annals of Mathematical Statistics. 1934;5(3):161–215.