I. GENERAL INTRODUCTION

Sometimes we may want to estimate more than one equations, which are closely related, The OLS and GLS estimation methods can be used to estimate the equations simultaneously, which has some advantages over estimation done one by one (Philip et al., 1990). As we will see later, estimating the system of equations is closely related to estimating models based on panel data (data from the same people/firms/countries for two or more periods).

One of the assumptions of a single equation model is that, there is one-way causation between the dependent variable Y and the independent variables Xs. When this assumption is not valid, as we have it in many econometric models, that is, the assumption of lack of correlation between the independent variables and the error terms (U) is further violated (i.e. $E(XU)\ne 0$ ). Thus, the function no longer belongs to a one-way causation model but rather a wider system of regression equations (multi-equation model), which describe the relationship among all the relevant variables. In a multi-equation model, the dependent variable Y and independent variables now appear as well as explanatory variables in other equation(s) of the model.

Moreover, in a multi-equation model, there are problems of Autocorrelation and Multicollinearity, together with the presence of correlation between the error terms, which may eventually lead to a seemingly unrelated regression model (Lang et al., 2003; Olanrewaju et al., 2017). Consequently, some degree of Autocorrelation and Multicollinearity may have to be allowed in the system of regression equations. Therefore, this study examined and compared the effect of correlation between the error terms ( $\lambda$ ) and Multicollinearity ( $\delta$ ) on the performances of seven methods of parameter estimation of a multi-equation model using the Monte Carlo approach.

a) Aim and Objectives of the Study

Consequently, the study examines the performances of some estimators of a single-equation and that of a system of Regression equation in the presence of correlation between the error terms, multicollinearity, and autocorrelation, study their effects on those estimators, and then, identify the preferred estimator(s) of the model parameters.

Very specifically, the study aims at the following:

(i) Examine the effect of sample size on the performance of the estimators (ii) Examine the effect of multicollinearity $(\lambda )$ and the correlation between the error terms $(\delta )$ jointly on the performance of seven estimators.

(iii) Identify the estimator that yields the most preferred estimates under the joint influence of the two correlation effects under consideration.

II. LITERATURE REVIEW AND THEORETICAL FRAME WORK

a) Estimation Methods under Multicollinearity in Single Equation

Olanrewaju et al. (2017) stated that, one of the major assumptions of the explanatory variables in the classical linear regression model is that they are independent (orthogonal). Orthogonal variables can be set up in experimental designs, but such variables are not often in business and economic data. Thus when the explanatory variables are strongly interrelated, we have the problem of multicollinearity. When multicollinearity is not exact (i.e., the linear relationship between two explanatory variables is not perfect) but strong, the regression analysis is not affected; however, its results become ambiguous. Consequently, interpreting a regression coefficient as measuring the change in the response variable when the corresponding independent variable is increased by one unit, while other predictor variables are held constant is incorrect. This is because the OLS estimator of $\beta$ given as;

and

are affected by the sample value of the explanatory variables. Precisely, in this case When multicollinearity is exact (perfect), the assumption that X has a full column rank break down and therefore\left|X^\prime X\right|=0. Consequently, the OLS estimate of equations (2.1) and (2.2) cannot be obtained. The concept of estimable function in which equations (2.1) and (2.2) now have an infinite solution of vectors is used. (Olanrewaju et al., 2017)

As long as multicollinearity is not perfect, the OLS estimates are still unbiased and BLUE (Johnson, 1984). Multicollinearity is associated with unstable estimated regression coefficients from the presence of strong linear relationships among the predictors. It is not a problem of model misspecification but rather that of data: and therefore, empirical study of this problem should only begin after the model has been satisfactorily specified (Charterjee, 2000). However, there may be some indications of the problem resulting from the process of adding, deleting, and transformation of variables or data points in search of a good model. Indications of multicollinearity that appear as a result of instability in regression coefficients are as follows.

i. Large changes in estimated OLS coefficients when a variable is dropped or added.

ii. Large changes in the estimated OLS coefficients when a data point is dropped or altered.

Once the residual plot indicates that the regression model has been satisfactorily specified, multicollinearity may be present if:

i. The algebraic signs of the estimated coefficients do not conform to prior expectations. This may be because greater covariance between the explanatory variable produces greater sampling covariance for the OLS coefficients. Comparing the off-diagonal terms in $X^{\prime }X$ and $(X^{\prime }X{)}^{-1}$ show that a positive covariance for the X's gives a negative covariance for the $\hat{\beta }$ 's, and vice versa. In a specific application, if ${\hat{\beta }}_2$ is below ${\hat{\beta }}_2,{\hat{\beta }}_3$ is most likely to exceed ${\hat{\beta }}_3$ and vice versa (provided that X's are positively correlated).

ii. Coefficients of variables X's that are expected to be important have large standard error (small t - value).

A thorough investigation of the presence of multicollinearity in a system of regression equations can be accomplished by several methods which include:

i. The use of variance inflation factor (VIF): Charterjee (2000).

ii. Principal component analysis approach: Seber (1984), Johnson and Wichern (1992), Charterjee (2000).

iii. The use of two-step procedure: Besley, et al. (1980).

iv. The Farrar - Glauber test: Farrar and Glauber (1967).

The assumption that the regressors X are treated as fixed variables in repeated samples is often violated by economists and other social scientists. The reason for this violation is because their X is often being generated by stochastic processes beyond the scientists' control. For instance, consider regressing daily bathing suit sales by a department store on the mean daily temperature. Surely, the department store cannot control daily temperature, so it would not be meaningful to think of repeated sampling when temperature levels are the same from sample to sample. Fomby et al. (1988) demonstrated that under general conditions, the essential results of the classical linear regression model remain intact even with stochastic regressors. Neter and Wasserman (1974) pointed out that all results on estimations, testing, and prediction obtained using the classical linear regression model still applies if the following conditions hold:

• The conditioned distribution of the dependent variable given the regressors are normal and independent, with means $X\beta$ and conditional variance ${\sigma }^2$

• The regressors are independent random variables, whose probability distribution does not involve the parameter of the classical linear regression model and the conditional variance ${\sigma }^2$.

However, they pointed out that modification would occur in the area of confidence interval calculated for each sample and the power of the test. This problem was also discussed and supported by Chartterjee et al. (2000).

The assumption that the values of X variables in a regression model are measured without error is hardly ever satisfied. Measurement errors may enter the value observed for the independent variable, 'for instance, when it is temperature, pressure, production line speed, or person's age. Consequently, the independent variable becomes correlated with the error terms (Neter and Wasserman, 1974). These measurement errors affect the residual variance, the multiple correlation coefficients, and the estimated regression coefficients. Its effects increase the residual variance and reduce the magnitude of the observed multiple correlations. The effects of these errors on the estimated regression coefficients are more difficult to assess (Chartterjee et al. 2000). A more extensive discussion of the aforementioned problem can be found in Cochran (1970), Fuller (1987), Chartterjee and Hadi (1988), and Chi – Lu and Van Ness (1999).

Dhrymes and Schwarz (1987) stated that "the heart of the problem is that the conditions on the parameters force the singularity of the covariance matrix-and to a certain degree the converse is true, i.e. the singularity of the covariance matrix implies certain restrictions." It is important to note that, as stated by Bewley (1986), "a necessary and sufficient condition for the OLS estimates to satisfy the adding-up criterion is that some linear combination of the regressors must be identically equal to the sum of regressands if the model is to be logically consistent."

Since the constraints in (2) depend on the values of the regressors, we postulate that the constraints are identically valid in the regressors, which induces restrictions on the parameters that are independent from the regressors. Thus, let Z be a T x P matrix of T-vectors $z_1,\dots ,z_p$, which constitute a base of the vector space containing the $\sum _i{k}_i$ regressors of all n equations. The obvious consequence is the existence of n matrices $c_i$ of order P x $k_i$ with $X_i=z.c_i$, for all i.

III. RESEARCH METHODOLOGY

a) The Monte - Carlo Approach

In a Monte-Carlo experiment, the experimenter artificially sets up a system (model) and specifies the distribution of the independent variables alongside with the values of the model parameters. More so, the values are then generated for the error terms and the independent variables as specified for a specified sample size. We then, make use of the generated and the parameter values to formulate data for the dependent variable. Next is to treat the generated data as if they are real-life data by estimating the parameters of the model via the estimation methods (estimators). This process of generating values for the disturbance term, independent variables, and estimating the parameters of the model which is then, replicated a large number of times. The experimenter then builds up empirical distributions of the parameter estimates, which are then used to evaluate the performance of those estimators in relation to the parameter values. (Olanrewaju et al. 2017)

The Monte - Carlo studies as stated in Olanrewaju et al. 2017, can be designed generally by using the following summarized five steps as given below:

(a) The researcher specifies a model and assigns specific numeric values as in parameters. The assigned values are assumed to be the real values of the parameters.

(b) There is need to specify the distribution of error terms.

(c) He also uses the distribution of the error terms (U's) with the random drawings to get new different values for it.

(d) The experimenter now selects or generates values for the regressors (X's) depending on the specifications of the model.

(e) The researcher obtains or generates values for the dependent variable using the computed values of the regressors and the error terms.

The five steps mentioned above are repeated several times, say R, to have R replications.

Thus, the experimenter obtains an estimate of the model parameters for each replication, treating the generated data as real-life data.

b) The Model Formulation

The system of regression equation used in this research work as stated in Olanrewaju, 2013, is given as

where- $u_{1t}=\rho u_{1(t-1)}+e_{1t},e_{1t}\approx (0,{\sigma }^2)$.

Note: (1) Multicollinearity exists between $X_1$ and $X_2$ in equation (3.1)

(3.2) appears as a control equation.

The models (3.1) and (3.2) were studied under two sub-divisions as given below:

1. There is no any form of correlation in the model i.e. $\delta =0$, $\rho =0$ and $\lambda =0$

2. There is correlation between the error term and presence of multicollinearity in the model i.e. $\delta \ne 0$, $\rho =0$ and $\lambda \ne 0$.

c) Specifications and Choice of Parameters for Simulation Study

For the simulation study in this research work, the parameters of the model in equations 3.1 and 3.2 are fixed as ${\beta }_{01}=0.4$; ${\beta }_{11}=1.8$; ${\beta }_{21}=2.5$; ${\beta }_{02}=2.0$; ${\beta }_{12}=4.5$; ${\beta }_{22}=-1.2$. The Multicollinearity ( $\delta$ ) levels are $-0.99$, $-0.9$, $-0.8$, $-0.6$, $-0.4$, $-0.2$, $0$, $0.2$, $0.4$, $0.6$, $0.8$, $0.9$, $0.99$ and that of Correlation between error terms ( $\lambda$ ) levels are $-0.99$, $-0.9$, $-0.8$, $-0.6$, $-0.4$, $-0.2$, $0$, $0.2$, $0.4$, $0.6$, $0.8$, $0.9$, $0.99$. The sample sizes (n) 20, 30, 50, 100 and 250 were used in the simulation. At a particular choice of sample size, multicollinearity level and correlation between the error terms, a Monte-Carlo experiment is performed 1000 times at two runs which were averaged at analysis stage.

d) The Data Generation for the Simulation Study

The generation of the data used in this simulation study is in three stages, which are: Generation of the (i) independent variables

(ii) error terms (iii) dependent variables

e) Computation of Data for the Independent Variables

The independent variables used in this study are fixed numbers at each trial of the simulation. They were computed using the equation provided by Ayinde (2007) to create normally distributed random variables with specified intercorrelations, i.e.

Notes

For the three normally distributed random variables given above, the newly derived equation is given as:

Since ${\rho }_{13}=0$ and ${\rho }_{23}=0$, then X becomes

where-, $g_{22}={\sigma }_2^2[1-{\rho }_{12}^2]$ , $g_{23}=0$ , $g_{33}={\sigma }_3^2$

f) Generation of the Error Terms

The two error terms, u1 and u2, assumed to be well behaved with a multivariate normal distribution $u\sim \text{NID }(0,\mathrm{\Sigma })$ as expressed in equations 3.1 and 3.2 were also generated to exhibit correlation $\lambda$ using the technique as provided by Ayinde (2007). Here is the equation in which the error terms values were generated

Suppose, $u_i\sim N(\mu ,{\sigma }_i^2)$ i = 1, 2. If these variables are correlated, then, $u_1$ and $u_2$ can be gotten by equations

Where $Z_i\sim N(0,1)$ for i = 1,2 and is the value of the correlation between the two variables.

g) Generation of Data for the Dependent Variables

Considering the system of regression equation in 3.1 and 3.2, we have

Notes

We can write 3.4 in matrix form as:

Where

Now, equation (3.6) becomes

Then, equation 3.1 and 3.2 become,

Therefore, equation (3.7) above was used to generate dependent variables $y_1$ and $y_2$ by substituting the values of model parameters, independent variables, and that of error terms as specified in the previous sections above.

h) The Evaluation, Comparative Analysis and Preference of Estimators

The evaluation and comparative analysis of the seven (7) estimation methods namely, Ordinary Least Squares (OLS), Cochran - Orcutt (CORC), Maximum Likelihood Estimator (MLE), Multivariate Regression (MR), Full Information Maximum Likelihood (FIML), Seemingly Unrelated Regression Model (SUR) and Three-Stage Least Squares (3SLS), were examined using the finite sampling properties of estimators, which include Bias (BB), Absolute Bias (AB), Variance (VAR), and the Mean Square Error (MSE) criteria.

Mathematically, for any estimator ${\hat{\beta }}_{ij}$ of Model (3.1) & (3.2)

(ii)

Notes

Using a computer program which was written with TSP software package to estimate all the model parameters and the criteria, the performances of the seven estimation methods considered in this work as mentioned above were examined by subjecting the results obtained from each finite properties of the estimators into a multi-factor analysis of variance model. Consequently, the highest order significant interaction effect, which has a "method" as a factor, is further examined using Duncan Multiple Range Test and the Least Significance Difference (LSD) test. The estimated marginal mean of the factor was investigated at a particular combination of levels of the correlations in which preferred estimators were chosen. An estimator is most preferred at a specific combination of levels of the correlation, if the marginal means is the smallest. Also, all estimators whose estimated marginal means are not significantly different from the most preferred are also preferred.

IV. ANALYSIS OF RESULTS AND DISCUSSIONS

a) Results when we do not have any Form of Correlation in the Model

The performances of the estimators under various sample sizes base on finite sampling properties of estimators using the Analysis of Variance technique are presented and discussed in the table below.

It was observed from Table1 above, that, all the estimators do not perform differently (P-value > 0.05) under all the criteria except under the variance criterion in some parameters of equation two. Thus, we concluded that all the estimators do not exhibit a significant difference in their performances under all the criteria in equation one. The results of the further test to identify those estimators that perform equivalently in equation2 are presented in Table 2. From the table, we observed that the performances of the OLS, CORC, MLE, MR, SUR, FIML, and 3SLS estimators are not significantly different. Meanwhile, the OLS, CORC, and MLE estimators are generally preferred.