Observing Variables

In Chapter 7 on Linear Correlation, Regression, and Prediction, we discussed determining the correlation and possible regression relationships between an independent variable X and a dependent variable Y. Specifically, in regression, the discussion was based on how the change in one variable (X) produced an effect on another (Y). This is the essence of regression. But from experience, we know that often multiple causes interact to produce a certain result. For example, yield from a crop is based on the amount of water a plant has to use, the soil fertility of the field, the potential of the seed to produce a plant, pest and pathogen pressures, and numerous other factors. In this lesson, we’ll explore how we can determine linear relationships between multiple independent variables and a single dependent variable.

Exploring Multiple Variables

Multiple regression functionally relates several continuous independent variables to one dependent variable. In the above example, barley yield per plot (Y) is shown as a function of the percentage of plants in the plot affected by rust (X₁) and the days to maturity required by the cultivar grown in a particular plot (X₂). Yield is modeled as a linear combination of these two X variables in the response surface on the previous slide. Before we can relate the dependent variable Y to the independent X variables, we need to know the interrelationships between all of the variables. Multiple correlation and partial correlation provide measures of the linear relationship among the variables.

Separating the individual factor’s effect on the whole result, such as the effect of rust infection or the number of days a particular cultivar requires to reach maturity, can be difficult and, at times, confounding. The objective of this chapter is to explain and illustrate the principles discussed in Chapter 7 on Linear Correlation, Regression and Prediction or correlating two variables or enumerating the effect of one variable on another, but now expanded to multiple variables.

Multiple Correlation and Regression

The correlation of multiple variables is similar to the correlation between two variables. The same assumptions apply; the sampled Y’s should be independent and of equal variance. Error (variance) is associated with the Y’s while X’s have no error, or the error is small. But now, since there are multiple factors involved, the correlations are somewhat more complex, and interactions between the X_i variables are expected. A note on notation: We now include a subscript with the “X” to indicate which independent variable to which we are referring. Three levels of correlation are used in determining the multi-faceted relationships; simple correlation, partial correlation, and total correlation.

Simple Correlation

The simple correlation between one of the X_i‘s and Y is computed for a simple correlation of X and Y. This calculation assumes a direct relationship between the particular X_i and Y. It is also useful in stating the simple relationship between two X_i‘s in the multiple correlation. When determining the significance of regression coefficients, the variable with the largest simple correlation with Y is usually the starting point. Some interaction among numerous X and Y variables is likely to occur. Because two X_i‘s have large simple correlations with a resulting Y does not necessarily indicate that their relationships to Y are independent of each other. They may be measuring the same effect on Y. The number of hours of sunlight (cloud-free skies) and GDDs both have a good (simple) correlation to the rate of crop development. But their effects would not be additive. There would be a significant interaction between these two variables in describing crop development. The two variables measure two different factors, light and temperature. However, the amount of sunlight and temperature are generally highly correlated during the summer. So there would be a significant relationship between the two variables. These individual effects can be separated using partial correlation.

Partial Correlation

Quantifying which continuous X variables are best correlated with the continuous Y-variable requires an understanding of the interactions between the X_i‘s. To break down the interaction requires partial correlation coefficients. These use the simple correlation coefficients to explain the correlation of two variables with all other variables held constant. One such example is, “How much yield will result from nitrogen applications assuming the seasonal amount of rainfall will be average?” Here, rainfall is held constant, and the effect of nitrogen on yield would be used for a partial correlation. This relationship is given for the partial coefficient of determination between Y and X₁ where two X’s are involved, as shown in Equation 1.

[latex]r_{YX_{1}X_{2}} =\frac{r_{YX_1}-r_{YX_2}r_{X_1X_2}}{\sqrt{1-r^2_{YX_2}}\sqrt{1-r^2_{YX_1}}}[/latex].

[latex]\textrm{Equation 1}[/latex] Formula for calculating partial coefficient of determination between Y and X.

where:
[latex]\small X_1, X_2[/latex] = continuous variables X,
[latex]\small Y[/latex] = continuous variable Y,
[latex]r[/latex] = correlation coefficient.

To calculate the partial coefficient of determination between Y and X₂, just reverse the equation, i.e. use r_YX1 and vice versa. Don’t panic! You will not be asked to hand-calculate this on the homework or exam. That is why we use R!

Correlation Matrix

The value r_YX1 is the simple correlation between Y and X₁. The whole equation describes the correlation between Y and X₁ with X₂ held constant. The relationship between the X₁ and Y is displayed within the effect of the interaction. Partial correlations can be calculated for all variables involved. They can also be calculated for more than three variables, but the equation becomes more complex. Often, the total and partial correlations are calculated and displayed in a table with the individual X_is and Y listed across the top and down the left side. The correlations for each variable pair are displayed at the intersection of the variables.

Total Correlation

The combination of these partial effects leads to a multiple correlation coefficient, R, which states how related the Y is to the combined effects of the X_i‘s. For X₁, X₂, and Y, the total correlation is determined once again using the simple correlations (Equation 2):

[latex]R^{2}_{Y\cdot X_{1}X_{2}} =\frac{r^{2}_{YX_1}+r^{2}_{YX_2}-2r_{YX_1}r_{YX_2}r_{Y_1X_2}}{1-r^{2}_{X_1X_2}}[/latex].

[latex]\textrm{Equation 2}[/latex] Formula for calculating multiple correlation coefficient, R.

In this equation,[latex]r^2_{YX_1}[/latex] and [latex]r^2_{YX_2}[/latex] are just the squares of partial correlation coefficients.

The calculations for 2 X_i‘s are relatively straightforward, but for three or more variables, the calculations involve a large number of terms with different correlations among individual variables. Consequently, total correlation is calculated with computer programs, such as R.

Similar to the linear correlation coefficient, the total correlation coefficient, when squared, produces the multiple coefficient of determination, R². This value explains the proportion of the Y variation, which can be accounted for by a multiple regression relationship. The partial correlation coefficients squared produce the partial coefficients of determination, r², or that proportion of variance which can be described by one variable, while the partial coefficients will be used in testing individual regression coefficients for significance.

Calculating the Correlation

Graphing data to visualize the correlation relationships in multiple dimensions is difficult. The graphing of data involving 2 X_i‘s with Y is possible in 3-dimensional space. Using the variables mentioned, the regression equation would be a plane in the X₁, X₂, Y space (Fig. 1). The partial regression coefficients for X₁ with Y and X₂ with Y in this space could be used to produce lines where the plane intersected a certain X value. For example, the following equation would produce a plane on a graph.

[latex]\small \textrm{Y}=2.4X_1+3.9X_2-7.1[/latex].

Setting X1 equal to 0 would reduce the above equation of a plane to a linear equation:

[latex]\small \textrm{Y}=3.9X_2-7.1[/latex].

Either X₁ or X₂ could be set to any value producing any number of different linear relationships in the plane. With more than two X values, graphing the relationship in 3 dimensions is not easily done. Instead of graphing, interpreting the data numerically and conceptually is the preferred method.

Multiple Regression

Relationships Among Multiple Variables

Multiple regression determines the nature of relationships among multiple variables. The resulting Y is based on the effect of several X’s (Equation 3). How much of an effect each has must be quantified to determine the equation (below). The degree of effect each X has on the Y is related through partial regression coefficients. The b-value estimate of each regression coefficient can be determined by solving simultaneous equations. Usually, computer programs determine these coefficients from the data supplied.

[latex]\normalsize \textrm{Y}=a + b_1x_1 + b_2x_2 + ... + b_ix_i + \epsilon[/latex].

[latex]\textrm{Equation 3}[/latex] Formula for calculating multiple regression for the relationship among multiple variables.

where:
[latex]a[/latex] = the [latex]\small Y[/latex] intercept,
[latex]b[/latex] = estimates of the true partial regression coefficients, β.

The [latex]a[/latex] is the Y-intercept, or Y estimate, when all of the X’s are 0. The [latex]b's[/latex] are estimates of the true partial regression coefficients β, the weighting of each variable’s effect on the resulting Y. The b’s are interpreted as the effect of a change in that X variable on Y, assuming the other X’s are held constant. These can be tested for significance. The weighting of effects now will be based on regression techniques.

The simplest example of multiple linear regression is where two X’s are used in the regression. The technique of estimating b₁ and b₂ minimizes the error sums of squares of the actual from estimated Y’s. The variability of the data (Y’s) can be partitioned into that caused by different X variables or into error.

Example of Multiple Correlation and Regression

The simplest example of multiple correlation involves two X’s. Calculations with more variables follow a similar method but become more complex. Computer programs have eased the computational problems. Proper analysis of the data and interpretation of analyses are still necessary and follow similar procedures.

The following two-variable research data were gathered relating the yield of inbred maize to the amount of nitrogen applied and the seasonal rainfall data (Table 2).

Review the Data

The first issue is to review how highly correlated the data are. Since visualization of multiple data is more difficult, numerical relationships must be emphasized. The first step is to examine the correlations among the variables. The simple correlations (calculated as in Chapter 7 on Linear Correlation, Regression and Prediction) may be computed for the three variables (see below).

Simple Correlations

[latex]\small r_{YX_1} = 0.895 \\ \small r_{YX_2} = 0.944 \\ \small r_{X_1X_2} = 0.905[/latex]

Partial Coefficients of Determination

All are highly correlated. But these simple correlations include the interactions among variables. To determine individual relationships, calculations of partial coefficients of determination are helpful (Equation 4).

[latex]r^2_{YX_1 \dot{}X_2} = \frac{(r_{YX_1} - r_{YX_2}r_{X_1X_2})}{(1 - r^2_{X_1X_2})(1-r^2_{X_1X_2})} = \small 0.081[/latex]

[latex]r^2_{YX_2 \dot{} X_1} = \small 0.499[/latex]

[latex]r^2_{YX_2 \dot{} X} = \small 0.170[/latex]

[latex]\textrm{Equation 4}[/latex] Example calculation of partial coefficient of determination.

These values are the additional variability that can be explained by a variable, such as that by X₁, after the variability of X₂ alone has been accounted for. These values are used in computing the ANOVA for multiple regression. The partial correlations may be found by taking the square root of these partial coefficients of determination.

Total Coefficients of Determination

The R²-value is the total coefficient of determination, which combines the X’s to describe how well their combined effects are associated with the Y’s. This is determined as shown below (Equation 5).

[latex]R^2_{Y \dot{}X_1X_2} = \frac{0.801+0.891-2(0.895)(0.944)(0.905)}{1-0.819} = \small 0.900[/latex]

[latex]\textrm{Equation 5}[/latex] Example calculation of total coefficient of determination.

The R² value is the proportion of variance in Y that is explained by the regression equation. This can be used to partition the variability in the ANOVA. The square root of this value gives the correlation of the X’s with Y. It is obvious that the correlations are not additive. The simple correlations are all greater than 0.8, and the correlation between X₁ and X₂ is 0.905. This is where partial correlation comes into play.

Partial Regression Coefficients

Before we can create an ANOVA and test the regression, we need a regression equation as determined by R. The estimate of the regression relationship is found to be in the below equation.

[latex]\small \textrm{Y}= 49.53 + 0.089x_1 + 0.515x_2[/latex].

The partial regression coefficients indicate that for the data gathered here, each additional pound of nitrogen applied per acre would produce an additional 0.089 bushels of maize per acre, and for each additional inch of rainfall, an additional 0.516 bushels per acre. An estimate of the yield is determined by entering the amount of nitrogen applied to the field and the amount of rainfall into the equation. The number produced is the regression equation estimate of the yield based on the data gathered.

The next issue is deciding if this equation is useful and explains the relationship in the gathered data. The sums of squares are partitioned in an ANOVA table, and the significance of the regression equation as a whole and the individual regression coefficient estimates are tested for significance in the next section.

Ex. 2: Multiple Regression and Anova Using R (1)

This exercise contains 26 steps (including this statement).

Ex. 2: Multiple Regression and Anova Using R (2)

R CODE FUNCTIONS

• anova
• summary
• lm
• pf
• ppcor

Multiple regression is used to determine the nature of relationships among multiple variables. The response variable (Y) is defined as the product of the effects of several explanatory variables (X’s). The level of effect each X has on the Y variable must be quantified before a regression equation can be constructed (i.e. equation 3). The degree of effect each X has on the Y is related through partial regression coefficients. The coefficient estimate of each explanatory variable can be determined by solving simultaneous equations. Usually, computer programs such as R determine these coefficients from the data supplied, [latex]\small \textrm{Y}=a + b_1X_1 + b_2X_2 + ... + b_iX_i + \epsilon[/latex].

In equation 3 above, the a term is the Y-intercept, or the estimate of Y when all of the X’s are 0. The b with each X is an estimate of the true partial regression coefficient ß for that X variable, the weighting of each variable’s effect on the resulting Y. The b’s are interpreted as the effect of a change in that X variable on Y, assuming the other X’s are held constant. These coefficients can also be tested for significance. The weighting of effects will now be based on regression techniques.

The simplest example of multiple linear regression is where two X variables are used in the regression. The technique of estimating b₁ and b₂ via multiple regression minimizes the error sums of squares of the actual data from the estimated Y’s. The variability in the data can be partitioned into that which is caused by different X variables or that which is caused by error.

In the file QM-Mod13-ex2.csv, we have yield data from one inbred maize line under all factorial combinations of 9 different levels of nitrogen treatment and 9 different levels of drought treatment. We’ll use these data to investigate correlations between the variables, to do a multiple regression analysis, and to carry out an analysis of variance (ANOVA).

Ex. 2: Multiple Regression and Anova Using R (3)

Ex. 2: Multiple Regression and Anova (4)

Simple Correlation

The correlation between X and Y, ([latex]\small P_{XY}[/latex]), is calculated as the covariance of X and Y divided by the product of the standard deviations of X and Y, i.e., [latex]\small P_{XY}=\frac{cov(X,Y)}{\sigma_{X}\sigma_{Y}}[/latex]
The first step is to review how highly correlated the data are. Since visualization of multiple data is more difficult, numerical relationships must be emphasized. Let’s examine the correlations among the variables. Calculate the simple correlations between the 3 variables by entering into the console window.

cor(data)

R returns the simple correlations matrix

             drought            N         yield
drought    1.0000000    0.0000000     0.7364989
N          0.0000000    1.0000000     0.2147159
yield      0.7364989    0.2147159     1.0000000

Ex. 2: Multiple Regression and Anova Using R (5)

Partial Correlation

Taking X₁ to be nitrogen, X₂ to be drought, and Y to be yield, we can list the simple correlation variables.

Simple Correlations

[latex]\small r_{YX_{1}}=0.2147159 \\ \small r_{YX_{1}}=0.7364989 \\ \small r_{Y_1X_{2}}=0[/latex]
Both nitrogen and irrigation are correlated with yield, but these simple correlations include interactions among the variables. To determine individual relationships, calculations of partial correlation coefficients are helpful. Partial correlation coefficient values are the additional variability in the response variable that can be explained by an independent variable, such as that by X₁, after the variability of another independent variable, such as X₂, alone has been accounted for. These values are also used in computing the ANOVA for multiple regression.

We will now do a quick investigation of the partial correlations between the variables in the dataset. If you haven’t already, load the ‘ppcor’ package. Then, use the pcor command to obtain the matrix of partial correlations between all variables in the data set.

library(ppcor)

pcor(data)

Ex. 2: Multiple Regression and Anova Using R (6)

R returns 3 matrices: a matrix with the partial correlation coefficient estimates ($estimate), a matrix with the test statistic for the estimate ($statistic), and a matrix for the p-value of the test statistic ($p.value).

$estimate

           drought            N        yield
dtm     1.0000000    -0.239363    0.7540868
N      -0.2393630     1.000000    0.3174210
yield   0.7540868     0.317421    1.0000000

$p.value

                drought             N          yield
drought   0.000000e+00   0.029458897   3.658519e-24
N         2.945890e-02   0.000000000   3.113831e-03
yield     3.658519e-22   0.003113831   0.000000e+00

$statistic

                drought           N       yield

 dtm           0.000000   -2.177291   10.140333
perc.inf     -2.177291    0.000000    2.956271
yield        10.140333    2.9562

Ex. 2: Multiple Regression and Anova Using R (7)

Let us calculate the partial correlation coefficient for nitrogen on yield by hand, using Equation 1, to check the calculation returned by R in the estimate matrix.

[latex]r_{YX_{1}X_{2}} =\frac{r_{YX_1}-r_{YX_2}r_{X_1X_2}}{\sqrt{1-r^2_{YX_2}}\sqrt{1-r^2_{YX_1}}}[/latex]
[latex]r_{YX_{1}X_{2}} =\frac{(0.2147159-0.7364989*0)}{\sqrt{(1-0.7364989^2)(1-0)}}=\frac{0.2147159}{0.6764387}=\small 0.317421[/latex].

You can see that the value in the estimate matrix for the partial correlation coefficient between nitrogen and yield is identical to the value obtained by our hand calculation. Also, based on the p-value matrix, all of the partial-correlation estimates are statistically significant.

The test-statistic matrix contains values calculated from the standard normal distribution (with a mean of 0 and standard deviation of 1). The test statistic for the partial correlation of nitrogen on yield is 12.12521. We can check that this value is correct by calculating the p-value for this value from the standard normal distribution by entering

(1-pnorm(2.95621, mean=0, sd=1))*2

Ex. 2: Multiple Regression and Anova Using R (8)

R returns

[1]0.00311834

The p-value for the partial correlation coefficient given in the R output from calculating the partial correlation coefficients is identical to that given in the R output using the pcor function.

The R² value is the total coefficient of determination, which combines the explanatory variables (X’s) to describe how well their combined effects are associated with the response variable (Y). This is determined by the following equation:

[latex]R^2_{Y \dot{}X_1X_2} = \frac{r^2_{YX_1} + r^2_{YX_2} - 2r_{YX_1}r_{YX_2}r_{X_1X_2}}{1-r^2_{X_1X_2}}[/latex]

The R² is very useful for interpreting how well a regression model fits. Its value is the proportion of variance in Y that is explained by the regression equation. The closer to 1.0, the better the fit; a value of 1 would mean all of the data points fall on the regression line. The square root of this value gives the correlation of the X’s with Y. It is obvious that the correlations are not additive. This is where partial correlation comes into play.

Ex. 2: Multiple Regression and Anova Using R (9)

The drawback of relying on the R2 value as a measure of fit for a model is that the value of R² increases with each additional term added to the regression model, regardless of how important the term is in predicting the value of the dependent variable. The Adjusted R² value (or R²_Adj) is a way to correct for this modeling issue. The formula for Adjusted R² is:

[latex]R^2_{Adj}=1-\frac{(1-R^2)-(N-1)}{n-K-1}[/latex].

 where R2 is the regression coefficient, n is the sample size, and k is the number of terms in the regression model. The R-squared value increases with each additional term added to the regression model, so taken by itself, can be misleading. The R²_Adj takes this into account and is used to balance the cost of adding more terms; i.e. it penalizes the R² for each additional term (k) in the model. The R²_Adj value is most important for comparing and selecting from a set of models with different numbers of regression terms. It is not of great concern until you are faced with choosing one model to describe a relationship over another. We’ll carry out hand calculations for both R² and R²_Adj after we run the regression in R.

Ex. 2: Multiple Regression and Anova Using R (10)

Regression Model Significance

The initial test is to determine if the total regression equation is significant. As in linear regression, “does the regression relationship explain enough of the variability in the response variable to be significant?”

The testing of the regression equation partitions the total sum of squares using the total coefficient of determination, R². Note that this is not the same as the square of total correlation.

[latex]\small R^2_{Adj}={\dfrac{\text{(Regression SS)}}{\text{ Total SS}}}[/latex].

Initially, the null hypothesis being tested is that the whole regression relationship is not significantly different from 0.

[latex]\small H_{0}:\beta _{1}=\beta _{2}=0 \\ \small H_{a}:\beta _{1} = \beta _{2}\neq 0[/latex]

Ex. 2: Multiple Regression and Anova Using R (11)

The F-test for multiple linear regression uses the regression mean square to determine the amount of variability explained by the whole regression equation. If the regression mean square is significant at your specified level, the null hypothesis that all of the regression coefficients are equal to 0 is rejected. This F-test does not differentiate between coefficients; all are significant, or none are according to the test.

Individual regression coefficients (b₁, b₂, etc.) may be tested for significance. The simple coefficient of determination between each X and Y explains the sum of squares associated with each regression coefficient, including interactions with other X’s. The partial coefficient of determination between each X and Y explains the additional variability without interaction. These can be tested with the residual error not explained by the regression model to test the significance of each X.

Each coefficient may also be tested with a t-test; R does this automatically when you run a multiple regression model using the lm function.

Ex. 2: Multiple Regression and Anova Using R (12)

MULTI-LINEAR REGRESSION

Let’s run a multiple regression analysis where yield is the response variable and drought and nitrogen are the explanatory variables. We will keep nitrogen and drought as numeric variables for this analysis but later will run the same analysis with these variables as factors.

In the console window, enter

summary(1m(data=data,yield~drought+N))

Let’s go through this command from the inside out.

1. data = data indicates that we want to run the linear model with dataset ‘data’
2. yield~ drought + N specifies that the regression equation we are analyzing is yield = ‘the amount of nitrogen applied’ + ‘the amount of drought applied’
3. lm indicates to R that we want to run a linear regression model
4. Summary indicates that we want R to return all of the useful information from the regression analysis back to us.

Ex. 2: Multiple Regression and Anova Using R (13)

Ex. 2: Multiple Regression and Anova Using R (14)

The R² value is given at the bottom of the R output as 0.5885. This means that the model explains 58.85% of the variation in yield. Let’s calculate the R² value by hand using the simple correlation coefficient matrix from above (Equation 2).

[latex]R^2_{Y \cdot{} X_1 X_2} = \frac{r^2_{YX_1}+r^2_{YX_2}-2r_{YX_1}r_{YX_2}r_{X_1X_2}}{1-r^2_{X_1X_2}}[/latex]

[latex]{R^2_{Y \cdot{}X_1X_2}}=\frac{0.2147159^2+0.7364989^2-2(0.2147159)(0.7364989)(0)}{1-0} = \small 0.5885[/latex]

The value obtained for R² obtained by our hand calculation is identical to the value returned by R.

Now, let’s calculate the [latex]R^2_{Adj}[/latex]for the model by hand. Use the value of R² from the R output (0.5885).

[latex]R^2_{Adj}=1-\frac{(1-R^2)-(N-1)}{n-K-1}[/latex] = [latex]1-\frac{(1-0.5885)-(80-1)}{(80-2-1)}=\small 0.5778117\sim 0.578[/latex].

 This is the same value for [latex]R^2_{Adj}[/latex] as given in the R output (under “Adjusted R-squared”).

Ex. 2: Multiple Regression and Anova Using R (15)

VARIABLE INTERACTION

Should we include a term in the linear model indicating the interaction between nitrogen and drought? Let’s run the regression again, this time adding a variable accounting for the interaction between the two independent variables into the model. (i.e. the amounts of drought and nitrogen applied). The interaction variable is specified using a multiplication sign (*) with the explanatory variables that you are analyzing for interaction.

summary(1m(data=data,yield~N+drought+N*drought))

Ex. 2: Multiple Regression and Anova Using R (16)

Ex. 2: Multiple Regression and Anova Using R (17)

Compare the R²_Adj value and the F-statistic of the model, including the interaction, to the model not including the interaction. Which model fits the data better?

With interaction: R²_Adj = 0.5754, F = 37.14

Without interaction: R²_Adj = 0.578, F = 55.78

The model without the interaction between Nitrogen and Irrigation has a slightly better fit for these data than the model including the interaction. Also, the regression coefficient on the interaction term has a very high p-value, indicating that is not statistically significant. Save the model without the interaction as ‘m1’.

m1<-1m(data=data,yield~N+drought)

Ex. 2: Multiple Regression and Anova Using R (18)

Calculate the ANOVA table for the multiple regression models with and without the interaction between N and drought.

First carry out the ANOVA for the model without the interaction.

Enter into the console window

anova(lm(data=data,yield~drought+N))

R returns the ANOVA table

Now run the ANOVA with the linear model excluding the interaction term.

anova(lm(data=data,Yield~drought+N))

Ex. 2: Multiple Regression and Anova Using R (19)

Ex. 2: Multiple Regression and Anova Using R (20)

The ANOVA table lists the model, error, and the sources of variation along with their respective degrees of freedom (df), sum of squares, mean squares, and an F-test for the model. Each model parameter has 1 df. The total df is 1 less than the total number of observations, in this case 80 (i.e. 1 + 1 + 78). This correction is for the intercept; the single df that is subtracted reflects this. We are most interested in the F-test for the model, which is calculated by dividing the model MS by the error MS. The F-statistic and p-value of the F-statistic for the model are listed at the bottom of the R output that we obtained from running the multiple regression model. The model MS is not listed in the ANOVA table R returned to us; however, we can easily calculate the F-statistic for the model using the ANOVA output as the mean of the F-statistics for the model parameters. The p-value for the model can also be calculated from the ANOVA table. The F-statistic value we obtain by averaging the drought and N F-statistics is [latex]\small {\dfrac{\text{(102.8263 + 8.7395)}}{\text{ 2}}}[/latex].

To get the p-value for this F-statistic, in the R console window, enter

1-pf(55.78,2,80)

Ex. 2: Multiple Regression and Anova Using R (21)

The value returned is 3.725908*10^-13. The probability of the F-statistic value of 78.4533 occurring by chance is only incredibly small, so we conclude that the model we have developed explains a significant proportion of the variation in the data set.

R² can be calculated from the ANOVA table as the model sum of squares (SS) divided by the corrected total SS [latex]\small {\dfrac{\text{81959 + 6966}}{\text{81959 + 6966 + 62171}}}=0.5885[/latex].

This is the same value as was reported for R² in the regression output.

MLR with factors

Let us run the same multiple regression model again, but this time having N and drought as factors instead of numbers. We must tell R that we want entries for these variables to be considered factors and not numbers. As factors, there are 9 specified treatment amounts for each of the 2 independent variables and 81 possible combinations between the 2-factor variables.

Convert the data for N and drought into factor variables.

Ex. 2: Multiple Regression and Anova Using R (22)

Ex. 2: Multiple Regression and Anova Using R (23)

Ex. 2: Multiple Regression and Anova Using R (24)

Ex. 2: Multiple Regression and Anova Using R (25)

Ex. 2: Multiple Regression and Anova Using R (26)

R CODE FUNCTIONS

• anova
• summary
• lm
• install.packages
• library(’ppcor’)
• cor
• pcor

You are a maize breeder in charge of developing an inbred line for use as the ‘female’ parent in a hybrid cross. Yield of the inbred female parent is a major factor affecting hybrid seed production; a high level of seed production from the hybrid cross leads to more hybrid seed that can be sold. Only 2 lines remain in your breeding program, and your boss wants you to determine which of the two lines has the best yield-response to variable Nitrogen fertilizer (N) applications under several different drought levels. The three-variable dataset relating the yield (per plot) of the 2 inbred lines to the amount of N and level of drought applied to each plot can be found in the file ex3.csv.

Determine the simple and partial correlation between yield and the amount of nitrogen fertilizer applied and drought for each of the lines. Then, develop a regression equation to predict yield from the independent variables. Test to see if an interaction between drought and N should be included in the linear model. Decide on a model to evaluate these data and decide which of the 2 lines should be selected.

Load the file ex3.csv into R.

data<-read.csv(“ex3.csv”,header=TRUE)

Check the head of the data to make sure the file was read into R correctly.

head(data)

         drought           N      yield     rep    line
1            -4       0.000   2991.842       1       1
2            -4      28.025   3533.566       1       1
3            -4      56.050   2900.837       1       1
4            -4      84.075   5759.073       1       1
5            -4     112.100   7630.583       1       1
6            -4     140.125   6723.432       1       1

Ex. 3: Correlation, Multiple Regression and Anova (3)

Ex. 3: Correlation, Multiple Regression and Anova (4)

There are only 2 lines in the data. The positive correlation between yield and line means that the two variables move in the same direction; a higher value for line (i.e. 2) corresponds to higher values of yield, and vice versa. This positive correlation provides evidence for line 2 being the higher-yielding line.

If there were more than 2 lines and more than 2 reps in these data, could we analyze the data in the same way (i.e. could ‘line’ and ‘rep’ be classified as numbers in the analysis)? Could we calculate the correlation between yield and line and rep and yield? If we had more than 2 lines and reps in these data, we’d have to reclassify the ‘line’ and ‘rep’ variables as factors. We would then not be able to calculate the correlation between yield and line and rep and yield.

You should’ve already installed the package ‘ppcor’. If you have, ignore the ‘install.package’ command and simply load the package using the ‘library’ command.

install.packages (‘ppcor’)

library(ppcor)

Calculate the partial correlations between the 3 variables.

pcor(data)

Ex. 3: Correlation, Multiple Regression and Anova (5)

Ex. 3: Correlation, Multiple Regression and Anova (6)

Ex. 3: Correlation, Multiple Regression and Anova (7)

Ex. 3: Correlation, Multiple Regression and Anova (8)

Ex. 3: Correlation, Multiple Regression and Anova (9)

Testing Multiple Regression

Regression Model Significance

Because multiple effects are involved in multiple regression, the determination of which terms and variables are of importance adds a level of difficulty to the analysis. Not only are there direct effects from certain variables, but combinations of effects among separate variables. These are caused by the interaction between several variables. The effects are estimated by using the associated regression coefficients.

The initial test is to determine if the total regression equation is significant. As in linear regression, “Does the regression relationship explain enough of the variability in Y to be significant?” Partitioning the sums of squares into an ANOVA table can be used to resolve the hypothesis test. The ANOVA table for multiple regression is similar to that in linear regression. Additional regression degrees of freedom are included for each X variable. Two df’s are used for a regression relationship with two variables.

The testing of the regression equation partitions the total sum of squares using the total coefficient of determination, R², (Equation 6). Note that this is the same as the square of the total correlation, as given in the Total Correction section above.

Formula for calculating total coefficient of determination.

[latex]\small R^2 ={\dfrac{\text{Regression SS}}{\text{ Total SS}}}[/latex].

[latex]\textrm{Equation 6}[/latex] Formula for calculating total coefficient of determination.

The Whole Regression Relationship

The hypothesis being tested, initially, is to test the whole regression relationship to see if it is significantly different from 0, Equation 7.

[latex]\small H_0: \beta_1\text{or}\beta_2\text{= 0}[/latex].

[latex]\small H_A: \beta_1\text{or}\beta_2{\neq 0}[/latex].

[latex]\textrm{Equation 7}[/latex] Null and alternative hypotheses formulae.

The F-test uses the regression mean square, RegMS, to determine the amount of variability explained by the whole regression equation. If the RegMS is significant at your alpha level, the null hypothesis that all of the partial regression coefficients equal 0 is rejected. This F-test does not differentiate any coefficients, all are significant, or none are according to the test.

The total sum of squares in this data set can be calculated as 338 (see Exercise 3). The R² was calculated in the last section as 0.900. The ANOVA table (Table 3) with two degrees of freedom is calculated.

The complete regression model is significant at a probability much less than 0.01. The regression equation is significant, explaining sufficient variability in the data.

Regression Coefficient Significance

Individual regression coefficients (b₁, b₂, etc.) may be tested for significance. The simple coefficient of determination between each X and Y explains the sum of squares associated with each regression coefficient, including interactions with other X’s. The partial coefficient of determination between each X and Y explains the additional variability without interaction. These can be tested with the residual error not explained by the regression to test the significance of each b.

Each coefficient may also be tested with a t-test, and this is done in the Parameter Estimates table of the lm() Output. Tested individually, X₂ is significant, while the coefficient for X₁ is not significant. The nitrogen term would probably be dropped because it explains little additional variance beyond that from the rainfall. The final equation would be a simpler linear equation obtained by just adding Rainfall to the Fit Model, not including N Fertilizer. This equation is:

[latex]\small \textrm{Y}=49.53+0.515X_2[/latex].

[latex]\textrm{Equation 8}[/latex] Linear model for fitting rainfall.

 You will note from the original regression equation that the X₁ coefficient was small. This does not necessarily mean that small regression coefficients are not significant. They need to be tested to determine their significance. The testing is not to attempt to remove terms but to remove terms which add to the complexity without explaining variability in the regression analysis. Dropping terms from an equation is not always done. All coefficients in the regression equation may be significant and may be kept to explain the variability in the response.

R CODE FUNCTIONS

• anova
• summary
• lm
• install.packages
• library
• cor
• pcor
• ggplot
• reshape2

If the assumptions necessary for multiple regression are not met, a number of problems can arise. These problems can usually be seen when examining the residuals, the difference between the actual Y’s from the predicted Y’s.

First, if the Y’s are not independent, serial correlation (or auto-correlation) problems can result. These can be seen if the residuals are plotted versus the X values, showing a consistently positive or negative trend over portions of the data. When collecting data over a period of time, this can be a problem since temporal data has some relationship to the value at the previous time. For instance, a temperature measurement 5 minutes after a previous one is going to be strongly correlated with the previous measurement because temperatures do not change that rapidly. These problems can be overcome by analyzing the data using different techniques. One of these is to take the difference of the value at the current time step from the value at the last time step as the Y value instead of the measured value.

Ex. 4: Non-Linear Regression and Model Comparison (2)

Second, violating the equal variances assumption leads to heteroscedasticity. Here the variance changes for changing values of X. A plot of residuals where the spread gradually increases toward lower or higher X’s can also occur.

The third problem is multicollinearity. Here two or more independent variables (X’s) are strongly correlated (for example, the growing degree days (GDD) and hours of sunlight). The individual effects are hard to separate and lead to greater variability in the regression. Large R² values with insignificant regression coefficients are seen with this problem. Eliminating the least significant variable after testing will often solve this problem without changing the R² very much.

Ex. 4, Non-Linear Regression and Model Comparison (3)

POLYNOMIAL FUNCTION

A set of functions that can be useful for describing quantitative responses are the various orders of polynomial functions. Polynomial functions have a general form:

[latex]Y=a + bx + cx^2 + dx^3 + ... + nx^{n-1} + \epsilon[/latex].

[latex]\textrm{Equation 9}[/latex] Polynomial function model.

A horizontal line is a polynomial function of order 0. Linear relationships are polynomial functions of the first order. The highest exponent of X in the function determines the order of the polynomial (0 for a horizontal line, 1 for simple linear regression equation). Each order has a distinctive shape. First-order polynomials produce a straight line, second-order polynomials produce a parabola, third-order polynomials produce a parabola with one inflection point and fourth-order polynomials produce a parabola with 2 inflection points. Graphs of the first 4 orders are shown Fig. 2 below.

Ex. 4, Non-Linear Regression and Model Comparison (4)

As with the other functions, an infinite number of curves may be created by carrying the coefficients. A polynomial function can usually be fit to most sets of data. The value of such relationships can be questioned at very high orders, though. Important in most functional relationships is the physical or biological relationship represented in the data. Higher-order relationships sometimes produce detailed equations which have a relatively limited physical or biological relevance.

ADDING ORDERS OF FUNCTIONS

Polynomial relationships are calculated to reduce the variability around the regression line, whatever the order. The usual technique is to begin with a linear equation. If the deviation from this line is significant, add a term to reduce the sum of squares (SS) about the line. Adding another order to the polynomial reduces the sum of squares. When the reduction of the sum of squares by adding another order becomes small, the limit of the equation has been reached. Enough terms can be added to fit any dataset. Generally, a third-order equation is the upper limit of terms in an equation to have any relevance. More terms often simply fit the error scatter of the data into the equation without adding additional relevance.

Ex. 4, Non-Linear Regression and Model Comparison (5)

Each additional order should be tested for significance using the hypothesis H₀: highest order coefficient = 0. This can be tested using the equation:

[latex]\small F={\dfrac{\text{Regression SS for higher degree model - Regression SS for lower degree model}}{\text{ Residual MS for higher degree model}}}[/latex].

where:
[latex]\small \textrm{Numerator df}[/latex] = 1,
[latex]\small \textrm{Denominator df}[/latex] = residual df for the higher order model.

Test additional models using the same approach.

Now we’ll look at some very simple data and try to find the best model to fit the data. In the file QM-Mod13-ex4.csv, you’ll find a very small data set giving the rate of runoff for various amounts of rainfall. Read the file QM-Mod13-ex4.csv into R and take a look at it (there are only 10 entries, so don’t use the “head” command).

data<-read.csv(“ex4.csv”,header=T)

data

Ex. 4, Non-Linear Regression and Model Comparison (6)

Ex. 4, Non-Linear Regression and Model Comparison (7)

R returns the plot in Fig. 3.

Let us run the regression models of the 1^st and 2^nd order (i.e. Runoff ~ Rainfall and Runoff ~ Rainfall^2) and compare them visually and statistically. We will plot the predictive function given by each regression model output on a scatterplot with these data and compare the models visually.

We use “I” in front of the “x” variable in the “lm” command to indicate to R that we want the higher-order of x included in the model (i.e., for a second-order model, we would indicate x² by entering: I(x^2).

Ex. 4, Non-Linear Regression and Model Comparison (8)

Ex. 4, Non-Linear Regression and Model Comparison (9)

Ex. 4, Non-Linear Regression and Model Comparison (10)

R returns the plots for the different-order polynomials in a single graph (Fig. 4).

1ST ORDER MODEL

Ex. 4, Non-Linear Regression and Model Comparison (11)

Ex. 4, Non-Linear Regression and Model Comparison (12)

Ex. 4, Non-Linear Regression and Model Comparison (13)

Ex. 4, Non-Linear Regression and Model Comparison (14)

Ex. 4, Non-Linear Regression and Model Comparison (15)

3RD ORDER MODEL

Let us take a look at the output for the 1st order regression model and ANOVA.

summary(m3)

anova(m3)

Call:

lm(formula = y ~ x + I(x^2) + I(x^3), data = data)

Residuals:

    Min       1Q    Median      3Q     Max

-2.7284  -1.2079    0.3878  1.1311  2.4531

Coefficients:

               Estimate  Std. Error  t value  Pr(>|t|)

(Intercept)    15.51888    10.04750    1.545     0.173

x              -6.94363     4.28634   -1.620     0.156

I(x^2)          0.63545     0.41826    1.519     0.180

I(x^3)         -0.01393     0.01166   -1.195     0.277

Residual standard error: 2.053 on 6 degrees of freedom

Multiple R-squared:   0.9368

   Adjusted R-squared:   0.9052

F-statistic:  29.66 on 3 and 6 df, p-value: 0.0005382

Ex. 4, Non-Linear Regression and Model Comparison (16)

Analysis of Variance Table

Response: y

            Df   Sum Sq  Mean Sq  F value     Pr(>F)

x            1  261.105  261.105  61.9225  0.0002229  ***

I(x^2)       1  108.085  108.085  25.6330  0.0023041  **

I(x^3)       1    6.017    6.017   1.4269  0.2773480

Residuals    8   25.300    4.217

—

Signif. codes:   0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1

Here, you see that not much more information about the response has been gained. The R² (and R²_Adj) increases little, and very little additional variability is explained in the third-order regression (Fig. 4). In the ANOVA table, the F-value for the third-order regression is not significant at even the 0.10 level. The second-order polynomial, therefore, is the best polynomial equation for describing the response. Physically, we are trying to fit a relationship between rainfall to runoff. The negative runoff or infiltration after rain begins makes sense. The x² relationship may be explainable since we are considering a volume of runoff from a depth of rainfall. The equation does fit the data well. Again, this fits only the data gathered. Use of this relationship beyond the scope of this dataset would be improper.

Problems in Multiple Regression

Examining Problems

Recall the assumptions for regression discussed at the beginning of the lesson and in Chapter 10 on Mean Comparisons. If the assumptions necessary for multiple regression are not met, a number of problems can arise. These problems can usually be seen when examining the residuals, the difference between the actual Y’s and the predicted Y’s.

First, if the Y’s are not independent, serial correlation or auto-correlation problems can result. These can be seen if the residuals are plotted versus the X values, showing a consistently positive or negative trend over portions of the data. When collecting data over a period of time, this can be a problem since temporal data has some relationship to the value at the previous time. For instance, a temperature measurement 5 minutes after a previous one is going to be strongly correlated with the previous measurement because temperatures do not change that rapidly. These problems can be overcome by analyzing the data using different techniques. One of these is to take the difference of the value at the current time step from the value at the last time step as the Y value instead of the measured value.

Second, violating the equal variances assumption leads to heteroscedasticity. Here the variance changes for changing values of X. A plot of residuals where the spread gradually increases toward lower or higher X’s can also occur. The residual plot from the replicated data regression in Chapter 7 on Linear Correlation, Regression and Prediction shows a hint of this. Notice how the residuals start to spread slightly as date of harvest, X, increases (Fig. 5).

Multicollinearity

The third problem is multicollinearity, as we discussed in the first part of the unit. Here two or more independent variables (x) are strongly correlated (for example, the GDD and hours of sunlight variables). The individual effects are hard to separate and lead to greater variability in the regression. Large R² values with insignificant regression coefficients are seen with this problem. Eliminating the least significant variable after testing will often solve this problem without changing the R² very much. The example just discussed showed such a property, where the X₁ and X₂ values were strongly correlated (r=.905). The insignificant coefficient can be eliminated, usually solving the problem (Fig. 6).

Polynomial Relationships

The application of the various orders of polynomial functions in describing quantitative responses has been covered in the earlier section – Polynomial Functions above. The equations, which are linear in the parameters (a, b, c, in Equation 9), are used to fit experimental data similar to the methods described earlier in this unit. Also covered earlier is enough terms can be added to fit any data set, with, in general, a third-order equation being the upper limit of terms in an equation to have any relevance. More terms often simply fit the error scatter of the data into the equation without adding additional relevance.

Each additional order should be tested for significance using the hypothesis H0: highest order coefficient = 0 using the F-test equation in section (Ex. 4. Non-Linear Regression and Model Comparison (5)) above.

Polynomial Regression

Polynomial Example

Let’s use the example from Chapter 7 on Linear Correlation, Regression and Prediction. The data set was approximated using a linear model (Fig. 7).

The R² value is 0.62, with a regression SS of 242.1 and a residual SS of 149.2. The linear regression is significant, but there is a scatter about the regression line. Fitting the same data with a second-order polynomial produces:

[latex]\small y =  4.32 - 2.05x + 0.15x^2[/latex].

[latex]r^2 = \small 0.93[/latex].

[latex]\small F=\frac{362.2-242.1}{4.15} = \small 28.9[/latex].

[latex]\small \textrm{Critical F} = \textrm{12.25; p-value = 0.01}[/latex].

[latex]\small \textrm{with df 1, 7}; p-value {\lt \lt \lt 0.01}[/latex].

Variance in the Data

Here you can see that the R2 value increased, indicating that more of the variance in the data is explained by the regression equation. Testing the reduction using the F-test produces a very significant decrease in unexplained variability as the residual SS drops from 149.2 to 29.1. The regression line follows the data closely (Fig. 8).

Going a step further to assure that most of the variance is explained by the regression equation, we fit a third order polynomial (Table 5).

[latex]\small y = 12.14 - 5.94x^2 + 0.01x^3[/latex].

[latex]r^2 = \small 0.93[/latex].

[latex]\small  F=\frac{365.6-262.2}{4.28} = \small 0.79[/latex].

[latex]\small \textrm{Critical F} = \textrm{3.29; p-value = 0.01}[/latex].

[latex]\small \textrm{with df 1, 7}; \textrm{not significant, even at p-value = 0.01}[/latex].

Calculating Polynomial Equations

Here, you see that not much more information about the response has been gained. The R² increases little, and very little additional variability is explained in the third-order regression (Fig. 9). The F-value for third-order regression is not significant at even the 0.10 level. The second-order polynomial, therefore, is the best polynomial equation for describing the response. Physically, we are trying to fit a relationship between rainfall to run-off. The negative run-off or infiltration after rain begins makes sense. The X² relationship may be explainable since we are considering a volume of run-off from a depth of rainfall. The equation does fit the data well. Again, this fits only the data gathered. Use of this relationship beyond the scope of this data set would be improper.
&nbsp;

R CODE FUNCTIONS

• anova
• summary
• lm
• install.packages
• library
• cor
• pcor

You are a maize breeder in charge of developing an inbred line for use as the ‘female’ parent in a hybrid cross. Yield of the inbred female parent is a major factor affecting hybrid seed production; a high level of seed production from the hybrid cross leads to more hybrid seed that can be sold. Only 3 lines remain in your breeding program, and your boss wants you to determine 1. Which is the best model to use to analyze the data, and 2. Which of the three lines should be selected for advancement in the breeding program? The three-variable data set relating the yield (per plot) of the 3 inbred lines (evaluated in 3 reps) to the amount of N and level of drought applied to each plot can be found in the file QM-mod13_ex5.csv.

Helpful questions to ask in writing the R code:

1. 1. Should “line” be classified as a numeric or factor variable?
   2. Should “rep” be classified as a numeric or factor variable?
   3. Should “rep” be included in the model?
   4. Should the interaction between N and drought be included?
   5. What higher orders of N and drought, if any, should be included in the model?

   Ex. 5: Non-Linear Multiple Regression Analysis (2)

   Answers:

   1. “Line” should be a factor
   2. “Rep” should be a factor
   3. “rep” should be included in the model (see ANOVA and Regression analysis below)
   4. Yes (see ANOVA and Regression analysis below)
   5. N²,drought² should be included. N³ has a slightly higher R²_Adj value, but it is only ~0.006 better than the model, including both N and drought as second-order variables.

   Students should test models on their own to find the best one.

   Ex. 5: Non-Linear Multiple Regression Analysis (3)

   The correct model is: yield ~ N + drought + line + rep + N*drought + N² + drought² “rep” and “line” should be factors, as the numeric values (1 to 3) are identifiers only and don’t indicate a treatment amount.

   The model including drought and N as a 2nd-order variable is the best. The model that has drought as a 2^nd order polynomial and N as a 3^rd order polynomial technically has a better R²_Adj, however since the difference between the R²_Adj values of the 2 models is incredibly small AND since the coefficient N² is not significant in the model with the higher-order polynomial, we choose the simpler of the 2.

   data$line<-as.factor(data$line)

   data$rep<-as.factor(data$rep)

   summary(1m(data,yield~N+drought+line+rep+N*drought+I(N^2)+I(drought^2)))

   Ex. 5: Non-Linear Multiple Regression Analysis (4)

   R outputs,

   Call:

   lm(formula = yield ~ N + drought + line + rep + N * drought + I(N^2) + I(drought^2), data = data)

   Residuals:

        Min        1Q    Median       3Q       Max

   -2872.09   -550.53     11.45   517.71   2123.26

   Coefficients:

                  Estimate Std. Error t value Pr(<|t|)

   (Intercept)   7.997e+03  9.944e+01  80.422  < 2e-16  ***

   N             6.936e+01  1.504e+00  46.120  < 2e-16  ***

   drought       7.172e+02  2.118e+01  33.864  < 2e-16  ***

   line2         6.026e+02  7.264e+01   8.296 5.31e-16  ***

   line3         8.256e+02  7.264e+01  11.366  < 2e-16  ***

   rep2         -2.080e+02  7.264e+01  -2.864  0.00431  **

   rep3         -8.158e+02  7.264e+01 -11.231  < 2e-16  ***

   I(N^2)       -2.730e-01  6.454e-03 -42.296  < 2e-16  ***

   I(drought^2) -1.920e+02  5.069e+00 -37.869  < 2e-16  ***

   N:drought     4.716e-01  1.587e-01   2.971  0.00306  **

   —

   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1

   Residual standard error:  800.7 on 719 degrees of freedom

   Multiple R-squared:   0.9211, Adjusted R-squared:   0.9201

   F-statistic:  933.1 on 9 and 719 df, p-value: < 2.2e-16

   Ex. 5: Non-Linear Multiple Regression Analysis (5)

   anova(m2)

   Analysis of Variance Table

   Response: yield

                  Df      Sum Sq     Mean Sq     F value     Pr(>F)

   N               1   254135518   254135518    396.4202  < 2.2e-16  ***

   drought         1  2881646910  2881646910   4495.0156  < 2.2e-16  ***

   line            2    88656727    88656727     69.1468  < 2.2e-16  ***

   rep             2    87339382    87339382     68.1194  < 2.2e-16  ***

   I(N^2)          1  1146830086  1146830086   1788.9142  < 2.2e-16  ***

   I(drought^2)    1   919337101   919337101   1434.0531  < 2.2e-16  ***

   N:drought       1     5659813     5659813      8.8286   0.003064  **

   Residuals     719   460933695      641076

   —

   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1

   Line 3 has the best predicted yield and should be advanced.

   Summary

   Multiple Regression

   • Effects of several continuous independent variables (Xs) on continuous dependent variable Y
   • With just X₁ and X₂, get the plane of best fit.

   Multiple Correlation

   • Start with all pairwise simple correlations of Y and Xs.
   • Partial correlation of Y and X₁ holds all other Xs at their average value.
   • Total multiple correlation of Y on Xs squared (R²) is the coefficient of determination

   Calculating Multiple Regression

   • In R, use the REG procedure
   • Get prediction equation Y = a + b₁X₁+…+b_kX_k from Parameter Estimates
   • Get Analysis of Variance for Regression

   Testing Multiple Regression

   • R² gives the proportion of variation accounted for by Regression
   • Overall F-test of all coefficients equal to zero
   • Each regression coefficient tested in Parameter Estimates or Effect Tests

   Problems in Multiple Regression

   • • Ys not independently distributed
   • Unequal variances
   • Either of these is seen in residual plots
   • Multicollinearity from high pairwise correlations of Xs

   Polynomial Regression

   • Add successive powers of X: X, X², X³, …
   • Test for significance with F-test.

   Acknowledgments

   How to cite this chapter: Mowers, R., D. Todey, K. Moore, L. Merrick, and  A. A. Mahama. 2023. Multiple Regression. In W. P. Suza, & K. R. Lamkey (Eds.), Quantitative Methods. Iowa State University Digital Press.