15.2 Linear and Polynomial Regression

Overview

Linear and polynomial regression calculate the best-fit line for one or more XY datasets. In addition, multiple linear regression can be used to study the relationship between several predictor variables and a response variable.

Tool Name Model Menu Entry
Linear Fit y_i=\beta _0+\beta _1x_i+\varepsilon_i Analysis: Fitting: Linear Fit
Fit Linear with X Error Y_i=\beta _0+\beta _1(X_i+\delta x_i)+\varepsilon_i Analysis: Fitting: Fit Linear with X Error
Polynomial Fit y_i=\beta _0+\beta _1x_i+\beta_2x_i^2+\beta_3x_i^3+...+\beta_kx_i^k+\varepsilon_i Analysis: Fitting: Polynomial Fit
Multiple Linear Regression y_i=\beta _0+\beta _1x_{1i}+\beta _2x_{2i}+\ldots +\beta _kx_{ki}+\varepsilon_i Analysis: Fitting: Multiple Linear Regression

Parameters are estimated using a weighted least-square method. The weight can be given to dependent variable in fitting to reduce the influence of the high leverage points. After fitting, the model is evaluated using hypothesis tests and plots of residuals.


How To Perform the Fitting

Fitting Control

To perform linear regression on data with X/Y Error
The Errors can exist for both dependent and independent values, Errors of dependent variable Y can be treated as weight in all Fitting Tools above by setting the Y Error column as Y Error in Input Data and designate the method of Error as Weight in Fit Control.
However, if you want to treat errors of independent variable X as weight, the Fit Linear with X Error tool should be used, in which you can set both X and Y Errors in Input Data, which can be treated as weight.
To perform linear/polynomial fit with parameters fixed
Fitting parameters can be fixed in tools above, For example, you can set the Intercept value to 0 by checking on the Fix Intercept in Fit Control dialog and set the Fix Intercept at = 0, which force the fitted line go through the origin point (0,0). Besides, the Model Degree of Freedom will be reduced due to the parameter fixed.
To perform linear/polynomial fit on a graph according to current axis settings
You can perform linear/polynomial fit on a graph according to current axis settings by checking on the Apparent Fit in Fit Control.
Apparent Fit will first transform your raw data into a new data space as specified by the graph axis type, and then fit the curve of the new data.
Here is an example of Finding the slope on the log-log plot by Apparent Fit
To define the x data type for the fitted curve plot
You can defined the X axis scales, fitted curves interval and range in Fitted Curve Plot: X data Types

Advice and Tips

To force fit curve through specified point
Two methods are provided to make fitted curve go through certain points for Linear and Polynomial Regression:
  • To force the fitted curve go through Origin (0,0), you can just fix the intercept to 0 for a linear or polynomial model.
  • To force the fitted curve go through a specific point in raw data, you can set a higher weight for the point.
For further information, please view this page.
To perform multiple linear regression with boundary or constraint
You can define a multiple linear regression function and set Constrain for it in the Nonlinear Curve Fit tool.
Please refer to this page for details.
To identify the outliers in fitting process
When we get the fitted curve, there may be a large difference between a few points and the fitted curve by the model, these points should be identified as Outliers.
In Linear Fit, the outliers can be shown in results table by checking on the Outliers checkbox in the Linear Fit dialog. Or, you can use the Q-Q plot to identify an outlier, which is introduced here. You can also exculde the outliers from an analysis routine.
To Find X from Y or Y from X
The Find X/Y table allows you to obtain either a dependent variable value given an independent variable value, or an independent variable value given a dependent variable value, from the fit you performed on the data.

See Also

Interpreting the Results

See Interpreting Regression Results Chapter for following information

Algorithm

Algorithms (Linear Regression)

Algorithms (Fit Linear with X Error)

Algorithm (Multiple Linear Regression)

Algorithms (Polynomial Regression)

Advanced: Linear fit for nonlinear model

You can get an analytical solution of an equation if the equation has multiple terms with linear parameters.

For example:the polynomial equation：y=a+bx+cx^2+dx^3 \,

or some more general formula y=ax+bx^2+ce^x \,

Instead of using the Nonlinear Curve Fit tool to get an iterative solution, you can treat the linear terms as different independent variables (such as x_1, x_2, x_3 \,, etc.) and use the Multiple Linear Regression or Ploynomial Fit to perform regression with multiple linear terms and obtain an analytical solution. Here is a case study of How to transfer a reciprocal function into a equation with linear parameters.

However, due to different contributions to residual sum of squares (RSS) for points in two models, the estimate results for parameters is different between nonlinear fit directly and transformed fit with linear way. Please view this page for details.

Topics covered in this section:

  • The Linear Regression Dialog Box
  • The Linear Fit with X Error Dialog (Pro Only)
  • The Polynomial Regression Dialog Box
  • The Multiple Linear Regression Dialog Box
  • Algorithms (Linear Regression)
  • Algorithms (Fit Linear with X Error)
  • Algorithm (Multiple Linear Regression)
  • Algorithms (Polynomial Regression)
    		•
    English | Deutsch | 日本語