NAG Library Function Document

nag_regsn_mult_linear_add_var (g02dec)

 Contents

    1  Purpose
    7  Accuracy

1
Purpose

nag_regsn_mult_linear_add_var (g02dec) adds a new independent variable to a general linear regression model.

2
Specification

#include <nag.h>
#include <nagg02.h>
void  nag_regsn_mult_linear_add_var (Integer n, Integer ip, double q[], Integer tdq, double p[], const double wt[], const double x[], double *rss, double tol, NagError *fail)

3
Description

A linear regression model may be built up by adding new independent variables to an existing model. nag_regsn_mult_linear_add_var (g02dec) updates the QR  decomposition used in the computation of the linear regression model. The QR  decomposition may come from nag_regsn_mult_linear (g02dac) or a previous call to nag_regsn_mult_linear_add_var (g02dec). The general linear regression model is defined by:
y = X β + ε  
where y  is a vector of n  observations on the dependent variable, X  is an n  by p  matrix of the independent variables of column rank k , β  is a vector of length p  of unknown arguments, and ε  is a vector of length n  of unknown random errors such that var ε = V σ 2 , where V  is a known diagonal matrix.
If V=I , the identity matrix, then least squares estimation is used.
If VI , then for a given weight matrix W V -1 , weighted least squares estimation is used.
The least squares estimates, β ^  of the arguments β  minimize y-X β T y-X β  while the weighted least squares estimates minimize y-X β T W y-X β .
The parameter estimates may be found by computing a QR  decomposition of X  (or W 1 2 X  in the weighted case), i.e.,
X = QR * or ​ W 1 2 X = QR *  
where R * = R 0  and R  is a p  by p  upper triangular matrix and Q  is an n  by n  orthogonal matrix. If R  is of full rank, then β ^  is the solution to:
R β ^ = c 1  
where c = QT y  (or QT W 1 2 y ) and c 1  is the first p  elements of c .
If R  is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of R .
To add a new independent variable, x p+1 , R  and c  have to be updated. The matrix Q p+1  is found such that Qp+1T R : QT x p+1  (or Qp+1T R : QT W 1 2 x p+1 ) is upper triangular. The vector c  is then updated by multiplying by Qp+1T .
The new independent variable is tested to see if it is linearly related to the existing independent variables by checking that at least one of the values QT x p+1 i , for i = p + 2 , p + 3 , , n  is nonzero.
The new parameter estimates, β ^ , can then be obtained by a call to nag_regsn_mult_linear_upd_model (g02ddc).
The function can be used with p=0 , in which case R  and c  are initialized.

4
References

Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley
Golub G H and Van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins University Press, Baltimore
Hammarling S (1985) The singular value decomposition in multivariate statistics SIGNUM Newsl. 20(3) 2–25
McCullagh P and Nelder J A (1983) Generalized Linear Models Chapman and Hall
Searle S R (1971) Linear Models Wiley

5
Arguments

1:     n IntegerInput
On entry: the number of observations, n .
Constraint: n1 .
2:     ip IntegerInput
On entry: the number of independent variables already in the model, p .
Constraint: ip0  and ip<n .
3:     q[n×tdq] doubleInput/Output
Note: the i,jth element of the matrix Q is stored in q[i-1×tdq+j-1].
On entry: if ip0 , then q must contain the results of the QR  decomposition for the model with p  arguments as returned by nag_regsn_mult_linear (g02dac) or a previous call to nag_regsn_mult_linear_add_var (g02dec).
If ip=0 , then the first column of q should contain the n  values of the dependent variable, y .
On exit: the results of the QR  decomposition for the model with p+1  arguments: the first column of q contains the updated value of c , the columns 2 to ip+1  are unchanged, the first ip+1  elements of column ip+2  contain the new column of R, while the remaining n - ip - 1  elements contain details of the matrix Q p+1 .
4:     tdq IntegerInput
On entry: the stride separating matrix column elements in the array q.
Constraint: tdq ip + 2 .
5:     p[ip+1] doubleInput/Output
On entry: p contains further details of the QR  decomposition used. The first ip elements of p must contain details of the Householder vector from the QR  decomposition. The first ip elements of array p are provided by nag_regsn_mult_linear (g02dac) or by previous calls to nag_regsn_mult_linear_add_var (g02dec).
On exit: the first ip elements of p are unchanged and the (ip+1)th element contains details of the Householder vector related to the new independent variable.
6:     wt[n] const doubleInput
On entry: optionally, the weights to be used in the weighted regression.
If wt[i-1]=0.0, then the ith observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.
If weights are not provided then wt must be set to NULL and the effective number of observations is n.
Constraint: if wtis notNULL, wt[i-1]=0.0, for i=1,2,,n.
7:     x[n] const doubleInput
On entry: the new independent variable, x .
8:     rss double *Output
On exit: the residual sum of squares for the new fitted model.
Note: this will only be valid if the model is of full rank, see Section 9.
9:     tol doubleInput
On entry: the value of tol is used to decide if the new independent variable is linearly related to independent variables already included in the model. If the new variable is linearly related then c  is not updated. The smaller the value of tol the stricter the criterion for deciding if there is a linear relationship.
Suggested value: tol=0.000001 .
Constraint: tol>0.0 .
10:   fail NagError *Input/Output
The NAG error argument (see Section 3.7 in How to Use the NAG Library and its Documentation).

6
Error Indicators and Warnings

NE_2_INT_ARG_GE
On entry, ip=value  while n=value . These arguments must satisfy ip<n .
NE_2_INT_ARG_LT
On entry, tdq=value  while ip + 2 = value. These arguments must satisfy tdq ip + 2 .
NE_INT_ARG_LT
On entry, ip=value.
Constraint: ip0.
On entry, n=value.
Constraint: n1.
NE_NVAR_NOT_IND
The new independent variable is a linear combination of existing variables. The ip+1 th column of q is, therefore, NULL.
NE_REAL_ARG_LE
On entry, tol must not be less than or equal to 0.0: tol=value .
NE_REAL_ARG_LT
On entry, wt[value]  must not be less than 0.0: wt[value] = value.

7
Accuracy

The accuracy is closely related to the accuracy of the QR decomposition.

8
Parallelism and Performance

nag_regsn_mult_linear_add_var (g02dec) is not threaded in any implementation.

9
Further Comments

It should be noted that the residual sum of squares produced by nag_regsn_mult_linear_add_var (g02dec) may not be correct if the model to which the new independent variable is added is not of full rank. In such a case nag_regsn_mult_linear_upd_model (g02ddc) should be used to calculate the residual sum of squares.

10
Example

A dataset consisting of 12 observations is read in. The four independent variables are stored in the array x while the dependent variable is read into the first column of q. If the character variable meanc indicates that a mean should be included in the model, a variable taking the value 1.0 for all observations is set up and fitted. Subsequently, one variable at a time is selected to enter the model as indicated by the input value of indx. After the variable has been added the parameter estimates are calculated by nag_regsn_mult_linear_upd_model (g02ddc) and the results printed. This is repeated until the input value of indx is 0.

10.1
Program Text

Program Text (g02dece.c)

10.2
Program Data

Program Data (g02dece.d)

10.3
Program Results

Program Results (g02dece.r)

© The Numerical Algorithms Group Ltd, Oxford, UK. 2017