GMM 估计

1Introduction

These notes povide an introduction to GMM estimation.Their primary purpose is to make the reader familiar enough with GMM to be able to solve problem set assignments.For the more theoretical foundations,properties and extensions of GMM,or to better understand its workings,interested reader should consult any of the standard graduate econometrics textbooks,e.g.,by Greene,Wooldridge, Hayashi,Hamilton,etc.,as well as the original GMM article by Hansen(1982). Available lecture notes for graduate econometrics courses,e.g.by Chamberlain (Ec2140),by Pakes and Porter(Ec2144),also contain very useful reviews of GMM.

Generalized Method of Moments provides asymptotic properties for estima-tors and is general enough to include many other commonly used techniques, like OLS and ML.Having such an umbrella to encompass many of the estimators is very useful,as one doesn’t have to derive each estimator property separately. With such a wide range,it is not surprising to see GMM used extensively,but one should also be careful when it is appropriate to apply.Since GMM deals with asymptotic properties,it works well for large samples,but does not pro-vide an answer when the samply size is small,or what is"large"enough sample size.Also,when applying GMM,one may forgo certain desirable properties, like e…iciency.

2GMM Framework

2.1De…nition of GMM Estimator

Let x i;i=1;:::;n be i.i.d.random draws from the unknown population distri-bution P.For a known function ;the parameter 02 (usually also in the interior of )is known to satisfy the key moment condition:

E[(x i; 0)]=0(1)

This equation provides the core of the GMM estimation.The appropriate function and the parameter 0are usually derived from a theoretical model. Both and 0can be vector valued and not necessarily of the same size.Let the size of be q,and the size of be p.The mean is0only at the true parameter value 0,which is assumed to be unique over some neighborhood around 0: Along with equation(1),one also imposes certain boundary conditions for the 2nd order moment and partial derivative one:

E (x i; 0)0(x i; 0) <1

and

@2j(x; )@ k@ l m(x)

1EC2610Fall2004 for all 2 ;where E[m(x)]<1:Also,de…ne

D E @ (x i; 0)@ 0

and assume,D has rank equal to p,the dimension of :

(Note:the above conditions are su¢cient,and properties of GMM estimators can also be obtained under weaker conditions).

The task of the econometrician lies in obtaining estimate b of 0from the key moment condition.Since there is sample of size n from the population distribution,one may try to obtain the estimate by replacing the population mean with a sample one:

1X i(x i;b )=0(2)

This is a system of q equations with p unknowns.If p=q,we’re"just-identi…ed,"and under some weak conditions,one can obtain a(unique)solution to(2)around the neighborhood of 0.When q>p,then we’re"over-identi…ed," and a solution will not exist for most functions :A natural approach for the latter case might be to try to get the left hand side as close to0as possible,

with"closeness"de…ned over some norm k k

A n

:

k y k A n=y0A 1n y

where A n is q-by-q symmetric,positive de…nite matrix.

Another approach could be to…nd the soltuion to(2),by making some linear combination of j equations equal to0.I.e.for some p-by-q matrix C n,of rank

p,solve for:

C n 1

n X i(x i;b )=0(3)

which will give us p equations with p unknowns.

In fact,both approaches are equivalent and GMM estimation is setup to do exactly that.That is,when p=q,GMM is just-identi…ed and we can usually solve for b exactly.When q>p,we’re in the over-identi…ed case and for some appropriate matrix A n(or C n),GMM estimate b is found by:

b =arg min 2 "1n X i(x i; )#0A 1n"1n X i(x i; )#(4)

(Or equivalently,solving for:equation(3)).The choice of A n will be discussed later,but for now assume A n ! a.s.,where is also symmetric and positive-de…nite.

2.2Asymptotic properties of GMM

Given the above setup,GMM provides two key results:consistency and as-ymptotic normality.Consistency shows that our estiamtor gives us the"right"

2EC2610Fall2004 answer,and asymptotic normality provides us with variance-covariance matrix, which we can use for hypothesis testing.More speci…cally,the estimator b , found via equation(3)satis…es b ! 0a.s.(consistency),and

p n(b 0)d !N(0; )(5) (asymptotic normality),where

= 0

and

=(D0 1D) 1D0 1

(Looking at above properties,one can draw obvious similarities between the GMM estimator,and the Delta Method).

To do hypothesis testing,let A denote the asymptotic distribtuion.Then, equation(5)implies:

b A N( 0;1n )

where

= 0(6) =(D0 1D) 1D0 1 1D(D0 1D) 1 and D are population means de…ned over true parameter values,and is the probability limit of A n::When computing the variance matrix for a given sample,one usually replaces the population mean with the sample mean;the true parameter value with the estimated value,and with A n:

=E (x i; 0)0(x i; 0)

1n X i(x i;b )0(x i;b )

D=E @ (x i; 0)@ 0

1n X i@ (x i; )@ 0j =b

and

A n

The standard errors are obtained from:

SE k=r1n kk

where kk is the k th diagonal entry of :

3EC2610Fall2004

2.3Optimal W eighting Matrices

2.3.1Choice of A n

Having established the properties of GMM,we now turn to the choice of the weighting matrix A n and C n:When GMM is just identi…ed,then one can usually solve for b from equation(2).This is equivalent for…nding a unique minimum point in equation(4)for any positive-de…nite matrix A n:Also,D will be square;

and since it has full rank,will be invertible.Then,the variance matrix will be:

= 0

=(D0 1D) 1D0 1 1D(D0 1D) 1

=D 1 D0 1D0 1 1DD 1 D0 1

=D 1 D0 1

As expected,the choice of A n doesn’t a¤ect the asymptotic distribution for the just-identi…ed case.

For the over-identi…ed case,the choice of the weight matrix will now matter for b :However,since the consistency and asymptotic normality results of GMM do not depend on the choice of A n(as long as it’s symmetric and positive de…nite),we should get our main results again for any choice of A n:In such a case,the most common choice is the identity matrix:

A n=I q

Then, =I q and

=(D0 1D) 1D0 1

=(D0D) 1D0

and the approximate variance-covariance matrix will be:

1 n =

1

n

=

1

n

(D0D) 1D0 D(D0D) 1

(This is the format of GMM variance-covariance matrix Prof.Pakes uses in the IO lecture notes.)

Given that one is free to choose which particular A n to choose,one can try pick the weighting matrix to give GMM other desirable properties as well,like e¢ciency.From equation6,we know that:

=(D0 1D) 1D0 1 1D(D0 1D) 1

4EC2610Fall2004

Since we’re now free to pick ;one can choose it to minimzes the variance:

=arg min

(D0 1D) 1D0 1 1D(D0 1D) 1

=arg min

It is easy to show that the minimum is equal to:

(D0 1D) 1D0 1 1D(D0 1D) 1=(D0 1D) 1 min

which is obtained at

=

The above solution has very intuitive appeal:indexes with larger variances are assigned smaller weights in the estimation.

2.3.22-Step GMM estimation

The above procedure then gives rise to2-step GMM estimation,in the spirit of FGLS.

1.Pick A n=I(equal weighting),and solve for the1st stage GMM estimate:

b 1:Since b 1is consistent,1n P i(x i;b 1)(x i;b 1)0will be consistent estimate of :

2.Pick A n=1

n P i(x i;b 1)(x i;b 1)0;and obtain the2nd stage GMM estimate b 2.The variance matrix1n b 2will then be the smallest.

2.3.3Choice of C n

It should be clear by now how the equations(3)and(4)are related to each, and correspondingly,how A n and C n are related.By actually di¤erentiating the minimization problem in equation(4),we obtain the FOC:

"1X

@ (x i;b )@ 0#0A 1n"1X i(x i;b )#=0(7)

i

If we now de…ne

C n "1n X i@ (x i;b )@ 0#0A 1n

we have equation(7)turning into(3).

One caveat should be pointed out.We speci…ed that equation(3)is linear combination of j(x;b );i.e.C n is a matrix of constants.But in equation(7) C n will in general depend on the solution of the equation:b :This can be easily circumvented if we look at the2nd stage GMM solution,and use the…rst stage 1st stage b 1for C n:That is,if in the second step,we’d normally solve:

"1n X

@ (x i;b 2)@ 0#0A 1n"1n X i(x i;b 2)#=0

i

5EC2610Fall2004 where A n is obtained from the1st stage.We can instead solve for a di¤erent

2nd stage estimate b 20:"1

n X i@ (x i;b 1)@ 0#0A 1n"1n X

i

(x i;b 20)#=0

Since b 1satis…es consistency,and asymptotic normality,b 20will once again be consistent,asymptotically normal,as well as e¢cient among the class of GMM estimators.And now C n is linear when solving for b 20:

3Applications of GMM

3.1Ordinary Least Squares

Since GMM does not impose any restrictions on the functional form of ;it can be easily applied to simple-linear as well as non-linear moment conditions. (It can also be extended to continuous,but non-di¤erentiable functions).The usefulness of GMM is perhaps more evident for non-linear estimations,but one can become more familiar with GMM by drawing similarities with other standard techniques.

For the case of OLS,we have:

y i=x0i +"i

with the zero covariance condition:

E(x i"i)=0

The latter is the key GMM moment condition,and can be rewritten as:

E((x i; ))=0

E(x i(y i x0i ))=0

The sample analog becomes:

1

n X i(x i(y i x0i b ))=0

Since these are k equations with k unknowns,GMM is just-identi…ed with the unique solution of:

b GMM=X i x i x0i! 1X i x i y i!

6

EC2610Fall 2004

which corresponds to the OLS solution.For the variance covariance matrix we need to compute only and D :

=E ( (x i ; ) (x i ; )0)=E (x i "i "i x 0i )

=E ("2i x i x 0

i )

b =1n X i e 2i x i x 0i

where e i =y i x 0i b :

D

=E

@ (x i ; 0)

@ 0

=E ( x i x 0i )

b D

= 1n X i

x i x 0i

Then,the variance-covariance matrix will equal:

1n

b

=1n b D 1b b D 0 1=1n 1n X i x i x 0i ! 1 1n X i e 2i x i x 0

i ! 1n X i x i x 0i

! 1

= X i

x i x 0i ! 1 X i

e 2i x i x 0

i ! X

i

x i x 0i

! 1This is also known as the White formula for heteroskedasticity-consistent

standard errors.

For a simpler OLS example,if we assume homoskedasticity,one can also obtain a simpler version of the variance matrix.With homoskedasticity,

=E ("2i x i x 0

i )

=E (E ("2i j x i )x i x 0

i )=E E ("2i )x i x 0i

=E ("2i )E (x i x 0i )b

= 1n X i e 2i ! 1

n X i

x i x 0i

!

Then,

1n

b

=

1n X i e 2

i

!

X

i

x i x 0i

! 1

which is the variance estimate for the homoskedastic case.

7

EC2610Fall 2004

3.2Instrumental Variables

Suppose again

y i =x 0i +"i

But for the IV estimation,we have

E (w i "i )=0

where w i is not necessarily equal to x i :We only require E (w 0

i x i )=0to be able to invert matrices.The sample analog now becomes:

1n X i

w i (y i x 0i b )=0If w i and x i have the same dimension,then we’re again in the just-identi…ed case,with the unique solution of:

b GMM

= X i

w i x 0i

! 1 X

i

w i y i !=b

IV

If the number of instruments exceeds the number of right-hand side variables,

we’re in the over-identi…ed case.We can go ahead with 2-stage estimation,but a particular choice of the weighting matrix deserves attention.If we set

A n =

1n X i

w i w 0

i

The FOC for GMM becomes:

"1n X i @ (x i ;b )@

0#0A 1n "1n X i (x i ;b )#

=0 1n X i w i x 0i !0 1n X i w i w 0i ! 1 1n X i w i (y i x 0i b )!

=0Let,

b

=

X

i w i w 0i

! 1 X

i

w i x 0i

!

(8)

b

is then the regression coe¢cients of x i on w i :Then we have:

b 0 X

i

w i (y i x 0i b )

!=0(9)

X

i b 0w i (y i x 0i b )

=0

8

EC2610Fall 2004

Note that:b 0w i = w 0i b 0

=b x i and

X

i

b x i x 0i =

X

i

b x i b x 0i

where b x i are the …tted values of x i from (8).Equation (9)then becomes:

X

i

b x i (y i b x 0i b )=0

This is the solution to 2Stage Least Squares (2SLS).The …rst stage is the

regression of the right hand side variables on the instruments;and the second stage is the regression of the dependent variable on the …tted values of the right-hand side variables.Thus,with GMM we’re able to obtain the 2SLS estimates and their correct standard errors.(The usual setting of 2SLS is to regress only the "problematic"right-hand side variables on the instruments,and then use their …tted values.The right-hand side variables,not correlated with the error term,are part of the instruments,and so their …tted values are equal to themselves.We’re then doing the exact same regressions).

3.3Maximum Likelihood

Suppose now we know the family of distributions,p ( ; );where the x i come from

but do not know the true parameter value 0.Maximum Likelihood solution to …nding 0is:

b ML =arg max 2 p (x 1;:::;x n j )

=arg max

2

Y

p (x i j )

The maximum point estimate is invariant to monotonic transformations,and so:

b ML

=arg max 2

log

Y

p (x i j )

=

arg max

2

1X

log p (x i j )The FOC becomes:

1n X @log p (x i j b )

@ 0=0(10)

If we let (x i j )=@log p (x i j )

@ 0

;the score function,then equation (10)can serve as the sample analog to a key moment condition of the form:

E @log p (x i j )@ 0

=0

When doing ML estimation,the above equation will usually hold.If in doubt,you should consult the references.

9