Lecture Advanced Econometrics (Part II) - Chapter 2: Hypothesis testing

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

Chapter 2

HYPOTHESIS TESTING

f Z (

θ , )

→

θ arg max ( )



ˆ θ MLE

= ∏ θ ( )

= 1

θ ( )

ln (

f Z

θ , )

θ ln ( ) 

= ∑

= i 1

I. MAXIMUM LIKELIHOOD ESTIMATORS:

• Asymptotic normality:

∂ L ∂ θ

− 1

ˆ θ MLE

∂ L ∂ ∂ θ θ '

  

  

  θ ~ N , -E  

   

= Eθ ( )

for Solve: MLEθˆ

( ×k

∂ L θ θ ∂ ∂ '

  

∂ ∂  L L  θ θ ∂ ∂ 

′   

 = −  E 

  

   

   

∂ L ∂ θ

θ  1  θ  2    kθ 

     

∂  L   ∂ θ   1 ∂  L   θ ∂=   2      ∂ L    ∂ θ  k



∂ L θ θ ∂ ∂ 1 2 ∂ L 2 θ ∂ 2

∂ L ′ θ θ ∂ ∂



 2

∂ L θ θ ∂ ∂ 1 k ∂ L θ θ ∂ ∂ 2 k  2



∂ ∂ L L θ θ θ θ ∂ ∂ ∂ ∂ k k 2

∂ L 2 θ ∂ k

 ∂ L  2 θ ∂ 1  ∂ L  = ∂ ∂ θ θ  2 1     2  

          

θ vector

(

Y × )1 n

(

+ εβ X × )1 n × × )1 )( kkn

University of New England

Nam T. Hoang UNE Business School

• For the linear model:

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

=→

N ,0(~

ˆ β+ 2I

)

−=

−

2 σ

−

− XY (

′ ()

− XY

β )

2 σβ ( ,

)

2ln

n 2

1 2 σ 2

= )0(

′

−

′ + X Y X X

(

β )

L 1 = − 2 β σ

= −

− Y X (

′ ) (

− Y X

β )

= )0(

∂ ∂  ∂ L ∂ 2 σ

n 1 + 2 σ σ 2

− 1

e 1 e

2  ne

     

     

− 1

(

X X '

)

−

4 σ 2

∂ L ′ ∂ ∂ θ θ

  

  

2  σ    

    

=)(θ

= ( ) ' YX → = = ()ˆ ′ β )ˆ β − ( XY − XY ˆ  β   2 σ ˆ   ' XX 1 n ' ee n

• We consider maximum likelihood estimator θ & the hypothesis:

II. WALD TEST

• Let θˆ be the vector of parameter estimator obtained without restrictions.

θˆ is restriction MLE of θ cH q • We test the hypothesis: =)(:0 θ

−)ˆ(θ

If the restriction is valid, then should be close to zero. We reject the •

hypothesis of this value significantly different from zero.

− 1

−

)ˆ([ θ

]

= cW

)ˆ([ θ c

( ′− ] q Var

) ]

• The Wald statistic is:

cH q =)(:0 θ Under:

• W has chi-squared distribution with degree of freedom equal to the number of

)ˆ( c θ

=− q

2 JX~W [ ]

University of New England

Nam T. Hoang UNE Business School

restrictions (i.e number of equations in )

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

III. LIKELIHOOD RATIO TEST:

cH q =)(:0 θ •

Uθˆ be the maximum likelihood estimator of θ obtained without

 Let

restriction.

 Let Rθˆ be the MLE of θ with restrictions.

ˆ&ˆ L L U

If are the likelihood functions evaluated at these two estimate. 

=λ

ˆ L ˆ L U

 The likelihood ratio:

)1 ≤≤ λ

( 0

=)(θ

RLˆ should be close to ULˆ .

ln2

θ )(:

−→= q

If the restriction is valid then 

cH 0

2 [ ] J

is chi-squared, with degree of freedom equal to the • Under

ln2 λ−=LR

2 JX~ [ ]

number of restrictions imposed.

IV. LAGRANGE MULTIPLIER TEST (OR SCORE TEST):

cH q =)(:0 θ

Let λ be a vector of Lagrange Multipliers, define the Lagrange function:

( ) θ

* L

[ ( ) ′+ θλθ )( c

= − L

′

( ) ∂ θ L ∂ θ

( ) ∂ θ c  ′ ∂ θ

  

− = q

( ) θ

( ) * ∂ θ L ∂ θ ( ) θ λ

    ∂ * L  ∂

University of New England

Nam T. Hoang UNE Business School

The FOC is:

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

)

If the restrictions are valid, then imposing them will not lead to a significant difference

ˆ( ∂ L θ R ˆ ∂ θ R

in the maximized value of the likelihood function. This means is close to 0 or λ

is close to 0. We can test this hypothesis:

− 1

∂

)

ˆ ∂ θ L ( R ˆ ∂ θ R

ˆ θ L ( ) R ˆ ˆ ′ ∂ ∂ θ θ R R

ˆ ∂ θ L ( R ˆ ∂ θ R

   

′    

   

   

   

   

 −  

   

leads to LM test. cH →= q )(:0 θ

Under the null hypothesis LM has a limiting chi-squared distribution with 0 =λH :0

degrees of freedom equal to the number of restrictions.

Graph

V. APPLICATION OF TESTS PROCEDURES TO LINEAR MODELS

(

R )kj ×

RH β 0 : ) ( × kj

qq ( )1 × j

= = ˆ β + Model: X Y × )1 n e )1 × n εβ + X × )1 n × × )1 )( kkn

βˆ is an MEE of β (unrestriction)

− 1

′

ˆ β

−

ˆ β

−

2 σ ˆ

(

) − 1 RXX

( = RW

)

′ [ ) Rq

1. Wald test:

] ( R

2 [ ] J

=2ˆσ

βˆ is an unrestriction estimator of β:

ee′ n

′− ee

)

2 JX~ [ ]

′ een ( RR ′ ee

It can be shown that:

−=

βˆ

(1)

Rβˆ is an estimator subject to the restriction βR .

University of New England

Nam T. Hoang UNE Business School

With

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

2. LR test:

)

λ=

,ˆ( β L R ,ˆ( β L X

)

RH q =β:0

−= = − λ LR L ln2 )ˆ( β ln

[ ln2

] JX~)ˆ( 2 β RL [ ]

−

(ln

′ ee

)

′ ee RR

βˆ

−=

It can be shown:

3. LM test:

0 :H R

qβ =

− 1

It can be shown:

) ′ eX ) ′ eX ) ′ R = = = (3) LM ′ XXXne ( 2 σ ˆ ′ ( XXXen ′ ee RR ′ ′− ( een ee RR ′ ′ ee RR

)

′− ee

−

It can be shown:

′ ( een RR ′ ee

′− ee ′ ee

2  = 

′  een RR  2 

′ ( een RR ′ ee RR

  

 ′ een  RR ′ 2 ee  RR

(2)

From (1), (2), (3) we have:

≥

For the linear models:

The tests are asymptotically equivalent but in general will give different numerical

results in finite samples.

Which test should be used?

The choice among would, LR & LM is typically made on the Basic of ease of

computation. LR require both restrict & unrestrict. Wald require only unrestrict & LM

University of New England

Nam T. Hoang UNE Business School

requires only restrict estimators.

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

VI. HAUSMAN SPECIFICATION TEST:

- Consider a test for endogeneity of a regressor in linear model.

ˆβ estimators. Where

ˆβ uses

- Test based on comparisons between two different estimators are called Hausman Test.

OLSβˆ & SLS

SLS

- Two alternative estimators are:

SLS

ˆ: β ≈ OLS

ˆ β 2

instruments to control for possible endogeneity of the regressor:

− 1

Hausman’s statistic:

− ˆ β − ˆ β − ˆ β ) ) ~)

[ ′ VarCov

]

2 χ [ ] r

SLS

OLS

SLS

OLS

ˆ( β 2 ˆ( β 2 ˆ( β 2

r: the number of endogenous regressors.

~ consider two estimators θˆ and θ

Model general:

plim(

ˆ = θ θ− )

We consider the test situation where:

plim(

ˆ ≠ θ θ− )

H0 :

~ˆ( →− θθ )

,0(

Var

~ˆ( − θθ

))

HA :

Assume under H0 :

− 1

~ˆ( − θθ )

Var

~ˆ( 2 − χθθ [ ] q

1 n

 ~ˆ( − θθ )  

  

The Hausman test statistic:

~ -ˆ( θθVar )

q is rank of

−

ˆ β

−

VarCov

)

ˆ( β

)

ˆ( β 2

SLS

OLS

ˆ( β 2

SLS

OLS

University of New England

Nam T. Hoang UNE Business School

For the linear model:

Advanced Econometrics - Part II

Chapter 2: Hypothesis Testing

VII. POWER AND SIZE OF TESTS:

Size of a test:

Size = Pr[type I error]

= Pr[reject H0 | H0 true]

05.0=α

Common choices: 0.01, 0.05 or 0.1,

Monte-Carlo: set H0 true, → see the probability of reject H0 → size

Power of a test:

Power = Pr [reject H0/H0 wrong]

= 1 - Pr[accept H0/H0 wrong]

= 1 - Pr[Type II error]

University of New England

Nam T. Hoang UNE Business School

Monte-Carlo: set H0 wrong, → see the probability of reject H0 → power size.

Lecture Advanced Econometrics (Part II) - Chapter 2: Hypothesis testing