Fixed Effects

FixedEffects
EITM2011
Christopher Berry
ChristopherBerry

SimplestExample:DiffinDiff
Themostbasicdesign,thedifferenceindifferences
(DD)method,isalsousedinmanyrandomizedexperimentsin
whichpretreatmentdataarecollected.
hi h
t t
td t
ll t d

Treatment
Control

Pre
Y1
Y3

Post
Y2
Y4

ATE
(Y2 Y1) (Y4 Y3)

Theaveragetreatmenteffect (ATE)isthedifferencein
differences.
differences
Innonexperimentalsetting,reliesonsomesortofexogenous
variationthatalterstreatmentforonegroupmorethan
another

Outcome

Average
Treatment
Effect

Y2
Treatment Group

Y1

Y4
ControlGroup

Y3

Treatment
Happens

Time

Outcome

Y2

Measuredeffect
without pre
withoutpre
measurement

Treatment Group

Y1

Y4
ControlGroup

Y3

Treatment
Happens

Time

Example: Berry & Gersen on Election Timing

Example:Berry&GersenonElectionTiming
Table5:DifferenceinDifferences
Before
After
Treatment
60,517 60,845
Control
62,516 63,433
Difference
1,999 2,588
Differenceindifferences

$589

Note:Treatment denotes di s tri cts tha ts wi tchedto

oncycl eel ecti ons ;Control thos etha tdi dnot.Before
denotes thel a s tyea rofpretrea tments a l a ry,1987;
After i s thea vera gea cros s pos ttrea tmentyea rs ,
1999to2008.Al l va l ues a rea djus tedto2008dol l a rs .

ComparetoAnzia(2011),whosecrosssectionalstudyusesonlythe
After dataandconcludesthattheposttreatmentdifferenceisthe
causaleffectofthetreatment.Effectively,sheisestimatingY2 Y4.
Whatassumptionstoyouneedinordertoclaimthat(Y2 Y4)is
equivalenttoATE?

Remember Why Were

RememberWhyWe
reDoingThis
Doing This
TheFixedEffectsdesignisageneralizationoftheDiffinDiff
approach.Beforegoingintothedetails,letsrememberthe
motivation.
Wewouldliketoestimatetheeffectofthebinarytreatment,D,on
theoutcomeY.Theproblemisthatwedonteverobservethesame
unit under both treatment and control rather we observe one
unitunderbothtreatmentandcontrol,ratherweobserveone
groupofunitsundertreatmentandanotherundercontrol.Using
OLS,wewouldestimate:
Yi = +D
i +
i
Theestimateof isjustthedifferencebetweenthemeanY forD =
1(thetreatmentgroup)andthemeanY forD =0(thecontrol
group)
Y1 = + + 1

Y0 = + 0
Thus,theOLSestimateis
Y 1 Y 0 = + ( 1 0

UsingRegressiontoEstimateTreatment
Effects(withRandomAssignment)
ff
( ih
d
i
)
Ifthetreatmentisrandomlyassigned,then
If
the treatment is randomly assigned then D isuncorrelated
is uncorrelated
with.
D isuncorrelatedwith
i
l t d ith ifandonlyif
if d l if 1 = 0
Y 1 Y 0 = + ( 1 0 ) =

SowhenDi isuncorrelatedwithi thestandardmethod(OLS)

g
givesanunbiasedestimateof,whichistheaverage

g
treatmenteffect
pp
g
,
ButsupposethereisaconcernthatD
i isendogenous,correlated
withi

RegressionwithoutRandomAssignment
(Si l C )
(SimpleCase)
Ifwedonothaverandomization,thereisnoguaranteethatD isuncorrelated
with (D maybeendogenous)
with
may be endogenous)
ThustheOLSestimateisstill

Y 1 Y 0 = + ( 1 0

IfD iscorrelatedwith,then
,
1

HenceY 1 Y 0 doesnotestimate, butsomeotherquantitythatdepends

onthecorrelationofD and.(Wewilldefinethisquantityshortly.)
Takeaway:IfD iscorrelatedwith,thenOLSgivesabiasedestimateof

Rescued by the CIA?

RescuedbytheCIA?
Howtogetcausalestimatesviaregressionwithoutrandomassignment?
SupposetheCIAholdsgivenavectorofobservedcovariates,Xi.
Decomposetherandompartofthepotentialoutcome,i,intoa
linearfunctionofXi andvi:
i = X i + v i
where isavectorofpopulationcoefficients.Since isdefinedbythe
regressionofi onXi theresidualvi and Xi areuncorrelatedby
construction. Moreover, by virtue of the CIA, the residual in the
construction.Moreover,byvirtueoftheCIA,theresidualinthe
regression:
Yi = + D i + X i + v i
isthereforeuncorrelatedwithDi andXi,,and
isthecausaleffectof
interest.
Thekeyassumption(CIA)isthatobservablecharacteristicsXi arethe
onlyreasonwhyi andDi arecorrelated(selectiononobservables;
unconfoundedness) A key question about any regression study is
unconfoundedness).Akeyquestionaboutanyregressionstudyis
whetherthisassumptionisplausible.

Omitted Variables Bias (OVB)

OmittedVariablesBias(OVB)
SupposeweleaveoutXi (forexample,becausewecannot
observe it).
observeit).
Thentheestimateof hasbias:
E ( ) = + XD
whereXD isthevectorofcoefficientsfromtheauxiliary
regressionsoftheelementsofXi onDi.
The bias is proportional to the correlation between X andD
ThebiasisproportionaltothecorrelationbetweenX
and D
(XD) andtotheeffectofX onY().
Toparaphrase:Shortequalslongplustheeffectofthe
omitted times the regression of omitted on included
omittedtimestheregressionofomittedonincluded.
Bottomline:ifthereisanomittedvariablecorrelatedwith
bothtreatmentandoutcome, willNOTgivethecausal
estimateweareinterestedin.

What Does OVB Mean?

WhatDoesOVBMean?
TheOVBformulaisoneofthemostimportant
eO
o u a s o e o t e ost po ta t
thingstoknowaboutregression
y
y
y y
IfyouclaimnoOVBforyourstudy,youre
effectivelysayingthattheregressionyouhaveis
theregressionyouwant
Inotherwords,youdependontheCIAfora
causalinterpretationofyourregressionestimates
aka,selectiononobservables,unconfoundedness
k
l ti
b
bl
f
d d

Ifyoudonttrulybelievetheassumption,then
you shouldntttrulybelieveyourregression
youshouldn
truly believe your regression

How Does Panel Data Help Us?

HowDoesPanelDataHelpUs?
Supposewehavemultipleobservationsonthesame
units(e.g.,individuals,states)overtime.Thenwecan
( g,
,
)
writethemotivatingequationas:
Yit =D
= Dit +X
+ Xit +Z
+ Zi +
+ it
wherei designatesunitsandt
g
designatestimeperiods.
g
p
Asbefore,Dit istreatmentstatus,whichcannowvary
withinunitsovertime.
Xit isavectorofcovariatesthatvaryovertimewithin
is a vector of covariates that vary over time within
units
Zi isavectorofcovariatesthatvaryacrossunitsbutnot
overtime;it istheerrorterm

FixedEffects
Nowtherearesomeattributesoftheunitsthatdontchangeover
h
b
f h
h d h
time,Zi.TheorytellsusthattheZisshouldbeinthemodel
(excludingthemleadstoOVB).
Butsupposewecannotobserve/measurealloftherelevantZ
But suppose we cannot observe/measure all of the relevant Zis.We
s We
canstillremovetheZis(observableandunobservable)by
estimatingthefollowingequation,whichremovesunitlevel
averagesfrombothsides:
Y it Y i = (D it D i ) + ( X

it

) + (Z

Z i ) + it i

NotethattheZisdropoutoftheequation becausetheydonotvary
overtime.
i
Importantly,
l notonlydoalltheZ
l d ll h isthatourtheorytells
h
h
ll
ustobeconcernedaboutdropout,butinfactanyZi thatwecould
everimagine,whetherobservableorunobsevable,dropsouttoo.
Notethatdeviatingfromunitlevelaveragesisequivalentto
Note that deviating from unitlevel averages is equivalent to
estimatingafixedeffect,ordummyvariable,forearchindividual
unit,whichwedenotei.
Y it = i + D it + X it + it
Whyarethesetwoequationsequivalent?

SomeDetails

Whyarethetwoapproachesdifferencingfrommeansandestimatingunit
h
h
h
diff
i f
d i
i
i
specificdummiesequivalent?Remembertheregressionanatomyformula.
(MostlyHarmless, sec.3.1.2and5.1)

Anysetofregressioncoefficientscanbeestimatedin2steps.Togetthe
multivariatecoefficientononesetofvariables,firstregressthemallontheother
includedvariables.ThenregressYontheresidualsfromthefirststep.The
residualsfromaregressiononafullsetofunitleveldummiesinapanelare
deviationsfromunitlevelmeans.

Conceptually,itisimportanttonotethatthetermfixed
C
ll i i i
h h
fi d effectdoesnot
ff d
actuallymeanthati isbeingtreatedasnonrandom;itmeansthati is
allowedtobearbitrarilycorrelatedwithDit andXit.(Forthisreason,
Wooldridgeadvocatesusing,ci,todenotethefixedeffectratherthanaGreek
l tt
letter,suchas.Hesprobablyright,butIfollowMostlyHarmlessinusing,
h
H
b bl i ht b t I f ll M tl H
l
i
i
asthatisourtextfortheweek.SeeWooldridge,sec.10.2)
Arelatedapproachtofixedeffectsiffirstdifferencing.AgaintheZiswilldrop
y
y
y
periods,FEandFD
p
outbecausetheydontvaryovertime.Withonlytwotime
areequivalent.Withmorethantwoperiodstheywillbedifferentand
estimatesubstantivelydifferentrelationships.Usetheregressionanatomy
forumlatothinkabouthowtheyaredifferent.TheFDmodelis:

Y it Y i ( t 1 ) = (D it D i ( t 1 ) ) + (X

it

i ( t 1)

)+

it

i ( t 1)

ThinkingaboutTiming

Source:Nichols2007

Interpretation
Conditioningoni,thefixedeffects,controlsforall
unitspecific
unit
specificfactors
factorswhether
whetherobservableor
observable or
unobservablethatareconstantovertime.
Thuswehaveremovedapotentiallylargesourceof
OVB The only omitted variables that threaten a FE
OVB.TheonlyomittedvariablesthatthreatenaFE
designarethosethatchangewithinunitsovertime.
Wecanruleoutall timeinvariantunitlevelfactorsasa
sourceofOVBeventhoughwemaynoteverbeableto
f OVB
h
h
b bl
observeormeasurethem.
Wenowinterprettheestimatedeffect,,astheeffect
p
, ,
ofawithinunitchangeintreatment.Forthisreason,
theFEestimatorisalsocalledthewithin estimator.
In effect each unit serves as its own control group
Ineffect,eachunitservesasitsowncontrolgroup.

Assumptions
AversionoftheCIA,specifiedinchangesratherthanlevels
Alltheselectiononunobservables isduetotimeinvariant
factors

Putdifferently,notimevarying OVB,ornotimevaryingOV
thatiscorrelatedwithchangesintreatmentandchangesin
outcome.Theomittedvariabledoesntchangeovertime.
Statednontechnically:Exceptforthechangeinpolicy,the
twogroupsshouldnototherwisehavehaddifferent
changesovertime.
Alsoneedafunctionalformassumption:effectofthe
p
treatmentislinearandadditive(needthisfortheZi todrop
out).

Summary of Approach
SummaryofApproach
Useacrosstimeorcohortvariationtocontrolforunobservedbut
fixedomittedvariables
Thisdesigncanbeextendedtotwowayandnwayfixedeffectsjust
astheDDcanbeextendedtothediffindiffindiffetc.
Cannotbeusedtocomparelevelsoftheoutcomebetween
t t
treatmentandcontrolgroups,justchangesortrends
t d
t l
j t h
t d
Eachuniteffectivelyservesasitsowncontrolgroup

IfthevariableyoucareabouttheoreticallyisinZi,i.e.,time
invariant then FE wontthelpyou.Butifwhatyoucareaboutis
invariant,thenFEwon
help you But if what you care about is
timevarying,Dit,thenFEsweepsoutabunchofstuffyoudont
careaboutinordertogiveyouamorecredibleestimateofthe
thingyoudocareabout.
Thekeyidentifyingassumptionisthatthecounterfactualtrendin
treatmentandcontrolgroupsisthesame
Notimevaryingomittedvariables

W
Wealsomustassumethattheeffectofthetreatmentisadditive
l
h h ff
f h
i ddi i
andconstant

Panel vs. Repeated CrossSections

Panelvs.RepeatedCross
Sections

Inmanycases,youcanusethesetechniqueswithrepeatedcrosssections
( p
(repeatedsamplesfromthesamepopulation)insteadofpaneldata
p
p p
)
p
(repeatedobservationsofthesame individuals)
Sincethediffindiffestimatorisdefinedintermsofsamplemeans,itcanbe
computedequallywellusingpanelorrepeatedcrosssectiondata(RCS)

You must have grouped data or group indicators in the data

Youmusthavegroupeddataorgroupindicatorsinthedata
GroupingdatatoobtainapseudopanelisanapplicationofIV

Thetreatmentvariablecanoftenbedefinedastheproportionofthe
groupreceivingthetreatment(whichmaybe100%,asinachangeinstate
law with individuals grouped by state)
lawwithindividualsgroupedbystate)
RCSanalysisrequirestheassumptionthatunobservableindividualeffects
aredrawnfromthesamepopulationdistributionacrossperiodsbefore
andafterthetreatment.Otherwisethereisthepossibilityof
compositionalbias.
lb
Essentially,assumethetruecausalrelationshipistableovertime.This
assumptionisnotrequiredforpanelanalysis.

AdvantagesofRCSarethatattritionandnonresponse
g
p
issuesareless
severeandsamplesizesareusuallybigger.OftenRCSdatasetsgofarther
backintimebecausethissortofdatahasbeencollectedlongerthantrue
paneldata.

Fixedvs.RandomEffects
OneofthedesirablefeaturesoftheFEdesignisthatitallowsforthe
unitspecificeffecttobecorrelatedwiththeXs.Thusitexplicitly
accountsforoneformofendogeneity,thatresultingfromtime
invariant omitted variables
invariantomittedvariables.
Analternativeapproachistouserandomeffects.TheREmodel
assumesthattheunitspecificeffectisuncorrelatedwiththeXs.
Therefore,thereisnoharminignoringit(noOVB)anditjustbecomes
,
g
g (
)
j
partoftheresidual.
WhentheassumptionsoftheREmodelaresatisfied,itwillbemore
efficientthanFE.However,whentheassumptionsdonthold,itwillbe
bi d B
biased.Bycontrast,FEwillbeunbiasedbutinefficient.
t t FE ill b
bi d b t i ffi i t
ThisleadstoaHausmantypetestforREvs.FE.IftheREassumptions
arevalid,theFEandREcoefficientsshouldbethesame.Testthat
hypothesis.
Bottomline:fromacausalinferenceperspective,REisnotparticularly
useful,sinceitrequiresassuming noOVB.
REiswayoverusedinpoliticalscienceandHausman
y
p
testrarelyreported
y p
ThechoiceofREvs.FEisnotamatterofpersonalpreferenceorstyle
DotheHausman test!Inmyownwork,IhaveneverseenacasewhereRE
isnotresoundinglyrejected

FEvs.LaggedDependentVariable
Usingalaggeddependentvariable(LDV)isanalternativetoFE,witha
similaridentificationframework(seeMostlyHarmless,sec.5.3)
LDVassumes,essentially,theOVBarisesnotfromatime
LDV assumes essentially the OVB arises not from a timeinvariant
invariant
unitlevelfactor,butfromtimevaryingpretreatmenttrends.
MostfamousexampleistheAshenfelter dip

Generallyspeaking,youdonotwanttousebothFEandLDVatthe
y p
g, y
sametime.Doingsorequiresdynamicpaneldatamodels(e.g.,
ArellanoBond)whichareverycomplexandrequirestrong
assumptions
Rather,estimateFEandLDVmodelsseparatelytochecktorobustness
Rather estimate FE and LDV models separately to check to robustness
ofyourresults.Youwanttoseesimilarresultsusingdifferent
identifyingassumptions.
FEandLDVhaveauseful
FE and LDV have a useful bracketing
bracketing property
property
IfthetruemodelisLDVbutyouuseFE,estimatesofapositivetreatment
effectwillbetoobig
IfthetruemodelisFEbutyouuseLDV,estimatesofapositivetreatment
effectwillbetoosmall
ff t ill b t
ll
ThereforeyoucanthinkofFEandLDVasboundingthecausaleffectof
interest(withsomeassumptions)

TestableImplicationsofAssumptions
Keyidentifyingassumptionisthattrendswouldbethe
samefortreatmentandcontrolgroupsintheabsenceof
the treatment
thetreatment
Wecanttestthisdirectly,butwecantestfordifferencesin
trendsbeforethetreatmentisgiven
Thistestcanbesimple:graphorsimplycomparemean
Thi t t
b i l
h
i l
outcomesbytreatmentandcontrolovertime,lookingfor
theeffecttostartafterthetreatment,notbefore
Itcanbedonemoreformally,asintheKuziemko
It
b d
f
ll
i th K i k &Werker
&W k
paper,byincludingleadsandlags
Anotherapproachistoaddaunitspecifictimetrendsto
th
themodel.Thetimetrendscanbelinear,ormorecomplex,
d l Th ti
t d
b li
l
aswithGentzkows 4th orderpolynomial
Worryifaddingsuchtrendscausesbigchangesinyourresults

Foranyoftheseapproaches,youneedenoughpre
treatmentdatatoestablishacleartrend

PotentialProblems
Forobviousreasons,youcannotestimatetimeinvariantcovariates
intheFEmodel.Ifthevariablewhoseeffectyoucareaboutdoesnt
changeovertime,FEisnotthedesignforyou.
Note,thislimitationdoesnotjustifytheuseofRE(unlessyouhave
otherreasonsforbelievingthenoOVB assumption).Rather,ittells
youthattheremaynotbeenoughvariationinyourdatatoestimate
y
g
theeffectyoucareabout.Findanotherdesign.

ThekeytoFEisthecommontrendsassumption.Inadditiontothe
sortsoftestsdescribedabove.Thinksubstantivelyaboutthe
problemandaskwhetherotherthingchangesimultaneouslywith
the treatment If so you need to find a way to control for them
thetreatment.Ifso,youneedtofindawaytocontrolforthem.
FEestimatesarenotoriouslysusceptibletoattenuationbiasdueto
measurementerror
Watchformiscodingofwithin
Watch for miscoding of withinunit
unitchanges
changes

Notappropriateincaseswherethecompositionofthetreatment
andcontrolgroupschangesasaresultofthetreatment
Example:generouswelfarebenefitsattractrecipientstoastate
p g
p

Evaluating a DD/FD/FE Paper

EvaluatingaDD/FD/FEPaper

Isthisreallyanaturalexperiment;isthepolicychangeuncorrelatedwith
thechangeinoutcomeotherthanthroughthechangeintreatment?
Are
Arethereotherthingsthatchangeasaresultofthepolicychange?Couldthey
there other things that change as a result of the policy change? Could they
affecttheoutcome?

Inessence,thecontrolgroup(counterfactual)foreachunitisitself
underadifferenttreatmentstatus(e.g.,beforethepolicychange).Isthisa
good counterfactual? No general answer depends on the application
goodcounterfactual?Nogeneralanswer;dependsontheapplication.
Aretherepreexistingtimetrendswithinunitsthatcouldaccountforthe
observedeffectsofthechangeintreatment?
g
y
g
p y g
g
Isthechangeintreatmentinducedbythechangeinpolicybigenoughto
producemeasurable,quantitativelyimportantchangesinoutcomes?
Doestheanalysisidentifyshorttermorlongtermeffects?
Whichdoyouwant,fortheoreticalreasonsorpolicyrelevance?
RelatedtothechoicebetweenFDandFE,anddependsonlengthofthepanel
R l t d t th h i b t
FD d FE d d
d
l th f th
l
andtheoreticalunderstandingofspeedofadjustment

Doestheanalysisaccountforserialcorrelation(orothertypesofnon
independence)intheresiduals?SeeMostlyHarmless,chap.8.

FixedEffects

EXAMPLE PAPER
EXAMPLEPAPER

HowMuchIsaSeatonthe
SecurityCouncilWorth?
Kuziemko andWerker

Overview
Thebigquestionisinthetitle.Thenarrowerquestion
addressedinthepaperis:
Whatisthecausaleffectofsecuritycouncilmembership
g
onUSforeignaid?
Istheevidenceconsistentwithvotebuying?

Whycantwejustregresssecuritycouncilmembership
on US Aid?
onUSAid?
Of15seats,5arereservedforpermanentmembers
Remaining10seatsarereservedforcountriesserving2
year terms elected from regional blocs Countries
yearterms,electedfromregionalblocs.Countries
campaignfornominationbytheirregion.
Neithercategoryofseatisrandomlyassigned.Sowehave
the usual OVB issues
theusualOVBissues.

Empirical Strategy
EmpiricalStrategy
UseFEmodeltostudydifferenceinUSaidduringyears
y
gy
whenacountryisonthecouncilvs.yearswhenthe
countryisoffthecouncil
Interactsecuritycouncilmembershipwithadummyfor
I t
t
it
il
b hi ith d
f
importantyears(lotsofNYtimescoverage).Ifvote
buyingisthemotive,weshouldexpectbiggeraid
y g
p
gg
effectsinyearswhencouncilisinvolvedinimportant
globalissues.
Useaneventtimespecificationtoconfirmthat
Use an event time specification to confirm that
changesinaiddidnotstartbeforejoiningthecouncil
anddonotcontinueafterleavingthecouncil.

ModelSpecification
p
MainModel

WhereXit includestimevaryingcountrylevel covariates,Wrt isaquartic

regionaltimetrend,t andi areyearandcountryFE,respectively.

EventtimeModel

WhereT1istheyearbeforecountryi iselectedtothecouncil, T0isyearof

election,T1T2isthetermofoffice,andT3T4aretwoyearsaftertermends.

Results

EventTimeGraph

Fixed Effects
FixedEffects
EITM2011
Christopher Berry
ChristopherBerry

Concluding Thoughts
ConcludingThoughts
EITM2011
Christopher Berry
ChristopherBerry

WhyWorryaboutResearchDesign?
Withobservationaldata,theassociationbetweenthe
outcomeandtheexplanatoryvariableofinterestislikelyto
be misleading because it partly reflects omitted factors that
bemisleadingbecauseitpartlyreflectsomittedfactorsthat
arerelatedtobothvariables.
Ifthesefactorscouldbemeasuredandheldconstantina
regression,theomittedvariablesbiaswouldbeeliminated.
i
th
itt d i bl bi
ld b li i t d
But,theorytypicallydoesnotspecifyallofthevariables
thatshouldbeheldconstantwhileestimatingarelationship
anditisdifficulttomeasurealltherelevantvariables
d it i diffi lt t
ll th
l
t
i bl
accuratelyeveniftheyarespecified.
Inotherwords,thereisalmostalwayssomeomitted
variablesbias,meaningthatthereisalmostalwayssome
i bl bi
i th t th
i l
t l
doubtaboutthecausalinterpretationofyourregression
Togetplausiblyunbiasedestimates,youneedagood
researchdesign.
hd i

ElementsofResearchDesign
Key Element
KeyElement

Example

Atheorythatleadstotestablehypotheses

Ifvotersdiscriminateagainstwomen,thewomen
whoareelectedmustbebetteronaveragethan
theirmalecounterparts

Anestimand
d
Often acomparativestatic/partialderivative
Inthepotentialoutcomesframework,acausaleffectof
somekind(ATE,ATT,ITT,etc.)

Thedifference
h d ff
inperformancebetweenwomen
f
b
andmen,allelseequal

D t
Data

Districtlevelfederalprojectspending
Di t i t l l f d l
j t
di
Bills sponsoredandcoponsored

Anestimator
Generated froma statisticalmodel
Involves animplicitdefinitionofthecounterfactual,or
an implicit definition of the counterfactual or
controlgroup

District fixedeffectsmodel
f representswithindistrictdifferenceinoutlays
or bills during years with woman legislator vs
orbillsduringyearswithwomanlegislatorvs.
yearswithman

AssumptionsforIdentification
Whatassumptionsarenecessaryforyoutoclaimthat
your estimator isavalidempiricalestimateofthe
yourestimator
is a valid empirical estimate of the
estimand?
UsuallysomeversionoftheCIA
Assumptionsshouldalwaysbeexplicitlystated

Outlays/bills arevalidproxiesforlegislator
performance
Nothing
Nothingelsewithin
else within districtischangingwhensex
district is changing when sex
ofrepresentativechanges
Note:weakerthanCIA,becausedistricttime
invariantunobservables maydiffer

Tests/validationofassumptions
/
p
Assumptionsareusuallynotdirectlytestable(orthey
wouldntbeassumptions)
Butoftenanimplicationoftheassumptioncanbetested
Orrobustnesstochangingassumptionscanbeshown

Includelineartimetrends
Shownochange intrendbeforewomanelected
Effectsdontholdforwidows
Effectsholdusingregressiondiscontinuitydesign

GoodDesign PlausibleAssumptions
Assumptionsofsomekindarealwaysnecessaryfor
identificationinobservationalstudies
Theplausibilityoftheresultsdependsontheplausibility
oftheassumptions(e.g.,CIA)
If
Ifyoudon
you donttbelieveyourassumptions,thenyoushouldn
believe your assumptions then you shouldntt
believeyourresults

Yougettopickyourdataandyourmodel,butyoudont
get to pick your assumptions
gettopickyourassumptions
Thedataandmodelimplythenecessaryassumptions

Ifyouareuncomfortablewithyourassumptions,you
shouldconsiderchangingsomeotherelementofyour
researchdesign
Changingdata,model,orevenestimand
g g
,
,
mayallowmore
y
plausibleassumptions

Identifyingassumptionsshouldalwaysbestatedexplicitly

SummaryofDesignsfromThisWeek
Design

KeyIdentifyingAssumption(Stated Informally)

Regression

Conditionalindependence:Conditionalonthecovariatesincluded
in the model the treatment is as good as randomly assigned No
inthemodel,thetreatmentisasgoodasrandomlyassigned.No
omittedvariablebias.

Matching

Seeabove.

IV

EExclusionrestriction:The
l i
t i ti
Th instrumentaffectsthetreatment,buthas
i t
t ff t th t t
t b th
noeffectonthedependentvariableotherwise.

RD

Continuity:Nothing elsechangesdiscontinuouslyatthecutoff.The
units just above and just below are the same but for the treatment
unitsjustaboveandjustbelowarethesamebutforthetreatment.

FixedEffects

Exceptforthechangeintreatment,thetwogroupsshouldnot
otherwisehavehaddifferentchangesovertime.NoOVBfromtime
varyingomitted
y g
variables.

Notheoremcantellyouwhetheranyoftheseassumptionsisplausibleinyourcase.
Theplausibilityoftheassumptionmustbeassessedbasedonyoursubstantive
k
knowledgeoftheproblemyouarestudying.
l d
f th
bl
t d i

WhatToDoIfYoureStuckwithOLS
Supposedatalimitationspreventyoufromhavingagreatresearch
design.Whentheusualtechniquesrequireidentifyingassumptions
(e g CIA) that are not tenable in a particular case ask what can be
(e.g.,CIA)thatarenottenableinaparticularcase,askwhatcanbe
learnedfromthedata(ifanything)withweaker(orno)
assumptions.Kosuketaughttwousefultoolsforthisearlierinthe
week.
Manskis partialidentificationtechniquesfocusonboundsofa
treatmenteffect,ratherthanpointestimates,basedonveryweak
(orno)assumptions.Whatcanwelearnfromthedatawehave
without making assumptions we donttbelieve?
withoutmakingassumptionswedon
believe?
Rosenbaumssensitivityanalysisattemptstoquantifyjusthow
strongtheCIAisinpractice.
The
ThekeyideaistorevisitonimplicationoftheCIA:controllingfor
key idea is to revisit on implication of the CIA: controlling for
covariates,thetreatmentandcontrolgroupsareequallylikelytohave
receivedthetreatment.Calltheratioofprobabilitiesgamma=1
Thenask,howdifferentwouldtheiroddsoftreatmenthavetobein
order to change the conclusions of the study Big gammamore
ordertochangetheconclusionsofthestudy.Biggammamore
threatfromOVB

ConcludingThoughts
Causalinferenceisfundamentallybasedonclearthinking
andsubstantiveknowledgeoftheselectionprocess,not
fancy econometrics
fancyeconometrics
Researchdesignistheartofcombiningdata,statistics,and
assumptions toproducecrediblecausalinference
Alwaysstateyouridentifyingassumptionsexplicitlyandask
Al
t t
id tif i
ti
li itl
d k
whetheryoucoulddefendthemwithastraightfacebefore
askepticalaudience
Concentrateondemonstratingtherobustnessofyour
C
t t
d
t ti th
b t
f
resultstoalternativeidentificationstrategiesratherthan
findingtheonetruemodel
Notallquestionsareanswerablegivenavailabledata.Just
N t ll
ti
bl i
il bl d t J t
acceptthat.
Pleasefillouttheonlinesurveywithasmuchfeedbackas
possible
ibl
Goforthandconquer!

ADifferentView
InthecurrentissueofPoliticalAnalysis,oneofthe
editorsoffersthefollowingviewoffixedeffects:
g
Theunitspecificinterceptsaresimplypropertiesofthe
variouspoliticalandeconomicsystemsthatwe
currentlycannotorhavenotmeasured.InthelargeN
panels used in labor economics, it is likely the case that
panelsusedinlaboreconomics,itislikelythecasethat
theunexplainedpersontopersonvariationis
idiosyncraticandlikelybothnotworthandimpossible
to model But in comparative political economy we
tomodel.Butincomparativepoliticaleconomy,we
oughttobeabletomodelwhynationsdifferandto
maketheunmodeled unitheterogeneitysmallenough
tobeignorable.Tomymind,thisshouldbeourgoal.