模式識別課件prch5part3ding.ppt

,PatternClassificationAllmaterialsintheseslidesweretakenfromPatternClassification(2nded)byR.O.Duda,P.E.HartandD.G.Stork,JohnWiley&Sons,2000withthepermissionoftheauthorsandthepublisher,5.8MinimumSquaredErrorProcedures,Criterionfunctioninvolvesallofthesamples,notjustmisclassifiedonesPreviouslywewereinterestedinmakingalloftheinnerproductspositiveNowtrytomakewherearesomearbitrarilyspecifiedpositiveconstantsThusreplacetheproblemofsolvingasetoflinearinequalitieswithmorestringentbutbetterunderstoodproblemoffindingasolutiontoasetoflinearequations,2,MinimumSquaredErrorandPseudoinverseForallthesampleswewantaweightvectorasothatforsomearbitrarilyspecifiedpositivenumbers.Thematrixnotation:Errorvector:,3,Sum-of-squared-errorcriterionfunction:ThegradientSetittozero,wegetIfisnonsingular,ThedbynmatrixiscallthepseudoinverseofY.Remarks:Foranarbitrarilyfixedb,MSEsolutionmaynotbeaseparatingvector.,4,5,TheWidrow-HofforLMSProcedureNeednotconvergetoaseparatinghyperplaneevenifthereexistone,6,5.9TheHo-KashyapProcedure,Takethecriterionfunctionasafunctionoftwovariablesaandb:Ifthetrainingsamplesarelinearlyseparable,thenthereshouldexistansuchthat:Ifweknewsuchbeforehand.WewouldgettheseparatingvectorusingtheMSEprocedure,7,8,5.12MulticategoryGeneralizations,MulticategorycaseLinearly-separableforMulticategorycasethereexistsasetofweightvectors,9,KeslersConstructionsupposethat,wehave,10,Remark:itallowsustoconvertmanymuylticategoryerror-correctionprocedurestotwo-categoryproceduresforthepurposeofobtainingaconvergenceproofFixed-IncrementRuleLetdenotethekthsamplerequiringcorrection,andsupposethat,theremustbeatleastonejiforwhichthenthefixed-incrementruleforcorrectingis,11,Remark:KeslersconstructioncanalsobeusedtoextendallofresultsforthePerceptronandrelaxationproceduretothemulticategorycase,12,GeneralizationforMSEProcedureconsidermulticategorycaseasasetofctwo-classproblem,