Thut ton my h tr vector

(support vector machine-SVM)

Ngi hng dn:CN.V Tin Thnh
Sinhvin :inh th Hng
Lp : k52CA
Ni dung chnh
I.Pht biu bi ton.
1.Trnh by tm tt v phn lp d liu
2.Ti sao li s dng thut ton SVM phn lp d
liu?
II. Thut ton my h tr vector(SVM)
1. nh ngha
2. Mc ch
3. M hnh gii thut
4.Cc cu hi
5.Gii quyt
6.Transductive support vector machine(TSVM)
III.ng dng ca SVM

I.Pht biu bi ton:
1.Trnh by tm tt v phn lp d liu
- Phn lp d liu l mt k thut trong khai
ph d liu c s dng rng ri nht v c
nghin cu m rng hin nay.
- Mc ch: d on nhng nhn phn lp
cho cc b d liu hoc mu mi.
- u vo: Mt tp cc mu d liu hun
luyn,vi mt nhn phn lp cho mi mu d
liu
- u ra:B phn lp da trn tp hun
luyn,hoc nhng nhn phn lp
- K thut phn lp d liu c tin
hnh bao gm 2 bc:
Bc 1:
Xy dng m hnh t tp hun luyn

Bc 2:
S dng m hnh kim tra tnh ng n
ca m hnh v dng n phn lp d liu
mi.



Bc 1. Xy dng m hnh
- Mi b/mu d liu c phn vo mt lp
c xc nh trc.
- Lp ca mt b/mu d liu c xc nh bi
thuc tnh gn nhn lp
- Tp cc b/mu d liu hun luyn-tp hun
luyn c dng xy dng m hnh.
- M hnh c biu din bi cc lut phn
lp,cc cy quyt nh hoc cc cng thc ton
hc.

Bc 2: S dng m hnh
- Phn lp cho nhng i tng mi hoc
cha c phn lp
- nh gi chnh xc ca m hnh
+ Lp bit trc ca mt mu/b d liu em
kim tra c so snh vi kt qu thu c t
m hnh.
+ T l chnh xc bng phn trm cc mu/b
d liu c phn lp ng bi m hnh trong
s cc ln kim tra.


- Cc thut ton phn lp d liu ph
bin:
+ Thut ton cy quyt nh
+ Thut ton SVM
+ Thut ton phn lp Bayes
+ Thut ton phn lp K ngi lng ging gn
nht.
2.Ti sao li s dng thut ton SVM
trong phn lp d liu?
- SVM rt hiu qu gii quyt bi ton d liu
c s chiu ln(nh ca d liu biu din
gene,protein, t bo)
- SVM gii quyt vn overfitting rt tt (d liu
c nhiu v tch di nhm hoc d liu hun
luyn qu t)
- L phng php phn lp nhanh
- C hiu sut tng hp tt v hiu sut tnh ton
cao.





II.Thut ton SVM

1.nh ngha:
- L phng php da trn nn tng ca l
thuyt thng k nn c mt nn tng ton hc
cht ch m bo rng kt qu tm c l
ti u
- L thut ton hc gim st (supervied
learning)c s dng cho phn lp d liu.
- L 1 phng php th nghim,a ra 1 trong
nhng phng php mnh v chnh xc nht
trong s cc thut ton ni ting v phn lp
d liu




- SVM l mt phng php c tnh tng qut cao
nn c th c p dng cho nhiu loi bi ton
nhn dng v phn loi

- tng chnh ca SVM: L chuyn tp mu
t khng gian biu din Rn ca chng sang mt
khng gian Rd c s chiu ln hn. Trong
khng gian Rd, tm mt siu phng ti u
phn hoch tp mu ny da trn phn lp ca
chng, cng c ngha l tm ra min phn b
ca tng lp trong khng gian Rn t xc
nh c phn lp ca 1 mu cn nhn dng




I.Thut ton SVM
2. Mc ch
- L tm ra hm phn lp hiu qu nht
phn bit thnh phn ca cc lp trong vic
hun luyn d liu.

+ V d trong tp d liu phn chia tuyn
tnh , hm phn loi tuyn tnh tng ng vi 1
siu phng f(x) phn chia 2 tp hp.khi hm
ny c xc nh th bt k 1 th hin mi s
c phn lp n gin bng vic xt du ca
hm f( ) nu thuc v tp cc gi dng th
f( )>0 ngc li th thuc tp cc gi tr m.



n
X
n
X
n
X
n
X

I. Thut ton SVM
3. M hnh gii thut
- Tnh cht ni tri ca SVM l ng thi cc tiu
li phn lp v cc i khong cch l gia cc lp
+ Gi s c 1 s im d liu thuc mt
trong hai lp,v mc tiu ca ta l xc nh xem d
liu mi thm vo s thuc lp no.Ta coi mi im
d liu nh mt vector p chiu v chng ta mun
bit l liu c tch c nhng im bng mt
siu phng p-1 chiu hay khng( c gi l phn
loi tuyn tnh)




+ Xem d liu u vo nh 2 tp vector n
chiu,mt SVM s xy dng mt siu phng ring
bit trong khng gian sao cho n ti a ha
bin l gia hai tp d liu.
+ tnh l,hai siu phng song song c xy
dng,mi ci nm mt pha ca siu phng
phn bit v chng c y v pha hai tp d
liu.
+ Mt cch trc quan,mt phn bit tt s thu
c bi siu phng c khong cch ln nht n
cc im ln cn ca hai lp,v l cng ln th sai
s tng qut ha ca b phn lp cng tt hn.










H3 (green) doesn't separate the 2 classes. H1 (blue) does, with a
small margin and H2 (red) with the maximum margin.

Trnh by vi b phn lp nh phn
- Cho tp d liu hc gm n d liu gn
nhn D={(x1, y1), (x1, y1),..., (xn, yn)} vi yi e {-
1,1} l mt s nguyn xc nh lp ca xi. Mi
xi l mt vn bn c biu din di dng mt
vector thc d chiu. B phn lp tuyn tnh (m
hnh phn lp) c xc nh thng qua mt
siu phng c dng:
f(x) = w.x b = 0
trong : w l vector php tuyn ca siu
phng v b ng vai tr l tham s m hnh
- B phn lp nh phn c xc
nh thng qua du ca f(x):




- tm c siu phng phn cch c l cc
i,xy dng cc vector h tr v cc siu
phng song song vi siu phng phn cch
v gn vector h tr nht, l cc hm :
w.x b = 1
w.x b = -1
- Khong cch gia 2 siu phng l do
cn phi cc tiu ha m bo vi
mi i ta c:
w.x b > 1 cho lp th nht
w.x b <-1 cho lp th hai
2
w
w









Hnh 1. Biu din siu phng l cc i cho b phn
lp SVM trn 2 lp
- m bo l lun tm c siu phng phn cch
c l cc i,hm xc nh SVM s lm cc i
hm sau vi vector v hng s b:
Lp =

Trong : + t l s lng mu th
+ vi i =1t l cc s nguyn dng
tha mn o hm Lp theo bin =0. c
gi l nhn t lagrange(im yn nga) v Lp
c gi l hm lagrange
+ vector v hng s b xc nh mt
phng
w
i o
i o
i o
w
1 1
(w. )
t t
i i
i i
yi xi b o o
= =
+

1
2
- Vi cch lm nh trn th ta c: yi(w.xi -b) -1 >0
vn ny c gii quyt bng cc k thut lp
trnh bc 2.vector w biu th s kt hp gia s
hng tuyn tnh v o to d liu:

vi xi l vector h tr,n nm trn l v tha mn
yi(w.xi -b) =1 b =w.xi yi
trn y l mt cch tng i xc nh b.
Trong thc t,n s chnh xc hn khi c tnh
theo cng thc:
b =

1
w . .
t
i
i
yi xi o
=
=

1
1
(w. )
Nsv
i
i
sv
x yi
N
=

4.Cu hi
a.C th m rng m hnh SVM x l trng
hp cho php li tn ti,khi no siu phng tt
nht cho php tn ti mt s li trong o to d
liu?
b.C th m rng m hnh SVM trong tnh trng
hp d liu khng phn chia tuyn tnh?
c.C th thit k thut gii trn cho vic tm siu
phng phn cch c l cc i cho hng ngn
hay hng triu thc th khng?



5.Tr li:

a.C th m rng m hnh SVM x l trng
hp cho php li tn ti,khi no siu phng tt
nht cho php tn ti mt s li trong o to d
liu?
- gi s c 1 s d liu m vt qu gii
hn,nhng im ny th hin s lng li ngay
c khi tn ti mt phng phn cch c l cc i.
- phng php l mm c s dng m
rng bi ton SVM do siu phng s cho php
mt s d liu nhiu tn ti.
- Phng php l mm (softmargin) c s
dng bng cch a thm mt bin ni lng
n o phn lp sai cho d liu x:

thamn cc rng buc

c tnh bng khong cch t lp d liu
b li n hm f(xn).do tng gi tr cc bin
tm c so snh vi gi tr ca hm cc
tiu ban u.



i








Vd minh ha d liu b nhiu
b.C th m rng m hnh SVM trong tnh
trng hp d liu khng phn chia tuyn
tnh?
- SVM ban u l mt thut ton phn lp tuyn
tnh,nh p dng cc hm kernel, thut ton c
th tm ra cc siu phng trong khng gian phi
tuyn c trng bin i.
- M rng tch v hng thng qua hm nh
x cho bin trong khng gian H ln hn
v thm ch c th v hn chiu,theo ng
thc vn c gi ng.
Trong mi ng thc,khi chng ta c tch v
hng th chng ta cng tnh c tch v
hng thng qua php bin i cc vectors
v n c gi l hm kernel

. xi yi
( ) xi |
xi
. xi yi
( ). ( ) xi yi | |

- Hm kernel c s dng xc nh nhiu
quan h u vo khng tuyn tnh.

- i vi hm kernel tuyn tnh ta c th xc nh
c nhiu hm bc hai hoc hm m.

- Trong nhng nm gn y ,nhiu nghin cu
i su vo nghin cu cc kernel khc nhau cho
s phn lp SVM v cho nhiu thng k th
nghim khc.





c.C th thit k thut gii trn cho vic tm
siu phng phn cch c l cc i cho hng
ngn hay hng triu thc th khng?
- Mt trong nhng tr ngi ban u ca SVM l
km hiu qu trong vic tnh ton.
- Tuy nhin vn ny ang c gii quyt
thnh cng.
+ Cch tip cn :chia cc vn ti u ha
ln thnh cc vn nh hn sao cho cui
cng ch gm mt vi bin c la chn cn
thn s ti u ha t hiu qu cao.Qu
trnh ny c lp i lp li cho n khi tt c
cc vn ti u ha c trin khai thnh
cng.



+Mt cch tip cn mi gn y l xem xt cc
vn nghin cu v SVM l tm ra mt bao
ng nh nht ca tp mu d liu.
Vi nhng mu d liu ny,khi nh x vo
khng gian n chiu,i din ca tp gc s c
s dng xy dng nhng bao ng xp x
nh nht ca tp mu d liu.Phng php ny
t hiu qu rt cao.ta c th s dng my core-
vector phn lp c hng triu d liu trong
vi giy.

5.Transductive support vector machine
(TSVM)
- L mt m rng ca support vector machine
chun vi d liu cha gn nhn.
- Mc tiu l tm mt gn nhn ca cc d liu
cha gn nhn, sao cho tn ti mt bin tuyn
tnh c l cc i trn c d liu gn nhn ban
u v d liu cha gn nhn.
-Gi s c L mu c gn nhn v U mu
cha c gn nhn gi l tp U




- Cc tiu ha theo cc bin ( ) hm
mc tiu:

p ng cc k :


trong C v C* l nhng tham s do ngi dng thit
lp,c ngha cn bng ln ca l vi nhng v d b
phn lp sai hoc nhng v d cha c gn nhn.
-Ban u dng b phn lp hc quy np gn nhn cho
cc v d tp U sau nng cp nghim bng cch i
li nhn cho cc v d trong tp U v L b gn sai ri
hun luyn li,n c lp cho n khi v hoc c
2 khng cn nh thua tham s ngi dng C* na.

*
C

*
+
C
V d minh ha TSVM

- cc d liu gn nhn,siu phng c phn
cch l cc i l ng chm chm,
- vi cc d liu cha gn nhn(cc im en)th
siu phng c phn cch l cc i l ng
thng mu en
III.ng dng ca SVM: S dng trong
phn loi v nhn dng
- Trong vic nhn dng ch vit tay ting Vit
+Da trn c s cc thnh phn lin thng
ca nh,phn tp k t ting Vit thnh 3 nhm
(box,dng ch ri,dng ch lin tc) v tch cc
k t c du thnh cc phn ri nhau.
+Sau xy dng my phn lp SVM
nhn dng cho tng phn ch v phn du.
+ Kt qu thc nghim cho thy m hnh
nhn dng c chnh xc tng i cao.
- Trong nhn dng mt ngi :
+Giai on hun luyn: cc nh mu c
vector ha x= { } ri dng phng
php PCA rt trch c trng thnh vector y
={ } ri a vo b hun luyn SVM

+Giai on nhn dng:Mu cn nhn dng
c vector ha v rt trch nh trn sau
a vo b nhn dng SVM xc nh lp cho
mu.
1 900
, , X X
1 100
, , Y Y
- Phn tch vi nh protein subcelluar location
+ nh u vo c tin x l thng qua cc
thut gii kh nhiu, iu chnh t ng cng
sng.
+ Thut gii MRF [9] cng c p ln nhnhm
thc hin segmentation phn bit r phn
nhn v cc phn khc ca nh.
+Bin i m t trong phn 2 c p trn cc
nh u vo vi ti a 8 t l phn r khc nhau
+ Bc phn lp c thc hin vi thut gii
SVM a t l cho ph hp vi tnh cht a phn
gii ca cc h s sau khi bin i


Ti liu tham kho:
- ti liu tham kho chnh: [3] Xindong Wu, Vipin Kumar, J.Ross
Quinlan, Joydeep Ghosh, Qiang Yang, Hiroshi Motoda,
Geoffrey J. McLachlan, Angus Ng, Bing Liu, Philip S. Yu , Zhi-
Hua Zhou, Michael Steinbach, David J. Hand, Dan Steinberg,
Top 10 algorithms in data mining, Knowl Inf Syst (2008)
- [CV95] Corinna Cortes, Vladimir Vapnik (1995). Support-Vector
Networks, Machine Learning, 20(3): 273-297
- http://en.wikipedia.org/wiki/Support_vector_machine
- Nguyn Th Hng Tho, Nguyn Th Thy Linh, Nguyn Thu
Trang, H Quang Thy (2007). Mt gii php hc bn gim
st SVM phn lp trang web ting Vit, Baos cao FAIR 07,
Nha Trang
- http://en.wikipedia.org/wiki/Regression_analysis
- http://en.wikipedia.org/wiki/Supervised_learning
- http://www.slideshare.net/khuongquynh/bai-4-phan-
lop-presentation
- http://en.wikipedia.org/wiki/Statistical_classification


Xin chn thnh cm n!