Krzysztof Sopyła – strona domowa Dla studentów, o moich badaniach.

LibSVM with CHI Squared Kernel and OpenMP support

This page describes the modified matlab interface to parallel implementation of LibSVM with CHI^2 Kernel and exponential CHI^2. Parallelization is done with use of  OpenMp (see LibSVM FAQ question – How can I use OpenMP to parallelize LIBSVM on a multicore/shared-memory computer?) . It is possible to pass number of threads through matlab parameters (this solves the problem describe in LibSVM FAQ - How do I use LIBSVM with OpenMP under MATLAB? )

This library contains two the most popular definition of Chi Squared kernels:

1. K(x,y)= 1-2*sum( (xi-yi)^2/(xi+yi)); # normal CHI^2
2. K(x,y)= sum( (xi*yi)/(xi+yi)); # normalized CHI^2
3. K(x,y)= exp( -gamma* Chi_Squared(x,y)); #Exponential CHI^2

Download LibSVM CHI^2

The library was tested on windows 7 x64 and matlab R20011b x64.

Build

There are already compiled .mex files for windows x64 architecture, but if you want build for different platform just write

matlab>cd 'libsvm_chi_openMP/matlab'
matlab>make

Usage

Two new parameter options was added, now parameter „t” can accept values equal 5,6,7

matlab> model = svmtrain(training_label_vector, training_instance_matrix, ‚libsvm_options’);
options:
-s svm_type : set type of SVM (default 0)”
0 — C-SVC”
1 — nu-SVC
2 — one-class SVM
3 — epsilon-SVR
4 — nu-SVR
-t kernel_type : set type of kernel function (default 2)
0 — linear: u’*v
1 — polynomial: (gamma*u’*v + coef0)^degree
2 — radial basis function: exp(-gamma*|u-v|^2)
3 — sigmoid: tanh(gamma*u’*v + coef0)
4 — precomputed kernel (kernel values in training_set_file)
5 — chi-squaree kernel k(x,y)=1-2*sum( (xi-yi)^2/(xi+yi))
6 — norm chi-squaree kernel k(x,y)=sum( xi*yi/(xi+yi))
7 — exponential chi-squaree kernel k(x,y)=exp(-gamma*sum( (xi-yi)^2/(xi+yi)))

-d degree : set degree in kernel function (default 3)
-g gamma : set gamma in kernel function (default 1/num_features)
-r coef0 : set coef0 in kernel function (default 0)
-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
-m cachesize : set cache memory size in MB (default 100)
-e epsilon : set tolerance of termination criterion (default 0.001)
-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)
-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)
-v n: n-fold cross validation mode
-q : quiet mode (no outputs)
-z nrThreads – number of threads passed to OpenMP

Example

See test_libsvm.m file included in project.

Classification with chi^2 kernel, with C=4

tr_path='a1a.train';
tst_path='a1a.train';
 
[trYY trXX]=libsvmread(tr_path);
[tstYY tstXX]=libsvmread(tst_path);
trXXn=trXX;
%l1 - norm
trXXn=bsxfun(@rdivide,trXXn,sum(trXXn,2));
tic;
model = svmtrain(trYY, trXXn,'-c 4 -t 5');
modelTime=toc;
 
tstXXn=tstXX;
tstXXn=bsxfun(@rdivide,tstXXn,sum(tstXXn,2)); %l1 - norm
 
tic
[pred, acc, dec_vals] = svmpredict(tstYY, tstXXn, model);
predTime = toc;
ss=sprintf('libsvm chi^2 acc=%0.5g modeltime=%g predtime=%g \n',acc(1),modelTime, predTime);
disp(ss);

Parallel RBF classification (see LibSVM_parallel.m file included in projcect)

dataFolder = 'D:\Data\';
 
tr_path=[dataFolder 'a9a.small.t'];
tst_path=[dataFolder 'a9a.t'];
 
[trYY trXX]=libsvmread(tr_path);
[tstYY tstXX]=libsvmread(tst_path);
 
maxCore = 8;
maxIter= floor(log2(maxCore))+1;
 
cores=ones(1,maxIter);
timeTest=zeros(1, maxIter);
timeTrain = zeros(1, maxIter);
 
for k=1:maxIter
 
    nrThreads = 2^(k-1);
    % new 'z' - option, determine the number of threads 
    optStr = sprintf('-c 4 -t 2 -g 0.5 -q -z %d',nrThreads);
    tic;
    model = svmtrain(trYY, trXX,optStr);
    modelTime=toc;
 
    tic
    [pred, acc, dec_vals] = svmpredict(tstYY, tstXX, model);
    predTime = toc;
 
    cores(k)=nrThreads;
    timeTrain(k)=modelTime;
    timeTest(k) = predTime;
    ss=sprintf('libsvm RBF acc=%0.5g modeltime=%g predtime=%g \n\n',acc(1),modelTime, predTime);
    disp(ss);
end
 
figure(1);
hold on
plot(cores,timeTrain,'ro:');
plot(cores, timeTest,'bs-');
hold off;