DGESVD slow coparing with intel implementation

[OtacilioNeto]

Dears, when I run this command on a scilab running on freebsd compiled with openblas I get a time of 37 seconds
A1=rand(1000, 1000); tic();[S, v, D]=svd(A1);toc()
ans =

37.276 

The same command on windows with intel library I get 1 second

A1=rand(1000, 1000); tic();[S, v, D]=svd(A1);toc()
ans =

1.382  

Could you please give me a hint why is so different?

[martin-frbg]

One trivial factor is probably that the OpenBLAS build system did not pass the "-O2" compiler option to the fortran compiler until very recently - as #843 showed this has a noticable impact on calculations at least compared to netlib LAPACK (practically identical code but using -O2 optimization level for the default build). You need to either correct the "override FFLAGS" line in Makefile.system or set the FFLAGS environment variable accordingly before invoking "make".
Less trivial could be that you are running OpenBLAS multithreaded and with too many threads to be efficient while MKL may be smart enough to use only one or two threads in this case. In that case running with OPENBLAS_NUM_THREADS=1 or 2 - or even building OpenBLAS without thread support may improve the timing.

[brada4]

Are you with OpenBLAS?

R> x<-matrix(rnorm(1e6),1e3,1e3)
R> system.time(r<-svd(x))
   user  system elapsed 
  2.516   0.824   1.032 

[brada4]

You are not with OpenBLAS by default
https://svnweb.freebsd.org/ports/head/math/scilab/Makefile?view=markup#l48

You have to rebuild scilab to use OpenBLAS, whose port in turn is ""good enough"" i.e likely half-speed LAPACK vs mkl without what @martin-frbg offered.
https://svnweb.freebsd.org/ports/head/math/openblas/Makefile?view=markup#l43

[OtacilioNeto]

I did a rebuild before open this issue. This is my system status:

[ota@nostromo /usr/ports/math/openblas]$ ldd /usr/local/bin/scilab-bin | grep blas
libopenblasp.so.0 => /usr/local/lib/libopenblasp.so.0 (0x808a00000)

[ota@nostromo /usr/ports/math/openblas]$ pkg info scilab
scilab-5.5.2_4
Name : scilab
Version : 5.5.2_4
Installed on : Thu Feb 2 15:28:49 2017 BRT
Origin : math/scilab
Architecture : freebsd:11:x86:64
Prefix : /usr/local
Categories : math java cad
Licenses :
Maintainer : makc@FreeBSD.org
WWW : http://www.scilab.org
Comment : Scientific software package for numerical computations
Options :
ATLAS : off
GUI : on
NETLIB : off
OCAML : on
OPENBLAS : on
TK : on
Shared Libs required:
libcurl.so.4
libgcc_s.so.1
libpcre.so.1
libfftw3.so.3
libarpack.so.2
libopenblasp.so.0
libumfpack.so.1
libstdc++.so.6
libxml2.so.2
libmatio.so.4
libcolamd.so.1
libintl.so.8
libtk86.so.1
libpcreposix.so.0
libcholmod.so.1
libtcl86.so.1
libsuitesparseconfig.so.1
libquadmath.so.0
libhdf5_hl.so.100
libamd.so.1
libhdf5.so.100
libgfortran.so.3
libomp.so.0

[brada4]

What is your CPU? Maybe it is not detected by 0.2.18? (shoud not be 40x slower)

[OtacilioNeto]

i7 3517U

I'm rebuilding openblas without OpenSMP support. I'm getting a feeling that OpenMP is running only two threads.

[brada4]

You can set OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1,1 to disable threading, no need to rebuild.

[OtacilioNeto]

1 thread A1=rand(1000, 1000);tic(); [S, v, D]=svd(A1); toc() -> ans = 46.151
2 thread A1=rand(1000, 1000);tic(); [S, v, D]=svd(A1); toc() -> ans = 45.244
3 thread A1=rand(1000, 1000);tic(); [S, v, D]=svd(A1); toc() -> ans = 44.917
4 thread A1=rand(1000, 1000);tic(); [S, v, D]=svd(A1); toc() -> ans = 45.274
5 thread A1=rand(1000, 1000);tic(); [S, v, D]=svd(A1); toc() -> ans = 47.004

This machine is virtualized under virtualbox and I'm running with 2 virtual CPUS. Maybe this is related?

[brada4]

Yes, virtualbox filters AVX2 and FMA4 always, and often AVX and SSE 4 depending on version and luck.

[OtacilioNeto]

I do not enable AVX2 because my processor do not have this set of instructions, but AVX and SSE 4 are enabled on compilation.

[brada4]

So far you discovered virtualisation overhead,
or impact of fake CPUID by virtual machine,
or simply very recent or very rare CPU that is handled by generic computation kernels.

Also CPUID (from real CPU) is overdue:
$ grep -e Features -e CPU /var/log/dmesg.boot > ~/cpuid.txt

Can you repeat measurement on real CPU?

[OtacilioNeto]

root@nostromo:~ # grep -e Features -e CPU /var/log/dmesg.today
CPU: Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz (2394.63-MHz K8-class CPU)
Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
Features2=0xdc982203<SSE3,PCLMULQDQ,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,XSAVE,OSXSAVE,AVX,RDRAND,HV>
AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
AMD Features2=0x1
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0: on acpi0
cpu1: on acpi0
SMP: AP CPU #1 Launched!

[OtacilioNeto]

I have a no virtualized machine, but it is a little old. I'm rebuilding in there to test.
Is this one.

root@squitch:/home/ota # grep -e Features -e CPU /var/log/dmesg.today
CPU: Intel(R) Core(TM)2 CPU T5300 @ 1.73GHz (1729.04-MHz K8-class CPU)
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0xe39d<SSE3,DTES64,MON,DS_CPL,EST,TM2,SSSE3,CX16,xTPR,PDCM>
AMD Features=0x20100800<SYSCALL,NX,LM>
AMD Features2=0x1
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
SMP: AP CPU #1 Launched!
cpu0: on acpi0
cpu1: on acpi0
coretemp0: on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
p4tcc0: on cpu0
coretemp1: on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
p4tcc1: on cpu1

[OtacilioNeto]

The rebuild in the virtualized machine without OpenMP is done. No luck, same behavior. So, it not OpenMP related.

[brada4]

Since your cpu supports no virtualisation - virtualbox completely emulates cpu seen in virtual machine. The more advanced instruction set you use the slower C emulation gets.
You can close the issue as it is just observing truths of life, not any particular software problem.