ifort + openmp + fftw problem 

Author 
Message 
foolcat

Posted: 200843 11:47:16 
Top 
fortran, ifort + openmp + fftw problem
Hi, folks,
After two days of trying, I almost give up.
What I want to do is very simple. I wanna using openmp to do fft in
parallel. But I kept get segfaults.
I have a do loop. In each do loop, I do ONE fft. The idea is to use
openmp to do FFTs in each processor at the same time.
Here is the fortran file.
main.F
program main
implicit none
#include "fftw3.inc"
integer nx,ny
parameter(nx=128,ny=8)
double precision val(nx,ny)
double precision tp(nx), tf(nx)
integer i,j,k
real*8 deltax,rlenx
real*8 pi,amp
integer*8 plan_r2c
integer tid,OMP_GET_THREAD_NUM
pi=4.d0*atan(1.d0)
rlenx=2.d0*pi
deltax=rlenx/dble(nx)
c write(*,*)'rlenx:',rlenx
amp=2.d0
do j=1,ny
do i=1,nx
val(i,j)=amp*dsin(2.d0*pi*2.d0*deltax*dble(i1)/rlenx)
enddo
enddo
CALL OMP_SET_NUM_THREADS(4)
!$omp parallel do private(tid,k,plan_r2c,tp,tf)
c!$omp parallel do default(private)
do k =1,ny
tid = OMP_GET_THREAD_NUM()
write(*,*) 'Thread ID = ', tid, ' Loop # = ', k
do i=1,nx
tp(i)=val(i,k)
tf(i)=dble(0.0)
enddo
call dfftw_plan_dft_r2c_1d(plan_r2c,nx,tp,tf,FFTW_MEASURE)
call dfftw_execute(plan_r2c)
call dfftw_destroy_plan(plan_r2c)
write(*,*) 'plan_r2c = ', plan_r2c
enddo
end
And fftw3.inc file
INTEGER FFTW_R2HC
PARAMETER (FFTW_R2HC=0)
INTEGER FFTW_HC2R
PARAMETER (FFTW_HC2R=1)
INTEGER FFTW_DHT
PARAMETER (FFTW_DHT=2)
INTEGER FFTW_REDFT00
PARAMETER (FFTW_REDFT00=3)
INTEGER FFTW_REDFT01
PARAMETER (FFTW_REDFT01=4)
INTEGER FFTW_REDFT10
PARAMETER (FFTW_REDFT10=5)
INTEGER FFTW_REDFT11
PARAMETER (FFTW_REDFT11=6)
INTEGER FFTW_RODFT00
PARAMETER (FFTW_RODFT00=7)
INTEGER FFTW_RODFT01
PARAMETER (FFTW_RODFT01=8)
INTEGER FFTW_RODFT10
PARAMETER (FFTW_RODFT10=9)
INTEGER FFTW_RODFT11
PARAMETER (FFTW_RODFT11=10)
INTEGER FFTW_FORWARD
PARAMETER (FFTW_FORWARD=1)
INTEGER FFTW_BACKWARD
PARAMETER (FFTW_BACKWARD=+1)
INTEGER FFTW_MEASURE
PARAMETER (FFTW_MEASURE=0)
INTEGER FFTW_DESTROY_INPUT
PARAMETER (FFTW_DESTROY_INPUT=1)
INTEGER FFTW_UNALIGNED
PARAMETER (FFTW_UNALIGNED=2)
INTEGER FFTW_CONSERVE_MEMORY
PARAMETER (FFTW_CONSERVE_MEMORY=4)
INTEGER FFTW_EXHAUSTIVE
PARAMETER (FFTW_EXHAUSTIVE=8)
INTEGER FFTW_PRESERVE_INPUT
PARAMETER (FFTW_PRESERVE_INPUT=16)
INTEGER FFTW_PATIENT
PARAMETER (FFTW_PATIENT=32)
INTEGER FFTW_ESTIMATE
PARAMETER (FFTW_ESTIMATE=64)
INTEGER FFTW_TIMELIMIT
PARAMETER (FFTW_TIMELIMIT=1073741824)
INTEGER FFTW_ESTIMATE_PATIENT
PARAMETER (FFTW_ESTIMATE_PATIENT=128)
INTEGER FFTW_BELIEVE_PCOST
PARAMETER (FFTW_BELIEVE_PCOST=256)
INTEGER FFTW_NO_DFT_R2HC
PARAMETER (FFTW_NO_DFT_R2HC=512)
INTEGER FFTW_NO_NONTHREADED
PARAMETER (FFTW_NO_NONTHREADED=1024)
INTEGER FFTW_NO_BUFFERING
PARAMETER (FFTW_NO_BUFFERING=2048)
INTEGER FFTW_NO_INDIRECT_OP
PARAMETER (FFTW_NO_INDIRECT_OP=4096)
INTEGER FFTW_ALLOW_LARGE_GENERIC
PARAMETER (FFTW_ALLOW_LARGE_GENERIC=8192)
INTEGER FFTW_NO_RANK_SPLITS
PARAMETER (FFTW_NO_RANK_SPLITS=16384)
INTEGER FFTW_NO_VRANK_SPLITS
PARAMETER (FFTW_NO_VRANK_SPLITS=32768)
INTEGER FFTW_NO_VRECURSE
PARAMETER (FFTW_NO_VRECURSE=65536)
INTEGER FFTW_NO_SIMD
PARAMETER (FFTW_NO_SIMD=131072)
INTEGER FFTW_NO_SLOW
PARAMETER (FFTW_NO_SLOW=262144)
INTEGER FFTW_NO_FIXED_RADIX_LARGE_N
PARAMETER (FFTW_NO_FIXED_RADIX_LARGE_N=524288)
INTEGER FFTW_ALLOW_PRUNING
PARAMETER (FFTW_ALLOW_PRUNING=1048576)
I compiled using
ifort c g C openmp main.F
ifort o test.x g C openmp main.o L/scratch/liu19/research/
fftw3.1.2/lib lfftw3
The output is something like:
[liu19@head new]$ ./test.x
Thread ID = 0 Loop # = 1
Thread ID = 1 Loop # = 3
Thread ID = 3 Loop # = 7
plan_r2c = 8036992
Thread ID = 0 Loop # = 2
*** glibc detected *** ./test.x: double free or corruption (!prev):
0x00002aaaac002340 ***
======= Backtrace: =========
/lib64/libc.so.6[0x396626e890]
/lib64/libc.so.6(cfree+0x8c)[0x3966271fac]





Steven G. Johnson

Posted: 200843 13:30:00 
Top 
fortran >> ifort + openmp + fftw problem
On Apr 2, 11:47 pm, foolcat <xiaofengli...@gmail.com> wrote:
> Hi, folks,
>
> After two days of trying, I almost give up.
>
> What I want to do is very simple. I wanna using openmp to do fft in
> parallel. But I kept get segfaults.
See the FFTW manual's section on threadsafety: http://www.fftw.org/doc/Threadsafety.html
In particular, plan creation is not threadsafe. You need to create
the plan(s) once and then only do fftw_execute in parallel.
Besides, if you care about performance so much that you are going to
multiple threads, you probably shouldn't be recreating the plan for
each transform, since they are all of the same size. You can reuse
the same plan for different arrays by using dfftw_execute_dft.
However, since Fortran provides no way (that I know of) to allocate 16
byte aligned memory (necessary for FFTW to exploit SIMD instructions),
you need to pass FFTW_UNALIGNED in the flags when creating the plan,
as explained in the FFTW manual.
Nor do you really want the completely unnecessary overhead of copying
the input to a temporary array. Nor do you need to initialize the
output array, since FFTW overwrites it anyway. You should just
transform with the column of your matrix as input to FFTW.
Even better would be to get rid of the loop entirely. (a) use FFTW's
advanced interface to do the multiple FFTs of the columns of your
matrix in a single call. (b) compile FFTW with enableopenmp, and
then use dfftw_plan_with_nthreads to tell it to do the multiple FFTs
in parallel.
Regards,
Steven G. Johnson





foolcat

Posted: 200843 23:27:00 
Top 
fortran >> ifort + openmp + fftw problem
On Apr 3, 12:30 am, "Steven G. Johnson" <stev...@alum.mit.edu> wrote:
> On Apr 2, 11:47 pm, foolcat <xiaofengli...@gmail.com> wrote:
>
> > Hi, folks,
>
> > After two days of trying, I almost give up.
>
> > What I want to do is very simple. I wanna using openmp to do fft in
> > parallel. But I kept get segfaults.
>
> See the FFTW manual's section on threadsafety:http://www.fftw.org/doc/Threadsafety.html
>
> In particular, plan creation is not threadsafe. You need to create
> the plan(s) once and then only do fftw_execute in parallel.
>
> Besides, if you care about performance so much that you are going to
> multiple threads, you probably shouldn't be recreating the plan for
> each transform, since they are all of the same size. You can reuse
> the same plan for different arrays by using dfftw_execute_dft.
> However, since Fortran provides no way (that I know of) to allocate 16
> byte aligned memory (necessary for FFTW to exploit SIMD instructions),
> you need to pass FFTW_UNALIGNED in the flags when creating the plan,
> as explained in the FFTW manual.
>
> Nor do you really want the completely unnecessary overhead of copying
> the input to a temporary array. Nor do you need to initialize the
> output array, since FFTW overwrites it anyway. You should just
> transform with the column of your matrix as input to FFTW.
>
> Even better would be to get rid of the loop entirely. (a) use FFTW's
> advanced interface to do the multiple FFTs of the columns of your
> matrix in a single call. (b) compile FFTW with enableopenmp, and
> then use dfftw_plan_with_nthreads to tell it to do the multiple FFTs
> in parallel.
>
> Regards,
> Steven G. Johnson
Thanks. It works until now.





Steve Lionel

Posted: 200844 7:11:00 
Top 
fortran >> ifort + openmp + fftw problem
Steven G. Johnson wrote:
> However, since Fortran provides no way (that I know of) to allocate 16
> byte aligned memory (necessary for FFTW to exploit SIMD instructions),
> you need to pass FFTW_UNALIGNED in the flags when creating the plan,
> as explained in the FFTW manual.
Standard Fortran doesn't, but ifort, which the OP is using, does.
!DEC$ ATTRIBUTES ALIGN:16 :: varname
For email address, replace "invalid" with "com"
User communities for Intel Software Development Products
http://softwareforums.intel.com/
Intel Fortran Support
http://support.intel.com/support/performancetools/fortran
My Fortran blog
http://www.intel.com/software/drfortran






