jdinan / xpmem-old Goto Github PK

What steps will reproduce the problem?
1. Looks like several functions are not exported in stock RHEL6.0 kernel:
per_cpu__mmu_gathers
free_pages_and_swap_cache
flush_tlb_mm
huge_pte_offset
__put_task_struct
zap_page_range

2. set_cpus_allowed() is not available because CONFIG_CPUMASK_OFFSTACK is 
defined in stock kernel

3. libxpemem.c fails to compile because <stropts.h> is missing. But it is not 
really needed to compile the library.


What version of the product are you using? On what operating system?
parent: 0:b81e55620a68 tip
 Initial commit to google code repository.
branch: default
commit: 5 modified, 99 unknown
update: (current)

OS: RHEL 6.0

Please provide any additional information below.

I build all my autotools packages in ./build so that it is easy to work with 
multiple compiler environments, etc.  Unfortunately, I cannot do this with 
xpmem.  

This is a low priority issue.

What steps will reproduce the problem?

jeff@megatron:/software/xpmem/xpmem> mkdir build && cd build && ../configure 
--with-default-prefix=/software/xpmem 
--with-module=/software/xpmem/xpmem.module && make

What is the expected output? What do you see instead?

make  all-recursive
make[1]: Entering directory `/software/xpmem/xpmem/build'
Making all in include
make[2]: Entering directory `/software/xpmem/xpmem/build/include'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/software/xpmem/xpmem/build/include'
Making all in kernel
make[2]: Entering directory `/software/xpmem/xpmem/build/kernel'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/software/xpmem/xpmem/build/kernel'
Making all in lib
make[2]: Entering directory `/software/xpmem/xpmem/build/lib'
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. 
-I../../lib -I..  -I../../include   -g -O2 -MT libxpmem.lo -MD -MP -MF 
.deps/libxpmem.Tpo -c -o libxpmem.lo ../../lib/libxpmem.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../lib -I.. -I../../include -g 
-O2 -MT libxpmem.lo -MD -MP -MF .deps/libxpmem.Tpo -c ../../lib/libxpmem.c  
-fPIC -DPIC -o .libs/libxpmem.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../lib -I.. -I../../include -g 
-O2 -MT libxpmem.lo -MD -MP -MF .deps/libxpmem.Tpo -c ../../lib/libxpmem.c -o 
libxpmem.o >/dev/null 2>&1
mv -f .deps/libxpmem.Tpo .deps/libxpmem.Plo
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2   -o libxpmem.la -rpath 
/usr/local/lib libxpmem.lo  
libtool: link: gcc -shared  .libs/libxpmem.o      -Wl,-soname -Wl,libxpmem.so.0 
-o .libs/libxpmem.so.0.0.0
libtool: link: (cd ".libs" && rm -f "libxpmem.so.0" && ln -s 
"libxpmem.so.0.0.0" "libxpmem.so.0")
libtool: link: (cd ".libs" && rm -f "libxpmem.so" && ln -s "libxpmem.so.0.0.0" 
"libxpmem.so")
libtool: link: ar cru .libs/libxpmem.a  libxpmem.o
libtool: link: ranlib .libs/libxpmem.a
libtool: link: ( cd ".libs" && rm -f "libxpmem.la" && ln -s "../libxpmem.la" 
"libxpmem.la" )
make[2]: Leaving directory `/software/xpmem/xpmem/build/lib'
Making all in test
make[2]: Entering directory `/software/xpmem/xpmem/build/test'
gcc -DHAVE_CONFIG_H -I. -I../../test -I..  -Iinclude -I../../include   -g -O2 
-MT xpmem_proc1.o -MD -MP -MF .deps/xpmem_proc1.Tpo -c -o xpmem_proc1.o 
../../test/xpmem_proc1.c
../../test/xpmem_proc1.c:19:24: fatal error: xpmem_test.h: No such file or 
directory
compilation terminated.
make[2]: *** [xpmem_proc1.o] Error 1
make[2]: Leaving directory `/software/xpmem/xpmem/build/test'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/software/xpmem/xpmem/build'
make: *** [all] Error 2

What version of the product are you using? 

I grabbed it a few minutes ago (11/27/2011 at ~1:55 PM central time).

On what operating system?

openSUSE 11.3 x86_64

> uname -a
Linux megatron 2.6.34.10-0.4-desktop #1 SMP PREEMPT 2011-10-19 22:16:41 +0200 
x86_64 x86_64 x86_64 GNU/Linux

Please provide any additional information below.

Sometimes xpmem_hugetlb_pte() is entered from xpmem_vaddr_to_pte_offset() 
because pmd_large(*pmd) is true. However the subsequent call to 
is_vm_hugetlb_page() is false.

This bug can be reproduced while running portals4 withe the PPE, and one of the 
tests in a loop:

  #!/bin/bash
  while :
  do
  yod -n 1 ./test_LE_swap  || break;
  done

Usually BUG() is hit within 20 minutes.

Run xpmem-th-server, then on another terminal run xpmem-th-client with
the two parameters given by the server --> oops



$ ./xpmem-th-server
Server parameters: 8589958440 16384


$ ./xpmem-th-client  8589958440 16384



The problem seems to be that xpmem_fault_handler is entered by both
threads at the same time. The first remap_pfn_range succeds, but the
second, which is called with the same parameters fails.

It is impossible to register with IB some memory that has been attached with 
XPMEM.
On that test case, with centos 6.2, ibv_reg_mr fails with errno=EFAULT

- process A does xpmem_make()
- process B does xpmem_get()/xpmem_attach()

If process A dies, then its entry in /proc/xpmem is still there. That's 
because, in XPMEM, process B has a refcount on process' A segment. XPMEM hasn't 
cleaned that ressource.

If process B keeps doing get/attach from other process that die/terminate, 
eventually a process with the same PID of A will come back. Then XPMEM spits 
errors that the proc entry already exists. I don't know whether that has 
another impact.