Jobs: MPICH-VMI

Home > User Info > Jobs > Multisite VMI

The following are general steps for setting up a MPICH-VMI (Virtual Machine Interface) run:

1) Setup softenv for vmi2, edit $HOME/.soft, then resoft or login again.

$ tail -2 $HOME/.soft
+mpich-vmi-intel
@teragrid

2) Compile with vmi2.

$ mpicc -g -o allall_vmi2 allall.c

3) For testing out cross site jobs, you can usually get a single node or two at a couple sites in an interactive PBS job.

# at a couple sites, try:
$ qsub -I -V -lnodes=2:ppn=2,walltime=01:00:00

4) Once the interactive PBS jobs have started, run with vmi2.

# site A
$ mpirun -np 4 -specfile xsite-myrinet-tcp -grid-procs 8 -grid-crm tg-master2.ncsa.teragrid.org -key arnoldg-testkey allall_vmi2 10 7500 500

# site B
$ mpirun -np 4 -specfile xsite-myrinet-tcp -grid-procs 8 -grid-crm tg-master2.ncsa.teragrid.org -key arnoldg-testkey allall_vmi2 10 7500 500

# sample output showing reasonable throughput with 7500k mpi message size
Node 0 Complete...
Host tg-cs03 to Host tg-c596(2): 192.239218MB/sec
Host tg-cs03 to Host tg-c596(3): 196.297191MB/sec
Host tg-cs03 to Host tg-c152(4): 37.764179MB/sec
Host tg-cs03 to Host tg-c152(5): 35.329025MB/sec
Host tg-cs03 to Host tg-c153(6): 37.440001MB/sec
Host tg-cs03 to Host tg-c153(7): 35.335636MB/sec
Node 1 Complete...
Host tg-cs03 to Host tg-c596(2): 198.585210MB/sec
Host tg-cs03 to Host tg-c596(3): 190.505130MB/sec
Host tg-cs03 to Host tg-c152(4): 38.251427MB/sec
Host tg-cs03 to Host tg-c152(5): 40.889033MB/sec
Host tg-cs03 to Host tg-c153(6): 38.246393MB/sec
Host tg-cs03 to Host tg-c153(7): 41.350169MB/sec
...