Parallel

Front 1

SISD

Back 1

single instruction, single data

Front 2

MISD

Back 2

multiple instruction, single data

Front 3

SIMD

Back 3

single instruction multiple data

GPGPU processing (general purpose GPU)

Front 4

MIMD

Back 4

multiple instruction, multiple data

Front 5

FIDFOE

Back 5

fetch instruction decode fetch operation execute

Front 6

SPMD

Back 6

single program multiple data

Front 7

things to keep in mind about PP

Back 7

logical organization of program

physical organization of hardware

how tasks are defined

how interactions occur between tasks

Front 8

UMA

Back 8

uniform memory

Front 9

NUMA

Back 9

non-uniform memory

(it says interconnection network)

Front 10

is distributed memory non-uniform or uniform or can be shared memory too

Back 10

NUMA

Front 11

diameter

Back 11

max distance between any 2 nodes

Front 12

connectivity

Back 12

how many cuts to split graph in 2

Front 13

bisection width

Back 13

min cuts to divide graph in half

Front 14

4 problems with cache

Back 14

cache coherence (invalid protocols, update, snoopy, directory based)

cache thrashing (FORTRAN)

algorithms are embarrassingly parallel

false sharing

Front 15

theta

Back 15

argonne national labs

dragonfly topology

bridge between mira and aurora

18 in top 500

8.6 petaflops

Front 16

sequoia

Back 16

lawrence livermore national labs

14 months to test, cards kept failing during thermal test

#1 when released now #4

20 petaflops

Front 17

tianhe2 (milky way 2)

Back 17

china

#2

54 petaflops

Front 18

piz daint

Back 18

sweden

water-cooled by lake

runs weather forecast simulations

#8

15.9 petaflops

Front 19

titan

Back 19

located in oakridge tn

#1 in america

#3 on top 500

27 petaflops

Front 20

pleiades

Back 20

nasa owned

runs kelper's quest to identify expoplanets

#13

7 petaflops

Front 21

lomonosov

Back 21

moscow, russia

most powerful in russia and eastern europe

research for 600 groups

#132

1.7 petaflops

Front 22

Oak forest PACS

Back 22

japans most powerful supercomputer

#6

24 petaflops

Front 23

HazelHen

Back 23

stuttgart, Germany

dragonfly topology

#14

7.4petaflops

Front 24

mira

Back 24

argonne national lab

illinois

5d torus

#9

10 petaflops

Front 25

DGX Saturn V

Back 25

worlds most efficient SC

made by NVidia

deep learning

#28

4.9 petaflops

Front 26

trinity

Back 26

los alamos national labs

new mexico

nuclear weapon simulations

#10

11 petaflops

Front 27

bus

Back 27

d = 1

c = 1

Front 28

complete graph

Back 28

d = 1

c = n - 1

Front 29

star

Back 29

d = 2

c = 1

Front 30

linear array

Back 30

d = n - 1

c = 1

bw = 1

Front 31

ring

Back 31

d = n/2

c = 2

bw = 2

Front 32

2D mesh

Back 32

d = 2 (sqrt(n) - 1)

c = 2

bw = sqrt(n)

Front 33

2d torus

Back 33

d = sqrt(n -1)

c = 4

bw = 2 * sqrt(n)

Front 34

hypercube

Back 34

d = log2 (n)

c = log2 (n)

bw = n / 2

Front 35

tree

Back 35

d = 2log2 (n)

c = log2 (N)

Front 36

fat tree

Back 36

Front 37

cross bar

Back 37

Front 38

omega

Back 38

Front 39

atomic in openmp

Back 39

has to happen at 1 time and can't be interrupted

Front 40

default (none)

Back 40

means nothing is shared

Front 41

openmp how to wait for all threads to finish

Back 41

#pragma omg barrier

Front 42

how to create threads in openmp

Back 42

#pragma omp parallel num_threads(#)

Front 43

things to know about communicating

Back 43

communicate in bulk (minimize start up time)

minimize volume of data

minimize distance that you send things

Front 44

what does single mean in openmp

Back 44

it means it has one thread do the code. whichever thread gets their first, then after finishing it continues

Front 45

wait statement in pthreads

Back 45

pthread_cond_wait(&cv, &mutex);

Front 46

creating a mutex in pthreads

Back 46

pthread_mutex_t nameofmutex = PTHREAD_MUTEX_INITIALIZER;

Front 47

creating a condition variable in pthreads

Back 47

pthread_cond_t cv = PTHREAD_COND_INITIALIZER;

Front 48

create threads in pthreads

Back 48

pthread_create(&threads[#], NULL, fund, #)

Front 49

destroy a mutex or condition in pthreads

Back 49

pthread_cond_destroy(&name);

pthread_mutex_destroy(&name);

Front 50

how to lock in pthreads and unlock

Back 50

pthread_mutex_lock(&nameOfMutex);

pthread_mutex_unlock(&nameOfMutex);

Front 51

how to broadcast in pthreads

Back 51

pthread_cond_broadcast(&cv);

Front 52

store and forward

Back 52

data network in which messages are routed to one or more intermediate stations where they may be stored before being forwarded to their destinations

Front 53

packet routing

Back 53

Packet forwarding is the transit of logically addressed network packets from one network interface to another. Intermediate nodes are typically network hardware devices such as routers, bridges, gateways, firewalls, or switches

Front 54

cut through routing

Back 54

cut-through switching is a method for packet switching systems, wherein the switch starts forwarding a frame (or packet) before the whole frame has been received, normally as soon as the destination address is processed.