Heterogeneous Computing
Heterogeneity introduces new bugs in parallel code
Slightly different fl pt can make data dependent branches go different ways when we expect identical behavior.
A “correct” algorithm on a network of identical workstations may fail if a slightly different machine is introduced.
Some easy to fix (compare s < tol on 1 proc and broadcast results)
Some hard to fix (handling denorms; getting the same answer independent of # of procs)