Parallel Computing using IPython (1)

We cannot understand the easiest programming skills unless we are able to manipulate them. Parallel computing first seemed to be hard, but once we could write even several lines of codes, it became so easy.

I started 15 engines, more than the times that the function would be called.

So we could speculate there will no delay for calling the functions. The result of above code was:

We see nearly no delay here. Let’s see the serial computing example:

We could expect there should be latency.

Each time, the function was called one second later due to the time.sleep(1).

Several import points:

  • Module importing was included in defining function, because each engine started was a “blank” engine before the codes came in. These engines were different from the “local” one.
  • If we use map_async, the process will not wait until the full execution is finished. In this case, the program will go on and we can let the program do other things. Later, we can use ready() and get() functions to see if the computing is finished and get the results.
  • If we use map_sync, the vice versa is true. This case is useful if we need the results as prerequisite for further analysis. Otherwise, use map_async.
  • We can wrap our programs in functions and call.

One Reply to “Parallel Computing using IPython (1)”

Comments are closed.