map a procedure over a dataset, using separate processes - Maple Help

Home : Support : Online Help : Toolboxes : Grid Computing : Grid/Map

Grid[Map] - map a procedure over a dataset, using separate processes

 Calling Sequence Map( f, expr ) Map( f, expr, arg1, arg2, ..., argN ) Map[tasksize=s]( f, expr, ... )

Parameters

 f - procedure or function expr - any expression arg1, ..., argN - (optional) further arguments to f s - posint

Description

 • The Map command applies f to the operands or elements of expr in parallel. This is similar to Threads[Map] except that application of f is done in separate processes, rather than separate threads.
 • The Map command applies f to the operands or elements of expr in parallel.  A typical calling sequence is Map(f, [1,2,...,n]) which applies f to each element of the given list to form the new list, [f(1), f(2), ..., f(n)]. Similarly, Map(f,[1,2,...,n],a,b) will return [f(1,a,b), f(2,a,b), ..., f(n,a,b)], using the extra arguments to Map in each call to f. Computation of each f(i) will be evenly distributed across all available nodes.
 • The Map command is similar to the top-level map command, and the Threads[Map] commands. The top-level seq command evaluates the entire sequence serially. The Threads[Map] command evaluates elements of the sequence in parallel on separate threads within the same process. The Grid[Map] command evaluates elements of the sequence in parallel on separate processes, and potentially on different machines.
 • The Map command attempts to initialize remote compute nodes by looking for named values inside of f and assigning those same names in the remote compute nodes. The expr parameter is not searched. If it depends on values that are not fully defined on the remote nodes, use the Grid:-Set command before calling Map to set those values. Similarly, Grid:-Set may be needed to complete the definition of f, in particular, when f is a procedure that calls other locally defined procedures.
 • Map will always try to use all compute nodes. It will typically subdivide the request into many more tasks than there are available nodes. This helps to more evenly distribute the load when the run-time of processing different requests is unevenly distributed.  However, in some situations, Map may not choose the optimal size. In particular, when there are a large number of short-running tasks, doing more work in the remote nodes in each task may be more efficient. In this case, you can specify the task size using the tasksize option. Map will divide the input into tasks that compute at most tasksize elements within a single task.
 • When using hpc or mpi modes, Map is designed to be called on all the nodes of a parallel computation, whereupon the result will only be available on node 0. If the resulting Maple expression is required for further computation on other nodes, it must be sent there explicitly using Send and Receive.
 • Map is a high-level command which uses Send and Receive calls. Therefore, evaluation of f should not involve other Send and Receive calls.
 • Because Map has to transfer portions of expr to each node, it will not be efficient when called with a trivial procedure f.
 • The result of a Map computation will only be available on node 0. If the resulting Maple expression is required for further computation on other nodes, it must be sent there explicitly using Send and Receive.
 • This function is part of the Grid package, so it can be used in the short form Map(..) only after executing the command with(Grid). However, it can always be accessed through the long form of the command by using Grid[Map](..).

Examples

This example shows some simple calling sequences

 > $\mathrm{Grid}:-\mathrm{Map}\left(x→{x}^{2},\left[1,2,3\right]\right)$
 $\left[{1}{,}{4}{,}{9}\right]$ (1)
 > $\mathrm{Grid}:-\mathrm{Map}\left(\left(x,y\right)→{x}^{2}+y,⟨⟨1|2⟩,⟨3|4⟩⟩,4\right)$
 $\left[\begin{array}{rr}{5}& {8}\\ {13}& {20}\end{array}\right]$ (2)
 > $\mathrm{expr}:=\mathrm{randpoly}\left(\left[x,y,z\right],\mathrm{terms}=20\right)$
 ${\mathrm{expr}}{:=}{-}{92}{}{{x}}^{{5}}{+}{6}{}{{x}}^{{4}}{}{y}{+}{74}{}{{x}}^{{4}}{}{z}{+}{72}{}{{x}}^{{3}}{}{y}{}{z}{+}{37}{}{{x}}^{{2}}{}{{y}}^{{2}}{}{z}{-}{23}{}{{x}}^{{2}}{}{y}{}{{z}}^{{2}}{+}{87}{}{{x}}^{{2}}{}{{z}}^{{3}}{+}{44}{}{x}{}{{y}}^{{3}}{}{z}{+}{29}{}{{y}}^{{4}}{}{z}{+}{98}{}{y}{}{{z}}^{{4}}{-}{23}{}{{z}}^{{5}}{-}{40}{}{{x}}^{{2}}{}{{z}}^{{2}}{+}{42}{}{x}{}{{y}}^{{3}}{-}{50}{}{x}{}{y}{}{{z}}^{{2}}{+}{23}{}{{y}}^{{3}}{}{z}{+}{75}{}{y}{}{{z}}^{{3}}{-}{10}{}{{y}}^{{3}}{-}{7}{}{{y}}^{{2}}{}{z}{-}{17}{}{{x}}^{{2}}{-}{75}{}{x}{}{y}$ (3)
 > $\mathrm{Grid}:-\mathrm{Map}\left(\mathrm{abs},\mathrm{expr}\right)$
 ${92}{}{\left|{x}\right|}^{{5}}{+}{6}{}\left|{{x}}^{{4}}{}{y}\right|{+}{74}{}\left|{{x}}^{{4}}{}{z}\right|{+}{72}{}\left|{{x}}^{{3}}{}{y}{}{z}\right|{+}{37}{}\left|{{x}}^{{2}}{}{{y}}^{{2}}{}{z}\right|{+}{23}{}\left|{{x}}^{{2}}{}{y}{}{{z}}^{{2}}\right|{+}{87}{}\left|{{x}}^{{2}}{}{{z}}^{{3}}\right|{+}{44}{}\left|{x}{}{{y}}^{{3}}{}{z}\right|{+}{29}{}\left|{{y}}^{{4}}{}{z}\right|{+}{98}{}\left|{y}{}{{z}}^{{4}}\right|{+}{23}{}{\left|{z}\right|}^{{5}}{+}{40}{}\left|{{x}}^{{2}}{}{{z}}^{{2}}\right|{+}{42}{}\left|{x}{}{{y}}^{{3}}\right|{+}{50}{}\left|{x}{}{y}{}{{z}}^{{2}}\right|{+}{23}{}\left|{{y}}^{{3}}{}{z}\right|{+}{75}{}\left|{y}{}{{z}}^{{3}}\right|{+}{10}{}{\left|{y}\right|}^{{3}}{+}{7}{}\left|{{y}}^{{2}}{}{z}\right|{+}{17}{}{\left|{x}\right|}^{{2}}{+}{75}{}\left|{x}{}{y}\right|$ (4)
 > $\mathrm{Grid}:-\mathrm{Map}\left(x→{x}^{2},\mathrm{table}\left(\left\{1=1,2=2,3=3,\mathrm{red}=4,\mathrm{blue}=y\right\}\right)\right)$
 ${\mathrm{table}}\left(\left[{1}{=}{1}{,}{2}{=}{4}{,}{3}{=}{9}{,}{\mathrm{blue}}{=}{{y}}^{{2}}{,}{\mathrm{red}}{=}{16}\right]\right)$ (5)

In this example we define two procedures such that Map will not be able to automatically find all definitions to send to external nodes.  The remote operation is not fully defined.

 > $\mathrm{myfunc1}:=x→{x}^{2}$
 ${\mathrm{myfunc1}}{:=}{x}{→}{{x}}^{{2}}$ (6)
 > $\mathrm{myfunc2}:=x→\mathrm{myfunc1}\left(x\right)+x$
 ${\mathrm{myfunc2}}{:=}{x}{→}{\mathrm{myfunc1}}{}\left({x}\right){+}{x}$ (7)
 > $\mathrm{Grid}:-\mathrm{Map}\left(\mathrm{myfunc2},\left[1,2,3\right]\right)$
 $\left[{\mathrm{myfunc1}}{}\left({1}\right){+}{1}{,}{\mathrm{myfunc1}}{}\left({2}\right){+}{2}{,}{\mathrm{myfunc1}}{}\left({3}\right){+}{3}\right]$ (8)

The myfunc1 procedure was not defined in the external nodes.  It needs to be set.

 > $\mathrm{Grid}:-\mathrm{Set}\left(\mathrm{myfunc1}\right)$
 > $\mathrm{Grid}:-\mathrm{Map}\left(\mathrm{myfunc2},\left[1,2,3\right]\right)$
 $\left[{2}{,}{6}{,}{12}\right]$ (9)

In this example the work required for the computation at j=1 is much less than the computation at j=9.  The sequence of heavy computations at the end of the list is may require an adjustment to the default tasksize

 > ${\mathrm{time}}_{\mathrm{real}}\left(\mathrm{Grid}:-\mathrm{Map}\left(j→\mathrm{add}\left(i,i=1..{10}^{\mathrm{min}\left(7,j\right)}\right),\left[\mathrm{}\left(1..9\right)\right]\right)\right)$
 ${1.204}$ (10)
 > ${\mathrm{time}}_{\mathrm{real}}\left({\mathrm{Grid}:-\mathrm{Map}}_{'\mathrm{tasksize}'=1}\left(j→\mathrm{add}\left(i,i=1..{10}^{\mathrm{min}\left(7,j\right)}\right),\left[\mathrm{}\left(1..9\right)\right]\right)\right)$
 ${1.266}$ (11)