The following image on the left depicts what I am explaining above. Blue lines are executed at the same time. Red lines are dependent on their blue parent's completion before they execute, and so on.
I then take the idea a step further and mix batches of parallel tasks with tasks dependent on the batch to complete.
This is a sample gist of personal work I am doing on a Project Management webApp. The goal of this gist is to show how non dependent tasks can be parallelized and dependent tasks can be run after those parallel dependencies are run. eachTask iterator takes a task object and a callback. It uses the Function prototype method call to pass scope to each parallel task.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
_helper = require '_helper' | |
### | |
completeIssue is a great example of parallel scripting. | |
req | |
- project ObjectId | |
- milestone ObjectId | |
- issue ObjectId | |
Since we know all three ObjectIds related to an Issue from the req | |
object, we can parallelize a few of the tasks: findIssueAndClose, socket.emit response, | |
getMilestoneTallyCompletedIssues, getProject info, getIssueAttachments | |
each pushed task has a cb which runs along side the cb of async.each iterator. each task | |
cb adds a particular result to the scope object. When all iterators are complete, the parallel | |
tasks end and execute the last few callback statements: findAccount of the issue owner, | |
and sendIssueCompletedMail | |
### | |
completeIssue = (req)-> | |
async = require 'async' | |
# Add other necessary references to this object | |
# Scope dependencies are hidden behind _helper classes | |
# to simplify this example | |
scope = | |
req : req | |
log = console.log | |
parallelTasks = [] | |
parallelTasks.push | |
fn : _helper.findIssueAndClose | |
opts : { _id: req.issue } | |
cb : (err, issue)-> | |
if err | |
log err | |
scope.issue = issue | |
parallelTasks.push | |
fn : _helper.getMilestoneTallyCompletedIssues | |
opts : {_id: req.milestone} | |
cb : (err, milestone)-> | |
if err | |
log err | |
scope.milestone = milestone | |
parallelTasks.push | |
fn : _helper.getProject | |
opts : { _id: req.project } | |
cb : (err, project)-> | |
if err | |
log err | |
scope.project = project | |
parallelTasks.push | |
fn : _helper.getIssueAttachments | |
opts : { issue: req.issue } | |
cb : (err, mailAttachments)-> | |
if err | |
log err | |
scope.mailAttachments = mailAttachments | |
parallelTasks.push | |
fn : _helper.getComments | |
opts : { issue: req.issue } | |
cb : (err, comments)-> | |
if err | |
log err | |
scope.comments = comments | |
eachTask = (task, cb)-> | |
task.fn.call scope, task.opts, (err, res)-> | |
task.cb(err, res) | |
cb() | |
async.each parallelTasks, eachTask, ()-> | |
_helper.findAccount.call scope, { _id: scope.issue.owner }, (err, account)-> | |
if err | |
log err | |
scope.account = account | |
if scope.project && scope.milestone && scope.issue | |
_helper.sendIssueCompletedMail.call scope, (err)-> | |
if err | |
log err | |
return @ | |
module.exports = completeIssue |