Kue, https://github.com/Automattic/kue
Document/Example:
https://www.linkedin.com/pulse/job-queue-nodejs-adrien-desbiaux
1. Install Redis
2. Install Kue
Usage:
Producer
1. 建立queue instance
var kue = require('kue'),
var jobs = kue.createQueue();
2. 建立job
job = jobs.create('bessel-filter-image', jobArgs).save();
3. 等待job 結束(callback)
job
.on('complete', function(result) { //whatever you want to do })
.on('failed', function() { console.log(job.id); });
Consumer:
1. 建立queue instance?
2. 處理job
3. 處理完呼叫done
console.log(job.data.whatever); // stored in jobArgs
var result = bessel(); // let's imagine applying the filter
done(null, result); // forward the result of your job
});
更新Job Progress....
http://stackoverflow.com/questions/15375126/fetching-the-result-of-a-kue-job-and-pushing-this-to-the-client-over-open-connec
actually this is covered in the documentation - https://github.com/LearnBoost/kue
"Job Events
Job-specific events are fired on the Job instances via Redis pubsub. The following events are currently supported:
failedthe job has failedcompletethe job has completedpromotionthe job (when delayed) is now queuedprogressthe job's progress ranging from 0-100 For example this may look something like the following:var job = jobs.create('video conversion', { title: 'converting loki\'s to avi' , user: 1 , frames: 200 }); job.on('complete', function(){ console.log("Job complete"); }).on('failed', function(){ console.log("Job failed"); }).on('progress', function(progress){ process.stdout.write('\r job #' + job.id + ' ' + progress + '% complete'); });
bare in mind that your job might not be processed immediatly (depends on your queue), so the client can wait some time for a result..
EDIT: as mentioned in the comments, a job doesn't return any results so you should store the result in the database along with the job id and query the database when the job is complete.
in order to keep the connection open, use
res.write and res.end instead of res.json which ends the connection (You'll have to JSON.stringify the data yourself). also, remember that the browser can timeout if this takes too long..Updating Progress
For a "real" example, let's say we need to compile a PDF from numerous slides with node-canvas. Our job may consist of the following data, note that in general you should not store large data in the job it-self, it's better to store references like ids, pulling them in while processing.
queue.create('slideshow pdf', {
title: user.name + "'s slideshow"
, slides: [...] // keys to data stored in redis, mongodb, or some other store
});
We can access this same arbitrary data within a separate process while processing, via the
job.data property. In the example we render each slide one-by-one, updating the job's log and progress.queue.process('slideshow pdf', 5, function(job, done){
var slides = job.data.slides
, len = slides.length;
function next(i) {
var slide = slides[i]; // pretend we did a query on this slide id ;)
job.log('rendering %dx%d slide', slide.width, slide.height);
renderSlide(slide, function(err){
if (err) return done(err);
job.progress(i, len, {nextSlide : i == len ? 'itsdone' : i + 1});
if (i == len) done()
else next(i + 1);
});
}
next(0);
});
