Presentation I gave to the July 2015 St. Louis Javascript Meetup

1. BEYOND THE CALLBACK:
Yield Control with Javascript Generators
JuLy 16, 2015
Darren Cruse darren.cruse@gmail.com

2. STORY TIME…
(put on your jammies)

3. Once upon a
time…

4. When you were young
• And learning to
program at your
momma’s knee…

5. She said
• Break your big long
sections of code into little
functions that do just one
thing each

6. Let the main routine call the little
ones
• And they will each run
• Until they’re done
• And return to the main
function

7. This is called “Control Flow”
• Can you say “control
flow”?

8. (I knew you could)

9. And then one day…
• (maybe in Colorado)

10. Momma said
• How about we try
something new?

11. Instead of that function you’ve been
used to…
• The one that starts at
the top
• And executes till it’s
done
• And then just returns

12. Let’s have us a function
• That’s starts at the
beginning like
normal

13. • But returns from
the middle
• Not when it’s
done done, but
maybe when it’s
just part way
done

14. • But if we call it again

15. • Let’s have it start up
in the middle again

16. • And then we can have it
do some stuff
• And then maybe return
from the middle again

17. • So when we call it again

18. • It can start up in the
middle again

19. • And then maybe we could
do this three or four times

20. • And maybe throw some
loops in there

21. • And then eventually when
the function is really
“done done”
• it can return
• And be finished
• Like we’re used to with
our other functions

22. So what do you think
About this new kind of function?

23. Sound good?

25. But wait a minute

26. Was your momma maybe kind of
an academic type?

27. Did she also use
words like…
“lazy sequences”
!
“coroutines”
!
“delimited continuations”
!
“callcc”

28. If so…

29. Then yo momma wasn’t on “the
pipe”.

30. Yo momma was talking about

31. Generator Functions!!!

32. (as added by the ecmascript 6 standard)

33. And yo momma deserves credit

34. (as does the es6 committee)

35. For trying to teach
your sorry a** about
alternative means of
control flow…

36. And thereby save you from
the fate of heavyweight
multithreading that
doesn’t scale, and shared
state concurrency with
rampant race conditions

37. Which is the true…

38. Enough silliness

39. In computer science, a generator is a special routine that can be
used to control the iteration behaviour of a loop. In fact, all
generators are iterators.[1] A generator is very similar to a
function that returns an array, in that a generator has
parameters, can be called, and generates a sequence of values.
However, instead of building an array containing all the values
and returning them all at once, a generator yields the values one
at a time, which requires less memory and allows the caller to
get started processing the first few values immediately. In short,
a generator looks like a function but behaves like an iterator.
From wikipedia:
https://en.wikipedia.org/wiki/Generator_(computer_programming)

40. 2.1 Lisp
2.2 CLU
2.3 Icon
2.4 C++
2.5 Perl
2.6 Tcl
2.7 Haskell
2.8 Racket
2.9 PHP
2.10 Ruby
2.11 Java
2.12 C#
2.13 XL
2.14 F#
Also from wikipedia:
I was surprised to see the number of languages with
generators (not all are “native” though)

41. The typical generator example is very often…
An infinite Fibonacci Sequence
function* fibonacci() {
let [prev, curr] = [0, 1];
while (true) {
yield curr;
[prev, curr] = [curr, prev + curr];
}
}
!
var gen = fibonacci();
console.log(gen.next().value); // 1
console.log(gen.next().value); // 1
console.log(gen.next().value); // 2
console.log(gen.next().value); // 3
console.log(gen.next().value); // 5
console.log(gen.next().value); // 8
again from: https://en.wikipedia.org/wiki/Generator_(computer_programming)

42. “yield” indicates to
yield the value “curr”
to the caller. This is
also where the function
will resume next time.
Picking it apart…
function* fibonacci() {
let [prev, curr] = [0, 1];
while (true) {
yield curr;
[prev, curr] =
[curr, prev + curr];
}
}
again from: https://en.wikipedia.org/wiki/Generator_(computer_programming)
The “*” in “function*” indicates
this is a generator function

43. Each call to “gen.next()”
runs the generator till
the next yield (or till the
function returns), and
gets back an object
containing the yielded
(or returned) value.
Picking it apart…
var gen = fibonacci();
console.log(gen.next().value); // 1
console.log(gen.next().value); // 1
console.log(gen.next().value); // 2
console.log(gen.next().value); // 3
console.log(gen.next().value); // 5
console.log(gen.next().value); // 8
again from: https://en.wikipedia.org/wiki/Generator_(computer_programming)
Calling the function does not run the function body.
Rather it’s like calling a constructor.
It returns a generator object that you can then use
to run (and resume) the function body.

44. { value: val, done: true }
What gen.next() returns is:
A little object like:
“value” is the value yielded/returned
“done” is true if the generator “returned” (or
exited without a “return”).
as opposed to a yield.
once a generator is “done” it should not be
“next’ed” (if so it simply returns undefined)

45. But add the --harmony option on the node command!!
node > v0.11.2
Recent versions of Chrome
Recent versions of Firefox
We can run this code as it is in
Also in the browser in:
For other browsers a transpiler is required…
io.js supports it (by default - no option needed)

46. The most commonly mentioned transpiler for
generators is from Facebook and is called
“Regenerator”
https://www.npmjs.com/package/regenerator
npm install -g regenerator
regenerator --include-runtime fibonacci-basic.js >
fibonacci-basic-es5.js

47. Regenerator works by converting your code to a
state machine…
function fibonacci() {
var prev, curr;
!
return regeneratorRuntime.wrap(
function fibonacci$(context$1$0) {
while (1) switch (context$1$0.prev = context$1$0.next) {
case 0:
prev = 0;
curr = 1;
case 2:
if (!true) {
context$1$0.next = 9;
break;
}
context$1$0.next = 5;
return curr;
case 5:
prev = curr;
curr = prev + curr;
context$1$0.next = 2;
break;
case 9:
case "end":
return context$1$0.stop();
}
}, marked0$0[0], this);
}

48. babel has support as well (that uses
Regenerator to do the magic)
!
Traceur supports them as well.

49. Even though there’s fibonacci
examples everywhere…
• What I found interesting was that almost
immediately after generators came on the scene
there was a spate of “task runners”.
• These are utilities that invert the relationship of
consumer and producer like you see in the fibonacci
example
• In order to simplify asynchronous code where it
looks like it’s “blocking”

50. Lots-o-Task-Runners
• co (TJ/Visionmedia)
• Q.async (Q)
• Promise.coroutine (bluebird)
• galaxy
• monocle
• suspend
• genny
• gen-run
• coro
• (and on…)

51. “Paving the cow paths?”
(how about a stampede!!)

52. Probably so many because:
• 1. People wanted them
• 2. They’re pretty easy to implement
once you have generators.

53. Remember when you first learned to
program how simple it was…
var myNumber = parseInt(Math.random() * 100);
!
while (true) {
var guess = read.question('What is your guess? ');
if (guess > myNumber) {
console.log('Guess lower.');
} else if (guess < myNumber) {
console.log('Guess higher.');
} else {
console.log('YOU WIN!');
break;
}
}

54. There are cases when blocking code
really is easier to follow
var myNumber = parseInt(Math.random() * 100);
!
rl.setPrompt('What is your guess? ');
rl.prompt();
!
rl.on('line', function(guess) {
if (guess > myNumber) {
console.log('Guess lower.');
} else if (guess < myNumber) {
console.log('Guess higher.');
} else {
console.log('YOU WIN!');
rl.close();
}
rl.prompt();
}).on('close', function() {
console.log('Thanks for playing.');
process.exit(0);
});

55. Looking at “co” as an example of a “task
runner”…
var myNumber = parseInt(Math.random() * 100);
!
co(function* playgame() {
while(true) {
var guess = yield prompt('What is your guess? ');
if (guess > myNumber) {
console.log('Guess lower.');
} else if (guess < myNumber) {
console.log('Guess higher.');
} else {
console.log('YOU WIN!');
return "Goodbye";
}
}
})

56. To clarify - when we used fibonacci we
were the consumer of the generator
function* fibonacci() {
let [prev, curr] = [0, 1];
while (true) {
yield curr;
[prev, curr] = [curr, prev + curr];
}
}
!
var gen = fibonacci();
console.log(gen.next().value); // 1
console.log(gen.next().value); // 1
console.log(gen.next().value); // 2
console.log(gen.next().value); // 3
console.log(gen.next().value); // 5
console.log(gen.next().value); // 8
“library” code
“our” code
(=“producer”)
(=“consumer”)

57. Here things are reversed
var myNumber = parseInt(Math.random() * 100);
!
co(function* playgame() {
while(true) {
var guess = yield prompt('What is your guess?');
!
if (guess > myNumber) {
console.log('Guess lower.');
} else if (guess < myNumber) {
console.log('Guess higher.');
} else {
console.log('YOU WIN!');
return "Goodbye";
}
}
})
We’ve written the “producing” (yielding) code, and hidden
inside the “co” routine is the gen.next() “consuming” code.
In this case “prompt”
would typically return a
promise which is yielded
back to “co”, then “co”
will “gen.next()” us
once the promise
resolves to a value.
!
This is the “secret sauce”
for appearing to block
yet remaining single
threaded as always.

58. AN EASY METAPHOR…
We “yield” the control flow and let other stuff
(like the event loop)
run until the asynchronous result is ready

59. But there’s another “metaphor” coming in es7…
Async/Await

60. Basically the story (more or less) is…
There was a strawman proposal for async/await
at the same time there was a proposal for generators
!
The generators proposal got approved in time for es6
!
The async/await proposal did not but is part of es7
!
The good news is async/await is pretty much the same
idea as “task runners” but with slightly simpler syntax
!
The syntax is known for being similar to what C# has
had - it doesn’t have e.g. function* is has async/await
!
(though some javascript async/await implementations are
using generators internally btw)
!
Also the async/await standard is designed to
complement the new promise standards i.e. to work
well with apis returning promises.

61. Same control flow - nicer syntax:
var prompt = require('./node_modules/prompt-promise');
!
var myNumber = parseInt(Math.random() * 100);
!
// note instead of function* below we have "async"
async function playgame() {
try {
var gameover = false;
while(!gameover) {
// and instead of yield we "await"
var guess = await prompt('What is your guess? ');
!
if (guess > myNumber) {
console.log('Guess lower.');
} else if (guess < myNumber) {
console.log('Guess higher.');
} else {
console.log('YOU WIN!');
gameover = true;
}
}
console.log('Thanks for playing.');
}
catch(err) {
console.log(err);
}
}

62. Yay Working Exceptions!!
• Yes those were standard try/catch blocks in that last example!!
• Yay (standard) exceptions work even with asynchronous code!!
• (no special .catch etc. as with promises)
• They work both with the generator “task runners” and with
async/await.
• I didn’t mention but in addition to gen.next() there’s also a
“gen.throw()” which throws an exception at the point the
generator has paused on a “yield”. This is the secret sauce.

63. SUPER BRIEFLY:
SOME INTERESTING USES
OF GENERATORS

64. Briefly: KOA (http://koajs.com)
• Another TJ/VisionMedia Project (as is “co”)
• Intended as the next generation of “Express” (which TJ/VisionMedia also
developed)
• It’s been able to simplify the writing of middleware greatly since you simply
“yield” to the downstream middleware:
app.use(function *(next){
var start = new Date;
yield next;
var ms = new Date - start;
console.log('%s %s - %s',
this.method, this.url, ms);
});
get the time before
yield to
downstream
get the time after

65. Briefly: js-csp
• A “communicating sequential processes” library in javascript - it uses generators in it’s
implementation.
• Modeled off of core.async and google go.
• A key concept is that concurrent processes communicate through channels
• And the default behavior of channels is to block (they can buffer if you want but the
default is to block).
• Think of the simplicity like you see in our “guess this number” example, but waiting
on concurrent processes as opposed to user input.
• See also:
• https://github.com/ubolonton/js-csp
• http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript
• http://swannodette.github.io/2013/08/24/es6-generators-and-csp/

66. Briefly: js-csp
var {chan,timeout} = csp =
require(‘js-csp');
!
var ch = chan();
!
go {
var val;
while((val = <- ch) !== csp.CLOSED) {
console.log(val);
}
}
!
go {
ch <- 1;
(<- timeout(1000));
ch <- 2;
ch.close();
}
This example uses js-csp with sweet.js macros I based on some
started by James Long. Note <- is a blocking channel operation
(i.e. it’s doing a yield behind it)
a channel to
talk over
run this
reader guy
concurrently
with
this writer guy
wait for the writer
wait for the reader

67. js-csp in the browser (w/out sweet macros this time)
(thanks David Nolen & James Long)
• How about being able to block on a channel of events?
Really changes how you think about UI programming!!
function listen(el, type) {
var ch = chan();
el.addEventListener(type, function(e) {
csp.putAsync(ch, e);
});
return ch;
}
!
go(function*() {
var el = document.querySelector('#ui1');
var ch = listen(el, 'mousemove');
while(true) {
var e = yield take(ch);
el.innerHTML = ((e.layerX||e.clientX)+', ' +
(e.layerY || e.clientY));
}
});
create channel of
“type” events on “el”
concurrent with
other stuff happening
on the page
wait for mouse
moves and display
them
http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript

68. THANKS FOR COMING