Javascript Generators

Generators are weird. They make programs behave in ways that are non-obvious. This weirdness easily turns into confusion because generator code looks like regular code. Generator syntax does not lend itself to easy discovery.

I’ve written about generators before. I’ve covered both the concepts and the specifics of PHP’s generator implementation.

Today I wanted to walk through javascript’s generators — specifically Node.js’s implementation (although this should apply to any modern javascript). We’ll be lighter than normal on concepts — if you’re looking to learn what the heck a generator is the afore-linked PHP Generators from Scratch is a decent place to start.

What is a Generator?

So what is a generator? A generator is a bit of code running inside a function

1. That will pause itself after returning a value, and
2. That the calling program can ask to un-pause and return another value

This “returning” isn’t a traditional return from a function. Because of this, it’s given a special name — yield.

Generator syntax varies from language to language. Javascript’s generator syntax is similar to PHP’s, but has enough differences that you’ll end up pretty confused if you expect them to work the same.

In javascript, if a programmer wants to use a generator, they’ll need to

1. Define a special generator function
2. Call that function to create a generator object
3. Use that generator object in a loop, or directly call its next method

We’ll run through each of these steps below, using this simple program as a starting point

// File: sample-program.js
function *createGenerator() {
  for(let i=0;i<20;i++) {
    yield i
  }
}

const generator = createGenerator()

console.log(generator.next())
console.log(generator.next())

If you run this program, you’ll get the following output.

$ node sample-program.js

{ value: 0, done: false }
{ value: 1, done: false }

The next few sections will explain how this program works.

Generator Functions

First, we have the generator function definition

function* createGenerator() {
  for(let i=0;i<20;i++) {
    yield i
  }
}

The extra * is what tells javascript this is a generator function. In a long tradition of * characters creating potentially confusing syntax, any of the following would be a valid definition of a generator function.

function*createGenerator
function* createGenerator
function *createGenerator

Despite this, the * is not a part of the function’s name. Instead it is the function* symbol that defines a generator.

Calling a Generator Function

After defining a generator function we call it like we would any other function.

// notice, no * when we call it.  * is not part of the function's name
// -- `function*` is the symbol used to define a generator function
const generator = createGenerator()

However, you’ll remember that the createGenerator function has no return value. That’s because generator functions do not have a traditional return value. Instead, when you call a generator function directly, the function always returns an instantiated Generator object.

This generator object has a next method. Calling next will run the code inside the generator function.

function* createGenerator() {
    for(let i=0;i<20;i++) {
        yield i
    }
}

This is important enough to call it out a second time. Calling a generator function directly does not run any code in the generator function. Instead it creates a generator object. It is calling next on the generator object that invokes the code inside the generator function.

The first time you call next on a generator object the code inside will run until there’s a yield statement. Once execution reaches a yield, javascript will pause execution of this code, and next will return (i.e. give you, or yield) an object that contains the value from the yield line.

When you call next a second time (or third time, or fourth time, etc) the code will un-pause and continue to run where it left off in the previous call. Variables (like the i in our example) will maintain their values. When the code reaches another yield statement the function will pause again, and return an object that contains the value yielded.

That’s why our two calls to next

console.log(generator.next())
console.log(generator.next())

returned the following output

{ value: 0, done: false }
{ value: 1, done: false }

Once the code inside the generator function has finished executing, any future call to next will return an object with an undefined value and done set to true.

{ value: undefined, done: true }

Generators and Loops

While it’s possible to manually call next on a generator object, we’re primarily meant to use them in loops. Consider this slightly modified program.

// File: sample-program.js
@highlightsyntax@jscript
function *createGenerator() {
  for(let i=0;i<5;i++) {
    yield i
  }
}

const generator = createGenerator()
for(const value of generator) {
  console.log(value)
}

When we use a generator object in a for ... of loop each trip through the loop will call next on the generator object and populate the variable (value above) with the value yielded. Running this program results in the following output

$ node sample-program.js
0
1
2
3
4

In our next article we’ll explore this for ... of loop in more depth, and discover how it’s designed to provide javascript with a built-in way to loop over any object in javascript.