Tony Finch – blog

Iterators are used in some programming languages to generalise the for loop. They were introduced by CLU in the 1970s and have been adopted by Python and Lua, amongst others. They are traditionally a special kind of function that has yield statements that return successive values in the iteration. This function is called at the start of the loop, and when it yields, the loop body is run. Each time round the loop the function is resumed until it yields again.

The iterator's activation frame is typically on top of the stack. This seems a bit backwards to me, because it leads to contortions to maintain the iterator's state between yields, e.g. using an auxiliary object or first-class closures. (See the first example here.)

I think it's simpler to desugar a for loop like this:

    for varlist in iterator(arglist) do
        statements
    loop

into

    iterator(
        (varlist): do
            statements
        end,
        arglist)

That is, you turn the loop body into an anonymous function which is passed to the iterator function. Instead of yielding, the iterator just calls the body function. The iterator can then be written as a normal loop without contortions to keep state between yields. (Pretty much like this, in fact.)

What makes this slightly interesting is how to desugar break, continue, and return statements inside the loop. A continue becomes just a return from the anonymous function, but the others have to escape from the iterator function as well as the anonymous function. This could be done with exceptions or continuation passing.