If you want to get to know them better, I would recommend playing around with a language that has them. The two most obvious ones are Scheme and Ruby. You might also find the paper Coroutines in C interesting (thanks to japhy for mentioning it once). It discusses an actual problem which coroutines would be a natural fit for, and then proceeds to show how you can mimic them in C. (The program that paper is about, PuTTY, is fairly widely used. If you are on Windows and occasionally need to telnet, or would prefer to ssh instead of telnet, then I strongly recommend looking into it.)

I've speculated about using "fibers" in Win32 to implement iterators that can yeild. Calling the function in its own super-lightweight thread means it can keep its stack, so it doesn't need to return but can yeild and keep looping. I never wrote up that article (yet) though.

You can also do iterator, today, using objects.

my $iterator= $container->iterate();
while (my $item= $iterator++) { process $$item; }
[download]

The code behind ++ would have to save its state and return each time, so it's harder to write than having a yeild. But you declare the iterator instance and can use that one (or another one).

What if yeild caused the call to return a token, and that is what you make subsequent calls on? You can have another one going without conflict.

I agree that it doesn't sound like co-routines as described in classic literature. Those would "call" each other, rather than having a yeild statement at all. Such co-routines as members of the same object would also make sense.

—John

Personally I don't think that the issue of avoiding conflict is that bad. If you really want multiple instances of the same thing, then just have a function that returns a closure, and each closure maintains its own state.

Anyways the yield idea can work really nicely. As I said before, try Ruby. Ruby combines yield with a nice piece of syntactic sugar. In Ruby you can call functions 2 ways. In one way you just call the function. In the other you call the function and pass a block as an extra argument. When you do the latter then any yield will call the block. (You can also detect which way you are being called and function appropriately.)

So what you get is effectively the same as creating a closure in your context and passing it to the function, and having it call the closure. However it works really nicely because, even though that is what you are doing, there is a lot less machinery needed to set it up. And it seems that people who have a hard time getting their heads around the idea, You encapsulate a closure and then pass it in have no problem with the idea of passing a block to a method named each and having your block called on each thing encapsulated in your object.

As Larry Wall has noted, sometimes sugar is worth it in its own right. Even if you cannot readily use yield for the full generality of what you want, it can work really nicely.

As I said before, you can play around with Ruby now to get a sense of what this feature will be like in Perl 6. (Of course many of the other features of Perl 6 are missing in Ruby...)

I also failed to understand the logic of the following example from the RFC:

  ... one could write: 
          %newhash = map {yield transform_key($_); transform_val($_)} 
+%oldhash;

  This flattens the %oldhash to a sequence of key/value pairs
[download]

   MeowChow                                   
               s aamecha.s a..a\u$&owag.print

I think that Damian's proposal assumes that Perl is going to maintain a global stack, and what is saved is just the internal state of the function. Then $_, being a global, will be whatever it is currently in Perl while where you start in the block will be preserved.

I believe that would create alot of nasty confusion, and make localized globals all but useless inside of coroutines. For instance, the following would be quite troublesome:

sub my_iter {
  for (@_) {
    yield $_;
    # upon return, $_ may no longer be localized. if we
    # modify it, we could be modifying a global value
    # outside our scope. there's no way to tell, and 
    # that's rather nasty.
  }
}
[download]

Of course, one could avoid using localized globals inside coroutines/generators in the first place, but where's the fun in that, I ask ;)

   MeowChow                                   
               s aamecha.s a..a\u$&owag.print

I couldn't follow the continuation lab you mentioned. But I think the idea of a continuation primitive is exactly what we need, and my comments to Tilly earlier on this thread seems to be that.

Specifically, the first yeild returns a continuation object. Use that object to "resume", and it knows exactly which instance you mean. The resume should pass parameters, too. It would re-bind @_ in the function, so the line after yeild could look at that (again) if desired.

The resumer thing could be an object with various members, including call and reset and who knows what else. Or, it could return a function reference and just calling that will resume.

my $next= $container->iterate ('inorder');
while (my $node= $next->()) { ... }
[download]

Details: the iterate() function would do set-up and yeild with a token, using another built-in function to generate it. Or it could be a separate statement that creates it and yeilds at once. Then it goes into the traversal loop.

What do you think? —John

The semantics of real continuations are less trivial than that, and they appear (to me, at least) substantially more convoluted when applied to procedural, as opposed to functional languages. What you're describing is still a form of generators, which can be built with continuations and anonymous subs, but doesn't provide the full power of continuations.

Additionaly, rebinding @_ is most likely a bad idea. There are better ways to pass values through to continuations. Changing your local context somewhat defeats the purpose of continuations to begin with.

   MeowChow                                   
               s aamecha.s a..a\u$&owag.print

Could you elaborate on the distinction between "coroutines", "generators", and "call with concurrent continuation"?

YAIUL (Yet Another Informative USENET Link ;-)

   MeowChow                                   
               s aamecha.s a..a\u$&owag.print

Basically, a line like

    while (my $node = $root->next_inorder()) {
        print $node->{data};
    }
[download]

found in a module will test fine, but will suddenly and mysteriously fail if that code is called (even indirectly, by many levels) from another use of the iterator, even if it's on a different collection.

You should also realize that someone who uses coroutines knows what he/she is doing. Programmers don't randomly call subs. They call subs because they know what the sub does - and they should also know a sub is a coroutine, and program appropriately.

We also need a "quit" feature. Say I'm iterating over a collection and break out of the loop when I find the element I want. Just last it, and don't look at the rest. Well, next time that code is called, it will still be incomplete...

-- Abigail

Yea, I know each has a similar problem, but it's not as bad because it has a separate state per hash. Called code can iterate over a different hash without messing it up.

Likewise, readdir is given a handle, and you can have multiple handles open. It's no different from a normal file read, really. m//g is even better, in that not only is it per string (and a string can be copied to a local temp so nobody else will even try to iterate over the same copy), but you can use pos() to reset the state, or cancel it by matching a RE without a /g. How is any of that "*exactly* the same"?

My issue with the proposal is that the iterator would be global for all instances. It would be like each maintaining a single global state, rather than per-hash. IAC, Perl 6 will fix nested each's on the same collection.

but more to the point, coroutine like behaviour only happens when yield is used. In the example you present, no yield happens, so it will behave like any other subroutine

Why do you say that? next_inorder() yeilds.

there is of course return ah, that makes sence. You can't pass parameters into the iteration in-progress, but you could have a separate reset function that sets a shared global and calls the iterator, which sees the flag and returns.

—John

My issue with the proposal is that the iterator would be global for all instances. It would be like each maintaining a single global state, rather than per-hash.

Ah, but not if you create your iterators as anonymous closures! Then you can have several instances, each working on separate data, and each keeping state.

but more to the point, coroutine like behaviour only happens when yield is used. In the example you present, no yield happens, so it will behave like any other subroutine

Why do you say that? next_inorder() yeilds.

I was referring to your:

Say I'm iterating over a collection and break out of the loop when I find the element I want. Just last it, and don't look at the rest.

No yield here.

-- Abigail

I just wanted to mention Coro by Marc Lehmann, a try at implementing coroutines in perl5.

-- Hofmator

I looked at Coro, and what is being done with the XS code in it is that the context of the current subroutine is being saved. These contexts can then be jumped back to. I don't know how well it works, but it passes its own tests <g>