http://qs321.pair.com?node_id=948107

Initially written Jan, 2012. Latest update Nov 2015.
(If you are what some call a "global warming denier", please bear with me. For the purposes of this post, let's imagine that global warming is real, and that the famous hockey stick graphs from a decade ago used good data.)

There is now increasing acceptance of the idea that the Perl Planet is again warming up after a brief cooling period. Perl 6 is a reinforcing factor in this warming, with an accelerating impact...

On #perl6 tonight, we see:

05:57  Coke     # 01/16/2012 - niecza at 100.02%
05:57             BOOYAH!
05:57  sorear     \o/
05:58  Coke     gist updated, google doc not. zzzzz.
06:04  moritz     \o/
06:05             sorear++ Coke++ colomon++
06:05             congratulations everybody

What did Coke++, a long time contributor to many Perl things, push to achieve tonight before he succumbed to exhaustion? I've been watching Perl 6 for over a decade, and to me no moment has looked more symbolically significant than this one.

Niecza, a Perl 6 compiler, overtook Rakudo, another Perl 6 compiler, as measured by the number of "spectests" passed. (This does NOT mean that Niecza is better than Rakudo. The 100.02% figure does NOT mean that Niecza is magically doing everything Larry specced, plus reassigning the colon. A comparison of the two compilers' features based on fairly recent data.)

Nov 2015: Niecza is now dead/dormant. Rakudo no longer supports Parrot. Rakudo on MoarVM passes ~48k of ~50k spectests. I'm ignoring the recently introduced 70k or so Unicode tests. As chromatic emphasized in some of his original responses to this post, don't read too much in to these numbers. I think they prove that Perl 6 isn't dead as some believe, but ymmv.

Outsiders might now be able to see that, although the effect is gradual -- so gradual many deny it is even happening -- Perl, and Perl 6, have been developing consistently for years now, and this development is subject to self-reinforcing acceleration, i.e. the Hockey Stick effect.

There are at least TWO Perl 6 Hockey Sticks: Rakudo and Niecza, with corresponding coding heroes jnthn++ and sorear++. (The new audrey++s. Not to forget pmichaud++, but he's being a hero of a different sort at the moment. And not to forget other Perl 6 compilers and their authors such as Perlito and fglock++.) Not only is development of each of these Perl 6 compilers accelerating individually, each is reinforcing the pace of development of the other.

Nov, 2015: jnthn continues to drive Rakudo dev. sorear wrote the plumbing needed on the JVM in mid 2013 that allowed jnthn to start building out the P6 concurrency story on Rakudo/JVM ... and hasn't been seen on #perl6 since. Larry Wall began landing Rakudo commits in 2014 and has continued that through 2015.

To get a sense of how the lead developer of a Perl 6 compiler sees things, check out Jonathan Worthington's post yesterday that summarized how 2011 went, and how 2012 should go, from the Rakudo perspective.

But better still, to check out how it feels from the perspective of the rest of us Perl 6 fans, check out Solomon Foster's Perl 6 post from yesterday in which, while describing the 400th Perl 6 Rosetta Code entry, he notes "the awkward situation with Rakudo Star has helped obscure the fantastic good news in Perl 6."

I don't want to raise expectations unduly. There is clearly still plenty to do before 6.0.0. But I was a contributor to Mozilla 2002 thru 2004, and I recall how most people had more or less written off the whole project by then. And then came Firefox.

So, if you know how to be nice to people, and to butterflies, why not come play hockey on irc chat at #perl6 on freenode?

Replies are listed 'Best First'.
Re: Perl 6 Hockey Sticks
by masak (Scribe) on Jan 16, 2012 at 11:13 UTC
    This post manages to put into words, and better than I've ever managed, what's going on right now with Perl 6 and its implementations. Thank you, raiph.
Re: Hockey Sticks
by chromatic (Archbishop) on Jan 17, 2012 at 01:06 UTC

    That's... not exactly what I understand statistics to mean.

    Way back in the day when Rakudo had graphs of spec test data every day, Solomon Foster spent a lot of time writing hundreds (thousands?) of tests for complex numerical operations. He put a lot of energy into a thankless task and deserves a lot of credit for that, but the best-kind-of-true fact that "Rakudo passes 10% more tests this month than last!" didn't translate into a Rakudo that was better for most people to use for serious purposes.

    Similarly the fact that Niezca passes more tests than Rakudo doesn't necessarily mean that Niezca is more usable or useful or complete than Rakudo. It's not even clear if the tests passed are the right tests or are meaningful to its utility. (Besides, given Rakudo's rewrite-induced regressions, it shouldn't be surprising that another implementation can overtake it.)

    Further, if you want to talk about trends and acceleration, you probably need more than a month's worth of data points to do so. You can't get much of a meaningful derivative from a graph with so much noise; you need to normalize it, and you need to normalize it to a normative period. Otherwise, fun with statistics means that we can predict that, based on the spectest measurements of one platform between mid-December 2011 and mid-January 2012, we expect:

    • Rakudo to pass 3.7% more of the spectests each month
    • The spectest suite test count to grow by 2.1% each month
    • Rakudo to mark 2.4% more spectests as TODO each month
    • Rakudo to mark 2.4% more spectests as SKIP each month
    • Rakudo to run 3.6% more of the spectest suite each month
    • Rakudo to fail 7.1% more spectests by count each month

    As for:

    "the awkward situation with Rakudo Star has helped obscure the fantastic good news in Perl 6."

    I hear that other than that, Mary Todd Lincoln rather liked the play.


    Improve your skills with Modern Perl: the free book.

      chromatic,

      You've said before that, despite their protestations to the contrary, the Rakudo team doesn't deliver what they say they're going to deliver. And that, as a result, you've experienced professional embarrassment and commercial/career loss. And all of that is on top of your feelings about Parrot, and your involvement in it, and the amazing 12-years-and-counting Perl 6 saga.

      I get all that, and that you are the Modern Perl (5) champion. (Thankyou.) I see why you might currently feel the need to play the role of skeptic in relation to Perl 6.

      Although I see your current need, I also see the future you, the one that will need to rebuild your bridges with Perl 6. You do NOT want to dig yourself an anti Perl 6 hole out of all proportion to what's justified. Perl 6 is progressing nicely. Perl 6.0.0 compilers will ship. Why are you so determined to throw away all the goodwill you built up before your disappointment?


      Late, fat, and ugly

      "There comes a time when the only merciful thing to do is pull the plug...

      Even Perl 6's most ardent supporters have to admit the possibility that their brand new language now more resembles a massive, festering cyst. Trapped in the womb for well over a decade, the Perl 6 Project has made a lot of noise, churned out a lot of code and has gone almost exactly nowhere. Late, fat and ugly, Perl 6 is hopelessly moribund, deeply mired in its own filth, with no end in sight. A quick and painless euthanasia is the best option for all concerned.

      As usual, the blame can be placed on the lack of responsible adult supervision. Told to create a utopia, the Perl 6 Project's programmers have done almost exactly the opposite, letting their far-thinking vision overwhelm such tedious, day-to-day tasks as actually getting a usable product out the door.

      Oblivious to the fact that their market share was disappearing faster than donuts in the break room, the Perl 6 Project programmers repeatedly abandoned real-world progress and accomplishments for -- and this is the technical term -- cool shit.

      With the sort of over-enthusiastic zeal that used to get missionaries attached to roasting spits, the Project couldn't satisfy itself with merely building a fast, efficient, CPAN compatible compiler. Instead, it set off on a quest to re-engineer the way Perl applications are built, to construct not just a new compiler, but the ultimate Perl, a be-all, end-all, goes-ping monster."

      (From Late, fat and ugly, with some minor editing.)

        Perl 6.0.0 compilers will ship.

        I've never claimed otherwise.

        You do NOT want to dig yourself an anti Perl 6 hole out of all proportion to what's justified.

        Thank you for the (I'm sure) well-intentioned advice on how to make you feel warm and fuzzy, but I believe you're operating from some very wrong assumptions.

        I'd like to use Perl 6, but the lack of usable and stable releases since the initial Rakudo Star announcement make it very unattractive. I have neither the time or energy to continue to devote to making that happen. Volunteers will work on what volunteers want to work on, and if they want to do only the fun things, so be it.

        I see why you might currently feel the need to play the role of skeptic in relation to Perl 6. Why are you so determined to throw away all the goodwill you built up before your disappointment?

        Why do people on #perl6 spend so much time performing long-distance psychoanalysis and so little time reading words as written? ("Professional embarrassment"? Really?)

        What are you talking about?

Re: Hockey Sticks
by Anonymous Monk on Jan 18, 2012 at 03:52 UTC

    So what is the conclusion at the end of this, "Perl 6 is still not ready".

    Sorry to be playing the bad guy part here, but measuring the progress of a project by measuring number of tests it passes, is very similar to measuring the progress of a project by lines of code.Unless you have uniformly distributed the test cases across feature sets. Test cases are probably a measure of stability not progress.

    Now lets come to what would actually constitute progress in Perl 6's case. At this time feature set with an acceptable performance will do. We need not have to ensure Rakudo or Niecza runs on a million virtual machines at this moment, Frankly that's the last thing any body wants. A full feature set implemented with decent performance will do.

    Most Perl 6 people say, Perl 6 specification is just like any other language, evolving always. That is true, but you must also know those other languages have 'stable' compiler releases. You are all welcome to try your experiments. But make a production release with a feature complete release. Then incrementally evolve your spec and continue your experiments in the period of two stable releases. I believe this is how all projects are run. They have a aim, once that is met then comes the 'next step'.

    Now Perl 6 project has become like the dichotomy paradox(http://en.wikipedia.org/wiki/Zeno's_paradoxes).

    That which is in locomotion must arrive at the half-way stage before it arrives at the goal.

    This is what is happening. In order to finish you need milestones. You need to plan and work towards those milestones within a time frame. Instead what you are doing is dividing a milestone into many milestones, and those milestones into further many milestones. So ultimately you never hit the original milestones because you now have some many several sub/sub-sub milestones in which you get lost and never go and achieve the original milestone as planned.

    While working on a project, Instead of completing the feature set in the project you find some interesting thing that can bring your in x% performance gains. You leave the feature set aside and go in its pursuit. Then you realize while making the x% performance gain, you can add something to the compiler that can bring some thing which enables it to run on multiples VM's. Now you go into the VM thing. While doing that you figure out, you figure out another thing which can help a native call interface.

    You have totally lost the sight of the goal by now. You have so many sub tasks to chase within a small milestone. That you can humanly never scale and come out of it. Since the original milestone never completes, you don't go to the next milestone. Since you don't go the next, you don't go the next after it and so on and so forth...

    Ultimately you never reach your goal.

Re: Hockey Sticks
by BrowserUk (Patriarch) on Jan 17, 2012 at 23:41 UTC

    From elesewhere in the thread:

    CONTRIBUTE!

    I hate to say this, but it needs to be said.

    You are just the latest in a fairly long line of enthusiastic P6 champion advocates.

    A line that at one point numbered in the low hundreds of active contributors -- many in minor ways, but a substantial few in major ways -- that have long since fallen by the wayside. What you have to ask yourself is: why?

    And when you've looked back, and digested the history, and reached the point of knowledge many of those here are already party to, you might understand why you, as the latest surrogate boy-who-cried-wolf, are not receiving the enthusiastic welcome to your 'news' that you perhaps expected.

    History (colloquially if not literally, his story) is a bitch. And in the modern world where the opportunity to rewrite it is all but gone, an unforgiving master.

    I sincerely wish you, and all those involved, the very best for the success of your endeavours, but the realist in me will trust in future history for the outcome.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      CONTRIBUTE!
      I was responding in kind to sundialsvc4's "hurry up, willya". What I meant to imply was that, if you want something to happen faster, then contribution is your best bet. (I'm intrigued by your addition of an exclamation point!)

      And when you've looked back, and digested the history, and reached the point of knowledge many of those here are already party to
      I hear you're trying to help me out, but I think you've got me (and by extension this Meditation) pegged wrong. I've been following Perl 6 closely since 2000. I got into it. I designed a product (parrodocs) I ended up having to shelve.

      The original Perl 6 conception was, arguably, too ridiculously ambitious. (I don't agree, but it's a reasonable view.) Larry got ill. Audrey burned out. Parrot (not Rakudo) went off the rails. It sucks to get cancer. But, as you yourself point out, shit happens and:

      History (colloquially if not literally, his story) is a bitch. And in the modern world where the opportunity to rewrite it is all but gone, an unforgiving master.
      You can't rewrite it. But you can write it.

      The Perl 6 project obviously hasn't had its Firefox moment, but it's as vibrant as it's ever been, partly due to what look to me like some self-reinforcing trends. That's what I found interesting, and decided to share in one of the very few posts I've ever made to PerlMonks.

      the realist in me will trust in future history for the outcome.
      Well, I sure can't argue with that! :)
        (I'm intrigued by your addition of an exclamation point!)

        I guess that as you wrote it in all caps, that's how I read it, even if it wasn't there.

        but it's as vibrant as it's ever been,

        Hm. That's not the picture I see.

        Author: Carl Masak <cmasak@gmail.com> Date: 2012-01-16 (Mon, 16 Jan 2012) Changed paths: M S02-bits.pod Log Message: ----------- [S02] fix tiny typo

        At one time my inbox was filled with those. Now it is just a sporadic trickle that comes in fits and starts.

        At its peek, it was ... I cannot describe it better than masak already did:

        Pugs: The golden age

        I remember stumbling into the #perl6 channel on freenode, still fairly dazed by the fact that someone was taking the Synopses and implementing them. Add to this that Audrey Tang turned out to be a frighteningly productive hacker with a magnetic personality which drew other people into the project like nothing I or many others had ever seen. Being on the #perl6 channel was like standing close to the eye of a hurricane; things just magically happened, either because Audrey had just landed another set of commits, or because someone had started a cool side project and was hacking on that, all the while bringing interesting ideas and thoughts into the channel.

        For all the very genuine reasons for various key individuals need to take time-outs or withdraw, there is an underlying problem that is best summed up by paraphrasing a speech given to me by one of my old bosses when he came in and found I'd pulled another all-nighter:

        I appreciate the productivity you achieve with your solitary way of working, but, the moment you appear to becoming indispensable to me, I will sack you.

        Just be sure to document where you are going and why; and to delegate each step as soon as you know where it is headed. That way, if you are hit by a bus, I'll be able to carry on moving forward, even if things slow to a crawl.

        Historically, I was lousy at letting go of my babies, but I learnt to delegate. I brought in a technical writer and made him project leader. He could take my scribbled, near incomprehensible ramblings, and with a few well targeted questions and some insistence, turn them into time-lines, milestones, action lists and progress reports. He ran the project better than I ever could, despite that he had little in the way of technical know-how or experience. But the key to his success was his insistence from about the 3rd week that I give him the authority to not just question me, but tell me I was wrong. I ceded that authority reluctantly, and benefited from having done so every day from that point forth.

        Not just on that project with that particular TW, but on every subsequent project because of what I learnt from it. There has to be someone, divorced from the oohs and ahhhs of the bleeding edge, with the authority to pull the threads of a project in a consistent direction. That's what I see missing from so many OSS projects.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Re: Hockey Sticks
by sundialsvc4 (Abbot) on Jan 16, 2012 at 23:53 UTC

    Perl-6 == still waiting.

    The wife of an IBM salesman divorced after many years because her marriage had never been consummated.   “All he ever did was, to sit on the edge of the bed and tell me how good it was going to be.”

        And the point is? If I'm waiting for a bus I am (prettymuch) standing still. It's the bus that moves. Hopefully. It's good to hear the bus nearing though.

        Jenda
        Enoch was right!
        Enjoy the last years of Rome.

        Shrug...   Whatever.

        I am the Customer.   I literally do this for a living, literally every weekday.   Give me a pragmatic, rugged, practical tool that actually does something dramatically useful.   “And hurry up, willya?”   Perl-5 is such a tool.   Nevertheless, to paraphrase a bunch of Biblical demons (Acts 19:15):   “Perl-5 I know, but who are you?”   I am encouraged by the promise of Perl-6 and am sensitive to the diligent efforts of those who are working on it ... I have tremendous respect for people who are capable of working at that level ... but, a product has to ship, if only so that the real-world professional community can kick the tires and discover what (else) needs to be changed or fixed.

        When it ships, you’re not done.   At this point, you will have barely started.   A tool is built for a craftsman’s use, but there have been no opportunities given yet for any of those craftsmen to determine if it is truly useful for anything.   Right now, from a purely market-driven point of view, you’re Perl-6 is just an armchair quarterback, known to be the offspring of a great and well-respected player, but himself never tested on the field of battle.

Re: Hockey Sticks
by tobyink (Canon) on Jan 24, 2012 at 01:06 UTC

    As much as I like Perl 6, all its implementations are right now missing the one killer feature that could drive adoption. And I don't mean heredocs in Rakudo.

    Imagine I wanted to port some of my Perl 5 modules to Perl 6. What exactly would that entail? A more or less line-by-line translation of a module might take, say, 30 seconds per line of code, so maybe 2 or 3 hours for a smallish module.

    And yay, I'm done. Or actually, maybe I've hardly begun because the first few lines of my script were:

    use LWP::UserAgent; use RDF::Trine; use XML::LibXML;

    ... d'oh! Now I've got another few months ahead of me in my porting exercise.

    This is a major problem for people hoping to use Perl 6 for actual work. People say that CPAN is the best feature of Perl. If Perl 6 doesn't have CPAN, then no matter what other features it does have, it's missing the best feature of Perl.

    I'm not saying that Perl 6 needs to reimplement all of Perl 5. It just needs to embed libperl so that Perl 6 code can load and use Perl 5 modules. While not trivial, this seems achievable.

    I think perhaps many of the Perl 6 developers are afraid to implement such a thing, because if Perl 6 can run Perl 5 code, people have less of an incentive to port their Perl 5 code to Perl 6. But I disagree - I think that the burden of having to port not only your own code, but any underlying libraries you use acts as a strong disincentive right now.

      Larry has planned use of Perl 5 and CPAN from within Perl 6 from the start. I am confident several approaches will get implemented, once the right pieces are in place. I agree it will make a huge difference.

      One approach is embedding the usual perl interpreter via libperl.

      jnthn's blizkost enables Perl 5 code to be executed within Parrot (and hence using Rakudo on Parrot). I think jnthn recently said that blizkost is currently broken and that he hopes to fix it in the next few weeks.

      Aiui, blizkost currently does very little, just eval'ing a single line of perl5 code, and not even making use of the result. I think that developing it toward supporting Perl 5 modules was blocked on other progress, in particular implementing native 6model. Maybe it'll move forward this year.

      Update August, 2014: jnthn implemented native 6model in 2012/2013 -- in a brand new VM MoarVM, not Parrot. nine (Stefan Seifert) then started Inline::Perl5 at YAPC::EU 2014. Within a couple days this new NativeCall based approach had already established itself as a great approach for delivering P5 interop, once P5 object destruction has been dealt with. It looks likely to me that this project will assimilate aspects of diakopter's unimplemented proposed P5 Interop design which in turn had assimilated blizkost.

      Alternatively, once Perl 6 is sufficiently complete, it might be "relatively easy" and compelling to implement a Perl 5 compiler in Perl 6. So that'll be another plausible approach to using some CPAN modules.

      When I last paid any attention, Perl 6 was intended to be able to run Perl 5 code unmodified, and allow new Perl 6 code to interact with old Perl 5 code. Of course, maybe the spec has changed since then.
Re: Hockey Sticks
by Xiong (Hermit) on Jan 24, 2012 at 12:28 UTC

    We're all different. I too have tremendous respect for anyone who is willing to spend years perfecting a language. But I also agree that it will be mostly useless at rollout unless it has some sort of "P5 compatibility mode" so that it doesn't propose to turn all of CPAN into so much junk.

    That is not at all to say it will remain useless. A new module archive and all the other machinery, tools, and infrastructure may eventually be built. For all I know it will be a fantastic language.

    But what is obvious is that it will not be Perl (certainly not Perl 5). It will be a language inspired by Perl; which could be said of Ruby.

    I would be much happier with the entire effort if it were simply given a new name instead of forcing P5 to minor-version indefinitely. This might also help to dispel any possible cloud of FUD surrounding the future of P5.

    I'm not the guy you kill, I'm the guy you buy. —Michael Clayton

      Conversely, I'd be happy to reconsider my current feelings once Perl6-core matches Perl5-core blow-for-blow.

      That means, equivalent or better functionality with equivalent storage and performance.

      At that point I'd enjoy the challenge of trying to recreate or better the key P5 modules I use.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

      fwiw, I concur 1000% with all the above.

      (Update: Except for the bit about turning CPAN into junk. Failing to utilize it isn't the same as turning it into junk. As has been pointed out many times, P5 will continue to exist for a long time.)

      I for one agree that the name should have been changed. The new language is a spiritual heir of Perl, but it's different enough to warrant a new name. Oberon was far more similar to Modula than P6 is to P5; Modula was also yet more similar to Pascal. You might argue that those represent a negative model, being, as they were, somewhat losers in the language race. I submit that they would have fared no better had they been named Pascal 7 and Pascal 9.:-)

Re: Hockey Sticks
by fisher (Priest) on Jan 20, 2012 at 09:38 UTC
    Excuse me, I'm honestly not asking to begin a flamewar, but. Could you please explain me why I must be interested in some language compiler which runs over .net / mono, If I never used neither of these and not planning to use it anyway? Even worse, I think than Mono has no future, cause it just trying to replay what other's did, in .NET.

    As of perl6, I think rakudo is great product, but it lacks something. When I talk about Rakudo Star with co-workers, almost everyone asks - what perl6 have, which others don't? Why should I use it? And they of course already have implied answer: there's nothing new.

    The great idea behind Java, for example, was VM, 'we can run everywhere'. Perl5 and alike were multipurpose scripting languages with dynamic types for quick devel-ent. The Haskell is _pure_ functional language. Erlang comes with idea of parallel execution on multinode cluster.

    So what have perl6, that others don't?

      Until Microsoft decides to port .Net to unix themselves, Mono has a future. Exactly because it replays what others did and allows you to use quite a bit of the stuff made for .Net in unix.

      Java has got (and had got) no great idea. Just great marketing.

      Jenda
      Enoch was right!
      Enjoy the last years of Rome.

      Could you please explain me why I must be interested in some language compiler which runs over .net / mono {given I am not interested in .net / mono}
      There's no need. Just ignore it.
      When I talk about Rakudo Star with co-workers
      Rakudo Star is intended primarily to make it easier for interested folk to try out Rakudo. Imo the Perl 6 project does not have anything useful to show or tell someone unless they are already interested in Perl 6, are willing to suspend the disbelief that reasonably follows from knowing the project has been in the works for 12 years, and are willing to visit #perl6 on freenode and ask about it with an open mind.

      So what have perl6, that others don't?
      Right now, to simplify things, I'll say it has nothing. It's a work-in-progress update of Perl. When a production ready version ships, there'll be efforts to create soundbites that describe what Perl 6 has, including what it has that is unique. Until then, I could say it integrates just about everything, does it right, and does it nicely, and that's what it has that others don't, but no one would believe me, so I'll not do that.
A reply falls below the community's threshold of quality. You may see it by logging in.