Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re^4: The future of Perl?

by eyepopslikeamosquito (Bishop)
on Nov 11, 2014 at 20:54 UTC ( #1106882=note: print w/replies, xml ) Need Help??

in reply to Re^3: The future of Perl?
in thread The future of Perl?

Anybody wanting to advance perl 5 seriously, should consider starting from scratch!

Though I too would love to see that, I doubt it is realistic. Remember, both Topaz and Ponie were attempted and abandoned. The best we can hope for is probably a slow, relentless refactoring.

Of course, the "old, huge, tangled code base problem" is not specific to the Perl 5 internals. It is a chronic problem in the software industry, even afflicting the blessed Perl Monks code base. :)

The problem isn't an infrastructure issue, however -- and speaking as one of the handful of people who've had a hand in developing the site's software: It's our own gosh-darn fault. Perlmonks is WAY more complex than when it originally launched. It does a crapload of perl evals and sql queries per page. It's vulnerable to resource hogs. Searching can cripple the database. And right now, I don't think we're gonna fix these problems any time soon. ... It's not a matter of computer resources, as much as human engineering resources.

-- Re: perlmonks too slow by nate (original co-author of the Everything Engine)

Some relevant quotes from Nobody Expects the Agile Imposition (Part VI): Architecture follow:

Have you ever played a game called Jenga? The idea behind Jenga is that you start by making a tower of blocks. Each player removes a block from somewhere in the tower, and moves it to the top of the tower. The top of the tower looks tidy, but it's very heavy and the bottom of the tower is growing more and more unstable. Eventually, someone's going to take away a block from the bottom and it'll all fall down.

I came into Perl development quite late, and I saw a very intricate, delicate interplay of ideas inside the Perl sources. It amazed me how people could create a structure so complex and so clever, but which worked so well. It was only much later that I realised that what I was seeing was not a delicate and intricate structure but the bottom end of a tower of Jenga. For example, fields in structures that ostensibly meant one thing were reused for completely unrelated purposes, the equivalent of taking blocks from the bottom and putting them on the top.

-- The Tower of Perl by Simon Cozens

The perl5 internals are a complete mess. It's like Jenga - to get the perl5 tower taller and do something new you select a block somewhere in the middle, with trepidation pull it out slowly, and then carefully balance it somewhere new, hoping the whole edifice won't collapse as a result.

-- Nicholas Clark

I remember one company that has about 120 engineers, developers of all kinds of whom 10 are still able to work on the core functionality. The other 110 are working on new stuff. We brought all the engineers into the room. We said, okay, the product manager for the first area and the lead engineer for the first area come on up here. Now select the people you need to do this work over the next month, including, of course, the core engineers. And they did and we said, okay, now leave, get out of here and start working. ... when we got to the fifth product manager and the lead engineer and they said we can't do anything. There's no core engineers left. We looked around the room and there were 60 engineers left. They were thoroughly constrained by the core piece of functionality.

If you have enough money, you rebuild your core. If you don't have enough money and the competition is breathing down your neck you shift into another market or you sell your company. Venture capitalists are into this now, buying dead companies. Design-dead software.

-- Ken Schwaber, Google tech talk on Scrum, Sep 5, 2006 (38:40)

Netscape 6.0 is finally going into its first public beta. There never was a version 5.0. The last major release, version 4.0, was released almost three years ago. Three years is an awfully long time in the Internet world. During this time, Netscape sat by, helplessly, as their market share plummeted. It's a bit smarmy of me to criticize them for waiting so long between releases. They didn't do it on purpose, now, did they? Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.

It's important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time. First of all, you probably don't even have the same programming team that worked on version one, so you don't actually have "more experience". You're just going to make most of the old mistakes again, and introduce some new problems that weren't in the original version.

-- Joel Spolsky on not Rewriting

Now the two teams are in a race. The tiger team must build a new system that does everything that the old system does. Not only that, they have to keep up with the changes that are continuously being made to the old system. Management will not replace the old system until the new system can do everything that the old system does. This race can go on for a very long time. I've seen it take 10 years. And by the time it's done, the original members of the tiger team are long gone, and the current members are demanding that the new system be redesigned because it's such a mess.

-- Robert C Martin in Clean Code (p.5)

That's not to say it can't be done though. The great Netscape rewrite (ridiculed by Spolsky above) -- though a commercial disaster -- metamorphosed into an open source success story. Another example of a successful rewrite is the Perl 5 rewrite of Perl 4.

Replies are listed 'Best First'.
Re^5: The future of Perl?
by BrowserUk (Pope) on Nov 11, 2014 at 23:41 UTC
    The best we can hope for is probably a slow, relentless refactoring.

    Relentless I agree with; but I question "slow".

    It is my (long) considered opinion that if you reduce the targets to Windows and *nix (I'd say POSIX, but that lives about a decade behind (at least) linux); then with just 10 people (They'd have to be the right people), you could refactor perl5 in one year, to be:

    • Faster.
    • Simpler.
    • More maintainable.

    The secret: Fcuk the rest; get it working on these two first.

    Rational: If it can be made to work -- ie. passes the entire perl build test suite and perlbench on those two platforms, whilst having reduce the kloc by 50% and the average function size by 50%-- on those two wildly disparate platforms, then it can be made to work anywhere where there is sufficient will and bodies to tackle the task. (If there ain't; c'est la vie!)

    Detail: Whilst neither kloc nor function size is directly proportional to understanding, correctness and maintainability ; the correlation is so strong, over many studies over many decades, that it would be obtuse not to recognise that simplification is inversely proportional to understanding; and understanding is directly proportional to both correctness and maintainability.

    Target: Start small in terms of platforms (just two); start small in terms of functionality (just does what p5 does now); start small in terms reduction in size. I've suggested 50% but I believe 70% is (quite easily) achievable.

    If you reduce the current code by 50%; twice as many people stand a chance of understanding it.

    Then you emulate the pugs model: everyone gets a commit bit; (a majority of) 3 people have to agree, to rescind a commit.

    The guiding logic

    Nothing substantial -- syntax or semantic -- changes until a 50% reduction (compiled kloc) occurs. Then you invite both requests-for-change, and patches.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re^5: The future of Perl?
by salva (Canon) on Nov 11, 2014 at 21:58 UTC
    Of course, the "old, huge, tangled code base problem" is not specific to the Perl 5 internals. It is a chronic problem in the software industry

    Yes, but perl 5 is an extreme case.

    In any case, IIRC topaz was not a fiasco but a prove of concept that derived into the Perl 6/Parrot project. Ponie intention was to bridge XS and parrot, something dammed from the start because XS is just a way to access the perl 5 internals directly. A high level API hidden the implementation details is completely missing.

    Corollary: anybody wanting to advance perl 5 seriously should forget about XS compatibility. It is a too heavy to carry stone.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1106882]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2021-02-28 22:48 GMT
Find Nodes?
    Voting Booth?

    No recent polls found