Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Open sourcing perlmonks

by BUU (Prior)
on May 28, 2005 at 02:06 UTC ( [id://461273] : monkdiscuss . print w/replies, xml ) Need Help??

I've noticed recently a fairly large numbers of posts in this dicussion section that mostly boil down to a request for features. Of course none of these get implemented because the people suggesting them don't produce any code to implement the feature and nobody else cares enough to produce the code for them. So if you want a feature to get included, you should write the code for it.

Here's the problem though, people suggesting the improvements can not write the code to implement them because they do not have access to the source code of perlmonks! That's right, perlmonks is effectively closed source. As it stands now, if you actually wanted to write code for a feature, first you have to come up with the idea based purely on using the website, then you have to design it completely, then you have to make a request to some group of people, then you have to convince some of those people that you are a good enough person to have access to the holy source code, then after that you finally get to see the source code and decide if your idea is possible or not.

Am I the only one who thinks this is silly? Isn't the entire point of this site to discuss perl, and isn't perl open source? If open source is good enough for perl, why isn't it good for perlmonks? Is there some kind of technical flaw that prevents easy access to the source code?

The only argument I've ever heard to counter open sourcing software is that closed source some how makes it more secure. That if people can't see the source code, they can't attack the site. Frankly I don't think that argument holds any water. Obviously there are hundreds of programs that get attacked on a daily basis by people who don't have the source code, so hiding it probably doesn't get you anywhere. And of course, if it was open then the actual members of this site, who number hundreds of very skilled programmers in their ranks, could actually examine the source code for errors and fix any problems before they were exploited.

In short, I think open sourcing of perlmonks would give several useful benefits, such as more features, less security holes and bugs in general. I feel that there is ample evidence of open source actually improving software such as linux and perl itself.

(Just a note to anyone who wants to mention the everydevel link at the bottom, yes perlmonks runs on the everything engine, but it runs on a very old and highly modified version of such, so the downloads at everydevel aren't particularly useful.)

Update:
To address a few concerns in the below posts:
A) I never intended that just anyone be able to modify the actual code that is running the site, I agree that would be silly. I'm all for having any patches being vetted by some community, I just think people should have to opportunity to write the patch.

B) It wasn't my intention to accuse the pmdevs of not doing any work, obviously they do a ton of work that is very rarely noticed, but just as obviously if only one person cares about a patch, they're the only people likely to work on the code.

Replies are listed 'Best First'.
Re: Open sourcing perlmonks
by Tanktalus (Canon) on May 28, 2005 at 04:42 UTC

    Hmmm... I think this is a reasonable example - which shows many aspects of where this thread could go, and I get an advantage as the pmdevil who was involved ;-)

    First off, it's incredibly trivial to become a pmdev member. Really. You make it sound difficult - it isn't. It does take some willpower to justify to oneself that you'll do something with it, but it doesn't seem, from looking at the list, that actually producing a patch is a requirement. I'm betting the promise to produce a patch isn't strictly required either.

    What this means is that the source code to PM is already nearly open source - you pretty much just have to ask to gain access. What is unfortunate is that you can't just grab a tarball of the whole thing and play with that, but even the GPL doesn't really say how an interested party may receive the source code, only that interested parties must be able to do so. Signing up for pmdev merely says you're interested. (I'm not saying that PM is GPL'd, either, so they are allowed to say no.)

    The real issues are, based on my limited experience, and a number of /msg's I've received:

    1. A desire to do something. Generally speaking, people fix their own itches. In the commercial world, this means that eating and a roof over one's head is an itch, for which one sells their programming expertise to solve someone else's itch. In the volunteer world, this means that they solve the problems that affect themselves first. Thus, if it itches you, you're the most likely person to scratch it.
    2. An ability to do something. Even once you write the patch(es), you need to convince someone that it's a good, useful patch. There is generally one person who will point out holes, whether that's coding, style, or usability, and block significant numbers of patches from people he doesn't know/trust. (If this is not the case, he's masking it very well, in my experience. As well as a number of others whose names/aliases I won't reveal.)
    The bottom line to me, having overcome the first hurdle, is that certain gods have a very tight hand in keeping changes from going in. At least, that's the perception others will get from dealing with him. I'm actually 100% positive that he believes he's keeping the chaos from grabbing hold of PM, leaving us with an ever-changing (the definition of instability) web site that will drive people away. At least on his usability comments. On his technical comments (code, style), I firmly believe that he knows more of the code than most other pmdevils, thus he probably is right about those anyway. (Although I'll continue to disagree about "unless is confusing".)

    So, on one hand, there are too few pmdevils submitting patches, yet on the other, there is a subconscious effort to stymy patches by anyone other than a select few (who generally can apply their own patches). It's the latter which demoralises the former, and it ends up in a violent circle which I cannot offer advice on how to fix.

      Tanktalus I feel that I owe you an explanation. As you know your vote patch has been applied for some time in a test configuration. As I am one of the people who experiences its effect I should say why I havent promoted it to a full patch: I am not sure if the patch is healthy for the site. For mature users seeing the downvotes is nice. But i find myself grumbling about certain things I see which makes me think that others will go far beyond grumbling. Its for this reason that I have been reluctant to promote the patch.

      Also you need to understand that this glacial pace (which I know to be frustrating) also effects stuff the gods themselves have written. I have written things that will never see the light of day here, I have written things that are in production but remain totally undocumented and thus for all intents and purposes non-existant. There are various reasons for this but who posted the patch is NOT one of them, being a god here only slightly raises your chance that a non-trivial patch will be accepted by the community as appropriate and needed.

      Basically I see it like this, you can break patches down into three classes:

      1. Controversial -- Patches that touch on controversial subjects like voting and experience. These will always be difficult patches to get into production just because of their potential to stir up trouble.
      2. Infrastructure -- Anything that is heavily used or involves multiple patches and table updates is going to take a while to get into production. Its difficult and time consuming to apply and test patches like this, and to do things like this properly takes considerable care and understanding of the sites behaviour and design to do right. Infrastructure changes also require heavy testing, and for one reason or another we still have problems in this area.
      3. Minor Feature or Enhancement -- These usually go in almost instantly. Assuming it doesnt fall into one of the other categories generally all this takes is to make a patch and ask somebody to apply it. Its quite possible that the code will be totally rewritten prior to being applied however the presence of an initial patch almost always results in something equivelent being applied. Cabal Matrix is an example of this. Arunbear had the idea, and put together an initial implementation, ysth rewrote part of it, and i added some more changes as well. But it got applied quite quickly.

      Lastly there is one thing that a patch producer needs to do: champion the patch and idea the patch represents. The gods are human, with various interests and responsibilities meaning we are just as forgetful as the next guy. If we havent applied your patch it may not be because of any particular reason it may simply be because we have forgetten that there is a patch that needs to be applied. Reminding us through /msg's or through PMD's about the proposed change can really help. With controversial nodes, PMD's discussing the state of the patch and the issues it raises can help the community come to a collective decision about the proposal.

      Anyway, the fact is that most patches that I see remain unapplied are in the first two categories. For some reason many peoples first patch tends to fall into one of the two (most often the reason being they are "controversial" patches). Also a potential patcher needs to understand that rejecting patches is part of our job as gods. However i personally beleive that the person posting the patch is owed an explanation as to why, and if one isnt forthcoming I beleieve the poster should just ask.

      ---
      $world=~s/war/peace/g

        Darn. Seems like I should have been more explicit. ;-)

        For starters, I am not looking for an explanation - I'm looking for a pmdev-guide (tye assures me there is one, but I've gone back in the wiki a fair bit without finding one or a link to one). One that explains what is expected from those in the pmdev, one that explains the rules of the gods on applying patches, one that shows this relationship. One that is also dynamic - as the rules are changed (whether evolution or revolution), somewhere for the appropriate god to update. One that can explain to the non-pmdevil what to expect should s/he put his/her name in the hat. Because I know at least one pmdevil who is yet to know what it's like to submit a patch; because I know I didn't really know what to expect when I offered to write that patch.

        This would also help the gods who are less than becoming in their manner or style of rejection. It's much less personal to point at a prewritten, yet living and dynamic, document than to gruffly say (paraphrasing here) "I don't like it, I'm not approving it." It will help promote writing patches because it gives a guideline to all. You get better behaviour out of people when they can predict, with 100% accuracy, the consequences of their actions. In this case, if the pmdevil can predict, with 100% accuracy, whether their patch will be accepted or not, they can then start producing acceptable patches. Or they can know whether to expect some sort of reticence, and provide reasonable explanation along with their patch.

        This should be a document that is referred to often - to the point where it's a common-knowledge document, like How (Not) To Ask A Question seems to be. Rather than pointing petitioners such as bofh_of_oz or artist (or even myself) to become pmdevils themselves to write and submit patches, they should be referred to this document, just as we point people who post bad questions to jeffa's How (Not) To Ask A Question. Because, as should be evident by the small number of people who are actually becoming pmdevils vs merely complaining about feature requests, we can't read the gods' minds no matter how much you may want us to.

        Now, as for actually initiating this document, yeah, yeah, become a member of the SDC and write it myself. I know. :-)

        On to a slight change in topic - below you wrote:

        Once this has been done reviewing the code is fairly easy and IMO mostly intuitive.
        I find this very entertaining. :-) To be honest, intuitive is only because you've been doing it a long time. I find the 15,000-line behemoth I designed/wrote at work to be fairly intuitive, too. The poor guy who is taking it over from me isn't so sure ;-)

Re: Open sourcing perlmonks
by castaway (Parson) on May 28, 2005 at 08:08 UTC
    I won't comment on your call for open sourcing, since it's been discussed already several times.

    But, the accusation that *none* of the proposed features get implemented, is not true. Some do, occasionally, some pmdev will jump right on a feature, that they would also use/think is a great idea, and create a patch for it right away. Usually the suggesting node gets replied to with "I patched this, wait for someone to apply it", or even "Patched and applied".

    Not every suggestion *can* get implemented. Some will just bring little gain while bogging down the server, some will wait on other changes which are still being tested, some will just not fit in to the site as a whole. Of the rest that can, and nobody jumps at, I would guess most of them are significantly complex that our volunteers need a bunch of time. Some of these get started on and not finished, which the person suggesting it will probably never find out about.

    So, having said that, my suggestion to people suggesting site improvements: Help the pmdevs, break down your suggestion into small chunks which are easier/shorter to add, try and think up reasons for and against them before you post, be ready for someone to say "sorry, we just cant do that".

    Ok, I will comment shortly on the other issue. Even if PMs code was available to all, we would keep pmdev group, the patch system and the gods-only-applying system, and thus a stable site. So the only gain you have is that people could try and figure out if their suggestions can be done, before posting them. The code is probably complex enough for that to take them a while, or disuade from even looking, though.

    And no matter what you say about security and obscurity, although it should never be used as the only deterrent, hiding the code *does* give us the advantage that anyone wanting to misuse the site has to try things out, or mine the data that gets sent between their browser and the site, and not just search for the appropriate piece of code in the source.

    C.

Re: Open sourcing perlmonks
by chromatic (Archbishop) on May 28, 2005 at 07:22 UTC
      Yes, people could submit patches. In my experience however, there aren't great masses of people lining up to submit patches to most open source projects.

      True, but putting any barrier in front of the source is going to make it less likely. I know I've been tempted to write some patches for perlmonk on occasion, but have been too lazy to jump through the necessary hoops to get at the source.

        The hoops involved are asking one of gods for membership and then activating several pmdev specific nodelets. And possibly reminding Corion or myself to give you access to the test server. Once this has been done reviewing the code is fairly easy and IMO mostly intuitive.

        ---
        $world=~s/war/peace/g

Re: Open sourcing perlmonks
by adrianh (Chancellor) on May 29, 2005 at 11:34 UTC
    As it stands now, if you actually wanted to write code for a feature, first you have to come up with the idea based purely on using the website, then you have to design it completely

    You don't have to have to have a feature in mind, or design it completely, to join pmdev (having joined pmdev to without any particular project in mind :-)

Re: Open sourcing perlmonks
by jacques (Priest) on May 28, 2005 at 20:08 UTC
    Well, obviously something is wrong. You don't see many other sites using the source code, even in a modified or generic form. And they have been hacking away at the code for years now.

    Personally, I like the perlmonks community, but I think the website just plain sucks from a design point of view.

    There might be several reasons for this, but the inescapable truth is:

    Perl hackers are some of the worse web designers in the world.

    The Perl community is infamous for having poor websites. Some of these poor sites have been recently updated. Take for example, www.pm.org www.perl.org. And I could single out the personal websites of many Perl programmers, including one famous one in particular, but I won't do that.

    There is such a thing as a professional web designer, a person who is highly skilled in the art of webpage design. It is an art and a profession. But it has been my experience that this profession is not very respected by systems administrators and programmers.

    Knowing basic HTML and being a life long systems administrator, who also happens to know a few things about Perl, doesn't make you a good web designer. System admins make the _worse_ web designers, but they just love designing crappy websites just to show that they can do it.

    If this website is to improve, the people in charge (and yes, there are people in charge, so don't give me any of that egalitarian crap) need to step aside, but that's not going to happen anytime soon IMHO. This is their baby and they have already demonstrated that they have a nasty case of separation anxiety.

    While I would like to see things improve, I have learned to enjoy the PM community and not to let the details bother me.

      The Perl community is infamous for having poor websites. Some of these poor sites have been recently updated. Take for example, www.pm.org. And I could single out the personal websites of many Perl programmers, including one famous one in particular, but I won't do that.

      Of course you won't. It's much easier to throw out vague complaints without actually doing anything about them.

      This is their baby and they have already demonstrated that they have a nasty case of separation anxiety.

      You're welcome to think that, but you're wrong. I've seen lots of patches applied to add CSS support, improve XML support, and put id attributes on all sorts of tags, making the resulting HTML more semantically useful.

      That doesn't mean things are perfect, but the system works and, as I said earlier, I haven't seen a flood of people knocking down the doors submitting patches to improve things. Thus, the imperfect system stays in place, a few people here and there jump up and down and say "Oooh, Ooh, I want to help!" and disappear a week later, and, every few months, someone complains that there's no legion of highly-paid volunteers working on the site full time to implement the shiny new feature he or she just invented.

      Take my personal web site, for example. It's not very attractive, but every day someone reads one of my talks or downloads some of my software. Mission accomplished.

      Update: Struck out a phrase I wish I hadn't used.

        It's not only what chromatic says, but that of all the major Perl websites out there, there are actually very few people involved with the design. The same people seem to do all of the work, and as chromatic says, there are plenty of volunteers whom we never hear from twice.

        People can blame me it they like, because www.pm.org, www.perl.org, and your "one famous person" website were or are my fault. It has nothing to do with Perl programming though: non-designers tend to suck at design no matter what they do.

        --
        brian d foy <brian@stonehenge.com>
        Of course you won't. It's much easier to throw out vague complaints without actually doing anything about them.

        No, I don't mention personal websites, because that would be silly. I probably shouldn't have brought them up in my OP.

        I haven't seen a flood of people knocking down the doors submitting patches to improve things.

        I see that as a failure in the system. Since this site is very popular, why isn't there that flood?

        and disappear a week later

        See above.

      Personally, I like the perlmonks community, but I think the website just plain sucks from a design point of view.

      Ah so. What is it that sucks? IMO its a lot easier saying that than actually offering a detailed review and critique that pmdev can use to improve things.

      I'd like to know what you consider to be good design. You mentioned a link in one of your other sites that I followed, but the page doesnt seem to load properly. Is that what you consider good design? Frankly a lot of sites that ive been pointed at by web designers as example of good stuff have themselve been really pretty, and basically useless. PM otoh, is not entirely pretty, (although given how customizable the user experience is this is only an issue for the anony monk), it is quite functional. Super Search works well unlike most sites search features. We provide XML feeds of just about everything we can think of, and seem to be able to provide an enviornment where programmers beginner and experienced can find a place to contribute or learn from. What exactly is it that we doing so wrong?

      ---
      $world=~s/war/peace/g

        Super Search does some things quite well but there are quite a few things that many people reasonably expect from a web search that Super Search mostly sucks at.

        Several things could be improved with some fairly simple features (like being able to build up the list of matches). But the problem space is rather strangely and somewhat severely constrained and so many things will never be done well by Super Search.

        But making PerlMonks robot-friendly will address many of those areas (because google is quite good at one common type of web search, especially if we define a few key "words"...)

        And Super Search is good at searching for punctuation, which is a rare treat for me; most searches suck at that, but then I wrote it so there should be something about it I like.

        This is all rather vague but most of it is covered in other nodes and I'm not going to spend more than a couple of minutes on this so you'll have to use it if you're curious.

        So there is much I like about it, it usually works quite well for me, I'm glad it works well for you, it treats the servers nicely, and I realize many of its significant faults and have plans for improvements for some and alternatives for others. But right now, there are several ways in which it sucks, and not just a little. (:

        - tye        

A reply falls below the community's threshold of quality. You may see it by logging in.