|
|
|
| Keep It Simple, Stupid | |
| PerlMonks |
Re: "Magic tools" that take the fun away (Releng/DevOps/Cloud/Server References)by eyepopslikeamosquito (Archbishop) |
| on Apr 10, 2021 at 09:29 UTC ( [id://11131082]=note: print w/replies, xml ) | Need Help?? |
|
I've had more time to reflect on your thought-provoking meditation. Let me start with some rambling anecdotes from twenty years ago. Around the year 2000, I was a keen C++ developer starting in a new company. Young (well, younger) and eager to make my mark. Our C++ software had to build and run successfully across many different Unix and Windows flavours. This was back when Cloud Computing was not talked about and, like everyone else back then, we owned our own set of dedicated on-premise build machines. We owned our own on-premise test lab too. :) A recent hire to our (newly formed) build team had just received a gold star from management for "successfully" implementing a brand new build system. On time and on budget. Only took a few months to develop. On Windows, he had crafted a truly horrifying .BAT script. Thousands of lines. Opening it in Notepad made my blood run cold. Oh the horror! :) This .BAT monstrosity drove our Unix builds from Windows via a telnet session (we even bought a commercial telnet product to support this approach!) that ran many Unix commands to build, test, and package our software. Back then, our Unix builds took many hours to complete. I would fire off his .BAT masterpiece from my PC every night ... only to arrive keen and early in the office the next morning (no working from home back then) to discover that my build had failed, due to a broken telnet connection. Every single time. Had to start a fresh build and manually nurse it through the day. I couldn't get my work done! Being young and reckless - and desperate to get my work done - I implemented a brand new build system over the weekend (written in Perl) that used remote shell, not telnet. No need to hold a telnet connection open for hours on our unreliable network. Instead it created a source tarball locally, ftp'ed it to each Unix box, then fired off a Perl script via remote shell to unpack the tarball, perform the build, run the tests and package our product. No need for a long-lived connection, simply check your build status occasionally via a rsh command to tail the build log. As you might expect, all developers switched to my system ... though I wasn't awarded a gold star from management. Though that was fun for me, I doubt it was fun for our gold-star winning .BAT maestro build engineer. Hardly an example of great teamwork. BTW, we've improved our communication since then; communicate early, often, and with trust. Lots of eye-gouging to be sure, but it usually results in everyone pulling in the same direction, as described at Conflict in Teams. Around this time, I remember developing a strong dislike of Unix make, forming the opinion that maintaining millions of lines of code built with the make build tool inevitably leads to misery. You need a real programming language (not a toy DSL) to deal effectively with huge complex builds. Sadly, I never did much about this, just suffered in silence, year after year. My only consolation is, after googling today, I see I'm not alone in my suffering. You see, way back in 1999, a mysterious unsung Perl hero called Bob Sidebotham lovingly crafted an award-winning Perl-based Cons build system. From this tpj article: Understandably, many Perl programmers don't like having to program in low-level languages like C and C++. Even more irritating is writing the Makefiles necessary to compile and install those programs with the Unix make utility. Worse still is being saddled with maintaining someone else's Makefiles. Perhaps the most egregious aspect of make is the make language itself. (In fairness, make has been around for a long time, predating even Perl. The standard for scripting in the Unix community at the time of make's emergence was the Bourne shell, and make represented a considerable advance over the then-common practice of using shell scripts to build software.) As a scripting vehicle, Makefiles are really just glorified collections of macros with no flow of control primitives, no complex substitution rules, no variable scoping, and poor control over import/export of environment variables and command arguments. If you want proof that make is inadequate, just look at the plethora of utilities that people use to avoid writing their own Makefiles, like MakeMaker and imake. I could provide a long list of the shortcomings of the make language, but I'd be surprised if any Perl programmer needs much persuasion.Thanks Bob! I can definitely relate to that! Sadly though, Cons seems to be in hibernation today. Rubbing salt into the wounds, it appears to have been superseded by a Python-based version called SCons: SCons software history started with the Cons software construction utility created by Bob Sidebotham in 1999. Cons was written in the Perl language. It served as a base for the ScCons build tool, a design which won the Software Carpentry project SC Build competition in August 2000. ScCons was the foundation for SCons. The long list of "Make-incompatible" build tools at handwiki suggests a lot of other people share my opinion on this. Though I now regret not doing more about this twenty years ago (rather than suffering in silence) it seems that reaching consensus on which "make-incompatible" build tool to use may prove problematic because of resistance to "having" to learn Perl (or Python or Ruby) just to work on your build system. Maybe that is why Cons lies dormant today. Of course, the software landscape has changed dramatically since then. Creating complex multi-machine test environments in the Cloud (using AWS in my case) has become astonishingly easy compared to twenty years ago in our good old on-premise test lab ... and BTW, astonishingly easy to run up huge AWS bills if you're careless and don't have proper cost controls in place. :) Though I agree with you that using these new DevOps tools and Banal Configuration Languages is not as enjoyable as crafting beautiful code in a real programming language, avoiding them nowadays seems problematic. I did notice one glimmer of hope for Perl in the DevOps space when I googled today and stumbled upon Rex, an open source remote execution, configuration management and software deployment tool that combines Perl and Secure Shell (SSH) for a portable, centralistic approach to its problem domain. Super-searching reveals that at least one Perl monk, namely nysus, is actively using it today (see Rex References below). Update: GNU parallel is another SysAdmin tool written in Perl (see GNU Parallel References below). General References
DevOps References
Cloud References
Build and Test Automation References
Cons References
Server, Operations, Administration and Management References
Issue Tracking/Version Control References
Rex References
GNU Parallel References
Other References
See Also
Updated: Dec-2021: Added GNU parallel references (mentioned here) - a Unix command line tool, written in Perl, that allows the user to execute shell scripts or commands in parallel. See also pexec and xargs and GNU Queue originally by Werner G Krebs (not heavily used anymore?). Sep 2023: Added "Build and Test Automation References" section.
In Section
Meditations
|
|
||||||||||||||||||||||||||