http://qs321.pair.com?node_id=236849

I found this interesting:
open the good and bad log files with Excel (take the defaults on the import wizard) and delete the first 3 columns (otherwise, the compare will show up every line as different, since the first 3 columns contain information that is different from run to run, such as the time and the process id). Finally, compare the resulting log files (for example with WinDiff
(from THE SYSINTERNALS NEWSLETTER http://www.sysinternals.com)

He massaged his text logs into usable form using a spreadsheet! Instead of running a text utility to pipe input and output, parsing out the columns, and rearranging them, deleting them, sorting the rows, or whatnot, he used the spreadsheet's ability to parse the text and simple features of the grid-like system to manipulate rows and/or columns. I never would have thought of that. Maybe that's inspirational, and I'll remember that next time I need to sort on something other than the leftmost characters.

—John

Replies are listed 'Best First'.
Re: How some people live without Perl
by djantzen (Priest) on Feb 20, 2003 at 03:16 UTC

    It was a humbling moment several years ago when, after writing a script to parse network latency test logs and generate a cute little histogram, my coworker simply copy-and-pasted the data into Excel, generating in no time flat far better graphs and stats than could my code. Perl is great, but it isn't the tool for every task.


    "The dead do not recognize context" -- Kai, Lexx
      True, but this ease of use can evaporate once you need to automate the process. Once you're through with the Windows task scheduler and OLE, etc, the effort might end up being the same. (I could be displaying my ignorance here, I'm not really much of a Windows sort -- feel free (anyone) to correct my impressions on this)

      You are absolutely spot on that we need a better graphing utility than the GD based solutions floating around. Excel rendering kicks their butt.

      Matt

Re: How some people live without Perl
by steves (Curate) on Feb 20, 2003 at 02:55 UTC

    One big adjustment for me at the current job is the heavy use of spreadsheets. Once you see a non-programmer type whip up some formulas and slice and dice you see why. The great upside of this for me is that Perl has great Excel support. I've integrated spreadsheet read/write/filtering into a Perl-based toolset we use using Spreadsheet::ParseExcel and Spreadsheet::WriteExcel and the results have been great.

Re: How some people live without Perl
by osfameron (Hermit) on Feb 19, 2003 at 22:40 UTC
    Yes! My 2 main tools for columnar data-munging are Excel and Perl.

    Of course, my ideal tool would be a spreadsheet with embedded Perl used for formulas, formatting (sprintf and other transforms), and as the internal macro language...

    Cheerio!
    Osfameron
    http://osfameron.perlmonk.org/chickenman/

      Sounds like it would be simple to write a spreadsheet in Perl/Tk that basically did that.
        Yes, it does sound simple...

        But I don't know enough about Tk. I did start trying to mock something like this up some time ago but I couldn't work out which Tk widget best represented that sort of grid...

        Cheerio!
        Osfameron
        http://osfameron.perlmonk.org/chickenman/

      I suppose you could put Perl code into formulas using Excel itself, and then use Spreadsheet::Excel and eval any non-Excel formulas.

      ----
      Reinvent a rounder wheel.

      Note: All code is untested, unless otherwise stated

Re: How some people live without Perl
by castaway (Parson) on Feb 20, 2003 at 07:24 UTC
    I suspect the advice is meant for more or less 'normal' Windows Admins, and I don't see a lot of them even knowing that Perl exists, let alone installing it on Windows machines ;)
    Just goes to show that having broad knowledge of tools and languages can be a lot more helpful than knowing just one language in depth.
    (Was it me, I'd probably have done it with Excel as well, automating with VBA to include the actual diffing..)

    C.
    (Hmm, is 'diffing' a word? ;)

Re: How some people live without Perl
by Aristotle (Chancellor) on Feb 20, 2003 at 14:24 UTC

    I see the point for non-programmer types, but I'm not very impressed, to be honest. If you know your way around the Unix toolset, such tasks will be more or less trivial even without Perl (and hardly any easier with it).

    Maybe it's my bias showing here, but I despise GUIs for any sort of largescale data munging.

    Another recommendation I can give you is to get to grips with Vim - it has so many and so powerful facilities to edit text (including a Perl binding in the recent version that enables you to let loose Perl commands on your text right inside your editor) that it can be used for just about any one-off data munging task that's relatively simple but too irregular to tackle with the usual tools.

    Makeshifts last the longest.

(jeffa) Re: How some people live without Perl
by jeffa (Bishop) on Feb 20, 2003 at 14:11 UTC
    One of my favorite Excel features is the 'Web Query' thingy. It's like LWP in that it will fetch a remote web page, but it actually 'tabulates' the contents. It is a very easy task to grab a page that contains tabular data and export the results to CSV. Not bad. Most of the time i'll use gnumeric if i need a spreadsheet. I haven't confirmed if it has this capability or not, though. If it does, kiss my Excel days goodbye. ;)

    jeffa

    L-LL-L--L-LL-L--L-LL-L--
    -R--R-RR-R--R-RR-R--R-RR
    B--B--B--B--B--B--B--B--
    H---H---H---H---H---H---
    (the triplet paradiddle with high-hat)
    
Re: How some people live without Perl
by iguanodon (Priest) on Feb 20, 2003 at 03:41 UTC
    Don't get me wrong, I like to bash Microsoft as much as the next guy :). But my dirty little secret is that I like Excel. And I love VLOOKUP.

Re: How some people live without Perl
by jdporter (Paladin) on Feb 21, 2003 at 06:04 UTC
    Then, of course, you could what I would do, which is to automate Excel from perl, using OLE. You could so easily stuff the data into Excel from perl, tell Excel what to do (delete columns, sort, etc.), and suck the data back, all programmatically.

    jdporter
    The 6th Rule of Perl Club is -- There is no Rule #6.

      Or do what I do and implement perlish things in VB and get the job done natively.

      Added You'd think people would glark from context that perl is my preferred language. Its just not what I get to use at work so I spend some time creating support perlisms in VB and get my job done easier and faster. And its not that I don't know how the VB-way would have me do it — I just happen to know that by using some sprinklings of perl-thought the resultant code will be clearer and not the ugly gobs it'd be otherwise. Sheesh. Its not as if I actually said "go use VB" or anything.


      Seeking Green geeks in Minnesota

Re: How some people live without Perl
by demerphq (Chancellor) on Mar 07, 2003 at 00:09 UTC

    One of my great regrets is that you cant use Perl as Excels macro language. Having said that, despite the fact that I used to do a lot of Excel VBA macros, I have found that when the job gets gritty Perl wins hands down. But for a quick and dirty I still use it. :-)


    ---
    demerphq


      One of my great regrets is that you cant use Perl as Excels macro language.
      But you can...

      Update: I'm posting the text of that link here, because I think its valuable to have in more than one place: