http://qs321.pair.com?node_id=268678


in reply to Re: Removing repeated lines from file
in thread Removing repeated lines from file

sigh, ++ to Anonymous

if you have cool text tools (some sort/uniq/cut are really lame) it's even easier to accomplish

$ cat -n config.log | head 1 This file contains any messages produced by compilers while 2 running configure, to aid debugging if configure makes a mista +ke. 3 4 It was created by configure, which was 5 generated by GNU Autoconf 2.57. Invocation command line was 6 7 $ ./configure 8 9 ## --------- ## 10 ## Platform. ## # lines 3,6,8 are the duplicates in this little test. # fields are tab seperated. $ cat -n config.log | head | # number the lines sort -k 2 | # sort on second field uniq -f 1 | # uniq skipping first field sort -k 1,1n | # numeric sort on first field cut -f2 # extract second field This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by configure, which was generated by GNU Autoconf 2.57. Invocation command line was $ ./configure ## --------- ## ## Platform. ##

Replies are listed 'Best First'.
Re: Re: Re: Removing repeated lines from file
by Notromda (Pilgrim) on Jun 24, 2003 at 22:18 UTC
    Spoken like a true unix fan. :) Simple tools and pipes are cool.