Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: Find duplicate based on specific fields while allowing 2 mismatch

by kcott (Archbishop)
on Aug 28, 2017 at 09:17 UTC ( [id://1198139]=note: print w/replies, xml ) Need Help??


in reply to Find duplicate based on specific fields while allowing 2 mismatch

G'day amitgsir,

There are a number of issues with your post:

  • What does "UMI" mean? I'm guessing that's domain-specific jargon. You should explain this sort of thing in your post or, at the very least, provide a link to a definition.
  • What does "2 mismatch" mean? Perhaps that's meaningful when "UMI" is explained.
  • CLUSTER and DELIMIT:
    • Both are mentioned in your textual description.
    • Only CLUSTER appears in what I assume is your expected output.
    • Only DELIMIT appears in your code.
    • Only DELIMIT appears in your actual output (as a result of the last point).
  • Using package variables for filehandles is a bad choice. Use lexical variables as shown in the open documentation.
  • The token 'DATA' is special to Perl; see "perldata: Special Literals". Using this (or any other special token) in your code for something other than its intended purpose should be avoided. I've used it in my code example below so you can see normal usage.
  • Don't comment out strict and warnings. Leave them in and fix the problems they highlight. If you don't understand the problem, or how to fix it, you can always ask here.
  • I'm pretty sure chomping elements of the @ARGV array is unnecessary. You can test that with something like 'print "|$var|"', which will show terminal newlines (and any other unexpected or unwanted characters). Also note that, in my example code below, I haven't used chomp at all: consider whether removing newlines from input, then adding them back for output, is actually gaining you anything — it's often just a lot of redundant processing.
  • Your code layout leaves much to be desired. Inconsistent indentation, and swathes of unnecessary whitespace, reduce readability. Choose a coding style that suits you and stick to it: "perlstyle - Perl style guide" has tips on how to do this.

The following code produces what I'm guessing is, at least, close to what you want. It produces the output in the same groupings you showshowed (see Update below), and it includes both the CLUSTER and DELIMIT tokens. Given the points I've already made regarding those tokens, this may not be exactly what you want: as stated, this is a guess on my part; modify to suit your needs.

#!/usr/bin/env perl use strict; use warnings; use constant { COL0 => 0, COL2 => 1, ALL => 2 }; my ($col0, $col2) = ('', ''); print map { if ($col0 ne $_->[COL0]) { ($col0, $col2) = @{$_}[COL0,COL2]; ("CLUSTER\n", $_->[ALL]); } elsif (substr($col2, 0, 3) ne substr($_->[COL2], 0, 3)) { $col2 = $_->[COL2]; ("DELIMIT\n", $_->[ALL]); } else { $_->[ALL]; } } sort { $a->[COL0] cmp $b->[COL0] || $b->[COL2] cmp $a->[COL2] } map { [ (split)[0,2], $_ ] } <DATA>; __DATA__ chrM:307 0 AGCGGGGA 129 chrM:307 0 AGCGGGGA 130 chrM:307 0 AGCGGGGA 129 chrM:308 0 AGCGGGGA 129 chrM:308 0 AGCGGGGA 130 chrM:308 0 AGCGGGGA 129 chrM:309 0 AGCGGGGA 129 chrM:309 0 AGCGGGGA 130 chrM:309 0 AGCGGGGA 129 chrM:307 0 TCAAAATG 130 chrM:308 0 TCAAAATG 130 chrM:309 0 TCAAAATG 130 chrM:307 0 TCACGGTG 130 chrM:308 0 TCACGGTG 130 chrM:309 0 TCACGGTG 130 chrM:307 0 TCAGCCTG 129 chrM:308 0 TCAGCCTG 129 chrM:309 0 TCAGCCTG 129 chrM:307 0 TCAGGGAG 130 chrM:308 0 TCAGGGAG 130 chrM:309 0 TCAGGGAG 130 chrM:307 1 TCAGGGTG 106 chrM:307 2 TCAGGGTG 130 chrM:307 2 TCAGGGTG 129 chrM:308 1 TCAGGGTG 106 chrM:308 2 TCAGGGTG 130 chrM:308 2 TCAGGGTG 129 chrM:309 1 TCAGGGTG 106 chrM:309 2 TCAGGGTG 130 chrM:309 2 TCAGGGTG 129

Output:

CLUSTER chrM:307 1 TCAGGGTG 106 chrM:307 2 TCAGGGTG 130 chrM:307 2 TCAGGGTG 129 chrM:307 0 TCAGGGAG 130 chrM:307 0 TCAGCCTG 129 chrM:307 0 TCACGGTG 130 chrM:307 0 TCAAAATG 130 DELIMIT chrM:307 0 AGCGGGGA 129 chrM:307 0 AGCGGGGA 130 chrM:307 0 AGCGGGGA 129 CLUSTER chrM:308 1 TCAGGGTG 106 chrM:308 2 TCAGGGTG 130 chrM:308 2 TCAGGGTG 129 chrM:308 0 TCAGGGAG 130 chrM:308 0 TCAGCCTG 129 chrM:308 0 TCACGGTG 130 chrM:308 0 TCAAAATG 130 DELIMIT chrM:308 0 AGCGGGGA 129 chrM:308 0 AGCGGGGA 130 chrM:308 0 AGCGGGGA 129 CLUSTER chrM:309 1 TCAGGGTG 106 chrM:309 2 TCAGGGTG 130 chrM:309 2 TCAGGGTG 129 chrM:309 0 TCAGGGAG 130 chrM:309 0 TCAGCCTG 129 chrM:309 0 TCACGGTG 130 chrM:309 0 TCAAAATG 130 DELIMIT chrM:309 0 AGCGGGGA 129 chrM:309 0 AGCGGGGA 130 chrM:309 0 AGCGGGGA 129

By the way, the construct I've used in the form of

... map { ... } sort { ... } map { ... } ...

is known as a Schwartzian Transform. It is particularly useful for sorting based on parts of a string while retaining the entire string, in an unmodified form, for eventual output.

Update: The original, expected output has changed. There is no notification of what changed or even that a change has been made. Here's the original (in the spoiler) that my response is based on:

— Ken

Replies are listed 'Best First'.
Re^2: Find duplicate based on specific fields while allowing 2 mismatch
by amitgsir (Novice) on Aug 29, 2017 at 00:51 UTC

    Thanks for your time and sorry I missed to update without keeping spoiler tag.

    I tried to explain the question again with more detail.

    It is interesting to learn the map contruct, I was trying to do with just using Arrays which was I guess not efficient. I am learning more about 'hash' and 'map' using perlmonks suggestion. Thanks!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1198139]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2024-04-20 01:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found