Removing similar lines and repeats

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • perlboy
    New Member
    • Jan 2012
    • 2

    #1

    Removing similar lines and repeats

    gi|282558029|ge ne000415 lga:LGAS_0495 1e-87 100....
    gi|282558029|ge ne000415 lga:LGAS_0495 1e-87 189...
    gi|282558029|ge ne000415 lmo:lmo0426 5e-06 29....
    How to compare lines for similar starting string. I want only first line out of many such lines with similar starting string.
    How to do that?
  • sathishkumar88
    New Member
    • Jan 2012
    • 4

    #2
    use below script for remove duplicate line in file
    $ftmp = 'Name.txt';
    my %match = ();
    {
    local @ARGV = ($ftmp);
    local $^I = '.tmp';
    while(<>)
    {
    $match{$_}++;
    next if $match{$_} > 1;
    print;
    }
    }

    Comment

    • perlboy
      New Member
      • Jan 2012
      • 2

      #3
      Dear sathishkumar88,
      I want to remove different lines having similar starting word. not completely identical lines....thanks for the reply.

      Comment

      • sathishkumar88
        New Member
        • Jan 2012
        • 4

        #4
        please use this one
        $ftmp = 'Name.txt';
        my %match = ();
        {
        local @ARGV = ($ftmp);
        local $^I = '.tmp';
        while (<>)
        {
        $row=$_;
        if($row=~m/([^<]*?)\s/is)
        {
        $word=$1;
        $match{$word}++ ;
        next if $match{$word} > 1;
        print;
        }

        }
        }

        Comment

        Working...