python script as an emergency mailbox cleaner

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Alex Martelli

    python script as an emergency mailbox cleaner

    All my mailboxes have been filling up with files of about 130k to 150k, no
    doubt copies of some immensely popular virus. So, I've no doubt lost lots
    of real mail because of "mailbox full" conditions (the proliferating fake
    bounce messages more or less ensure nobody knows their mail to me has
    bounced, either).

    As an emergency response I and Anna developed, over the last half hour, a
    small Python script to be run from cron every few minutes and automatically
    scrub any POP3 mailbox from files in the target size range. I'm saving them
    to a local file for potential later perusal, but that's obviously easy to
    comment out if needed. Here's the tiny script in question...:

    import poplib
    import time

    print 'Start at', time.asctime()

    host = 'pop.mail.yahoo .com'
    port = 110
    user = 'aleaxit'
    pasw = 'secret'

    logfilename = 'bigjunk'
    minsize = 130000
    maxsize = 180000
    fromtag = 'From aleaxit@yahoo.c om %s\n'

    ps = poplib.POP3(hos t, port)
    ps.user(user)
    ps.pass_(pasw)

    messages = ps.list()
    print '%d messages, %d bytes' % (len(messages[1]), messages[-1])

    logfile = open(logfilenam e, 'a')

    for sms in messages[1]:
    sid, ssize = sms.split()
    if minsize <= int(ssize) < maxsize:
    message = ps.retr(sid)
    print 'retrieving and deleting msg#%s, %d bytes, %d lines' % (
    sid, message[-1], len(message[1]))
    logfile.write(f romtag % time.asctime())
    for line in message[1]:
    logfile.write(l ine)
    logfile.write(" \n")
    logfile.write(' \n')
    ps.dele(sid)

    ps.quit()

    print 'Done at', time.asctime()
    print


    Hope it can come in useful to somebody...!!!

    Alex & Anna

  • Phil Weldon

    #2
    Re: python script as an emergency mailbox cleaner

    It's a worm. Worm.Automat.AG H. This is going to be a bad one. The worm
    installs, among other things, an SMPT engine, searches an infected system
    for email address, and sends two types of e-mail: the first is HTML and is
    a fake "security patch" supposedly from Microsoft. It looks very official,
    but the attachment, 104 KBytes long, is infectious. Norton Antivirus
    definitions only began to identify it with the 18SEP03 manual definition
    update. The worm also posts to usenet newsgroups. The other type of e-mail
    is a fake notification of undeliverable e-mail. This one is a real bear.
    There seem to be hundreds variations in the body content and thousands of
    variations in the header. The infectious package is also about 104 KBytes.
    I'm getting nearly 100 of the two types per hour. Norton Antivirus does not
    detect the worm in usnet posts read by Outlook Express Newsreader or Outlook
    Newsreader. Only when you attempt to open the attachment or save the
    attachment to disk will Norton identify it. Norton will NOT detect the
    virus in the newsgroup posts folder NOR will it detect the newsgroup folder
    in a full system scan. It will not remove the infected file from the
    newsgroup folder, but it will prevent execution of the vermal payload.

    Microsoft Outlook with the SP3 security update when used as your e-mail
    reader protects against infection. Prior to 18SEP03 Norton did not.

    The worm is also retrieving additional variations, so you can expect the
    payload size to begin changing soon. The HTML message is easy to identify;
    it is always the same (so far), and includes the phrase 'Run attached file'.
    The bogus 'Undeliverable e-mail' variations have no commonality but the
    payload attachment (that purports to be your bounced e-mail.) This will
    likley change soon.

    My guess is that the internet will not open on Monday.

    Phil Weldon, pweldon@mindspr ing.com

    "Alex Martelli" <aleax@aleax.it > wrote in message
    news:G6Zab.1273 13$R32.3854946@ news2.tin.it...[color=blue]
    > All my mailboxes have been filling up with files of about 130k to 150k, no
    > doubt copies of some immensely popular virus. So, I've no doubt lost lots
    > of real mail because of "mailbox full" conditions (the proliferating fake
    > bounce messages more or less ensure nobody knows their mail to me has
    > bounced, either).
    >
    > As an emergency response I and Anna developed, over the last half hour, a
    > small Python script to be run from cron every few minutes and[/color]
    automatically[color=blue]
    > scrub any POP3 mailbox from files in the target size range. I'm saving[/color]
    them[color=blue]
    > to a local file for potential later perusal, but that's obviously easy to
    > comment out if needed. Here's the tiny script in question...:
    >
    > import poplib
    > import time
    >
    > print 'Start at', time.asctime()
    >
    > host = 'pop.mail.yahoo .com'
    > port = 110
    > user = 'aleaxit'
    > pasw = 'secret'
    >
    > logfilename = 'bigjunk'
    > minsize = 130000
    > maxsize = 180000
    > fromtag = 'From aleaxit@yahoo.c om %s\n'
    >
    > ps = poplib.POP3(hos t, port)
    > ps.user(user)
    > ps.pass_(pasw)
    >
    > messages = ps.list()
    > print '%d messages, %d bytes' % (len(messages[1]), messages[-1])
    >
    > logfile = open(logfilenam e, 'a')
    >
    > for sms in messages[1]:
    > sid, ssize = sms.split()
    > if minsize <= int(ssize) < maxsize:
    > message = ps.retr(sid)
    > print 'retrieving and deleting msg#%s, %d bytes, %d lines' % (
    > sid, message[-1], len(message[1]))
    > logfile.write(f romtag % time.asctime())
    > for line in message[1]:
    > logfile.write(l ine)
    > logfile.write(" \n")
    > logfile.write(' \n')
    > ps.dele(sid)
    >
    > ps.quit()
    >
    > print 'Done at', time.asctime()
    > print
    >
    >
    > Hope it can come in useful to somebody...!!!
    >
    > Alex & Anna
    >[/color]


    Comment

    • John Roth

      #3
      Re: python script as an emergency mailbox cleaner


      "Phil Weldon" <pweldon@mindsp ring.com> wrote in message
      news:ZCZab.4545 0$Aq2.39773@new sread1.news.atl .earthlink.net. ..[color=blue]
      > It's a worm. Worm.Automat.AG H. This is going to be a bad one. The worm
      > installs, among other things, an SMPT engine, searches an infected system
      > for email address, and sends two types of e-mail: the first is HTML and[/color]
      is[color=blue]
      > a fake "security patch" supposedly from Microsoft. It looks very[/color]
      official,[color=blue]
      > but the attachment, 104 KBytes long, is infectious. Norton Antivirus
      > definitions only began to identify it with the 18SEP03 manual definition
      > update. The worm also posts to usenet newsgroups. The other type of[/color]
      e-mail[color=blue]
      > is a fake notification of undeliverable e-mail. This one is a real bear.
      > There seem to be hundreds variations in the body content and thousands of
      > variations in the header. The infectious package is also about 104[/color]
      KBytes.[color=blue]
      > I'm getting nearly 100 of the two types per hour. Norton Antivirus does[/color]
      not[color=blue]
      > detect the worm in usnet posts read by Outlook Express Newsreader or[/color]
      Outlook[color=blue]
      > Newsreader. Only when you attempt to open the attachment or save the
      > attachment to disk will Norton identify it. Norton will NOT detect the
      > virus in the newsgroup posts folder NOR will it detect the newsgroup[/color]
      folder[color=blue]
      > in a full system scan. It will not remove the infected file from the
      > newsgroup folder, but it will prevent execution of the vermal payload.
      >
      > Microsoft Outlook with the SP3 security update when used as your e-mail
      > reader protects against infection. Prior to 18SEP03 Norton did not.
      >
      > The worm is also retrieving additional variations, so you can expect the
      > payload size to begin changing soon. The HTML message is easy to[/color]
      identify;[color=blue]
      > it is always the same (so far), and includes the phrase 'Run attached[/color]
      file'.[color=blue]
      > The bogus 'Undeliverable e-mail' variations have no commonality but the
      > payload attachment (that purports to be your bounced e-mail.) This will
      > likley change soon.
      >
      > My guess is that the internet will not open on Monday.[/color]

      So far, I have seen no copies of the worm on usenet. This may be
      the result of my paying $$$ to a good usenet provider (Supernews.)
      Unfortunately, my e-mail provider got the stupid idea that "delete"
      meant "save a complete copy for 14 days just in case you want
      to see it." Most of the stuff is going into two mailboxes that I need
      to clean out manually every two or three hours (they're not my inbox,
      so the POP3 script won't do it.)

      It looks like two worms that just happened to hit at one time,
      doesn't it?

      John Roth



      Comment

      • Ng Pheng Siong

        #4
        Re: python script as an emergency mailbox cleaner

        According to Phil Weldon <pweldon@mindsp ring.com>:[color=blue]
        > The other type of e-mail
        > is a fake notification of undeliverable e-mail. This one is a real bear.
        > There seem to be hundreds variations in the body content and thousands of
        > variations in the header.[/color]

        So far I've caught all of these using this procmail recipe:

        :0 B
        * ^<BR><BR><BR>(< BR>)*Undeliver
        /dev/null

        This scans the body for a string matching the regex in the second line. "*"
        is a delimiter, not part of the regex.

        The first several hundred I got had 3 <BR>'s, then it started coming with
        4.

        Similarly, the fake MS update first said "September 2003 Cumulative Patch",
        then after several hundred I started to see "July 2003". I just saw one
        "January 2000". The thing feels like it is mutating in response to filters!
        [color=blue]
        > My guess is that the internet will not open on Monday.[/color]

        Civilisation is under attack.


        --
        Ng Pheng Siong <ngps@netmemeti c.com>

        http://firewall.rulemaker.net -+- Manage Your Firewall Rulebase Changes
        http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL

        Comment

        • Phil Weldon

          #5
          Re: python script as an emergency mailbox cleaner

          No, it is only one worm with multiple methods of attack.

          #1. The fake 'security update' announcement purporting to be from Microsoft
          (this e-mail is in HTML) and that includes an infected attachment. If you
          don't have protection, opening this e-mail runs the attachment even if you
          don't run it. So far the body of this vector doesn't vary, though the
          header information does.

          #2. The bogus 'Undeliverable e-mail' message in which everything seems to
          vary except, so far, the infected attachment that purports to be your
          bounced e-mail.

          #3. The worm scans PtoP file-sharing data to spread further.

          #4. The worm hijacks servers to act as a source from which to download
          packages to vary the infectious e-mail.

          #5. The worm can post to usenet. Here's the header of a post that appeared
          this morning on alt.comp.periph s.mainboard.abi t

          Path:
          newsspool1.news .atl.earthlink. net!stamper.new s.atl.earthlink .net!elnk-atl-nf
          1!newsfeed.eart hlink.net!newsh osting.com!news-xfer2.atl.newsh osting.com!prox
          ad.net!proxad.n et!news-hub.cableinet.n et!blueyonder!i nternal-news-hub.cablei
          net.net!news-binary.blueyond er.co.uk.POSTED !53ab2750!not-for-mail
          FROM: "Clive Skingle" <mufchfohaauqze _qset@ykmke.com >
          NEWSGROUPS:
          alt.comp.lang.p hp,alt.comp.mai l.postfix,alt.c omp.mail.qmail, alt.comp.malays i
          a,alt.comp.peri phs.cdr,alt.com p.periphs.mainb oard.abit,alt.c omp.periphs.mai n
          board.asus,alt. comp.periphs.ma inboard.gigabyt e,alt.comp.peri phs.videocards. a
          ti
          SUBJECT: Watch this critical update from the M$
          Mime-Version: 1.0
          Content-Type: multipart/mixed; boundary="hilgh fawbnhbqydk"
          Lines: 2182
          Message-ID: <3RWab.214$_z.9 @news-binary.blueyond er.co.uk>
          Date: Sat, 20 Sep 2003 11:27:59 GMT
          NNTP-Posting-Host: 82.32.208.238
          X-Complaints-To: abuse@blueyonde r.co.uk
          X-Trace: news-binary.blueyond er.co.uk 1064057279 82.32.208.238 (Sat, 20 Sep
          2003 11:27:59 GMT)
          NNTP-Posting-Date: Sat, 20 Sep 2003 11:27:59 GMT
          Organization: blueyonder (post doesn't reflect views of blueyonder)
          Xref: news.earthlink. net alt.comp.lang.p hp:12068 alt.comp.mail.q mail:7299
          alt.comp.malays ia:17767 alt.comp.periph s.cdr:403271
          alt.comp.periph s.mainboard.abi t:567500
          alt.comp.periph s.mainboard.asu s:661135
          alt.comp.periph s.mainboard.gig abyte:41619
          alt.comp.periph s.videocards.at i:145487
          X-Received-Date: Sat, 20 Sep 2003 04:28:01 PDT
          (newsspool1.new s.atl.earthlink .net)

          Phil Weldon, pweldon@mindspr ing.com


          "John Roth" <newsgroups@jhr othjr.com> wrote in message
          news:vmorgph6n3 lg12@news.super news.com...[color=blue]
          >
          > "Phil Weldon" <pweldon@mindsp ring.com> wrote in message
          > news:ZCZab.4545 0$Aq2.39773@new sread1.news.atl .earthlink.net. ..[color=green]
          > > It's a worm. Worm.Automat.AG H. This is going to be a bad one. The[/color][/color]
          worm[color=blue][color=green]
          > > installs, among other things, an SMPT engine, searches an infected[/color][/color]
          system[color=blue][color=green]
          > > for email address, and sends two types of e-mail: the first is HTML and[/color]
          > is[color=green]
          > > a fake "security patch" supposedly from Microsoft. It looks very[/color]
          > official,[color=green]
          > > but the attachment, 104 KBytes long, is infectious. Norton Antivirus
          > > definitions only began to identify it with the 18SEP03 manual definition
          > > update. The worm also posts to usenet newsgroups. The other type of[/color]
          > e-mail[color=green]
          > > is a fake notification of undeliverable e-mail. This one is a real[/color][/color]
          bear.[color=blue][color=green]
          > > There seem to be hundreds variations in the body content and thousands[/color][/color]
          of[color=blue][color=green]
          > > variations in the header. The infectious package is also about 104[/color]
          > KBytes.[color=green]
          > > I'm getting nearly 100 of the two types per hour. Norton Antivirus does[/color]
          > not[color=green]
          > > detect the worm in usnet posts read by Outlook Express Newsreader or[/color]
          > Outlook[color=green]
          > > Newsreader. Only when you attempt to open the attachment or save the
          > > attachment to disk will Norton identify it. Norton will NOT detect the
          > > virus in the newsgroup posts folder NOR will it detect the newsgroup[/color]
          > folder[color=green]
          > > in a full system scan. It will not remove the infected file from the
          > > newsgroup folder, but it will prevent execution of the vermal payload.
          > >
          > > Microsoft Outlook with the SP3 security update when used as your e-mail
          > > reader protects against infection. Prior to 18SEP03 Norton did not.
          > >
          > > The worm is also retrieving additional variations, so you can expect the
          > > payload size to begin changing soon. The HTML message is easy to[/color]
          > identify;[color=green]
          > > it is always the same (so far), and includes the phrase 'Run attached[/color]
          > file'.[color=green]
          > > The bogus 'Undeliverable e-mail' variations have no commonality but the
          > > payload attachment (that purports to be your bounced e-mail.) This will
          > > likley change soon.
          > >
          > > My guess is that the internet will not open on Monday.[/color]
          >
          > So far, I have seen no copies of the worm on usenet. This may be
          > the result of my paying $$$ to a good usenet provider (Supernews.)
          > Unfortunately, my e-mail provider got the stupid idea that "delete"
          > meant "save a complete copy for 14 days just in case you want
          > to see it." Most of the stuff is going into two mailboxes that I need
          > to clean out manually every two or three hours (they're not my inbox,
          > so the POP3 script won't do it.)
          >
          > It looks like two worms that just happened to hit at one time,
          > doesn't it?
          >
          > John Roth
          >
          >
          >[/color]


          Comment

          • Dennis Lee Bieber

            #6
            Re: python script as an emergency mailbox cleaner

            John Roth fed this fish to the penguins on Saturday 20 September 2003
            08:11 am:
            [color=blue]
            >
            > So far, I have seen no copies of the worm on usenet. This may be
            > the result of my paying $$$ to a good usenet provider (Supernews.)
            > Unfortunately, my e-mail provider got the stupid idea that "delete"
            > meant "save a complete copy for 14 days just in case you want
            > to see it." Most of the stuff is going into two mailboxes that I need
            > to clean out manually every two or three hours (they're not my inbox,
            > so the POP3 script won't do it.)
            >[/color]
            Sounds like the "suspect spam" folder I had active on Earthlink... I
            was getting "mail box full" emails faster than I could empty that
            folder (since it is only accessible via HTTP, and I'm on a dial-up).
            I've essentially had to leave my home system running 24 hours a day
            with email checks every 8 minutes (the dial-up logs off if idle 10
            minutes).

            One would think with the rather common format of the subject/from
            lines (at least for the M$ Patch variations) the ISP could identify
            them as known "spam" (which doesn't count against the 10MB mailbox
            limit).

            I'd sent an email to their support address... only to receive a form
            letter saying they don't accept emails that weren't initiated via their
            web site form... And the response to /that/ was never viewed by a
            person -- some program scanned it, saw the word "spam" and sent another
            form letter specifying they need the full headers of the message
            (singular) I was complaining about. Almost makes me want to rig a
            filter to send ALL of these to their spam evaluation address -- but if
            downloading at ~44K is bad, think of the uplink at ~31K...

            --[color=blue]
            > =============== =============== =============== =============== == <
            > wlfraed@ix.netc om.com | Wulfraed Dennis Lee Bieber KD6MOG <
            > wulfraed@dm.net | Bestiaria Support Staff <
            > =============== =============== =============== =============== == <
            > Bestiaria Home Page: http://www.beastie.dm.net/ <
            > Home Page: http://www.dm.net/~wulfraed/ <[/color]

            Comment

            • James Kew

              #7
              Re: python script as an emergency mailbox cleaner

              "Phil Weldon" <pweldon@mindsp ring.com> wrote in message
              news:ZCZab.4545 0$Aq2.39773@new sread1.news.atl .earthlink.net. ..[color=blue]
              > My guess is that the internet will not open on Monday.[/color]

              Yeah, well, death of the Internet's been predicted many times before and
              hasn't happened yet.

              F-Secure's writeup is pretty good at describing the various attacks and
              texts the worm uses:
              Technical details and removal instructions for programs and files detected by F-Secure products.


              FWIW, for me Bayesian filtering is doing a pretty good job on both the fake
              security advisories and the fake bounce messages. (Somewhat ashamedly, I'll
              admit to using POPFile (Perl) rather than the SpamBayes (Python) -- as a
              POP3 proxy solution it seemed slightly slicker at the time...)

              James


              Comment

              • netvegetable

                #8
                OT: Re: python script as an emergency mailbox cleaner

                On Sat, 20 Sep 2003 14:37:45 +0000, Phil Weldon wrote:
                [color=blue]
                > It's a worm. Worm.Automat.AG H. This is going to be a bad one. The worm
                > installs, among other things, an SMPT engine, searches an infected system
                > for email address, and sends two types of e-mail: the first is HTML and
                > is a fake "security patch" supposedly from Microsoft. It looks very
                > official, but the attachment, 104 KBytes long, is infectious. Norton
                > Antivirus definitions only began to identify it with the 18SEP03 manual
                > definition update. The worm also posts to usenet newsgroups. The other
                > type of e-mail is a fake notification of undeliverable e-mail. This one
                > is a real bear. There seem to be hundreds variations in the body content
                > and thousands of variations in the header. The infectious package is also
                > about 104 KBytes. I'm getting nearly 100 of the two types per hour.
                > Norton Antivirus does not detect the worm in usnet posts read by Outlook
                > Express Newsreader or Outlook Newsreader. Only when you attempt to open
                > the attachment or save the attachment to disk will Norton identify it.
                > Norton will NOT detect the virus in the newsgroup posts folder NOR will it
                > detect the newsgroup folder in a full system scan. It will not remove the
                > infected file from the newsgroup folder, but it will prevent execution of
                > the vermal payload.
                >
                > Microsoft Outlook with the SP3 security update when used as your e-mail
                > reader protects against infection. Prior to 18SEP03 Norton did not.
                >
                > The worm is also retrieving additional variations, so you can expect the
                > payload size to begin changing soon. The HTML message is easy to
                > identify; it is always the same (so far), and includes the phrase 'Run
                > attached file'. The bogus 'Undeliverable e-mail' variations have no
                > commonality but the payload attachment (that purports to be your bounced
                > e-mail.) This will likley change soon.
                >
                > My guess is that the internet will not open on Monday.[/color]


                The worm uses newsgroup info from Outlook Express as well.

                What's to stop a worm from retrieving header file info, and using the NNTP
                posting header to actually hack people's computers?

                --
                to email me remove underscore _
                death to spammers

                Comment

                • Phil Weldon

                  #9
                  Re: python script as an emergency mailbox cleaner

                  Inboxer for Outlook is a plugin written with Python that will analyze
                  collections of what you consider legitimate e-mail and and what you consider
                  illegitimate e-mail. I downloaded it and ran it against a collection of
                  1500 messages generated by the Worm.Automat.AH B and 265 the latest
                  legitimate e-mails I've received. After the analysis, Inboxer has detected
                  about 250 Worm.Automat.AH B generated messages with no false negatives and no
                  false positives (granted there were only three new legitimate e-mails.

                  Phil Weldon

                  "Alex Martelli" <aleax@aleax.it > wrote in message
                  news:G6Zab.1273 13$R32.3854946@ news2.tin.it...[color=blue]
                  > All my mailboxes have been filling up with files of about 130k to 150k, no
                  > doubt copies of some immensely popular virus. So, I've no doubt lost lots
                  > of real mail because of "mailbox full" conditions (the proliferating fake
                  > bounce messages more or less ensure nobody knows their mail to me has
                  > bounced, either).
                  >
                  > As an emergency response I and Anna developed, over the last half hour, a
                  > small Python script to be run from cron every few minutes and[/color]
                  automatically[color=blue]
                  > scrub any POP3 mailbox from files in the target size range. I'm saving[/color]
                  them[color=blue]
                  > to a local file for potential later perusal, but that's obviously easy to
                  > comment out if needed. Here's the tiny script in question...:
                  >
                  > import poplib
                  > import time
                  >
                  > print 'Start at', time.asctime()
                  >
                  > host = 'pop.mail.yahoo .com'
                  > port = 110
                  > user = 'aleaxit'
                  > pasw = 'secret'
                  >
                  > logfilename = 'bigjunk'
                  > minsize = 130000
                  > maxsize = 180000
                  > fromtag = 'From aleaxit@yahoo.c om %s\n'
                  >
                  > ps = poplib.POP3(hos t, port)
                  > ps.user(user)
                  > ps.pass_(pasw)
                  >
                  > messages = ps.list()
                  > print '%d messages, %d bytes' % (len(messages[1]), messages[-1])
                  >
                  > logfile = open(logfilenam e, 'a')
                  >
                  > for sms in messages[1]:
                  > sid, ssize = sms.split()
                  > if minsize <= int(ssize) < maxsize:
                  > message = ps.retr(sid)
                  > print 'retrieving and deleting msg#%s, %d bytes, %d lines' % (
                  > sid, message[-1], len(message[1]))
                  > logfile.write(f romtag % time.asctime())
                  > for line in message[1]:
                  > logfile.write(l ine)
                  > logfile.write(" \n")
                  > logfile.write(' \n')
                  > ps.dele(sid)
                  >
                  > ps.quit()
                  >
                  > print 'Done at', time.asctime()
                  > print
                  >
                  >
                  > Hope it can come in useful to somebody...!!!
                  >
                  > Alex & Anna
                  >[/color]


                  Comment

                  • Karlheinz klingbeil

                    #10
                    Re: python script as an emergency mailbox cleaner

                    Alex Martelli wrote:
                    [color=blue]
                    > All my mailboxes have been filling up with files of about 130k to 150k, no
                    > doubt copies of some immensely popular virus. So, I've no doubt lost lots
                    > of real mail because of "mailbox full" conditions (the proliferating fake
                    > bounce messages more or less ensure nobody knows their mail to me has
                    > bounced, either).
                    >
                    > As an emergency response I and Anna developed, over the last half hour, a
                    > small Python script to be run from cron every few minutes and
                    > automatically
                    > scrub any POP3 mailbox from files in the target size range. I'm saving
                    > them to a local file for potential later perusal, but that's obviously[/color]


                    I have made a quick and dirty pop-cleaner, mor configurable with size, regex
                    to deny and regular pattern match to explicitly allow mails
                    look at http://www.lunqual.de/popclean.zip (only 3kb)

                    --
                    Greetz.... lunqual

                    Comment

                    Working...