An unobtrusive partial alternative to CAPCHA

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Ben Bacarisse

    An unobtrusive partial alternative to CAPCHA

    A while back I suggested a method of using timestamps to filter out at
    least some automatic form postings. Now that I have tried it for
    about 10 months, I thought it might useful to report back.

    Briefly, the current time is encoded in a hidden form field when the
    page containing it is served. The script that processes the form
    checks the (new) current time against that in the form and rejects the
    submission if it is either too fast or too slow. Unless the user
    is super fast they see no effect at all. There are no accessibility
    issues unless one sets the maximum permitted time too low. I currently
    allow submissions from 5 seconds up to an hour after the form was
    sent. Results suggest that this upper limit can safely be increased.

    Even if a user is "caught", in both cases one can re-present the
    form. In the case of a slow submission one would regenerate the time
    stamp[1] and the user need only hit submit again.

    Of course, the time stamp must be protected so that tampering could be
    detected, although no examples of altered or missing timestamps showed
    up in this test (which it hardly surprising, why would a bot alter
    some mysterious hidden field?).

    The method does seem to work well. The people seeing the submissions
    report very little "spam" and there have been no complaints from users
    being inconvenienced. I have not been able to study the submissions
    that got through to count the failures, so all I have to go on is the
    reports of "very little" spam and counts of blocked submissions.

    Of the 1369 submissions in 42 weeks 518 were blocked by this method.
    159 of these for being too fast and a surprising 359 for being too
    slow -- way too slow. Only two of these are at all close to the one
    hour cut-off time. I image many bots queue up the forms for
    submission later. It seems they also keep trying without requesting
    the form again since there are increasing time values later in the
    test period.

    The 5 second minimum look about right. The spread for fast reply
    rejections was:

    0s: 16
    1s: 72
    2s: 38
    3s: 26
    4s: 7

    I can supply code if anyone else wants to try it.

    [1] The ideal setting would be exactly 5 seconds (the minimum
    submission time) old. A submit would then be accepted no matter how
    fast the user was.

    --
    Ben.
  • David Quinton

    #2
    Re: An unobtrusive partial alternative to CAPCHA

    On Fri, 09 May 2008 03:33:13 +0100, Ben Bacarisse
    <ben.usenet@bsb .me.ukwrote:
    >A while back I suggested a method of using timestamps to filter out at
    >least some automatic form postings.
    Thanks for this, Ben. very interesting and thorough research.
    --
    Locate your Mobile phone: <http://www.bizorg.co.u k/news.html>
    Great gifts: <http://www.ThisBritain .com/ASOS_popup.html >

    Comment

    • Pavel Lepin

      #3
      Re: An unobtrusive partial alternative to CAPCHA


      Ben Bacarisse <ben.usenet@bsb .me.ukwrote in
      <87abj0fbpy.fsf @bsb.me.uk>:
      A while back I suggested a method of using timestamps to
      filter out at least some automatic form postings. Now
      that I have tried it for about 10 months, I thought it
      might useful to report back.
      That's a brilliant idea, thanks a lot for your research. I'm
      going to adopt this method for filtering out the
      spambots... unless you consider this your IP and are going
      to patent this. But that would be very, very evil of
      you. :-)

      --
      I'm not dead, just pinin' for the fnords.

      Comment

      • Andrew Heenan

        #4
        Re: An unobtrusive partial alternative to CAPCHA

        "Ben Bacarisse" <ben.usenet@bsb .me.ukwrote ...
        Of the 1369 submissions in 42 weeks 518 were blocked by this method.
        159 of these for being too fast and a surprising 359 for being too
        slow -- way too slow. Only two of these are at all close to the one
        hour cut-off time. I image many bots queue up the forms for
        submission later. It seems they also keep trying without requesting
        the form again since there are increasing time values later in the
        test period.
        What does the system do about a failed form?
        Does the bounce page say why?
        --

        Andrew
        seo2seo.com
        sick-site-syndrome.com

        UK Residents:
        STOP THE "10p Tax Ripoff"
        Sign the petition to stop the government stealing from the
        very poorest tell your friends about this petition:



        Comment

        • Ben Bacarisse

          #5
          Re: An unobtrusive partial alternative to CAPCHA

          "Andrew Heenan" <andrew3@heenan .netwrites:
          "Ben Bacarisse" <ben.usenet@bsb .me.ukwrote ...
          >Of the 1369 submissions in 42 weeks 518 were blocked by this method.
          >159 of these for being too fast and a surprising 359 for being too
          >slow -- way too slow. Only two of these are at all close to the one
          >hour cut-off time. I image many bots queue up the forms for
          >submission later. It seems they also keep trying without requesting
          >the form again since there are increasing time values later in the
          >test period.
          >
          What does the system do about a failed form?
          Does the bounce page say why?
          For a "fast" or a "slow" submission, the page gives the user a message
          and re-presents the form (with the user-provided data, of course) and
          asks that the user re-submit. When the time-stamp is missing or
          corrupt (i.e. the hashed version does not match the plain text
          version) I also re-present the form (with a general-sounding failure
          message) though the logs show that this has not yet actually happened.

          --
          Ben.

          Comment

          • Ben Bacarisse

            #6
            Re: An unobtrusive partial alternative to CAPCHA

            Pavel Lepin <p.lepin@ctncor p.comwrites:
            Ben Bacarisse <ben.usenet@bsb .me.ukwrote in
            <87abj0fbpy.fsf @bsb.me.uk>:
            >A while back I suggested a method of using timestamps to
            >filter out at least some automatic form postings. Now
            >that I have tried it for about 10 months, I thought it
            >might useful to report back.
            >
            That's a brilliant idea, thanks a lot for your research. I'm
            going to adopt this method for filtering out the
            spambots... unless you consider this your IP and are going
            to patent this. But that would be very, very evil of
            you. :-)
            .... and would be inconsistent with reporting it here. It is simple to
            implement and almost entirely invisible to users so, although it is
            only partially effective, I hope it does get used. That was the point
            of the post.

            Thank you for your kind words.

            --
            Ben.

            Comment

            • Adrienne Boswell

              #7
              Re: An unobtrusive partial alternative to CAPCHA

              Gazing into my crystal ball I observed Ben Bacarisse
              <ben.usenet@bsb .me.ukwriting in news:87abj0fbpy .fsf@bsb.me.uk:
              A while back I suggested a method of using timestamps to filter out at
              least some automatic form postings. Now that I have tried it for
              about 10 months, I thought it might useful to report back.
              >
              Briefly, the current time is encoded in a hidden form field when the
              page containing it is served. The script that processes the form
              checks the (new) current time against that in the form and rejects the
              submission if it is either too fast or too slow. Unless the user
              is super fast they see no effect at all. There are no accessibility
              issues unless one sets the maximum permitted time too low. I
              currently
              allow submissions from 5 seconds up to an hour after the form was
              sent. Results suggest that this upper limit can safely be increased.
              >
              Even if a user is "caught", in both cases one can re-present the
              form. In the case of a slow submission one would regenerate the time
              stamp[1] and the user need only hit submit again.
              >
              Of course, the time stamp must be protected so that tampering could be
              detected, although no examples of altered or missing timestamps showed
              up in this test (which it hardly surprising, why would a bot alter
              some mysterious hidden field?).
              >
              The method does seem to work well. The people seeing the submissions
              report very little "spam" and there have been no complaints from users
              being inconvenienced. I have not been able to study the submissions
              that got through to count the failures, so all I have to go on is the
              reports of "very little" spam and counts of blocked submissions.
              >
              Of the 1369 submissions in 42 weeks 518 were blocked by this method.
              159 of these for being too fast and a surprising 359 for being too
              slow -- way too slow. Only two of these are at all close to the one
              hour cut-off time. I image many bots queue up the forms for
              submission later. It seems they also keep trying without requesting
              the form again since there are increasing time values later in the
              test period.
              >
              The 5 second minimum look about right. The spread for fast reply
              rejections was:
              >
              0s: 16
              1s: 72
              2s: 38
              3s: 26
              4s: 7
              >
              I can supply code if anyone else wants to try it.
              >
              [1] The ideal setting would be exactly 5 seconds (the minimum
              submission time) old. A submit would then be accepted no matter how
              fast the user was.
              >
              You could enhance it by placing the time into a db, and upon submission,
              compare the value in the db. Generate a unique identifier as a hidden
              field, and compare that to the one in the db with the time submitted.

              --
              Adrienne Boswell at Home
              Arbpen Web Site Design Services

              Please respond to the group so others can share

              Comment

              • Brian Cryer

                #8
                Re: An unobtrusive partial alternative to CAPCHA

                "Ben Bacarisse" <ben.usenet@bsb .me.ukwrote in message
                news:87abj0fbpy .fsf@bsb.me.uk. ..
                >A while back I suggested a method of using timestamps to filter out at
                least some automatic form postings. Now that I have tried it for
                about 10 months, I thought it might useful to report back.
                >
                Briefly, the current time is encoded in a hidden form field when the
                page containing it is served. The script that processes the form
                checks the (new) current time against that in the form and rejects the
                submission if it is either too fast or too slow. Unless the user
                is super fast they see no effect at all. There are no accessibility
                issues unless one sets the maximum permitted time too low. I currently
                allow submissions from 5 seconds up to an hour after the form was
                sent. Results suggest that this upper limit can safely be increased.
                <snip>

                Fascinating Ben. This is an area where I have an active interest, so I may
                borrow your idea (no need for code, but the concept is priceless.) I know
                its not a complete answer to spam, but it all helps.

                One thing I would say is that I wouldn't advertise the idea. Once spammers
                catch on to it it shouldn't take them much effort to get round it. That
                said, most spammers seem to be absolute idiots so the idea may be sound for
                many years to come.
                --
                Brian Cryer
                Brian Cryer's home page, a collection of notes on my professional and personal interests. Including C#, VB.Net, Windows, DB admin, Delphi and more.


                Comment

                • Jerry Stuckle

                  #9
                  Re: An unobtrusive partial alternative to CAPCHA

                  Adrienne Boswell wrote:
                  Gazing into my crystal ball I observed Ben Bacarisse
                  <ben.usenet@bsb .me.ukwriting in news:87abj0fbpy .fsf@bsb.me.uk:
                  >
                  >A while back I suggested a method of using timestamps to filter out at
                  >least some automatic form postings. Now that I have tried it for
                  >about 10 months, I thought it might useful to report back.
                  >>
                  >Briefly, the current time is encoded in a hidden form field when the
                  >page containing it is served. The script that processes the form
                  >checks the (new) current time against that in the form and rejects the
                  >submission if it is either too fast or too slow. Unless the user
                  >is super fast they see no effect at all. There are no accessibility
                  >issues unless one sets the maximum permitted time too low. I
                  currently
                  >allow submissions from 5 seconds up to an hour after the form was
                  >sent. Results suggest that this upper limit can safely be increased.
                  >>
                  >Even if a user is "caught", in both cases one can re-present the
                  >form. In the case of a slow submission one would regenerate the time
                  >stamp[1] and the user need only hit submit again.
                  >>
                  >Of course, the time stamp must be protected so that tampering could be
                  >detected, although no examples of altered or missing timestamps showed
                  >up in this test (which it hardly surprising, why would a bot alter
                  >some mysterious hidden field?).
                  >>
                  >The method does seem to work well. The people seeing the submissions
                  >report very little "spam" and there have been no complaints from users
                  >being inconvenienced. I have not been able to study the submissions
                  >that got through to count the failures, so all I have to go on is the
                  >reports of "very little" spam and counts of blocked submissions.
                  >>
                  >Of the 1369 submissions in 42 weeks 518 were blocked by this method.
                  >159 of these for being too fast and a surprising 359 for being too
                  >slow -- way too slow. Only two of these are at all close to the one
                  >hour cut-off time. I image many bots queue up the forms for
                  >submission later. It seems they also keep trying without requesting
                  >the form again since there are increasing time values later in the
                  >test period.
                  >>
                  >The 5 second minimum look about right. The spread for fast reply
                  >rejections was:
                  >>
                  > 0s: 16
                  > 1s: 72
                  > 2s: 38
                  > 3s: 26
                  > 4s: 7
                  >>
                  >I can supply code if anyone else wants to try it.
                  >>
                  >[1] The ideal setting would be exactly 5 seconds (the minimum
                  >submission time) old. A submit would then be accepted no matter how
                  >fast the user was.
                  >>
                  >
                  You could enhance it by placing the time into a db, and upon submission,
                  compare the value in the db. Generate a unique identifier as a hidden
                  field, and compare that to the one in the db with the time submitted.
                  >
                  Or, better yet, in the session.

                  --
                  =============== ===
                  Remove the "x" from my email address
                  Jerry Stuckle
                  JDS Computer Training Corp.
                  jstucklex@attgl obal.net
                  =============== ===

                  Comment

                  • Ben Bacarisse

                    #10
                    Re: An unobtrusive partial alternative to CAPCHA

                    Adrienne Boswell <arbpen@yahoo.c omwrites:
                    Gazing into my crystal ball I observed Ben Bacarisse
                    <ben.usenet@bsb .me.ukwriting in news:87abj0fbpy .fsf@bsb.me.uk:
                    >
                    >A while back I suggested a method of using timestamps to filter out at
                    >least some automatic form postings.
                    <snip>
                    >Briefly, the current time is encoded in a hidden form field when the
                    >page containing it is served. The script that processes the form
                    >checks the (new) current time against that in the form and rejects the
                    >submission if it is either too fast or too slow.
                    <snip>
                    >Of course, the time stamp must be protected so that tampering could be
                    >detected, although no examples of altered or missing timestamps showed
                    >up in this test (which it hardly surprising, why would a bot alter
                    >some mysterious hidden field?).
                    <snip>
                    You could enhance it by placing the time into a db, and upon submission,
                    compare the value in the db. Generate a unique identifier as a hidden
                    field, and compare that to the one in the db with the time
                    submitted.
                    I am not sure that would add anything. Currently, the server sets the
                    hidden field to:

                    time + ":" + md5(time + "some secret string")

                    When the form comes back, the server splits the string at the ":" and
                    it computes md5(part-before-the-colon + "some secret string").
                    Checking that this md5 hash matches the part after the colon is
                    equivalent, I think, to looking up a unique ID in a server-side DB
                    (but simpler to do).

                    --
                    Ben.

                    Comment

                    • Ben Bacarisse

                      #11
                      Re: An unobtrusive partial alternative to CAPCHA

                      Jerry Stuckle <jstucklex@attg lobal.netwrites :
                      Adrienne Boswell wrote:
                      >Gazing into my crystal ball I observed Ben Bacarisse
                      ><ben.usenet@bs b.me.ukwriting in news:87abj0fbpy .fsf@bsb.me.uk:
                      >>
                      >>A while back I suggested a method of using timestamps to filter out at
                      >>least some automatic form postings.
                      <snip>
                      >You could enhance it by placing the time into a db,
                      <snip>
                      Or, better yet, in the session.
                      See my reply to Adrienne Boswell. I don't think you gain much by
                      using session data. There is no reason not to store the data in the
                      session, but given the checks I make, I don't think it adds much.

                      --
                      Ben.

                      Comment

                      • Jerry Stuckle

                        #12
                        Re: An unobtrusive partial alternative to CAPCHA

                        Ben Bacarisse wrote:
                        Jerry Stuckle <jstucklex@attg lobal.netwrites :
                        >
                        >Adrienne Boswell wrote:
                        >>Gazing into my crystal ball I observed Ben Bacarisse
                        >><ben.usenet@b sb.me.ukwriting in news:87abj0fbpy .fsf@bsb.me.uk:
                        >>>
                        >>>A while back I suggested a method of using timestamps to filter out at
                        >>>least some automatic form postings.
                        <snip>
                        >>You could enhance it by placing the time into a db,
                        <snip>
                        >Or, better yet, in the session.
                        >
                        See my reply to Adrienne Boswell. I don't think you gain much by
                        using session data. There is no reason not to store the data in the
                        session, but given the checks I make, I don't think it adds much.
                        >
                        Just one more layer of security - it isn't in the web page.

                        --
                        =============== ===
                        Remove the "x" from my email address
                        Jerry Stuckle
                        JDS Computer Training Corp.
                        jstucklex@attgl obal.net
                        =============== ===

                        Comment

                        • Chuck Anderson

                          #13
                          Re: An unobtrusive partial alternative to CAPCHA

                          Jerry Stuckle wrote:
                          Ben Bacarisse wrote:
                          >
                          >Jerry Stuckle <jstucklex@attg lobal.netwrites :
                          >>
                          >>
                          >>Adrienne Boswell wrote:
                          >>>
                          >>>Gazing into my crystal ball I observed Ben Bacarisse
                          >>><ben.usenet@ bsb.me.ukwritin g in news:87abj0fbpy .fsf@bsb.me.uk:
                          >>>>
                          >>>>
                          >>>>A while back I suggested a method of using timestamps to filter out at
                          >>>>least some automatic form postings.
                          >>>>>
                          ><snip>
                          >>
                          >>>You could enhance it by placing the time into a db,
                          >>>>
                          ><snip>
                          >>
                          >>Or, better yet, in the session.
                          >>>
                          >See my reply to Adrienne Boswell. I don't think you gain much by
                          >using session data. There is no reason not to store the data in the
                          >session, but given the checks I make, I don't think it adds much.
                          >>
                          >>
                          >
                          Just one more layer of security - it isn't in the web page.
                          >
                          >
                          With any use of sessions I always have to wonder; what about people who
                          have cookies disabled?

                          Do you insist they enable cookies, or go with the flawed trans_sid method?

                          --
                          *************** **************
                          Chuck Anderson • Boulder, CO

                          Nothing he's got he really needs
                          Twenty first century schizoid man.
                          *************** *************** *****

                          Comment

                          • Jerry Stuckle

                            #14
                            Re: An unobtrusive partial alternative to CAPCHA

                            Chuck Anderson wrote:
                            Jerry Stuckle wrote:
                            >Ben Bacarisse wrote:
                            >>
                            >>Jerry Stuckle <jstucklex@attg lobal.netwrites :
                            >>>
                            >>>
                            >>>Adrienne Boswell wrote:
                            >>>>
                            >>>>Gazing into my crystal ball I observed Ben Bacarisse
                            >>>><ben.usenet @bsb.me.ukwriti ng in news:87abj0fbpy .fsf@bsb.me.uk:
                            >>>>>
                            >>>>>
                            >>>>>A while back I suggested a method of using timestamps to filter
                            >>>>>out at
                            >>>>>least some automatic form postings.
                            >>>>>>
                            >><snip>
                            >>>
                            >>>>You could enhance it by placing the time into a db,
                            >>>>>
                            >><snip>
                            >>>
                            >>>Or, better yet, in the session.
                            >>>>
                            >>See my reply to Adrienne Boswell. I don't think you gain much by
                            >>using session data. There is no reason not to store the data in the
                            >>session, but given the checks I make, I don't think it adds much.
                            >>>
                            >>>
                            >>
                            >Just one more layer of security - it isn't in the web page.
                            >>
                            >>
                            >
                            With any use of sessions I always have to wonder; what about people who
                            have cookies disabled?
                            >
                            Do you insist they enable cookies, or go with the flawed trans_sid method?
                            >
                            PHP will handle the session id through a get parameter. Others do
                            similarly.

                            But then, people who surf with cookies disabled are uses to sites which
                            don't work.

                            --
                            =============== ===
                            Remove the "x" from my email address
                            Jerry Stuckle
                            JDS Computer Training Corp.
                            jstucklex@attgl obal.net
                            =============== ===

                            Comment

                            Working...