Problem with ifilters, regex and large text files

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Mobious
    New Member
    • Mar 2009
    • 1

    Problem with ifilters, regex and large text files

    Hello,

    I am currently designing a console app that will run and search our network for any files that contain 16 digit numbers. I'm having to utilise iFilters to properly index each file, which was gratefully based on the great article I found on CodeProject.

    The problem I'm having is that it basically seems to hang after a certain amount of time and after some debugging, it seems to always happen on the same file. This file is a ~40MB text file and is the second or third one that it has processed. I've included the code snippet that is doing the ifilter processing below.

    using (TextReader reader = new FilterReader(f) )
    {
    char[] buffer = new char[512 * 512];
    int position = 0;
    int charsRead = 0;

    while ((charsRead = reader.Read(buf fer, 0, buffer.Length)) > 0)
    {
    string contentToProces s = new string(buffer, 0, charsRead);

    //Check it against the provided regular expression
    if (regexp.IsMatch (contentToProce ss))
    {
    //Update the global counter
    matchCounter++;

    //Write the filename out to a log
    writeFileName(f );

    //Skip over the remainign data chunks as we have found a match
    goto skipChunk;
    }
    position += charsRead;
    }
    skipChunk: ;
    }

    I do not doubt that I am doing something fairly silly as I'm kinda self taught at C#, but any constructive help would be greatly appreciated. Any abuse at being rubbish I can get for free from my wife when I get home! :)
Working...