How to determine position within a text file under FSO model?

[Killer42]

Hi all.

In the past I've done most file IO using the old built-in VB statements such as Open, Line Input #, Get, Put and so on. In a recent project I decided to try and update a bit, so I'm using a File object to represent each file, and OpenAsTextStrea m to create a TextStream object, then reading that using ReadLine.

It's working just fine, except for one thing. How can I tell where I am in the file? I'm dealing with large files (hundreds of MB) so I like to provide a progress indicator, based on the file size and my position in the file. That was perfectly simple before, using LOF and Seek functions. But when using a TextStream in this way, how can I determine my current position? I can get the line number of course, but for that to be any use I need to know the total number of lines.

At the moment I just start out by doing a complete read through the file (using .ReadLine) to count the lines. This is fairly quick (anywhere from 5 to 30 seconds), then I can use that count to provide the progress indicator during the "real" processing (which obviously takes much longer, or I wouldn't bother). But it seems silly to have to do a double pass through the file like this.

[Ali Rizwan]

Hello,
Do you want to add a progressbar or any slider type thing.

[QVeen72]

Hi,

I had answered this a few days ago.

Use "ReadAll" Method, and Split for vbCrLF, you will get Array's ubound as No. of Lines..:

[code=vb]
Dim FSO As New Scripting.FileS ystemObject
Dim TArr
Dim fsStr As TextStream
'
Set fsStr = FSO.OpenTextFil e("C:\MyText.tx t", ForReading)
TArr = Split(fsStr.Rea dAll, vbCrLf)
MsgBox "Total Number Of Lines : " & UBound(TArr)
fsStr.Close
Set FSO = Nothing
[/code]

Regards
Veena

[Killer42]

Use "ReadAll" Method, and Split for vbCrLF, you will get Array's ubound as No. of Lines..:

Thanks for the idea.

I'd rather not, though. I am already counting the number of lines, so I don't need a way to do that (though yours may be faster). Since each file is at least 100-200MB in size, I don't want to be making copies of it all over my RAM. That's why I'm processing it line by line.

[Killer42]

Do you want to add a progressbar or any slider type thing.

I already have a progress bar, thanks. What I'm looking for is a better way to determine where I'm up to in the file. In other words, I'm looking for an equivalent of the old Seek() function.

[QVeen72]

Hi,

FSO dosent have any equivalent of "Seek" operation, you have to settle for IO's Seek. Or write a Seek Function of your own.

Regards
Veena

[hariharanmca]

I already have a progress bar, thanks. What I'm looking for is a better way to determine where I'm up to in the file. In other words, I'm looking for an equivalent of the old Seek() function.

I think this Article page will help you

How To Seek Past VBA's 2GB File Limit

[Killer42]

Thanks for trying, people, but I guess I'll just have to accept that with the FSO model, MS in their wisdom have thrown out the whole concept of "where I'm up to". No wonder their progress indicators are so inaccurate.

Hari, thanks for the link, but I've already seen that article (some years ago). I'm not trying to go beyond 2GB, just want to know my position within a file. I can't use things like Seek() because I'm not using Open.

Looks as though I only have two real options...

1. Live with the two-step process I'm using now. That is, first count all the lines, then count them again while processing them to track position, or
2. Go back to good old Open and Line Input # processing, and use Seek() function to track progress.
3. Just thought of another alternative while I was typing. Since I can easily determine the file size, I can just add up the length of each line as I go, plus 2 for the delimiters, to track my progress. Yippee!

[hariharanmca]

If you don’t mind! Can you post some part of code and let me do some research? (Of-course it will help me to learn.)

[hariharanmca]

Hari, thanks for the link, but I've already seen that article (some years ago). I'm not trying to go beyond 2GB, just want to know my position within a file. I can't use things like Seek() because I'm not using Open.

In text file there is no Row or column ID to indicate where we are. But not willing to say this will b the answer. I want to try more.

It's working just fine, except for one thing. How can I tell where I am in the file? I'm dealing with large files (hundreds of MB) so I like to provide a progress indicator, ........

we cannot show progressbar without knowing Min,CurrentPosi tion,Max value

Till working on that....

[QVeen72]

[*]Just thought of another alternative while I was typing. Since I can easily determine the file size, I can just add up the length of each line as I go, plus 2 for the delimiters, to track my progress. Yippee![/LIST]

YES, Good one, U can get Total Size of the file thru FSO,
and Just get total bytes read till now using the "LenB( )" function,
Convert to kb, and Increment the progress bar accordingly....
Real good option, thanx for that..

Regards
Veena

[hariharanmca]

How about My Program?

I hope this will be usefull!

[QVeen72]

Hi Hari,

Read Killer's previous posts, he dont want to use "ReadAll"..

[hariharanmca]

Hi Hari,

Read Killer's previous posts, he dont want to use "ReadAll"..

Yha i replied to that post.

In text file there is no Row or column ID to indicate where we are. But not willing to say this will b the answer. I want to try more.

Without row or column id we cannot set directly to any row or column.
But just gussing.....

[Killer42]

I'm on a different computer and don't have access to my code. So I'll throw together a demo function to show the sort of process I'm using. First is the FSO version. The second function shows how I would normally do this sort of thing, using Open and Line Input # statements.

[CODE=vb]Sub ReadFile_FSO_ve rsion(FileName As String, PrgBar As ProgressBar)
' For simplicity, I'm going to assume the progress bar
' has Min = 0, Max = 100.

Dim fso As New FileSystemObjec t
Dim fil As File
Dim str As TextStream
Dim TotalLines As Long
Dim OnePercent As Long


Set fil = fso.GetFile(Fil eName)

' Step 1 - count the lines in the file.
Set str = fil.OpenAsTextS tream(ForReadin g)
With str
Do Until .AtEndOfStream
.ReadLine
TotalLines = TotalLines + 1
Loop
.Close
End With
OnePercent = TotalLines / 100

' Step 2 - process the data in the file.
Set str = fil.OpenAsTextS tream(ForReadin g)
With str
Do Until .AtEndOfStream
.ReadLine
' Process the data from this line.

If .Line Mod 500 = 0 Then
PrgBar.Value = .Line / OnePercent
DoEvents
End If
Loop
.Close
End With
Set fil = Nothing


End Sub


Sub ReadFile_NonFSO _version(FileNa me As String, PrgBar As ProgressBar)
' For simplicity, I'm going to assume the progress bar
' has Min = 0, Max = 100.

Dim OnePercent As Long
Dim FileSize As Long
Dim FileNo As Long, LineNo As Long
Dim Text As String

FileNo = FreeFile
Open FileName For Input Access Read Shared As #FileNo
FileSize = LOF(FileNo)
OnePercent = FileSize / 100

Do Until EOF(FileNo)
Line Input #FileNo, Text
LineNo = LineNo + 1

' Process the data from this line.

If LineNo Mod 500 = 0 Then
PrgBar.Value = Seek(FileNo) / OnePercent
DoEvents
End If

Loop

Close #FileNo

End Sub[/CODE]