How to load a big XML file in Java

[ThatThatGuy]

What is the best way to load a big XML file that's 20-40 MB in size?
I have tried using DOMParser and DocumentBuilder but, all I get is the following as an error:

java.lang.OutOf MemoryError: Java heap space

[jkmyoung]

1. Do you really need the whole XML document to be loaded into a DOM structure? What are you trying to do with it?
If you can look at alternatives, see SAX, or VTD, other alternatives that don't require full loading into memory. Perhaps all you need is some sort of XMLStreamReader .

2. You could increase the allocated heap space of your java runtime.

[ThatThatGuy]

1. Do you really need the whole XML document to be loaded into a DOM structure? What are you trying to do with it?
If you can look at alternatives, see SAX, or VTD, other alternatives that don't require full loading into memory. Perhaps all you need is some sort of XMLStreamReader .

2. You could increase the allocated heap space of your java runtime.

Yes I need the whole XML at a stretch.

[jkmyoung]

I meant, all at once?
If so then how are you running your java program?

java -Xms<initial heap size> -Xmx<maximum heap size>
eg: java -Xms32m -Xmx128m

[ThatThatGuy]

I meant, all at once?
If so then how are you running your java program?

java -Xms<initial heap size> -Xmx<maximum heap size>
eg: java -Xms32m -Xmx128m

No it needs to load up 65000 lines of xml tags... i need all of them for my pos application...

Any other methods i can try.... the file is just 7MB

[jkmyoung]

So even when you increase the heap size to 512, eg
java -Xms32m -Xmx512m
you still get out of memory errors?

[ThatThatGuy]

So even when you increase the heap size to 512, eg
java -Xms32m -Xmx512m
you still get out of memory errors?

I dropped the idea of the creating the app in java thanks.

[Dheeraj Joshi]

Because you could not load entire XML in one shot? or you dropped this idea because of some other reason?

Regards
Dheeraj Joshi

[ThatThatGuy]

Because you could not load entire XML in one shot? or you dropped this idea because of some other reason?

Regards
Dheeraj Joshi

Since Java can't load XML file that's 7MB(65000 lines long). The thing is that I need to load the xml file at a stretch because I want every data in XML. It's not a setting file it's a whole bunch of data. When I did this in .NET using XmlDocument, it didn't seemed to have a problem.

[jkmyoung]

Another alternative I neglected to mention: jdom.

It's still hard to tell what you're doing with the data; thus hard to recommend blindly.
Other possible suggestion:
Is it possible to load this into a db and let the db do the gruntwork of whatever you want to do?

[Dheeraj Joshi]

I don't know why you are not able to load 7 Mb in one shot. Once I successfully loaded 15 Mb XML file in one shot.

Regards
Dheeraj Joshi