Parsing huge data

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • psbasha
    Contributor
    • Feb 2007
    • 440

    Parsing huge data

    Hi ,

    I have to parse a huge data of 3GB and store it in the dictnoray.After storing the data ,I need to do lot of calculations and write to the output file.The application is working for 500 MB file data,but for 3GB file it throwing Out of Memory message on the Python Editor.

    Could you please help me with the best approach of data handling and traversing.

    Thanks
    PSB
  • bvdet
    Recognized Expert Specialist
    • Oct 2006
    • 2851

    #2
    Please show us what you were doing codewise when you received the memory error. Will you end up with a dictionary of 3 GB? Consider storing the data in a database such as MySQL.

    BV

    Comment

    Working...