Read large data files

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • clairePuj
    New Member
    • Aug 2007
    • 2

    Read large data files

    Hi everybody,
    Please can you tell me how can I read large data file (> 15 millions line) in C/C++ languge.

    Thanks for your help,
    Claire
  • Meetee
    Recognized Expert Contributor
    • Dec 2006
    • 928

    #2
    Originally posted by clairePuj
    Hi everybody,
    Please can you tell me how can I read large data file (> 15 millions line) in C/C++ languge.

    Thanks for your help,
    Claire
    You can use vectors for this:

    1. declare/open a file-stream for reading (using std::ifstream )
    2. declare a vector of strings (std::vector<st d::string>)
    3. if you haven't already reached the end of file, then do the following, else goto 8:
    4. declare a temporary string that will hold the next line (std::string temp;).
    5. fill this temporary string with the current line of your file (std::getline)
    6. add the string to your string-vector (using push_back function)
    7. go back to 3
    8. close the file

    If you are not aware of vectors then http://www.cppreference.com/cppvector/index.html

    Regrads

    Comment

    • gpraghuram
      Recognized Expert Top Contributor
      • Mar 2007
      • 1275

      #3
      Gi,
      If the file size is huge then reading it at a short wont be very effective.
      Try to split the file into smaller sizes and then have a logic to to open these files in a loop on by one and then read it

      Raghuram

      Comment

      • clairePuj
        New Member
        • Aug 2007
        • 2

        #4
        Thanks a lot.
        best,[

        QUOTE=zodilla58]You can use vectors for this:

        1. declare/open a file-stream for reading (using std::ifstream )
        2. declare a vector of strings (std::vector<st d::string>)
        3. if you haven't already reached the end of file, then do the following, else goto 8:
        4. declare a temporary string that will hold the next line (std::string temp;).
        5. fill this temporary string with the current line of your file (std::getline)
        6. add the string to your string-vector (using push_back function)
        7. go back to 3
        8. close the file

        If you are not aware of vectors then http://www.cppreference.com/cppvector/index.html

        Regrads[/QUOTE]

        Comment

        Working...