Filtering a large data file

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • ariane92
    New Member
    • Jun 2018
    • 2

    Filtering a large data file

    Hi I have a large data file and I'm only interested in rows with specific x values of 4.125 as shown below. Now because the value of 4.125 relates to the stop position of the ion, the corresponding start position is also of interest to me and I want to keep this information in the array. How do I write a program which effectively finds the x stop position of 4.125 and retains the ion start position? I have attached the data file I'm using.

    This is the code I have developed so far but does not work:

    Code:
    import pandas as pd 
    import numpy as np
    
    opts = pd.read_csv('Ambre_2.dat',sep = ',', low_memory = False)
    df = pd.DataFrame(opts)
    
    X = df.iloc[:,3]
    IonN = df.iloc[:,0]
    tol = 1e-6
    Fltr = 4.125
    
    filterreddata = df[abs(df.X-Fltr)<tol,:]
    filteredions = df(np.in1d(df.IonN, filterreddata.IonN), :]
    filteredions[2:2:end, :] = []
    f = open('ions.csv', 'w')
    f.write(tabulate(filteredions))
    f.close()
  • ariane92
    New Member
    • Jun 2018
    • 2

    #2
    In the example shown below I would be interested in keeping the information of ion # 3849096.

    "Ion N","Mass","Char ge","X","Y","Z" ,"Azm","Elv","K E" 3849094,0.00054 858,-1,66.5216,-51,-3.8,-180,88.7,18160 3849094,0.00054 858,-1,27.3925,30.35 32,-4.07076,-177.1,41.5494,1 7697.2 3849095,0.00054 858,-1,66.5216,-51,-3.7,-180,88.7,18160 3849095,0.00054 858,-1,26.6277,31.00 39,-3.91402,-177.096,40.8293 ,17699.4 3849096,0.00054 858,-1,66.5216,-51,-3.6,-180,88.7,18160 3849096,0.00054 858,-1,4.125,44.9887 ,-2.47517,-176.363,25.715, 17711.1

    Comment

    • dwblas
      Recognized Expert Contributor
      • May 2008
      • 626

      #3
      Cross posted at https://python-forum.io/Thread-How-t...arge-data-file and

      Comment

      Working...