List confusion

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • eso40043
    New Member
    • Feb 2007
    • 15

    List confusion

    Hello,

    I am a newbie at python, as will shortly become painfully apparent. I wrote some code that goes through a list of file names that contain names of stars. I put the star names in 'tgtnamelist'. The corresponding file names are in 'imagelist'. 'imagelist' and 'tgtnamelist' have the same length. 'tgtnamelist' does not contain len(tgtnamelist ) different star names, so I want to group all the filenames that correspond to a unique star name in one list. Basically I want to have subset lists of filenames and which star they contain.


    Now, the code I've written works just fine and does exactly what I want, but it's a part of a much larger code where I haev to go through tgtnamelist and iamgelist repeatedly and a whole lot of nested 'for' and 'if' loops so at the end everything becomes very confusing!! Which is why I want to trim down the code a little, so I figure I'll start by getting rid of some unnecessary lists.

    Is there a way to do this? To match two lists, one with entries that repeat themselves in subsets of unknown length, to another list with unique entires. The only thing connecting the lists is that target 0 is in file 0, target 1 is in file 1, etc. Target names are unordered, so the same star name can be spread out throughout the list.

    Any help, tips and hints would be greatly appreciated!!


    #first target
    currtgtname=tgt namelist[0]
    #located in first file
    currimname=imag elist[0]
    del imagelist[0]
    del tgtnamelist[0]
    #all files containing one star
    subsets=[]
    #all subsets of stars
    alltargets=[]

    for curr in range(len(image list)):
    subsets.append( currimname)
    if tgtnamelist[curr]==currtgtname:
    currimname=imag elist[curr]
    else:
    currtgtname=tgt namelist[curr]
    currimname=imag elist[curr]
    alltargets.appe nd(list(subsets ))
    del subsets[:]
  • bvdet
    Recognized Expert Specialist
    • Oct 2006
    • 2851

    #2
    Originally posted by eso40043
    Hello,

    I am a newbie at python, as will shortly become painfully apparent. I wrote some code that goes through a list of file names that contain names of stars. I put the star names in 'tgtnamelist'. The corresponding file names are in 'imagelist'. 'imagelist' and 'tgtnamelist' have the same length. 'tgtnamelist' does not contain len(tgtnamelist ) different star names, so I want to group all the filenames that correspond to a unique star name in one list. Basically I want to have subset lists of filenames and which star they contain.


    Now, the code I've written works just fine and does exactly what I want, but it's a part of a much larger code where I haev to go through tgtnamelist and iamgelist repeatedly and a whole lot of nested 'for' and 'if' loops so at the end everything becomes very confusing!! Which is why I want to trim down the code a little, so I figure I'll start by getting rid of some unnecessary lists.

    Is there a way to do this? To match two lists, one with entries that repeat themselves in subsets of unknown length, to another list with unique entires. The only thing connecting the lists is that target 0 is in file 0, target 1 is in file 1, etc. Target names are unordered, so the same star name can be spread out throughout the list.

    Any help, tips and hints would be greatly appreciated!!


    #first target
    currtgtname=tgt namelist[0]
    #located in first file
    currimname=imag elist[0]
    del imagelist[0]
    del tgtnamelist[0]
    #all files containing one star
    subsets=[]
    #all subsets of stars
    alltargets=[]

    for curr in range(len(image list)):
    subsets.append( currimname)
    if tgtnamelist[curr]==currtgtname:
    currimname=imag elist[curr]
    else:
    currtgtname=tgt namelist[curr]
    currimname=imag elist[curr]
    alltargets.appe nd(list(subsets ))
    del subsets[:]
    We can help you, but let me ask you a couple of things first. Please repost your code with CODE TAGS (see reply guidelines). That would make your code much easier to follow. Also, would you post samples of your lists that you need to process, and a sample file. Dictionaries may be better suited for your project.

    Comment

    • eso40043
      New Member
      • Feb 2007
      • 15

      #3
      Ok, pardon my ignorance, I'll repost the code.

      this is the way it looks now:

      Code:
      #first target
      currtgtname=tgtnamelist[0]
      #located in first file
      currimname=imagelist[0]
      del imagelist[0]
      del tgtnamelist[0]
      #all files containing a certain star
      subsets=[]
      #all subsets of all stars
      alltargets=[]
      
      for curr in range(len(imagelist)):[INDENT]subsets.append(currimname)
      if tgtnamelist[curr]==currtgtname:[INDENT]currimname=imagelist[curr][/INDENT]
      else:[INDENT]currtgtname=tgtnamelist[curr]
      currimname=imagelist[curr]
      alltargets.append(list(subsets))
      del subsets[:][/INDENT][/INDENT]
      The basic problem is this: I have images with header information which contains fields such as target name, etc. I have extracted the values of target names from those fields in tgtnamelist with PyFits. tgtnamelist can look like this:
      Code:
      tgtnamelist = ['pg2203+051', 'MarkA', 'pg2203+051', 'pg2203+051', 'sa95', 'MarkA', 'MarkA']
      imagelist simply contains the names of the corresponding image files, e.g.:

      Code:
      imagelist = ['image1.fits', 'image2.fits', 'image3.fits', 'image4.fits', 'image5.fits', 'image6.fits', 'image7.fits']

      where
      image1.fits contains pg2203+051
      image2.fits contains MarkA
      image3.fits contains pg2203+051
      image4.fits contains pg2203+051
      image5.fits contains sa95
      image6.fits contains MarkA
      image7.fits contains MarkA

      my code groups the filenames like this:

      Code:
      subset1=['image1.fits', 'image3.fits', 'image4.fits']
      subset2=['image2.fits', 'image6.fits', 'image7.fits']
      subset3=['image5.fits']
      and then returns alltargets as

      Code:
      alltargets=[subset1, subset2, subset3]
      which is what I want, but combined with the rest of the code there's just too many lists to keep track of and I'm finding it very difficult to continue expanding the code which is why I'm here. I thought of using dictionaries but don't they take keys with unique values only?


      anyway, just to show you the list confusion I'm talking about , here the entire procedure. You don't have to look at it in detail at all, the excerpt above works as a stand-alone. But as you can see I loop through the list repeatedly just because when I tried to do everything in one loop I completely lost track of what happens, when and why.





      Code:
      #####-----------------------############## 
      ##   group images into bias,flat   #####
      ##   object and std                    #####
      #####-----------------------############## 
      def groupimages(headers,imagenames,stdnames): [INDENT]
          exptimelist=[] 
          tgtnamelist=[] 
          for curr in headers:[INDENT]
              if curr.has_key("EXPTIME"): [INDENT]
                  exptime=curr.get("EXPTIME")
                  exptimelist.append(exptime)
      [/INDENT]
              if curr.has_key("TCSTGT"): [INDENT]
                  tgtname=curr.get("TCSTGT")
                  tgtnamelist.append(tgtname) 
      [/INDENT]
              else: [INDENT]
                  print "FIX TCSTGT keywords..."		 
                  system.exit(0) 
      [/INDENT][/INDENT]
          
          biaslist=[] 
          flatlist=[] 
          imagelist=list(imagenames) 
          objlist=[] 
          delindex=[]
          
          for curr in range(len(headers)):[INDENT]
              if exptimelist[curr]==0.0: [INDENT]
                  biaslist.append(imagenames[curr])
                  delindex.append(curr)
      [/INDENT]
              elif tgtnamelist[curr].find("Blank")!=-1: [INDENT]
                  flatlist.append(imagenames[curr])
                  delindex.append(curr)
      [/INDENT]
      [/INDENT]
          for i in reversed(delindex):[INDENT]
              del imagelistcopy[i]
              del exptimelist[i] 
              del tgtnamelist[i]
      [/INDENT]
          del delindex[:]
      
          #now have only object files remaining
          currtgtname=tgtnamelist[0]
          currimname=imagelistcopy[0]
          del imagelistcopy[0]
          del tgtnamelist[0]
          subsets=[]
          alltargets=[]
          
          for curr in range(len(imagelistcopy)):[INDENT]
              subsets.append(currimname)
              if tgtnamelist[curr]==currtgtname:[INDENT]
                  currimname=imagelistcopy[curr]
      [/INDENT]
              else:[INDENT]
                  currtgtname=tgtnamelist[curr]
                  currimname=imagelistcopy[curr]
                  alltargets.append(list(subsets))
                  del subsets[:]
      [/INDENT]
      [/INDENT]
          #now all objects are groupped.
          #separate stds from science frames
          stdlist=[]
          delindex=[]
          for i in range(len(alltargets)):[INDENT]
              objimage=alltargets[i][0]
              hdr=PF.getheader(objimage)
              if hdr.get("TCSTGT").upper() in stdnames:[INDENT]
                  stdlist.append(alltargets[i])
                  delindex.append(i)
                  print "found an STD, ", hdr.get("TCSTGT")
      [/INDENT]
      [/INDENT]
          #remove stds from object list         
          for i in reversed(delindex):[INDENT]
              del alltargets[i]
       [/INDENT]
          return biaslist,flatlist,alltargets,stdlist
      [/INDENT]
      #pfew!

      Comment

      • eso40043
        New Member
        • Feb 2007
        • 15

        #4
        I forgot to mention,

        imagelist

        does not have to contain filenames which can be ordered in some logical fashion, eg. it doesn't have to be
        Code:
        imagelist=['image1.fits','image2.fits','image3.fits',....]
        it could be

        Code:
        imagelist=['std1.fits','li100043.fits','ONT-12:34:54.fits',....]

        Comment

        • bvdet
          Recognized Expert Specialist
          • Oct 2006
          • 2851

          #5
          Originally posted by eso40043
          I forgot to mention,

          imagelist

          does not have to contain filenames which can be ordered in some logical fashion, eg. it doesn't have to be
          Code:
          imagelist=['image1.fits','image2.fits','image3.fits',....]
          it could be

          Code:
          imagelist=['std1.fits','li100043.fits','ONT-12:34:54.fits',....]
          I can see how you are confused. I may not understand what you are trying to accomplish, but I believe a dictionary of dictionaries would be a better choice. The image file names could be your keys for your primary dictionary. Iterate once for each file - each iteration adds a secondary dictionary. Your required lists can be extracted from this dictionary. Here's a simple example:
          Code:
          imagelist = ['image1.fits', 'image2.fits', 'image3.fits', 'image4.fits', 'image5.fits', 'image6.fits', 'image7.fits']
          
          datalist = [['pg2203+051', 'Z', '275'],
                      ['MarkA', 'A', '150'],
                      ['pg2203+051', 'Z', '275'],
                      ['pg2203+051', 'A', '275'],
                      ['sa95', 'T', '500'],
                      ['MarkA', 'B', '150'],
                      ['MarkA', 'C', '150']
                      ]
          
          #  imagelist and datalist correspond
          
          dd = {}
          for i, item in enumerate(imagelist):
              dd[item] = dict(zip(['star', 'type', 'dist'], datalist[i]))
          
          for key in dd:
              print '%s = %s' % (key, dd[key])
          
          '''
          >>> image2.fits = {'type': 'A', 'star': 'MarkA', 'dist': '150'}
          image6.fits = {'type': 'B', 'star': 'MarkA', 'dist': '150'}
          image1.fits = {'type': 'Z', 'star': 'pg2203+051', 'dist': '275'}
          image7.fits = {'type': 'C', 'star': 'MarkA', 'dist': '150'}
          image5.fits = {'type': 'T', 'star': 'sa95', 'dist': '500'}
          image3.fits = {'type': 'Z', 'star': 'pg2203+051', 'dist': '275'}
          image4.fits = {'type': 'A', 'star': 'pg2203+051', 'dist': '275'}
          '''
          
          # Python 2.3 - learned this from ghostdog74
          starList = [x['star'] for x in dd.values() if x['star'] not in locals()['_[1]'].__self__]
          
          # Python 2.4
          # starList = list(set([x['star'] for x in dd.values()]))
          
          print starList
          
          '''
          >>> ['MarkA', 'pg2203+051', 'sa95']
          '''
          
          # Star, image files
          starImageDict = {}
          
          for key in dd:
              try:
                  starImageDict[dd[key]['star']].append(key)
              except:
                  starImageDict[dd[key]['star']] = [key, ]
          
          print starImageDict
          
          '''
          >>> {'sa95': ['image5.fits'], 'MarkA': ['image2.fits', 'image6.fits', 'image7.fits'], 'pg2203+051': ['image1.fits', 'image3.fits', 'image4.fits']}
          >>>
          '''
          I don't know PyFits, but I am assume the attributes for the secondary dictionaries are readily available to create datalist.

          Comment

          • eso40043
            New Member
            • Feb 2007
            • 15

            #6
            I apologize for bumping this thread, I just wanted to thank you!! Thank you very much, this will seriously reduce the amount of code!!

            Comment

            • bvdet
              Recognized Expert Specialist
              • Oct 2006
              • 2851

              #7
              Originally posted by eso40043
              I apologize for bumping this thread, I just wanted to thank you!! Thank you very much, this will seriously reduce the amount of code!!
              You are welcome. The positive comments are much appreciated. Let us know if we can help again!

              Comment

              Working...