Using Python with COM to communicate with proprietary Windows software

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Joakim Persson

    #1

    Using Python with COM to communicate with proprietary Windows software

    Hello all. I am involved in a project where we have a desire to
    improve our software testing tools, and I'm in charge of looking for
    solutions regarding the logging of our software (originating from
    embedded devices). Currently, we are using a heavyweight, proprietary
    log tool developed by another part of the company. This tool contains
    all "standard" logging functionality, but we also need to insert
    "debug" log points in the software of the embedded device, and we
    cannot get new releases of this big log tool very often (typically,
    these debug log points might be very temporarily inserted into the
    software during the development phase).

    Therefore, we have added a requirement to the log tool where we made
    them implement a COM server in the log tool, so that we can access
    arbitrary log points in the embedded device. My thoughts then, as
    someone who wants to bring in more interpreted, lightweight languages
    (and SW development processes) into the organization, was to use
    Python to easily add log parsing and log presentation in different
    ways.

    This means that I must come up with a way to interface Python with
    this log tool. After some investigations, I have run inte problems --
    my inital approach was to use Python + win32com to communicate with
    the COM interface. Unfortunately, it seems as if win32com, even if I
    makepy the log tool executable (which gives me the full range of
    available functions of the interface, just as expected), cannot be
    used, since the interface is not a simple IDispatch interface. Also, I
    discovered that the Python client must also implement a COM server to
    receive the log information as it is generated by the embedded device,
    passed into the log tool and then sent by the log tool COM server.

    I have a rather bloated C++ client prototype which "works", so I
    should have all the code needed to invoke the interface, but this
    client is of course rather bloated in itself and not suitable for my
    experimental programming approach. Am I right in believing that the
    COM interface I am trying to use is of what is called a "custom" type,
    that I cannot access using Python + win32com? I can use Python +
    win32com just fine when accessing most normal COM servers (typical
    example: MS Office programs implementing IDispatch interfaces), but I
    cannot access the proprietary interface -- I get "interface not
    supported"-type messages (I can query the CoClass for IUnknown and
    IDispatch interfaces, but not using the known IID for the custom
    interface).

    So, I'm trying to sum up my options. I have looked at Boost to try to
    merge C++ and Python code, take care of the COM server/client creation
    and messaging by the C++ part, and then take care of the data
    processing/presentation part in Python, like so:

    +----+
    |PRES| <-- Python and friends (wxPython could be interesting)
    +----+
    ||||
    +----+
    |DATA| <-- Pure python
    +----+
    ||||
    +----+
    |COM | <-- C++
    +----+

    .... all linked together into, eventually, one Python executable
    (linking in the C++ through a DLL which is then imported by Python).

    I haven't fully explored how to do this, but it seems as if Boost is
    usable using MSVC++ (the organizations standard IDE) -- has anyone
    tried something similar? My final idea is to "give up" on one single
    program for this, and use a C++ intermediate program to communicate
    with the log tool and then let it forward its data through sockets or
    named pipes, which Python seems to deal with more easily.

    I have tried using Java + jawin as well just for a quick exploratory
    spike, and ran into the same problems ("interface not supported" etc).
    The jawin documentation talks a bit about wrapping the custom
    interface into java objects, but this is rather obscure and
    undocumented, and untangling it would take quite some time -- so I
    figured that somebody could have done something similar in the past.

    Does anyone have any hints on how to go forward? The whole point with
    my project is to demonstrate the usefulness of rapid prototyping using
    Python (this has already been successful for smaller test tools, test
    scripts and smaller intermediary programs), but I don't want something
    like COM client/server obscurity getting in the way...

    Thanks in advance,

    --
    Joakim Persson
    M.Sc student, CS/E @ LTH
  • Thomas Heller

    #2
    Re: Using Python with COM to communicate with proprietary Windowssoftware

    Joakim Persson <zaladin@home.s e> writes:

    [...][color=blue]
    > This means that I must come up with a way to interface Python with
    > this log tool. After some investigations, I have run inte problems --
    > my inital approach was to use Python + win32com to communicate with
    > the COM interface. Unfortunately, it seems as if win32com, even if I
    > makepy the log tool executable (which gives me the full range of
    > available functions of the interface, just as expected), cannot be
    > used, since the interface is not a simple IDispatch interface. Also, I
    > discovered that the Python client must also implement a COM server to
    > receive the log information as it is generated by the embedded device,
    > passed into the log tool and then sent by the log tool COM server.
    >
    > I have a rather bloated C++ client prototype which "works", so I
    > should have all the code needed to invoke the interface, but this
    > client is of course rather bloated in itself and not suitable for my
    > experimental programming approach. Am I right in believing that the
    > COM interface I am trying to use is of what is called a "custom" type,
    > that I cannot access using Python + win32com?[/color]

    Sounds like a perfect job for comtypes, which is a COM library
    implemented in pure Python, based on ctypes. comtypes should make it
    easy to access custom (non-dispatch derived) com interfaces, or the
    vtable based part of dual interfaces - it would be good however, if you
    have a type library for the interfaces.

    http://sourceforge.net/projects/comtypes/

    No docs yet, but there are tests included which should get you started.

    (I have released and announced this 3 weeks ago, but haven't got a
    single feedback. So it seems the need to access custom interfaces is
    very low.)

    Thommas

    Comment

    • Joakim Persson

      #3
      Re: Using Python with COM to communicate with proprietary Windows software

      On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <theller@python .net>
      wrote:
      [color=blue]
      >Joakim Persson <zaladin@home.s e> writes:
      >[color=green]
      >> I have a rather bloated C++ client prototype which "works", so I
      >> should have all the code needed to invoke the interface, but this
      >> client is of course rather bloated in itself and not suitable for my
      >> experimental programming approach. Am I right in believing that the
      >> COM interface I am trying to use is of what is called a "custom" type,
      >> that I cannot access using Python + win32com?[/color]
      >
      >Sounds like a perfect job for comtypes, which is a COM library
      >implemented in pure Python, based on ctypes. comtypes should make it
      >easy to access custom (non-dispatch derived) com interfaces, or the
      >vtable based part of dual interfaces - it would be good however, if you
      >have a type library for the interfaces.
      >
      >http://sourceforge.net/projects/comtypes/
      >
      >No docs yet, but there are tests included which should get you started.
      >
      >(I have released and announced this 3 weeks ago, but haven't got a
      >single feedback. So it seems the need to access custom interfaces is
      >very low.)
      >
      >Thommas[/color]

      Looks good, I think I downloaded ctypes but didn't put in the effort
      to wrap the custom COM interface.

      I have the type library and full documentation of the interface, so
      IMO it _should_ be possible to make a 100% Python application for
      testing this particular interface, which would definitely speed up
      prototyping, which is the entire goal of my work. I will give it a
      spin tomorrow, and I'll get back on the NG once I have tested it. Many
      thanks!

      It seems most Python uses for COM rely on "simple" automation of COM
      interfaces of the IDispatch type, but I need something more (full
      client<->server communication and threading, for instance). Hopefully
      your module will at least let me get one step further...

      My remaining two options, seeing as I would really like to at least
      put the data processing and presentation part in Python, are linking
      Python with C++ for the COM part and building (Data + Presentation) in
      Python, or making a complete standalone C++ part for the COM part and
      then _another_ client<->server solution using e.g. named
      pipes/sockets.

      --
      Joakim Persson
      M.Sc student, CS/E @ LTH

      Comment

      • Andrew MacIntyre

        #4
        Re: Using Python with COM to communicate with proprietary Windowssoftware

        On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <theller@python .net>
        wrote:

        {...}
        [color=blue]
        >(I have released and announced this 3 weeks ago, but haven't got a
        >single feedback. So it seems the need to access custom interfaces is
        >very low.)[/color]

        I have downloaded it and am trying to find the time to play with it
        (unsuccessfully so far).

        As someone working with a large, complex, COM library with minimal
        IDispatch support, I'm really looking forward to this.

        However, at the moment I'm limited to Python 2.2 and ctypes 0.6.3 (which
        is allowing me to get the job done!!).

        Regardless, I thank you for what you have released!

        Cheers,
        Andrew.

        -------------------------------------------------------------------------
        Andrew I MacIntyre "These thoughts are mine alone..."
        E-mail: andymac@bullsey e.apana.org.au (pref) | Snail: PO Box 370
        andymac@pcug.or g.au (alt) | Belconnen ACT 2616
        Web: http://www.andymac.org/ | Australia

        Comment

        • Joakim Persson

          #5
          Re: Using Python with COM to communicate with proprietary Windows software

          On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <theller@python .net>
          wrote:
          [color=blue]
          >Sounds like a perfect job for comtypes, which is a COM library
          >implemented in pure Python, based on ctypes. comtypes should make it
          >easy to access custom (non-dispatch derived) com interfaces, or the
          >vtable based part of dual interfaces - it would be good however, if you
          >have a type library for the interfaces.
          >
          >http://sourceforge.net/projects/comtypes/
          >
          >No docs yet, but there are tests included which should get you started.
          >
          >(I have released and announced this 3 weeks ago, but haven't got a
          >single feedback. So it seems the need to access custom interfaces is
          >very low.)
          >
          >Thommas[/color]

          After some testing today, it does seem to do exactly what I wanted --
          I can now access the "custom" but IDispatch-like COM interface that I
          couldn't access with win32com (or with Java + jawin). It might be
          possible in other ways, but using comtypes was definitely the most
          painfree way (everything, including the return values from the
          methods, worked as expected). Thank you very much.

          Of course, this does not complete my task -- although I can now use my
          interface to send messages and commands to the big log tool, I still
          need to implement a COM server and pass a pointer to its interface
          through one of the messages to the com server to be able to receive
          data:

          BridgeInterface .StartLogging(f ilename) <--- works fine, didn't work
          before
          BridgeInterface .Advise(ptr) <--- Now, I need to create a new
          interface for receiving the data sent from the log application, so
          that I can (at first) print it

          This _shouldn't_ be too difficult -- I know which methods must be
          implemented (basically just some kind of event handling to deal with
          randomly arriving log points, should be implemented as "onMsg()" on my
          COM server side, and some other similar methods), but I don't really
          know how. I have tried doing simple COM servers using win32com, but is
          it equally possible to implement such a simple thing in comtypes? I
          didn't find any server side examples in comtypes, but perhaps there is
          a way?

          --
          Joakim Persson
          M.Sc student, CS/E @ LTH

          Comment

          • Thomas Heller

            #6
            Re: Using Python with COM to communicate with proprietary Windowssoftware

            Andrew MacIntyre <andymac@bullse ye.apana.org.au > writes:
            [color=blue]
            > On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <theller@python .net>
            > wrote:
            >
            > {...}
            >[color=green]
            >>(I have released and announced this 3 weeks ago, but haven't got a
            >>single feedback. So it seems the need to access custom interfaces is
            >>very low.)[/color]
            >
            > I have downloaded it and am trying to find the time to play with it
            > (unsuccessfully so far).
            >
            > As someone working with a large, complex, COM library with minimal
            > IDispatch support, I'm really looking forward to this.
            >
            > However, at the moment I'm limited to Python 2.2 and ctypes 0.6.3
            > (which is allowing me to get the job done!!).
            >
            > Regardless, I thank you for what you have released![/color]

            I knew there are fans out there ;-)

            Thomas

            Comment

            • Thomas Heller

              #7
              Re: Using Python with COM to communicate with proprietary Windowssoftware

              Joakim Persson <zaladin@home.s e> writes:
              [color=blue]
              > On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <theller@python .net>
              > wrote:
              >[color=green]
              >>Sounds like a perfect job for comtypes, which is a COM library
              >>implemented in pure Python, based on ctypes. comtypes should make it
              >>easy to access custom (non-dispatch derived) com interfaces, or the
              >>vtable based part of dual interfaces - it would be good however, if you
              >>have a type library for the interfaces.
              >>
              >>http://sourceforge.net/projects/comtypes/
              >>
              >>No docs yet, but there are tests included which should get you started.
              >>
              >>(I have released and announced this 3 weeks ago, but haven't got a
              >>single feedback. So it seems the need to access custom interfaces is
              >>very low.)
              >>
              >>Thommas[/color]
              >
              > After some testing today, it does seem to do exactly what I wanted --
              > I can now access the "custom" but IDispatch-like COM interface that I
              > couldn't access with win32com (or with Java + jawin). It might be
              > possible in other ways, but using comtypes was definitely the most
              > painfree way (everything, including the return values from the
              > methods, worked as expected). Thank you very much.
              >
              > Of course, this does not complete my task -- although I can now use my
              > interface to send messages and commands to the big log tool, I still
              > need to implement a COM server and pass a pointer to its interface
              > through one of the messages to the com server to be able to receive
              > data:
              >
              > BridgeInterface .StartLogging(f ilename) <--- works fine, didn't work
              > before
              > BridgeInterface .Advise(ptr) <--- Now, I need to create a new
              > interface for receiving the data sent from the log application, so
              > that I can (at first) print it
              >
              > This _shouldn't_ be too difficult -- I know which methods must be
              > implemented (basically just some kind of event handling to deal with
              > randomly arriving log points, should be implemented as "onMsg()" on my
              > COM server side, and some other similar methods), but I don't really
              > know how. I have tried doing simple COM servers using win32com, but is
              > it equally possible to implement such a simple thing in comtypes? I
              > didn't find any server side examples in comtypes, but perhaps there is
              > a way?[/color]

              There is not yet any server support in comtypes, but it will be added.
              In the meantime you could use the ctypes.com package that is included in
              ctypes itself, there are even samples.

              Thomas

              Comment

              • Joakim Persson

                #8
                Re: Using Python with COM to communicate with proprietary Windows software

                On Tue, 13 Sep 2005 10:15:18 +0200, Thomas Heller <theller@python .net>
                wrote:
                [color=blue][color=green]
                >> This _shouldn't_ be too difficult -- I know which methods must be
                >> implemented (basically just some kind of event handling to deal with
                >> randomly arriving log points, should be implemented as "onMsg()" on my
                >> COM server side, and some other similar methods), but I don't really
                >> know how. I have tried doing simple COM servers using win32com, but is
                >> it equally possible to implement such a simple thing in comtypes? I
                >> didn't find any server side examples in comtypes, but perhaps there is
                >> a way?[/color]
                >
                >There is not yet any server support in comtypes, but it will be added.
                >In the meantime you could use the ctypes.com package that is included in
                >ctypes itself, there are even samples.
                >
                >Thomas[/color]

                Thank you, I will try that. Right now I've been trying to use win32com
                for the server creation, but I haven't been successful yet (have not
                explored it that much yet though).

                I have registered a COM server (it should only implement two methods,
                "onMsg()" and "onNotify() ", which should print whatever is sent to
                them directly to stdout), but I need to create an interface which is
                not an IDispatch interface and pass it on to the proprietary COM
                server. This connection is needed so that the proprietary COM server
                knows where to send the log and control data.

                Registering the COM object in the registry works fine, but how do I
                make it look like a custom interface from the proprietary COM servers
                point of view? Oh well, I will look at it some more tomorrow and
                hopefully come up with a solution...

                --
                Joakim Persson
                M.Sc student, CS/E @ LTH

                Comment

                • Paul Casteels

                  #9
                  Re: Using Python with COM to communicate with proprietary Windowssoftware

                  Joakim Persson wrote:[color=blue]
                  > On Tue, 13 Sep 2005 10:15:18 +0200, Thomas Heller <theller@python .net>
                  > wrote:
                  >
                  >[color=green][color=darkred]
                  >>>This _shouldn't_ be too difficult -- I know which methods must be
                  >>>implemente d (basically just some kind of event handling to deal with
                  >>>randomly arriving log points, should be implemented as "onMsg()" on my
                  >>>COM server side, and some other similar methods), but I don't really
                  >>>know how. I have tried doing simple COM servers using win32com, but is
                  >>>it equally possible to implement such a simple thing in comtypes? I
                  >>>didn't find any server side examples in comtypes, but perhaps there is
                  >>>a way?[/color]
                  >>
                  >>There is not yet any server support in comtypes, but it will be added.
                  >>In the meantime you could use the ctypes.com package that is included in
                  >>ctypes itself, there are even samples.
                  >>
                  >>Thomas[/color]
                  >
                  >
                  > Thank you, I will try that. Right now I've been trying to use win32com
                  > for the server creation, but I haven't been successful yet (have not
                  > explored it that much yet though).
                  >
                  > I have registered a COM server (it should only implement two methods,
                  > "onMsg()" and "onNotify() ", which should print whatever is sent to
                  > them directly to stdout), but I need to create an interface which is
                  > not an IDispatch interface and pass it on to the proprietary COM
                  > server. This connection is needed so that the proprietary COM server
                  > knows where to send the log and control data.
                  >
                  > Registering the COM object in the registry works fine, but how do I
                  > make it look like a custom interface from the proprietary COM servers
                  > point of view? Oh well, I will look at it some more tomorrow and
                  > hopefully come up with a solution...
                  >[/color]
                  Hi Joakim,

                  This server I wrote for the Palm Desktop with win32com, it creates a none IDispatch COMserver.
                  Maybe it can be of help. I am interested in what you find out on this.

                  Regards,

                  Paul Casteels


                  import pythoncom
                  import pywintypes
                  import winerror
                  import types,time,stri ng
                  import win32com
                  import win32ui,win32co n
                  import rfc822
                  import XmlWriter

                  from win32com.server .exception import COMException
                  from win32com.server .util import wrap
                  from win32com.client import gencache,Dispat ch,constants
                  from win32com import universal



                  # The following 3 lines may need tweaking for the particular server
                  # Candidates are pythoncom.Missi ng and pythoncom.Empty
                  defaultNamedOpt Arg=pythoncom.M issing
                  defaultNamedNot OptArg=pythonco m.Missing
                  defaultUnnamedA rg=pythoncom.Mi ssing

                  palmApp = 0
                  outfileName = r'c:\Documents And Settings\paul\M y Documents\Pytho n\PDA\cal2.xml'

                  universal.Regis terInterfaces(' {C9B354D8-4A1C-11D5-81D2-00C04FA03755}', 0, 1, 0, ["IDesktopAd din"])
                  universal.Regis terInterfaces(' {C9B354D8-4A1C-11D5-81D2-00C04FA03755}', 0, 1, 0, ["IDesktopComman d"])

                  def d2s(inDate):
                  return rfc822.mktime_t z(rfc822.parsed ate_tz(inDate))
                  def s2d(inDate):
                  return time.asctime(ti me.localtime(in Date))



                  class DesktopCommand:
                  global palmApp

                  _public_methods _ = ['OnCommand']
                  _com_interfaces _ = ['IDesktopComman d']

                  def OnCommand(self) :
                  print "Hello, command"
                  print palmApp
                  win32ui.Message Box("PC-Hello","PC-Title",win32con .MB_OK )

                  class DesktopAddin:
                  _public_methods _ = ['GetFrameworkVe rsion',
                  'OnInitialize',
                  'OnUninitialize ',
                  'ShowAboutBox',
                  'ShowProperties ']
                  _reg_progid_ = "PythonPalm.Des ktopAddin"
                  _reg_clsid_ = "{6F36C1D4-571C-484E-B47B-61297761B3D1}"
                  _com_interfaces _ = ['IDesktopAddin']

                  def GetFrameworkVer sion(self, plVersion=defau ltNamedNotOptAr g):
                  """method GetFrameworkVer sion"""
                  return 2

                  def OnInitialize(se lf, pUnk=defaultNam edNotOptArg, dwCookie=defaul tNamedNotOptArg , \
                  dwDesktopVersio n=defaultNamedN otOptArg, lParam=defaultN amedNotOptArg, \
                  pbSuccess=defau ltNamedNotOptAr g):
                  """method OnInitialize"""
                  global palmApp
                  self.cookie = dwCookie
                  palmApp = Dispatch(pUnk)
                  # add a command
                  self.cmd = wrap(DesktopCom mand())
                  addinMgr = palmApp.AddinMa nager
                  cmdId = 0
                  cmdId = addinMgr.AddAdd inCommand(self. cookie, "AddressCommand ", \
                  self.cmd, cmdId)
                  cmdDesc = "&Address Command"
                  # & vbTab & _
                  # "Displays a list of 10 contacts." & vbTab & _
                  # "Displays 10 contacts"

                  addinMgr.AddAdd inCommandDetail s(cmdId, 0, 0, cmdDesc, constants.ePimA ddressBook)

                  # no data here yet available
                  return 1

                  def OnUninitialize( self):
                  """method OnUninitialize" ""
                  global palmApp

                  of = XmlWriter.XmlFi le(outfileName, palmApp.Current User.Name)

                  dbPim = palmApp.DateBoo kPim
                  startDate = '1 Jan 2000 0:0:0'
                  endDate = '1 Jan 2010 0:0:0'
                  events = dbPim.Events(d2 s(startDate),d2 s(endDate))
                  for j in range(events.Co unt):
                  startDate = time.localtime( events[j].StartTime)
                  day = of.day(startDat e)
                  app = {}
                  app['description'] = events[j].Description
                  if events[j].Untimed:
                  app['untimed'] = 1
                  else:
                  app['untimed'] = 0
                  app['starttime'] = s2d(events[j].StartTime)
                  app['endtime'] = s2d(events[j].EndTime)
                  if events[j].Note:
                  app['note'] = events[j].Note
                  # app['id'] = 'id%d' % (events[j].RecordId)
                  of.appointment( app,day)
                  of.close()

                  # adPim = palmApp.Address BookPim
                  # contacts = adPim.Contacts( "All")
                  # for j in range(contacts. Count):
                  # print contacts[j].LastName
                  return None

                  def ShowAboutBox(se lf, hWndParent=defa ultNamedNotOptA rg):
                  """method ShowAboutBox"""
                  print 'ShowAboutBox'
                  print hWndParent
                  return None

                  def ShowProperties( self, hWndParent=defa ultNamedNotOptA rg):
                  """method ShowProperties" ""
                  print 'ShowProperties '
                  return None

                  _prop_map_get_ = {
                  }
                  _prop_map_put_ = {
                  }

                  if __name__ == "__main__":
                  print "Registerin g COM server"
                  import win32com.server .register
                  win32com.server .register.UseCo mmandLine(Deskt opAddin)

                  print "Registerin g our server with Palm desktop"
                  palmReg = Dispatch("PalmD esktop.Registra tionServices")
                  lRegistered = palmReg.IsAddin Registered ("PythonPalm.De sktopAddin")
                  if lRegistered == 1:
                  print "Removing PythonPalm.Desk topAddin"
                  addinInfo = palmReg.GetAddi nRegInfo("Pytho nPalm.DesktopAd din")
                  addinInfo.Delet e()
                  print "Registerin g PythonPalm.Desk topAddin"
                  palmReg.Registe rAddin ("PythonPalm.De sktopAddin","Py thon Addin")

                  Comment

                  Working...