Singleton Problem

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Sudesh Sawant

    Singleton Problem

    Hello,
    We have an application which communicates using remoting. There is a server
    which is a Windows Service. The server exposes an object which is a
    singleton. The client is a Web Application which makes calls to the service.
    We are using tcp channel which is using binaryformatter by default. The
    problem is that after a certain number of remoting calls the calls dont get
    through to the server. The client application makes the call and hangs up.
    The call never reaches the service. The initial calls get through. Earlier we
    used http channel. Then the problem was worse. The client could not make more
    than 20 calls. Now it hangs up after about 100 or so calls if made
    sucessively. We had come to conclusion that it is some threadpool issue as we
    observed that after some time the thread count goes high. The observation in
    case of both http and tcp was that if the thread count crosses a certain
    limit, the server does not accept more calls. If the calls don't come
    successively, ie with in a short amount of time, the service does accept
    calls. I believe this is not a synchronization problem as we have use sync
    objects to protect in remoting calls. Also another point in this favor is
    that the same code hangs up earlier with http channel.
    The configuration for remote server is
    <system.runtime .remoting>
    <application>
    <service>
    <wellknown mode="Singleton " displayName="** ***"
    type="***.***,S ervice.exe" objectUri="***. rem" />
    </service>
    <channels>
    <channel ref="tcp" port="1978"></channel>
    </channels>
    </application>
    </system.runtime. remoting>

    // This is in the shared assembly
    public __gc __interface IInterface
    {
    virtual System::String* Request(System: :String* a_pRequest) = 0;
    };

    // This is in an assembly with the service
    public __gc class CInterface : public IInterface, public
    System::Marshal ByRefObject
    {
    public:
    // Singleton Pattern applied
    virtual System::String* Request(System: :String* a_pRequest) {
    privateLock(); Process(); privateUnLock() ;
    System::GC::Col lect(); // After ading this we get to process more calls
    return "****" };
    virtual System::Object* InitializeLifet imeService() {return 0;};
    void privateRegister ForRemoting()
    {
    try
    {

    System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
    System::Runtime ::Remoting::Rem otingServices:: Marshal(this, "***.rem");
    }
    catch(System::E xception* a_pException)
    {
    WRITETOSYSTEMLO G(a_pException) ;
    }
    };
    What could be the problem and how can this be addressed? We strongly think
    this is some threadpool issue. Or there may be some problem the way we use
    remoting.
    We can change it to singlecall if needed, but first we want to check the issue
  • Brian Gideon

    #2
    Re: Singleton Problem

    Sudesh,

    The Request method calls privateLock, Process, and privateUnLock in
    that order. One thing that jumps out at me is that you don't have the
    Process method call wrapped in a try/catch/finally block. If an
    unhandled exception is thrown in the Process method then the
    privateUnLock method won't be called and all subsequent remoting calls
    will be blocked indefinitely.

    Brian

    Sudesh Sawant wrote:[color=blue]
    > Hello,
    > We have an application which communicates using remoting. There is a server
    > which is a Windows Service. The server exposes an object which is a
    > singleton. The client is a Web Application which makes calls to the service.
    > We are using tcp channel which is using binaryformatter by default. The
    > problem is that after a certain number of remoting calls the calls dont get
    > through to the server. The client application makes the call and hangs up.
    > The call never reaches the service. The initial calls get through. Earlier we
    > used http channel. Then the problem was worse. The client could not make more
    > than 20 calls. Now it hangs up after about 100 or so calls if made
    > sucessively. We had come to conclusion that it is some threadpool issue as we
    > observed that after some time the thread count goes high. The observation in
    > case of both http and tcp was that if the thread count crosses a certain
    > limit, the server does not accept more calls. If the calls don't come
    > successively, ie with in a short amount of time, the service does accept
    > calls. I believe this is not a synchronization problem as we have use sync
    > objects to protect in remoting calls. Also another point in this favor is
    > that the same code hangs up earlier with http channel.
    > The configuration for remote server is
    > <system.runtime .remoting>
    > <application>
    > <service>
    > <wellknown mode="Singleton " displayName="** ***"
    > type="***.***,S ervice.exe" objectUri="***. rem" />
    > </service>
    > <channels>
    > <channel ref="tcp" port="1978"></channel>
    > </channels>
    > </application>
    > </system.runtime. remoting>
    >
    > // This is in the shared assembly
    > public __gc __interface IInterface
    > {
    > virtual System::String* Request(System: :String* a_pRequest) = 0;
    > };
    >
    > // This is in an assembly with the service
    > public __gc class CInterface : public IInterface, public
    > System::Marshal ByRefObject
    > {
    > public:
    > // Singleton Pattern applied
    > virtual System::String* Request(System: :String* a_pRequest) {
    > privateLock(); Process(); privateUnLock() ;
    > System::GC::Col lect(); // After ading this we get to process more calls
    > return "****" };
    > virtual System::Object* InitializeLifet imeService() {return 0;};
    > void privateRegister ForRemoting()
    > {
    > try
    > {
    >
    > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
    > System::Runtime ::Remoting::Rem otingServices:: Marshal(this, "***.rem");
    > }
    > catch(System::E xception* a_pException)
    > {
    > WRITETOSYSTEMLO G(a_pException) ;
    > }
    > };
    > What could be the problem and how can this be addressed? We strongly think
    > this is some threadpool issue. Or there may be some problem the way we use
    > remoting.
    > We can change it to singlecall if needed, but first we want to check the issue[/color]

    Comment

    • Sudesh Sawant

      #3
      Re: Singleton Problem

      Hi Brian,
      Process is just a dummy function. The actual processing function has try
      catch blocks.
      My main problem is that after a certain number of remoting calls the calls
      dont get through to the server. I had tied searching on the net for the
      problem and found from a couple of sites that people where facing a similar
      problem. We got improvements in performance when we changed the channel to
      tcp, but still it doesnot suffice. The explanation given for this was that
      the clr threadpool gets exhausted and cant fork more threads for more
      requests. I had tried to check with my code if there are any leaks but then i
      get the threadcount decreasing after the request has been completed, so there
      may be no such leak. But somewhere the requests have piledup at lower level
      and the pool gets exhausted
      My last approach is to try singlecall.
      Cheers,
      Sudesh
      "Brian Gideon" wrote:
      [color=blue]
      > Sudesh,
      >
      > The Request method calls privateLock, Process, and privateUnLock in
      > that order. One thing that jumps out at me is that you don't have the
      > Process method call wrapped in a try/catch/finally block. If an
      > unhandled exception is thrown in the Process method then the
      > privateUnLock method won't be called and all subsequent remoting calls
      > will be blocked indefinitely.
      >
      > Brian
      >
      > Sudesh Sawant wrote:[color=green]
      > > Hello,
      > > We have an application which communicates using remoting. There is a server
      > > which is a Windows Service. The server exposes an object which is a
      > > singleton. The client is a Web Application which makes calls to the service.
      > > We are using tcp channel which is using binaryformatter by default. The
      > > problem is that after a certain number of remoting calls the calls dont get
      > > through to the server. The client application makes the call and hangs up.
      > > The call never reaches the service. The initial calls get through. Earlier we
      > > used http channel. Then the problem was worse. The client could not make more
      > > than 20 calls. Now it hangs up after about 100 or so calls if made
      > > sucessively. We had come to conclusion that it is some threadpool issue as we
      > > observed that after some time the thread count goes high. The observation in
      > > case of both http and tcp was that if the thread count crosses a certain
      > > limit, the server does not accept more calls. If the calls don't come
      > > successively, ie with in a short amount of time, the service does accept
      > > calls. I believe this is not a synchronization problem as we have use sync
      > > objects to protect in remoting calls. Also another point in this favor is
      > > that the same code hangs up earlier with http channel.
      > > The configuration for remote server is
      > > <system.runtime .remoting>
      > > <application>
      > > <service>
      > > <wellknown mode="Singleton " displayName="** ***"
      > > type="***.***,S ervice.exe" objectUri="***. rem" />
      > > </service>
      > > <channels>
      > > <channel ref="tcp" port="1978"></channel>
      > > </channels>
      > > </application>
      > > </system.runtime. remoting>
      > >
      > > // This is in the shared assembly
      > > public __gc __interface IInterface
      > > {
      > > virtual System::String* Request(System: :String* a_pRequest) = 0;
      > > };
      > >
      > > // This is in an assembly with the service
      > > public __gc class CInterface : public IInterface, public
      > > System::Marshal ByRefObject
      > > {
      > > public:
      > > // Singleton Pattern applied
      > > virtual System::String* Request(System: :String* a_pRequest) {
      > > privateLock(); Process(); privateUnLock() ;
      > > System::GC::Col lect(); // After ading this we get to process more calls
      > > return "****" };
      > > virtual System::Object* InitializeLifet imeService() {return 0;};
      > > void privateRegister ForRemoting()
      > > {
      > > try
      > > {
      > >
      > > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
      > > System::Runtime ::Remoting::Rem otingServices:: Marshal(this, "***.rem");
      > > }
      > > catch(System::E xception* a_pException)
      > > {
      > > WRITETOSYSTEMLO G(a_pException) ;
      > > }
      > > };
      > > What could be the problem and how can this be addressed? We strongly think
      > > this is some threadpool issue. Or there may be some problem the way we use
      > > remoting.
      > > We can change it to singlecall if needed, but first we want to check the issue[/color]
      >
      >[/color]

      Comment

      • Jared Parsons [MSFT]

        #4
        Re: Singleton Problem

        Have you attached a debbuger to the service when the call hangs? If so how
        many threads are running on the service?

        --
        Jared Parsons [MSFT]
        jaredpar@online .microsoft.com

        "This posting is provided "AS IS" with no warranties, and confers no rights"
        "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
        news:5DA515D2-D5E6-4F56-ACB2-5F071A9EC73E@mi crosoft.com...[color=blue]
        > Hi Brian,
        > Process is just a dummy function. The actual processing function has try
        > catch blocks.
        > My main problem is that after a certain number of remoting calls the calls
        > dont get through to the server. I had tied searching on the net for the
        > problem and found from a couple of sites that people where facing a
        > similar
        > problem. We got improvements in performance when we changed the channel to
        > tcp, but still it doesnot suffice. The explanation given for this was that
        > the clr threadpool gets exhausted and cant fork more threads for more
        > requests. I had tried to check with my code if there are any leaks but
        > then i
        > get the threadcount decreasing after the request has been completed, so
        > there
        > may be no such leak. But somewhere the requests have piledup at lower
        > level
        > and the pool gets exhausted
        > My last approach is to try singlecall.
        > Cheers,
        > Sudesh
        > "Brian Gideon" wrote:
        >[color=green]
        >> Sudesh,
        >>
        >> The Request method calls privateLock, Process, and privateUnLock in
        >> that order. One thing that jumps out at me is that you don't have the
        >> Process method call wrapped in a try/catch/finally block. If an
        >> unhandled exception is thrown in the Process method then the
        >> privateUnLock method won't be called and all subsequent remoting calls
        >> will be blocked indefinitely.
        >>
        >> Brian
        >>
        >> Sudesh Sawant wrote:[color=darkred]
        >> > Hello,
        >> > We have an application which communicates using remoting. There is a
        >> > server
        >> > which is a Windows Service. The server exposes an object which is a
        >> > singleton. The client is a Web Application which makes calls to the
        >> > service.
        >> > We are using tcp channel which is using binaryformatter by default. The
        >> > problem is that after a certain number of remoting calls the calls dont
        >> > get
        >> > through to the server. The client application makes the call and hangs
        >> > up.
        >> > The call never reaches the service. The initial calls get through.
        >> > Earlier we
        >> > used http channel. Then the problem was worse. The client could not
        >> > make more
        >> > than 20 calls. Now it hangs up after about 100 or so calls if made
        >> > sucessively. We had come to conclusion that it is some threadpool issue
        >> > as we
        >> > observed that after some time the thread count goes high. The
        >> > observation in
        >> > case of both http and tcp was that if the thread count crosses a
        >> > certain
        >> > limit, the server does not accept more calls. If the calls don't come
        >> > successively, ie with in a short amount of time, the service does
        >> > accept
        >> > calls. I believe this is not a synchronization problem as we have use
        >> > sync
        >> > objects to protect in remoting calls. Also another point in this favor
        >> > is
        >> > that the same code hangs up earlier with http channel.
        >> > The configuration for remote server is
        >> > <system.runtime .remoting>
        >> > <application>
        >> > <service>
        >> > <wellknown mode="Singleton " displayName="** ***"
        >> > type="***.***,S ervice.exe" objectUri="***. rem" />
        >> > </service>
        >> > <channels>
        >> > <channel ref="tcp" port="1978"></channel>
        >> > </channels>
        >> > </application>
        >> > </system.runtime. remoting>
        >> >
        >> > // This is in the shared assembly
        >> > public __gc __interface IInterface
        >> > {
        >> > virtual System::String* Request(System: :String* a_pRequest) = 0;
        >> > };
        >> >
        >> > // This is in an assembly with the service
        >> > public __gc class CInterface : public IInterface, public
        >> > System::Marshal ByRefObject
        >> > {
        >> > public:
        >> > // Singleton Pattern applied
        >> > virtual System::String* Request(System: :String* a_pRequest) {
        >> > privateLock(); Process(); privateUnLock() ;
        >> > System::GC::Col lect(); // After ading this we get to process more calls
        >> > return "****" };
        >> > virtual System::Object* InitializeLifet imeService() {return 0;};
        >> > void privateRegister ForRemoting()
        >> > {
        >> > try
        >> > {
        >> >
        >> > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
        >> > System::Runtime ::Remoting::Rem otingServices:: Marshal(this, "***.rem");
        >> > }
        >> > catch(System::E xception* a_pException)
        >> > {
        >> > WRITETOSYSTEMLO G(a_pException) ;
        >> > }
        >> > };
        >> > What could be the problem and how can this be addressed? We strongly
        >> > think
        >> > this is some threadpool issue. Or there may be some problem the way we
        >> > use
        >> > remoting.
        >> > We can change it to singlecall if needed, but first we want to check
        >> > the issue[/color]
        >>
        >>[/color][/color]


        Comment

        • Sudesh Sawant

          #5
          Re: Singleton Problem

          Hello Jared,
          I had attached the debugger to the service. Infact I had tried to trace it
          through the client to my servce. The request goes ahead from the client and
          does not reach the service. I had tried to solve the problem in the following
          steps.
          1. Called GC::Collect from the methods (methods which are being remotely
          called). This improved the performance a lot.
          2. Changed from http channel to tcp channel. Even this improved the
          performance, but still the requests are limited.

          When looking for number of threads, the thread count at which it stops
          functioning in case of tcp is 91 and in case of http it is 54. Out of these
          around 33 threads are created by the service. The other threads are spawned
          at every request. If there is some time lag in between requests, the thread
          count goes down. but the service stops remoting requests after the threshold
          mentioned above is reached. I had found a link which recommendd changing the
          threadpool size, but i want to try this as the last solution.

          Cheers,
          sudesh


          "Jared Parsons [MSFT]" wrote:
          [color=blue]
          > Have you attached a debbuger to the service when the call hangs? If so how
          > many threads are running on the service?
          >
          > --
          > Jared Parsons [MSFT]
          > jaredpar@online .microsoft.com
          > http://blogs.msdn.com/jaredpar
          > "This posting is provided "AS IS" with no warranties, and confers no rights"
          > "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
          > news:5DA515D2-D5E6-4F56-ACB2-5F071A9EC73E@mi crosoft.com...[color=green]
          > > Hi Brian,
          > > Process is just a dummy function. The actual processing function has try
          > > catch blocks.
          > > My main problem is that after a certain number of remoting calls the calls
          > > dont get through to the server. I had tied searching on the net for the
          > > problem and found from a couple of sites that people where facing a
          > > similar
          > > problem. We got improvements in performance when we changed the channel to
          > > tcp, but still it doesnot suffice. The explanation given for this was that
          > > the clr threadpool gets exhausted and cant fork more threads for more
          > > requests. I had tried to check with my code if there are any leaks but
          > > then i
          > > get the threadcount decreasing after the request has been completed, so
          > > there
          > > may be no such leak. But somewhere the requests have piledup at lower
          > > level
          > > and the pool gets exhausted
          > > My last approach is to try singlecall.
          > > Cheers,
          > > Sudesh
          > > "Brian Gideon" wrote:
          > >[color=darkred]
          > >> Sudesh,
          > >>
          > >> The Request method calls privateLock, Process, and privateUnLock in
          > >> that order. One thing that jumps out at me is that you don't have the
          > >> Process method call wrapped in a try/catch/finally block. If an
          > >> unhandled exception is thrown in the Process method then the
          > >> privateUnLock method won't be called and all subsequent remoting calls
          > >> will be blocked indefinitely.
          > >>
          > >> Brian
          > >>
          > >> Sudesh Sawant wrote:
          > >> > Hello,
          > >> > We have an application which communicates using remoting. There is a
          > >> > server
          > >> > which is a Windows Service. The server exposes an object which is a
          > >> > singleton. The client is a Web Application which makes calls to the
          > >> > service.
          > >> > We are using tcp channel which is using binaryformatter by default. The
          > >> > problem is that after a certain number of remoting calls the calls dont
          > >> > get
          > >> > through to the server. The client application makes the call and hangs
          > >> > up.
          > >> > The call never reaches the service. The initial calls get through.
          > >> > Earlier we
          > >> > used http channel. Then the problem was worse. The client could not
          > >> > make more
          > >> > than 20 calls. Now it hangs up after about 100 or so calls if made
          > >> > sucessively. We had come to conclusion that it is some threadpool issue
          > >> > as we
          > >> > observed that after some time the thread count goes high. The
          > >> > observation in
          > >> > case of both http and tcp was that if the thread count crosses a
          > >> > certain
          > >> > limit, the server does not accept more calls. If the calls don't come
          > >> > successively, ie with in a short amount of time, the service does
          > >> > accept
          > >> > calls. I believe this is not a synchronization problem as we have use
          > >> > sync
          > >> > objects to protect in remoting calls. Also another point in this favor
          > >> > is
          > >> > that the same code hangs up earlier with http channel.
          > >> > The configuration for remote server is
          > >> > <system.runtime .remoting>
          > >> > <application>
          > >> > <service>
          > >> > <wellknown mode="Singleton " displayName="** ***"
          > >> > type="***.***,S ervice.exe" objectUri="***. rem" />
          > >> > </service>
          > >> > <channels>
          > >> > <channel ref="tcp" port="1978"></channel>
          > >> > </channels>
          > >> > </application>
          > >> > </system.runtime. remoting>
          > >> >
          > >> > // This is in the shared assembly
          > >> > public __gc __interface IInterface
          > >> > {
          > >> > virtual System::String* Request(System: :String* a_pRequest) = 0;
          > >> > };
          > >> >
          > >> > // This is in an assembly with the service
          > >> > public __gc class CInterface : public IInterface, public
          > >> > System::Marshal ByRefObject
          > >> > {
          > >> > public:
          > >> > // Singleton Pattern applied
          > >> > virtual System::String* Request(System: :String* a_pRequest) {
          > >> > privateLock(); Process(); privateUnLock() ;
          > >> > System::GC::Col lect(); // After ading this we get to process more calls
          > >> > return "****" };
          > >> > virtual System::Object* InitializeLifet imeService() {return 0;};
          > >> > void privateRegister ForRemoting()
          > >> > {
          > >> > try
          > >> > {
          > >> >
          > >> > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
          > >> > System::Runtime ::Remoting::Rem otingServices:: Marshal(this, "***.rem");
          > >> > }
          > >> > catch(System::E xception* a_pException)
          > >> > {
          > >> > WRITETOSYSTEMLO G(a_pException) ;
          > >> > }
          > >> > };
          > >> > What could be the problem and how can this be addressed? We strongly
          > >> > think
          > >> > this is some threadpool issue. Or there may be some problem the way we
          > >> > use
          > >> > remoting.
          > >> > We can change it to singlecall if needed, but first we want to check
          > >> > the issue
          > >>
          > >>[/color][/color]
          >
          >
          >[/color]

          Comment

          • Jared Parsons [MSFT]

            #6
            Re: Singleton Problem

            You could try isolating the problem by hacking up your service to give back
            a default answer and not spawn any of it's own threads and see if you still
            hit the problem

            --
            Jared Parsons [MSFT]
            jaredpar@online .microsoft.com

            "This posting is provided "AS IS" with no warranties, and confers no rights"
            "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
            news:4D58161F-09A4-4109-9E7A-12AA69F9A410@mi crosoft.com...[color=blue]
            > Hello Jared,
            > I had attached the debugger to the service. Infact I had tried to trace it
            > through the client to my servce. The request goes ahead from the client
            > and
            > does not reach the service. I had tried to solve the problem in the
            > following
            > steps.
            > 1. Called GC::Collect from the methods (methods which are being remotely
            > called). This improved the performance a lot.
            > 2. Changed from http channel to tcp channel. Even this improved the
            > performance, but still the requests are limited.
            >
            > When looking for number of threads, the thread count at which it stops
            > functioning in case of tcp is 91 and in case of http it is 54. Out of
            > these
            > around 33 threads are created by the service. The other threads are
            > spawned
            > at every request. If there is some time lag in between requests, the
            > thread
            > count goes down. but the service stops remoting requests after the
            > threshold
            > mentioned above is reached. I had found a link which recommendd changing
            > the
            > threadpool size, but i want to try this as the last solution.
            >
            > Cheers,
            > sudesh
            >
            >
            > "Jared Parsons [MSFT]" wrote:
            >[color=green]
            >> Have you attached a debbuger to the service when the call hangs? If so
            >> how
            >> many threads are running on the service?
            >>
            >> --
            >> Jared Parsons [MSFT]
            >> jaredpar@online .microsoft.com
            >> http://blogs.msdn.com/jaredpar
            >> "This posting is provided "AS IS" with no warranties, and confers no
            >> rights"
            >> "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
            >> news:5DA515D2-D5E6-4F56-ACB2-5F071A9EC73E@mi crosoft.com...[color=darkred]
            >> > Hi Brian,
            >> > Process is just a dummy function. The actual processing function has
            >> > try
            >> > catch blocks.
            >> > My main problem is that after a certain number of remoting calls the
            >> > calls
            >> > dont get through to the server. I had tied searching on the net for the
            >> > problem and found from a couple of sites that people where facing a
            >> > similar
            >> > problem. We got improvements in performance when we changed the channel
            >> > to
            >> > tcp, but still it doesnot suffice. The explanation given for this was
            >> > that
            >> > the clr threadpool gets exhausted and cant fork more threads for more
            >> > requests. I had tried to check with my code if there are any leaks but
            >> > then i
            >> > get the threadcount decreasing after the request has been completed, so
            >> > there
            >> > may be no such leak. But somewhere the requests have piledup at lower
            >> > level
            >> > and the pool gets exhausted
            >> > My last approach is to try singlecall.
            >> > Cheers,
            >> > Sudesh
            >> > "Brian Gideon" wrote:
            >> >
            >> >> Sudesh,
            >> >>
            >> >> The Request method calls privateLock, Process, and privateUnLock in
            >> >> that order. One thing that jumps out at me is that you don't have the
            >> >> Process method call wrapped in a try/catch/finally block. If an
            >> >> unhandled exception is thrown in the Process method then the
            >> >> privateUnLock method won't be called and all subsequent remoting calls
            >> >> will be blocked indefinitely.
            >> >>
            >> >> Brian
            >> >>
            >> >> Sudesh Sawant wrote:
            >> >> > Hello,
            >> >> > We have an application which communicates using remoting. There is a
            >> >> > server
            >> >> > which is a Windows Service. The server exposes an object which is a
            >> >> > singleton. The client is a Web Application which makes calls to the
            >> >> > service.
            >> >> > We are using tcp channel which is using binaryformatter by default.
            >> >> > The
            >> >> > problem is that after a certain number of remoting calls the calls
            >> >> > dont
            >> >> > get
            >> >> > through to the server. The client application makes the call and
            >> >> > hangs
            >> >> > up.
            >> >> > The call never reaches the service. The initial calls get through.
            >> >> > Earlier we
            >> >> > used http channel. Then the problem was worse. The client could not
            >> >> > make more
            >> >> > than 20 calls. Now it hangs up after about 100 or so calls if made
            >> >> > sucessively. We had come to conclusion that it is some threadpool
            >> >> > issue
            >> >> > as we
            >> >> > observed that after some time the thread count goes high. The
            >> >> > observation in
            >> >> > case of both http and tcp was that if the thread count crosses a
            >> >> > certain
            >> >> > limit, the server does not accept more calls. If the calls don't
            >> >> > come
            >> >> > successively, ie with in a short amount of time, the service does
            >> >> > accept
            >> >> > calls. I believe this is not a synchronization problem as we have
            >> >> > use
            >> >> > sync
            >> >> > objects to protect in remoting calls. Also another point in this
            >> >> > favor
            >> >> > is
            >> >> > that the same code hangs up earlier with http channel.
            >> >> > The configuration for remote server is
            >> >> > <system.runtime .remoting>
            >> >> > <application>
            >> >> > <service>
            >> >> > <wellknown mode="Singleton " displayName="** ***"
            >> >> > type="***.***,S ervice.exe" objectUri="***. rem" />
            >> >> > </service>
            >> >> > <channels>
            >> >> > <channel ref="tcp" port="1978"></channel>
            >> >> > </channels>
            >> >> > </application>
            >> >> > </system.runtime. remoting>
            >> >> >
            >> >> > // This is in the shared assembly
            >> >> > public __gc __interface IInterface
            >> >> > {
            >> >> > virtual System::String* Request(System: :String* a_pRequest) = 0;
            >> >> > };
            >> >> >
            >> >> > // This is in an assembly with the service
            >> >> > public __gc class CInterface : public IInterface, public
            >> >> > System::Marshal ByRefObject
            >> >> > {
            >> >> > public:
            >> >> > // Singleton Pattern applied
            >> >> > virtual System::String* Request(System: :String* a_pRequest) {
            >> >> > privateLock(); Process(); privateUnLock() ;
            >> >> > System::GC::Col lect(); // After ading this we get to process more
            >> >> > calls
            >> >> > return "****" };
            >> >> > virtual System::Object* InitializeLifet imeService() {return 0;};
            >> >> > void privateRegister ForRemoting()
            >> >> > {
            >> >> > try
            >> >> > {
            >> >> >
            >> >> > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
            >> >> > System::Runtime ::Remoting::Rem otingServices:: Marshal(this,
            >> >> > "***.rem");
            >> >> > }
            >> >> > catch(System::E xception* a_pException)
            >> >> > {
            >> >> > WRITETOSYSTEMLO G(a_pException) ;
            >> >> > }
            >> >> > };
            >> >> > What could be the problem and how can this be addressed? We strongly
            >> >> > think
            >> >> > this is some threadpool issue. Or there may be some problem the way
            >> >> > we
            >> >> > use
            >> >> > remoting.
            >> >> > We can change it to singlecall if needed, but first we want to check
            >> >> > the issue
            >> >>
            >> >>[/color]
            >>
            >>
            >>[/color][/color]


            Comment

            • Sudesh Sawant

              #7
              Re: Singleton Problem

              Ok, Will try this.
              Cheers,
              Sudesh Sawant

              "Jared Parsons [MSFT]" wrote:
              [color=blue]
              > You could try isolating the problem by hacking up your service to give back
              > a default answer and not spawn any of it's own threads and see if you still
              > hit the problem
              >
              > --
              > Jared Parsons [MSFT]
              > jaredpar@online .microsoft.com
              > http://blogs.msdn.com/jaredpar
              > "This posting is provided "AS IS" with no warranties, and confers no rights"
              > "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
              > news:4D58161F-09A4-4109-9E7A-12AA69F9A410@mi crosoft.com...[color=green]
              > > Hello Jared,
              > > I had attached the debugger to the service. Infact I had tried to trace it
              > > through the client to my servce. The request goes ahead from the client
              > > and
              > > does not reach the service. I had tried to solve the problem in the
              > > following
              > > steps.
              > > 1. Called GC::Collect from the methods (methods which are being remotely
              > > called). This improved the performance a lot.
              > > 2. Changed from http channel to tcp channel. Even this improved the
              > > performance, but still the requests are limited.
              > >
              > > When looking for number of threads, the thread count at which it stops
              > > functioning in case of tcp is 91 and in case of http it is 54. Out of
              > > these
              > > around 33 threads are created by the service. The other threads are
              > > spawned
              > > at every request. If there is some time lag in between requests, the
              > > thread
              > > count goes down. but the service stops remoting requests after the
              > > threshold
              > > mentioned above is reached. I had found a link which recommendd changing
              > > the
              > > threadpool size, but i want to try this as the last solution.
              > >
              > > Cheers,
              > > sudesh
              > >
              > >
              > > "Jared Parsons [MSFT]" wrote:
              > >[color=darkred]
              > >> Have you attached a debbuger to the service when the call hangs? If so
              > >> how
              > >> many threads are running on the service?
              > >>
              > >> --
              > >> Jared Parsons [MSFT]
              > >> jaredpar@online .microsoft.com
              > >> http://blogs.msdn.com/jaredpar
              > >> "This posting is provided "AS IS" with no warranties, and confers no
              > >> rights"
              > >> "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
              > >> news:5DA515D2-D5E6-4F56-ACB2-5F071A9EC73E@mi crosoft.com...
              > >> > Hi Brian,
              > >> > Process is just a dummy function. The actual processing function has
              > >> > try
              > >> > catch blocks.
              > >> > My main problem is that after a certain number of remoting calls the
              > >> > calls
              > >> > dont get through to the server. I had tied searching on the net for the
              > >> > problem and found from a couple of sites that people where facing a
              > >> > similar
              > >> > problem. We got improvements in performance when we changed the channel
              > >> > to
              > >> > tcp, but still it doesnot suffice. The explanation given for this was
              > >> > that
              > >> > the clr threadpool gets exhausted and cant fork more threads for more
              > >> > requests. I had tried to check with my code if there are any leaks but
              > >> > then i
              > >> > get the threadcount decreasing after the request has been completed, so
              > >> > there
              > >> > may be no such leak. But somewhere the requests have piledup at lower
              > >> > level
              > >> > and the pool gets exhausted
              > >> > My last approach is to try singlecall.
              > >> > Cheers,
              > >> > Sudesh
              > >> > "Brian Gideon" wrote:
              > >> >
              > >> >> Sudesh,
              > >> >>
              > >> >> The Request method calls privateLock, Process, and privateUnLock in
              > >> >> that order. One thing that jumps out at me is that you don't have the
              > >> >> Process method call wrapped in a try/catch/finally block. If an
              > >> >> unhandled exception is thrown in the Process method then the
              > >> >> privateUnLock method won't be called and all subsequent remoting calls
              > >> >> will be blocked indefinitely.
              > >> >>
              > >> >> Brian
              > >> >>
              > >> >> Sudesh Sawant wrote:
              > >> >> > Hello,
              > >> >> > We have an application which communicates using remoting. There is a
              > >> >> > server
              > >> >> > which is a Windows Service. The server exposes an object which is a
              > >> >> > singleton. The client is a Web Application which makes calls to the
              > >> >> > service.
              > >> >> > We are using tcp channel which is using binaryformatter by default.
              > >> >> > The
              > >> >> > problem is that after a certain number of remoting calls the calls
              > >> >> > dont
              > >> >> > get
              > >> >> > through to the server. The client application makes the call and
              > >> >> > hangs
              > >> >> > up.
              > >> >> > The call never reaches the service. The initial calls get through.
              > >> >> > Earlier we
              > >> >> > used http channel. Then the problem was worse. The client could not
              > >> >> > make more
              > >> >> > than 20 calls. Now it hangs up after about 100 or so calls if made
              > >> >> > sucessively. We had come to conclusion that it is some threadpool
              > >> >> > issue
              > >> >> > as we
              > >> >> > observed that after some time the thread count goes high. The
              > >> >> > observation in
              > >> >> > case of both http and tcp was that if the thread count crosses a
              > >> >> > certain
              > >> >> > limit, the server does not accept more calls. If the calls don't
              > >> >> > come
              > >> >> > successively, ie with in a short amount of time, the service does
              > >> >> > accept
              > >> >> > calls. I believe this is not a synchronization problem as we have
              > >> >> > use
              > >> >> > sync
              > >> >> > objects to protect in remoting calls. Also another point in this
              > >> >> > favor
              > >> >> > is
              > >> >> > that the same code hangs up earlier with http channel.
              > >> >> > The configuration for remote server is
              > >> >> > <system.runtime .remoting>
              > >> >> > <application>
              > >> >> > <service>
              > >> >> > <wellknown mode="Singleton " displayName="** ***"
              > >> >> > type="***.***,S ervice.exe" objectUri="***. rem" />
              > >> >> > </service>
              > >> >> > <channels>
              > >> >> > <channel ref="tcp" port="1978"></channel>
              > >> >> > </channels>
              > >> >> > </application>
              > >> >> > </system.runtime. remoting>
              > >> >> >
              > >> >> > // This is in the shared assembly
              > >> >> > public __gc __interface IInterface
              > >> >> > {
              > >> >> > virtual System::String* Request(System: :String* a_pRequest) = 0;
              > >> >> > };
              > >> >> >
              > >> >> > // This is in an assembly with the service
              > >> >> > public __gc class CInterface : public IInterface, public
              > >> >> > System::Marshal ByRefObject
              > >> >> > {
              > >> >> > public:
              > >> >> > // Singleton Pattern applied
              > >> >> > virtual System::String* Request(System: :String* a_pRequest) {
              > >> >> > privateLock(); Process(); privateUnLock() ;
              > >> >> > System::GC::Col lect(); // After ading this we get to process more
              > >> >> > calls
              > >> >> > return "****" };
              > >> >> > virtual System::Object* InitializeLifet imeService() {return 0;};
              > >> >> > void privateRegister ForRemoting()
              > >> >> > {
              > >> >> > try
              > >> >> > {
              > >> >> >
              > >> >> > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
              > >> >> > System::Runtime ::Remoting::Rem otingServices:: Marshal(this,
              > >> >> > "***.rem");
              > >> >> > }
              > >> >> > catch(System::E xception* a_pException)
              > >> >> > {
              > >> >> > WRITETOSYSTEMLO G(a_pException) ;
              > >> >> > }
              > >> >> > };
              > >> >> > What could be the problem and how can this be addressed? We strongly
              > >> >> > think
              > >> >> > this is some threadpool issue. Or there may be some problem the way
              > >> >> > we
              > >> >> > use
              > >> >> > remoting.
              > >> >> > We can change it to singlecall if needed, but first we want to check
              > >> >> > the issue
              > >> >>
              > >> >>
              > >>
              > >>
              > >>[/color][/color]
              >
              >
              >[/color]

              Comment

              • Sudesh Sawant

                #8
                Re: Singleton Problem

                Hello Jared,
                Still the problem is not resolved.
                Cheers,
                Sudesh

                "Sudesh Sawant" wrote:
                [color=blue]
                > Ok, Will try this.
                > Cheers,
                > Sudesh Sawant
                >
                > "Jared Parsons [MSFT]" wrote:
                >[color=green]
                > > You could try isolating the problem by hacking up your service to give back
                > > a default answer and not spawn any of it's own threads and see if you still
                > > hit the problem
                > >
                > > --
                > > Jared Parsons [MSFT]
                > > jaredpar@online .microsoft.com
                > > http://blogs.msdn.com/jaredpar
                > > "This posting is provided "AS IS" with no warranties, and confers no rights"
                > > "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
                > > news:4D58161F-09A4-4109-9E7A-12AA69F9A410@mi crosoft.com...[color=darkred]
                > > > Hello Jared,
                > > > I had attached the debugger to the service. Infact I had tried to trace it
                > > > through the client to my servce. The request goes ahead from the client
                > > > and
                > > > does not reach the service. I had tried to solve the problem in the
                > > > following
                > > > steps.
                > > > 1. Called GC::Collect from the methods (methods which are being remotely
                > > > called). This improved the performance a lot.
                > > > 2. Changed from http channel to tcp channel. Even this improved the
                > > > performance, but still the requests are limited.
                > > >
                > > > When looking for number of threads, the thread count at which it stops
                > > > functioning in case of tcp is 91 and in case of http it is 54. Out of
                > > > these
                > > > around 33 threads are created by the service. The other threads are
                > > > spawned
                > > > at every request. If there is some time lag in between requests, the
                > > > thread
                > > > count goes down. but the service stops remoting requests after the
                > > > threshold
                > > > mentioned above is reached. I had found a link which recommendd changing
                > > > the
                > > > threadpool size, but i want to try this as the last solution.
                > > >
                > > > Cheers,
                > > > sudesh
                > > >
                > > >
                > > > "Jared Parsons [MSFT]" wrote:
                > > >
                > > >> Have you attached a debbuger to the service when the call hangs? If so
                > > >> how
                > > >> many threads are running on the service?
                > > >>
                > > >> --
                > > >> Jared Parsons [MSFT]
                > > >> jaredpar@online .microsoft.com
                > > >> http://blogs.msdn.com/jaredpar
                > > >> "This posting is provided "AS IS" with no warranties, and confers no
                > > >> rights"
                > > >> "Sudesh Sawant" <SudeshSawant@d iscussions.micr osoft.com> wrote in message
                > > >> news:5DA515D2-D5E6-4F56-ACB2-5F071A9EC73E@mi crosoft.com...
                > > >> > Hi Brian,
                > > >> > Process is just a dummy function. The actual processing function has
                > > >> > try
                > > >> > catch blocks.
                > > >> > My main problem is that after a certain number of remoting calls the
                > > >> > calls
                > > >> > dont get through to the server. I had tied searching on the net for the
                > > >> > problem and found from a couple of sites that people where facing a
                > > >> > similar
                > > >> > problem. We got improvements in performance when we changed the channel
                > > >> > to
                > > >> > tcp, but still it doesnot suffice. The explanation given for this was
                > > >> > that
                > > >> > the clr threadpool gets exhausted and cant fork more threads for more
                > > >> > requests. I had tried to check with my code if there are any leaks but
                > > >> > then i
                > > >> > get the threadcount decreasing after the request has been completed, so
                > > >> > there
                > > >> > may be no such leak. But somewhere the requests have piledup at lower
                > > >> > level
                > > >> > and the pool gets exhausted
                > > >> > My last approach is to try singlecall.
                > > >> > Cheers,
                > > >> > Sudesh
                > > >> > "Brian Gideon" wrote:
                > > >> >
                > > >> >> Sudesh,
                > > >> >>
                > > >> >> The Request method calls privateLock, Process, and privateUnLock in
                > > >> >> that order. One thing that jumps out at me is that you don't have the
                > > >> >> Process method call wrapped in a try/catch/finally block. If an
                > > >> >> unhandled exception is thrown in the Process method then the
                > > >> >> privateUnLock method won't be called and all subsequent remoting calls
                > > >> >> will be blocked indefinitely.
                > > >> >>
                > > >> >> Brian
                > > >> >>
                > > >> >> Sudesh Sawant wrote:
                > > >> >> > Hello,
                > > >> >> > We have an application which communicates using remoting. There is a
                > > >> >> > server
                > > >> >> > which is a Windows Service. The server exposes an object which is a
                > > >> >> > singleton. The client is a Web Application which makes calls to the
                > > >> >> > service.
                > > >> >> > We are using tcp channel which is using binaryformatter by default.
                > > >> >> > The
                > > >> >> > problem is that after a certain number of remoting calls the calls
                > > >> >> > dont
                > > >> >> > get
                > > >> >> > through to the server. The client application makes the call and
                > > >> >> > hangs
                > > >> >> > up.
                > > >> >> > The call never reaches the service. The initial calls get through.
                > > >> >> > Earlier we
                > > >> >> > used http channel. Then the problem was worse. The client could not
                > > >> >> > make more
                > > >> >> > than 20 calls. Now it hangs up after about 100 or so calls if made
                > > >> >> > sucessively. We had come to conclusion that it is some threadpool
                > > >> >> > issue
                > > >> >> > as we
                > > >> >> > observed that after some time the thread count goes high. The
                > > >> >> > observation in
                > > >> >> > case of both http and tcp was that if the thread count crosses a
                > > >> >> > certain
                > > >> >> > limit, the server does not accept more calls. If the calls don't
                > > >> >> > come
                > > >> >> > successively, ie with in a short amount of time, the service does
                > > >> >> > accept
                > > >> >> > calls. I believe this is not a synchronization problem as we have
                > > >> >> > use
                > > >> >> > sync
                > > >> >> > objects to protect in remoting calls. Also another point in this
                > > >> >> > favor
                > > >> >> > is
                > > >> >> > that the same code hangs up earlier with http channel.
                > > >> >> > The configuration for remote server is
                > > >> >> > <system.runtime .remoting>
                > > >> >> > <application>
                > > >> >> > <service>
                > > >> >> > <wellknown mode="Singleton " displayName="** ***"
                > > >> >> > type="***.***,S ervice.exe" objectUri="***. rem" />
                > > >> >> > </service>
                > > >> >> > <channels>
                > > >> >> > <channel ref="tcp" port="1978"></channel>
                > > >> >> > </channels>
                > > >> >> > </application>
                > > >> >> > </system.runtime. remoting>
                > > >> >> >
                > > >> >> > // This is in the shared assembly
                > > >> >> > public __gc __interface IInterface
                > > >> >> > {
                > > >> >> > virtual System::String* Request(System: :String* a_pRequest) = 0;
                > > >> >> > };
                > > >> >> >
                > > >> >> > // This is in an assembly with the service
                > > >> >> > public __gc class CInterface : public IInterface, public
                > > >> >> > System::Marshal ByRefObject
                > > >> >> > {
                > > >> >> > public:
                > > >> >> > // Singleton Pattern applied
                > > >> >> > virtual System::String* Request(System: :String* a_pRequest) {
                > > >> >> > privateLock(); Process(); privateUnLock() ;
                > > >> >> > System::GC::Col lect(); // After ading this we get to process more
                > > >> >> > calls
                > > >> >> > return "****" };
                > > >> >> > virtual System::Object* InitializeLifet imeService() {return 0;};
                > > >> >> > void privateRegister ForRemoting()
                > > >> >> > {
                > > >> >> > try
                > > >> >> > {
                > > >> >> >
                > > >> >> > System::Runtime ::Remoting::Rem otingConfigurat ion::Configure( System::AppDoma in::CurrentDoma in->SetupInformati on->ConfigurationF ile);
                > > >> >> > System::Runtime ::Remoting::Rem otingServices:: Marshal(this,
                > > >> >> > "***.rem");
                > > >> >> > }
                > > >> >> > catch(System::E xception* a_pException)
                > > >> >> > {
                > > >> >> > WRITETOSYSTEMLO G(a_pException) ;
                > > >> >> > }
                > > >> >> > };
                > > >> >> > What could be the problem and how can this be addressed? We strongly
                > > >> >> > think
                > > >> >> > this is some threadpool issue. Or there may be some problem the way
                > > >> >> > we
                > > >> >> > use
                > > >> >> > remoting.
                > > >> >> > We can change it to singlecall if needed, but first we want to check
                > > >> >> > the issue
                > > >> >>
                > > >> >>
                > > >>
                > > >>
                > > >>[/color]
                > >
                > >
                > >[/color][/color]

                Comment

                • Rogas69

                  #9
                  Re: Singleton Problem

                  So even the dummy call of service hangs after 91 times? if so - maybe the
                  problem lays on client side. how many clients do you use to test this? are
                  they multithreaded? how many proxies do you create on client side?

                  Peter


                  Comment

                  Working...