Determining the size of data being sent from Socket Server

[Frinavale]

I'm implementing a Silverlight application that uses Sockets to receive data that is pushed to it from a Socket Server. (Silverlight only supports the TCP protocol)

The Socket Server pushes a series of images to the Silverlight application which displays them. Every 2 seconds the next image in the series is sent until all of the images have been sent, at which time the Socket Server just starts over again.

Everything's working fine right now but that's only because I've made the receive buffer size for the SocketAsyncEven tArgs large enough to accept the whole image file.

When the receive buffer size is smaller than the file more than one packet is sent from the server to the client until the whole file's been sent.

My question is: how do I know how large the file is that's being sent?


Thanks for your time,

-Frinny

[Plater]

Do you control the file sending? Send a datasize first?

[Frinavale]

Thanks Plater,

Yes I control both sides.

I have thought of that solution too but it seems that there should be another way to do this....

The BeginSend method even takes a parameter that indicates the size of the data being sent.

Why can't I see this value client side?

I was actually thinking about serializing each packet so that the size would be included at the beginning....I was reading an article on this last week but when I tried access it this morning it wasn't there any more.....

Maybe this serialization thing isn't the best idea?

-Frinny

[Plater]

Well msdn.com and my local msdn library both seem to be broken when it comes to looking up that SocketAsyncEven tArgs object.
If you have access to the socket, you can see how much data is currently on the stream though, that might help a bit?
I generally avoid using ASync stuff for these very "grouping" reasons.

[Frinavale]

Well msdn.com and my local msdn library both seem to be broken when it comes to looking up that SocketAsyncEven tArgs object.
If you have access to the socket, you can see how much data is currently on the stream though, that might help a bit?
I generally avoid using ASync stuff for these very "grouping" reasons.

I have no choice but to use the async stuff because Silverlight requires it.

I have access to the socket, and I can see how much data is currently on the stream but I have no idea how much data's supposed to be sent....how big the end result is supposed to be...so I have no idea how large to grow my "data collection" buffer or when the file actually ends and new data begins. I'm quite a newbie when it comes to Sockets so I might even be over looking a property/method that tells me this...but I did go through every single property/method intellisense displays looking for anything that could help me out.

-Frinny

[Plater]

Well sending a header with the datasize is still my recomended action. This backwards client/server business boggles my mind (I prefer the polling process)
If you send the datasize first, your async thing can go "Ok I'm getting this many bytes" and you can keep appending the bytes to some sort of object until you have them all and create you image object from it.

How do you currently know what seperates image A from image B?

[Frinavale]

How do you currently know what seperates image A from image B?

The whole image fits into the receive buffer right now because I've made the buffer large enough to accommodate the whole file... so it's pretty simple, when I receive the data (in the OnReceive event) I know that it's the whole file.

Right now, for practice, the Socket Server sending static image files (images that exist in a folder) to the client. This is how I was able to determine how big to make the buffer (just used the largest image size); however, in my real world application the socket server is going to be receiving raw byte data from a physical device and pushing that to the Silverlight application. So you can understand why it's important that I some how be able to tell how big the data is.

Well sending a header with the datasize is still my recomended action.

I have no idea how to do this...what should I be keywords would you recommend that I research with to find the answer?

If you send the datasize first, your async thing can go "Ok I'm getting this many bytes" and you can keep appending the bytes to some sort of object until you have them all and create you image object from it.

Would you do this in a header or in a separate "init" push to the client?

[Plater]

However you want really. You could steal the idea from how http works with a robust header system (you can include things like the name of the image, the size, the type, date modified, any meta-data, etc) or just something simple like the first 8 bytes represent a size and anything after it are bytes for the image.

What happens when a 2nd image is sent before the first one is done being received? Wouldn't the OnReceive events inter-mix with each other? There mose be some way to tell them apart? Perhaps the remote port from the Socket object could tell you which "image" the receive event belongs to?

[Frinavale]

However you want really. You could steal the idea from how http works with a robust header system (you can include things like the name of the image, the size, the type, date modified, any meta-data, etc) or just something simple like the first 8 bytes represent a size and anything after it are bytes for the image.

I think I'll try sending the file information first and then send the file because at least I understand how that works...I don't know how to get at the header information using the SocketAsyncEven tArgs right now and google's not helping me today.

What happens when a 2nd image is sent before the first one is done being received? Wouldn't the OnReceive events inter-mix with each other? There mose be some way to tell them apart? Perhaps the remote port from the Socket object could tell you which "image" the receive event belongs to?

I've thought about this problem too but I'm not ready to solve it yet. That's why I have the timer for 2 seconds...and sometimes I extend this

When I'm debugging and have the client paused for more than 2 seconds the server throws an exception saying something or other...I'm not sure what happens when more than one file is sent within milliseconds of each other yet...I'm sure that the packets are going to be all mixed up. Asynchronous Socekts are pretty hard to work with.

I'll look at that later. First I just want to figure out how big the expected data is. Then the bigger can of worms will be opened.

One step at a time :)

-Frinny

[Frinavale]

What happens when a 2nd image is sent before the first one is done being received? Wouldn't the OnReceive events inter-mix with each other? There mose be some way to tell them apart? Perhaps the remote port from the Socket object could tell you which "image" the receive event belongs to?

On second thought I think I know how I'll be able to tell them apart......I Think.

When the application calls BeginSend, it will use a separate thread to execute the specified callback method, and will block on EndSend until the Socket sends the number of bytes requested.

I would assume that if a second file were to be sent before the first file it would be queued. I don't know for sure though.

This would kind of defeat the purpose of sending via asyc though.
I'll have to look into this later.

Err, I'm off track now...back to the original problem: determining the size of the file.

[Plater]

Oh! When I said header i didn't specifically mean something relating to the Socket and TCP transfering protocol, I meant like create your own "header"

[Frinavale]

Thanks for your help Plater :)

I sent an "init package" to the client which contained the size of the data before I sent the actual data. From there I was able to allocate the correct amount of memory to the array I'm using to appended the received data to.

[Frinavale]

Hmm this solution is not working well at all.

[Plater]

Are they getting mixed up now?

[Frinavale]

Are they getting mixed up now?

How can you tell?

All I know is it's easy to crash this.

I change the implementation to append 2 bytes to the data being sent to the client. The 2 bytes contains an Int16 used to indicate the length of data that's being sent to the client....this way I don't have to send an "init package" before the real send begins.

Now that I've been playing with this more I'm thinking of trying the other way again...where I send an "init package" to the client with the length, and then the data.

The problem that I was having with the original implementation is that if anything takes a bit longer to process than usual (lag) in the client everything sent from the server gets thrown out of sync.

I can't figure out when the begin send/end send is happening to get it back into sync.

I think the something similar is happening with this new implementation when I increase the the timer to a value of 100th of a second...some how the data sent from the server is getting out of sync with the client after some time passes and I open more socket connections to the server.

If I knew when the begin send and end send was happening in the client I would be happy...I think.

And I thought sockets were easy when I first started playing with them :P
It only took me 5 minutes to figure out how to get a synchronized solution to work using a normal windows application (client) and console application (server). Between using asynchronous sockets and Silverlight I'm starting to feel like a newbie.