Overlapped IO problem on x64

[=?Utf-8?B?TmljayBCdXJraXR0?=]

Hi.

I have a C# app that uses named pipes via InteropServices . I have no
problems running on a 32-bit machine, or when targeting x86 architecture, but
it fails on a 64-bit machine when targeting x64 or AnyCPU.
The problem appears to be that ReadFile() sets lpNumberOfBytes Read to 0,
even though the receive buffer contains the data I'm expecting.
Is this a known issue, or am I just doing something dumb?
I can provide a small test app via email if needed.
Thanks,

-Nick

[Jialiang Ge [MSFT]]

RE: Overlapped IO problem on x64

Good morning Nick,

I appreciate it if you can send the test app to my mailbox:
jiagle@microsof t.com, and this will give me a clearer picture of the
symptom and I will try to figure out the reason. Thanks

Jialiang Ge (jialge@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.

This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====

[=?Utf-8?B?TmljayBCdXJraXR0?=]

RE: Overlapped IO problem on x64

Hi Jialiang.

Thanks for your reply.
I don't seem to be able to send you email. Here's the bounce message:

<jiagle@microso ft.com>:
216.32.181.22 does not like recipient.
Remote host said: 554 <jiagle@microso ft.com>: Recipient address rejected:
Access denied
Giving up on 216.32.181.22.

-Nick


""Jialiang Ge [MSFT]"" wrote:

Good morning Nick,
>
I appreciate it if you can send the test app to my mailbox:
jiagle@microsof t.com, and this will give me a clearer picture of the
symptom and I will try to figure out the reason. Thanks
>
Jialiang Ge (jialge@online. microsoft.com, remove 'online.')
Microsoft Online Community Support
>
=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.
>
This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====
>
>

[Jialiang Ge [MSFT]]

RE: Overlapped IO problem on x64

My appology, Nick. I typed my mail address wrong. It should be
jialge@microsof t.com.

Best Regards,
Jialiang Ge (jialge@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.

This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====

[=?Utf-8?B?TmljayBCdXJraXR0?=]

RE: Overlapped IO problem on x64

That seems to be working, thanks. Check your email...

-Nick


""Jialiang Ge [MSFT]"" wrote:

My appology, Nick. I typed my mail address wrong. It should be
jialge@microsof t.com.
>
Best Regards,
Jialiang Ge (jialge@online. microsoft.com, remove 'online.')
Microsoft Online Community Support
>
=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.
>
This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====
>
>

[Jialiang Ge [MSFT]]

RE: Overlapped IO problem on x64

Hello Nick

Thank you for the sample. I spent the past four hours on this issue.
Although I have not figured out the cause of the symptom, I'd like to
update you about my current research results.

1. I debugged your sample in x64 and x86 machines and noticed that the
_OVERLAPPED struct's Internal and InternalHigh values were set differently:

x64:

KERNEL32!_OVERL APPED
+0x000 Internal : 0x103
+0x008 InternalHigh : 0 ' always 0
+0x010 Offset : 0
+0x014 OffsetHigh : 0
+0x010 Pointer : (null)
+0x018 hEvent : 0x00000000`0000 0220

x86:

KERNEL32!_OVERL APPED
+0x000 Internal : 0
+0x004 InternalHigh : 0x10 ' this is the right value.
+0x008 Offset : 0
+0x00c OffsetHigh : 0
+0x008 Pointer : (null)
+0x010 hEvent : 0x000001f8

"Internal" holds the processed I/O's error code. Both 0x103 and 0 mean that
the operation is successful (SUCCEEDED(Inte rnal) == TRUE). However, I feel
that there must be some meaning behind 0x103. I am asking the OS team about
it and hope this can give some insights of the cause.

2. I also discussed the issue with the .NET team. They reminded me that we
added the built-in managed pipe APIs in .NET 3.5 (see System.IO.Pipes
http://msdn.microsoft.com/en-us/libr...io.pipes.aspx), so we
recommend just using that instead of rolling our own support for pipes.
Doing low-level async I/O in managed code is very hard.

In addition, the .NET team tells me that NativeOverlappe d should work just
fine on both architectures.

3. I suspected that the problem here is that we are not pinning the object
containing the NativeOverlappe d. CAsyncHelper is not blittable
(http://msdn.microsoft.com/en-us/library/75dwhxf7.aspx), and thus cannot be
pinned. I manually pinned the NativeOverlappe d struct inside it and
allocate a GCHandle on the ManualResetEven t object, but it does not work
out the problem. I'm still researching in this aspect.

4. Another suggestion we'd give is to look at binding the pipe to the
ThreadPool (via ThreadPool.Bind Handle) and using Overlapped instead of
using NativeOverlappe d directly. This makes the pinning stuff somewhat
easier, and gives a much nicer completion model.

Best Regards,
Jialiang Ge
Microsoft Online Community Support

=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.

This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====

[Jialiang Ge [MSFT]]

RE: Overlapped IO problem on x64

Hello Nick

I'm reminded that System.IO.FileS tream.BeginRead/Write is implemented by
using the NativeOverlappe d struct, thus NativeOverlappe d should work for
x64 platform. I then checked the source code of
System.IO.FileS tream.BeginRead/Write, and found that it uses the
NativeOverlappe d struct in this way:

// Create a managed overlapped class
// We will set the file offsets later
Overlapped overlapped = new Overlapped(0, 0, IntPtr.Zero, asyncResult);

// Pack the Overlapped class, and store it in the async result
NativeOverlappe d* intOverlapped;
if (userCallback != null)
intOverlapped = overlapped.Pack (IOCallback, bytes);
else
intOverlapped = overlapped.Unsa fePack(null, bytes);

Overlapped.Pack packs an instance of System.Threadin g.Overlapped into a
NativeOverlappe d struct. The unmanaged pointer returned by this method can
be passed to the operating system in overlapped I/O operations. The
NativeOverlappe d structure is fixed in physical memory until Unpack is
called. This is much safer than directly creating a NativeOverlappe d struct
in that Overlapped.Pack manages the complex pinning logic for us.

Based on this finding, I updated your code to use Overlapped.Pack . I've
sent the new project to your mailbox. The test project has been tested in
my Windows 2003 x64 box, and it works fine. Please try it on your side and
let me know the result.

Have a very nice day!

Best Regards,
Jialiang Ge (jialge@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.

This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====

[=?Utf-8?B?TmljayBCdXJraXR0?=]

RE: Overlapped IO problem on x64

Hi Jialiang.

Thanks for the obviously considerable effort you put into finding this
solution! This appears to solve the problem completely.
Thanks again for a truly great job!

-Nick


""Jialiang Ge [MSFT]"" wrote:

Hello Nick
>
I'm reminded that System.IO.FileS tream.BeginRead/Write is implemented by
using the NativeOverlappe d struct, thus NativeOverlappe d should work for
x64 platform. I then checked the source code of
System.IO.FileS tream.BeginRead/Write, and found that it uses the
NativeOverlappe d struct in this way:
>
// Create a managed overlapped class
// We will set the file offsets later
Overlapped overlapped = new Overlapped(0, 0, IntPtr.Zero, asyncResult);
>
// Pack the Overlapped class, and store it in the async result
NativeOverlappe d* intOverlapped;
if (userCallback != null)
intOverlapped = overlapped.Pack (IOCallback, bytes);
else
intOverlapped = overlapped.Unsa fePack(null, bytes);
>
Overlapped.Pack packs an instance of System.Threadin g.Overlapped into a
NativeOverlappe d struct. The unmanaged pointer returned by this method can
be passed to the operating system in overlapped I/O operations. The
NativeOverlappe d structure is fixed in physical memory until Unpack is
called. This is much safer than directly creating a NativeOverlappe d struct
in that Overlapped.Pack manages the complex pinning logic for us.
>
Based on this finding, I updated your code to use Overlapped.Pack . I've
sent the new project to your mailbox. The test project has been tested in
my Windows 2003 x64 box, and it works fine. Please try it on your side and
let me know the result.
>
Have a very nice day!
>
Best Regards,
Jialiang Ge (jialge@online. microsoft.com, remove 'online.')
Microsoft Online Community Support
>
=============== =============== =============== ====
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg@microsof t.com.
>
This posting is provided "AS IS" with no warranties, and confers no rights.
=============== =============== =============== ====
>
>
>