x64 and BSTR allocation, what has changed?

**David Lowndes** · Mar 21 '06, 07:55 PM

Re: x64 and BSTR allocation, what has changed?

>In win32 mode, a BSTR was UINT length prefixed and terminated exactly by a[color=blue]
>zero char.
>
>So if you allocated "Hello World" that would allocate 28 bytes.
>
>In x64 and (IA64 as well) it would become 32 bytes, because of the fact that
>a pointer, is 8 bytes instead of 4 bytes.[/color]

How do you come to that conclusion? Where does a pointer come into a
BSTR?

Dave

**Egbert Nierop \(MVP for IIS\)** · Mar 21 '06, 08:26 PM

Re: x64 and BSTR allocation, what has changed?

"David Lowndes" <DavidL@example .invalid> wrote in message
news:6ol022tvfk 39j1ao0fi065d8h blf3n34mt@4ax.c om...[color=blue][color=green]
> >In win32 mode, a BSTR was UINT length prefixed and terminated exactly by
> >a
>>zero char.
>>
>>So if you allocated "Hello World" that would allocate 28 bytes.
>>
>>In x64 and (IA64 as well) it would become 32 bytes, because of the fact
>>that
>>a pointer, is 8 bytes instead of 4 bytes.[/color]
>
> How do you come to that conclusion? Where does a pointer come into a
> BSTR?[/color]
ok,

I'll explain this.
Not the pointer -itself- comes into but the pointer is to the UINT length
prefix.

On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
UINT_PTR length prefix.

But but... however, if you want the length, you need a UINT prefix, not a
UINT_PTR prefix.

I'm not stating this, just hesitating. Maybe someone knows the real facts...

BSTR __stdcall SysAllocString( const OLECHAR * input)
{
PWSTR retval = NULL;

if (input != NULL)
{
UINT_PTR slen = lstrlenW(input) ;
UINT_PTR newlen = slen * sizeof(OLECHAR) + sizeof(UINT_PTR ) +
sizeof(OLECHAR) ;
PWSTR temp = NULL;
temp = (PWSTR)::CoTask MemAlloc(newlen );

if (temp != NULL)
{
UINT* plen = (UINT*)temp;
plen[0] = (UINT)slen * sizeof(OLECHAR) ;
retval = &temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)];
if (slen > 0)
CopyMemory(retv al, input, (slen + 1) * sizeof(OLECHAR) );
}
}
return retval;
}
[color=blue]
> Dave[/color]

**David Lowndes** · Mar 21 '06, 09:15 PM

Re: x64 and BSTR allocation, what has changed?

>> How do you come to that conclusion? Where does a pointer come into a[color=blue][color=green]
>> BSTR?[/color]
>ok,
>
>I'll explain this.
>Not the pointer -itself- comes into but the pointer is to the UINT length
>prefix.
>
>On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
>UINT_PTR length prefix.[/color]

So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
length portion rather than a 4 byte length? I'd be surprised if it
was.

I don't see any evidence for that assumption in the MSDN documentation
or the header files. SysStringLen is defined to return an UINT not
UINT_PTR.

Dave

**Jochen Kalmbach [MVP]** · Mar 21 '06, 10:05 PM

Re: x64 and BSTR allocation, what has changed?

Hi David![color=blue][color=green]
>>On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
>>UINT_PTR length prefix.[/color]
>
> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
> length portion rather than a 4 byte length? I'd be surprised if it
> was.[/color]

A BSTR does not have a "length portion"... it is just a pointer...

Or did I miss something?

BSTR is just "wchar_t*". ..

--
Greetings
Jochen

My blog about Win32 and .NET

Jochen Kalmbach's Blog | Infos about Windows development and dotNET Framework

http://blog.kalmbachnet.de/

**Alexander Nickolov** · Mar 21 '06, 10:15 PM

Re: x64 and BSTR allocation, what has changed?

Let's not forget that the memory must be 8-byte aligned.
I would guess it allocates 8 bytes for the length prefix, and
ignores the first four. And also let's not forget that the
allocation strategy for BSTR is officially undocumented...
Only the layout is documented.

--
=============== =============== =======
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: agnickolov@mvps .org
MVP VC FAQ: http://www.mvps.org/vcfaq
=============== =============== =======

"David Lowndes" <DavidL@example .invalid> wrote in message
news:kmp022lus0 mo2ha6v60usl7kg ok5qlsf2n@4ax.c om...[color=blue][color=green][color=darkred]
>>> How do you come to that conclusion? Where does a pointer come into a
>>> BSTR?[/color]
>>ok,
>>
>>I'll explain this.
>>Not the pointer -itself- comes into but the pointer is to the UINT length
>>prefix.
>>
>>On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
>>UINT_PTR length prefix.[/color]
>
> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
> length portion rather than a 4 byte length? I'd be surprised if it
> was.
>
> I don't see any evidence for that assumption in the MSDN documentation
> or the header files. SysStringLen is defined to return an UINT not
> UINT_PTR.
>
> Dave[/color]

**Igor Tandetnik** · Mar 21 '06, 10:25 PM

Re: x64 and BSTR allocation, what has changed?

Jochen Kalmbach [MVP] <nospam-Jochen.Kalmbach @holzma.de> wrote:[color=blue]
> A BSTR does not have a "length portion"... it is just a pointer...[/color]

Does too. The length in bytes is stored in the 4 bytes preceding the
byte referred to by the pointer. How do you expect SysStringLen to work
otherwise? Recall that BSTR may contain embedded NULs.
--
With best wishes,
Igor Tandetnik

With sufficient thrust, pigs fly just fine. However, this is not
necessarily a good idea. It is hard to be sure where they are going to
land, and it could be dangerous sitting under them as they fly
overhead. -- RFC 1925

**Egbert Nierop \(MVP for IIS\)** · Mar 21 '06, 10:35 PM

Re: x64 and BSTR allocation, what has changed?

"Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in message
news:uG3RvFTTGH A.4792@TK2MSFTN GP14.phx.gbl...[color=blue]
> Hi David![color=green][color=darkred]
>>>On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
>>>UINT_PTR length prefix.[/color]
>>
>> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
>> length portion rather than a 4 byte length? I'd be surprised if it
>> was.[/color]
>
> A BSTR does not have a "length portion"... it is just a pointer...
>
> Or did I miss something?[/color]

Yes, I'm saying that on 64-bit platforms, the length portion is 8 bytes!
But only the lower 4 bytes are effective.

I can prove this since I used a BSTR stub. This had special reasons. I had a
benchmark, that has proven that avoiding BSTR caching, was good on a Wintel
platform I did this by redefining all SysAlloc* strings to my own written
replacements.

On the 64-bit platform, this stub is working in a stand-alone EXE but not
inside a service and I can't figure why.

**Egbert Nierop \(MVP for IIS\)** · Mar 21 '06, 10:35 PM

Re: x64 and BSTR allocation, what has changed?

"Alexander Nickolov" <agnickolov@mvp s.org> wrote in message
news:uk0coJTTGH A.776@TK2MSFTNG P09.phx.gbl...[color=blue]
> Let's not forget that the memory must be 8-byte aligned.
> I would guess it allocates 8 bytes for the length prefix, and
> ignores the first four. And also let's not forget that the
> allocation strategy for BSTR is officially undocumented...
> Only the layout is documented.[/color]

Your guess is right about the 8 bytes length prefix.

The BSTR is just allocated by CoTaskMemAlloc and and CoTaskMemReallo c that
is using RtlHeap functions.

I have no source code, but my COM components, have been working for 3 years
on this. Except on the 64-bit platform that I am testing now.

**Egbert Nierop \(MVP for IIS\)** · Mar 21 '06, 10:35 PM

Re: x64 and BSTR allocation, what has changed?

"Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in message
news:uG3RvFTTGH A.4792@TK2MSFTN GP14.phx.gbl...[color=blue]
> Hi David![color=green][color=darkred]
>>>On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
>>>UINT_PTR length prefix.[/color]
>>
>> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
>> length portion rather than a 4 byte length? I'd be surprised if it
>> was.[/color]
>
> A BSTR does not have a "length portion"... it is just a pointer...
>
> Or did I miss something?
>
> BSTR is just "wchar_t*". ..[/color]

You missed a lot by thinking this :)

**Alexander Nickolov** · Mar 21 '06, 10:55 PM

Re: x64 and BSTR allocation, what has changed?

Oh, I know how it's allocated :). I'm just pointing out that
this is officially undocumented.

So in Win64 we have:

byte len content
-8 4 unused
-4 4 length
0 n string

Or did I miss something?

--
=============== =============== =======
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: agnickolov@mvps .org
MVP VC FAQ: http://www.mvps.org/vcfaq
=============== =============== =======

"Egbert Nierop (MVP for IIS)" <egbert_nierop@ nospam.invalid> wrote in
message news:OtJDoZTTGH A.5500@TK2MSFTN GP12.phx.gbl...[color=blue]
>
> "Alexander Nickolov" <agnickolov@mvp s.org> wrote in message
> news:uk0coJTTGH A.776@TK2MSFTNG P09.phx.gbl...[color=green]
>> Let's not forget that the memory must be 8-byte aligned.
>> I would guess it allocates 8 bytes for the length prefix, and
>> ignores the first four. And also let's not forget that the
>> allocation strategy for BSTR is officially undocumented...
>> Only the layout is documented.[/color]
>
> Your guess is right about the 8 bytes length prefix.
>
> The BSTR is just allocated by CoTaskMemAlloc and and CoTaskMemReallo c that
> is using RtlHeap functions.
>
> I have no source code, but my COM components, have been working for 3
> years on this. Except on the 64-bit platform that I am testing now.[/color]

**David Lowndes** · Mar 21 '06, 11:16 PM

Re: x64 and BSTR allocation, what has changed?

>Let's not forget that the memory must be 8-byte aligned.[color=blue]
>I would guess it allocates 8 bytes for the length prefix, and
>ignores the first four.[/color]

OK.

.... so why does this matter to Egbert?

Dave

**Egbert Nierop \(MVP for IIS\)** · Mar 21 '06, 11:55 PM

Re: x64 and BSTR allocation, what has changed?

"Alexander Nickolov" <agnickolov@mvp s.org> wrote in message
news:OSrXQgTTGH A.5656@TK2MSFTN GP11.phx.gbl...[color=blue]
> Oh, I know how it's allocated :). I'm just pointing out that
> this is officially undocumented.
>
> So in Win64 we have:
>
> byte len content
> -8 4 unused
> -4 4 length
> 0 n string[/color]
and add a '\0' after the n string.
(unicode)

[color=blue]
> Or did I miss something?[/color]

It's working. I had still somewhere an invalid memory location calculation
(-4 instead of -8)

#include "stdafx.h"
#include "bstrnocach e.h"

void __stdcall FreeString(BSTR * theString) throw()
{
if (theString != NULL && *theString != NULL )
{
SysFreeString2( *theString);
*theString = NULL;
}
}

int __stdcall SysReAllocStrin gByteLen2(BSTR * bstr, const char* input, UINT
cch) throw()
{
// ATLASSERT(bstr != NULL);
if (*bstr == NULL)
*bstr = SysAllocStringB yteLen(input, cch);
else
{
UINT copyIntoLen = input != NULL ? lstrlenA(input) : 0;
if (copyIntoLen > cch)
copyIntoLen = cch;
UINT_PTR newlen = cch + sizeof(OLECHAR) + sizeof(UINT_PTR );

PCHAR temp = (PCHAR)(*bstr) - sizeof(UINT_PTR );

temp = (PCHAR)::CoTask MemRealloc(temp , newlen);

if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp ;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrecti on] = cch ; //asign bytelength for BSTR
if (copyIntoLen > 0) //copy chars
memcpy(&plen[1],
input,
(copyIntoLen + 1));

temp[sizeof(UINT_PTR ) + cch ] = 0; //terminate LPWSTR with a wide_string
temp[sizeof(UINT_PTR ) + cch + 1] = 0;
*bstr = (BSTR)&temp[sizeof(UINT_PTR )];
}
}
return *bstr == NULL ? FALSE : TRUE;
}
int __stdcall SysReAllocStrin gLen2(BSTR * bstr, const OLECHAR* input, UINT
cch) throw()
{
//ATLASSERT(bstr != NULL);
if (*bstr == NULL)
{
*bstr = SysAllocStringL en2(input, cch);
}
else
{
UINT_PTR copyIntoLen = input != NULL ? lstrlenW(input) : 0;
if (copyIntoLen > cch)
copyIntoLen = cch;
UINT_PTR newlen = cch * sizeof(OLECHAR) + sizeof(OLECHAR) +
sizeof(UINT_PTR );

PWSTR temp = *bstr - sizeof(UINT_PTR ) / sizeof(OLECHAR) ;

temp = (PWSTR)::CoTask MemRealloc(temp , newlen);

if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp ;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrecti on] = cch * sizeof(OLECHAR) ; //asign bytelength for
BSTR
if (copyIntoLen > 0) //copy values
memcpy(&temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)],
input,
(copyIntoLen + 1) * sizeof(OLECHAR) );

temp[sizeof(UINT_PTR ) / sizeof(OLECHAR) + cch ] = 0; //terminate LPWSTR
*bstr = &temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)];

}

}
return *bstr == NULL ? FALSE : TRUE;
}
BSTR __stdcall SysAllocStringL en2(const OLECHAR * input, UINT cch) throw()
{
UINT_PTR copyIntoLen = input != NULL ? lstrlenW(input) : 0;
if (copyIntoLen > cch)
copyIntoLen = cch;

UINT_PTR newlen = cch * sizeof(OLECHAR) + sizeof(OLECHAR) +
sizeof(UINT_PTR );

PWSTR temp = (PWSTR)::CoTask MemAlloc(newlen );
if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp ;
UINT*plen2 = (UINT*)temp;
plen2[0] =0;
plen2[sizePtrCorrecti on] = cch * sizeof(OLECHAR) ;
if (copyIntoLen > 0)
memcpy(&plen[1],
input,
(copyIntoLen + 1) * sizeof(OLECHAR) );

temp[sizeof(UINT_PTR ) / sizeof(OLECHAR) + cch ] = 0;
return &temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)];
}
else
return NULL;

}

BSTR __stdcall SysAllocStringB yteLen2(const char* input, UINT cch)
throw()
{
UINT_PTR copyIntoLen = input != NULL ? lstrlenA(input) : 0;
if (copyIntoLen > cch)
copyIntoLen = cch;

UINT_PTR newlen = cch + sizeof(OLECHAR) + sizeof(UINT_PTR );

PWSTR temp = (PWSTR)::CoTask MemAlloc(newlen );
if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp ;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrecti on] = cch;
if (copyIntoLen > 0)
memcpy(&temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)],
input,
copyIntoLen + 1);
char* zeroit = (char*)temp;
zeroit[sizeof(UINT_PTR ) + cch ] = 0; //terminate LPWSTR with a wide_string
zeroit[sizeof(UINT_PTR ) + cch + 1] = 0;
//temp[sizeof(unsigned int) / sizeof(wchar_t) + cch ] = 0;
return &temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)];
}
else
return NULL;

}

///<summary>
/// Identical to SysAllocString
///</summary>
BSTR __stdcall SysAllocString2 (const OLECHAR * input) throw()
{
PWSTR retval = NULL;

if (input != NULL)
{
UINT_PTR slen = lstrlenW(input) ;
UINT_PTR newlen = slen * sizeof(OLECHAR) + sizeof(UINT_PTR ) +
sizeof(OLECHAR) ;
PWSTR temp = NULL;
temp = (PWSTR)::CoTask MemAlloc(newlen );

if (temp != NULL)
{
UINT* plen = (UINT*)temp;
plen[sizePtrCorrecti on] = (UINT)slen * sizeof(OLECHAR) ;
retval = &temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)];
if (slen > 0)
memcpy(retval, input, (slen + 1) * sizeof(OLECHAR) );
}
}
return retval;
}

int __stdcall SysReAllocStrin g2(BSTR * bstr, const OLECHAR * input) throw()
{
//ATLASSERT(bstr != NULL);
bool didEmpty = FALSE;
if (*bstr == NULL)
{
*bstr = SysAllocString2 (input);
}
else
{

if (input == NULL)
{
FreeString(bstr );
didEmpty = true;
}
else
{
UINT_PTR slen = lstrlenW(input) ;
UINT_PTR newlen = slen * sizeof(OLECHAR) + sizeof(OLECHAR) +
sizeof(UINT_PTR );

BSTR temp = *bstr - sizeof(INT_PTR) / sizeof(OLECHAR) ; //get real address
temp = (BSTR)::CoTaskM emRealloc(temp, newlen);

if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp ;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrecti on] = (UINT)slen * sizeof(OLECHAR) ;
*bstr = &temp[sizeof(UINT_PTR ) / sizeof(OLECHAR)];
memcpy(*bstr, input, (slen + 1) * sizeof(OLECHAR) );
}
}

}
return *bstr == NULL && didEmpty == false? FALSE : TRUE;
}

UINT __stdcall SysStringByteLe n2(BSTR theString) throw()
{
UINT retval = 0;
if (theString != NULL)
{
UINT* temp = (UINT*)theStrin g;
temp--;
retval = *temp;
}
return retval;

}

UINT __stdcall SysStringLen2(B STR theString) throw()
{
UINT retval = 0;
if (theString != NULL)
{
UINT* temp = (UINT*)theStrin g;
temp--;
retval = *temp / sizeof(OLECHAR) ;
}
return retval;
}

**Jochen Kalmbach [MVP]** · Mar 22 '06, 07:15 AM

Re: x64 and BSTR allocation, what has changed?

Hi Egbert!
[color=blue][color=green][color=darkred]
>>> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
>>> length portion rather than a 4 byte length? I'd be surprised if it
>>> was.[/color]
>>
>> A BSTR does not have a "length portion"... it is just a pointer...
>>
>> Or did I miss something?
>>
>> BSTR is just "wchar_t*". ..[/color]
>
> You missed a lot by thinking this :)[/color]

No problem for me... I never rely on undecomented features...

(As "Alexander said: "And also let's not forget that the
allocation strategy for BSTR is officially undocumented... ")

Greetings
Jochen

**Egbert Nierop \(MVP for IIS\)** · Mar 22 '06, 07:25 AM

Re: x64 and BSTR allocation, what has changed?

"David Lowndes" <DavidL@example .invalid> wrote in message
news:jk1122h28j tb1ttlprk11tvhf grli6oa3o@4ax.c om...[color=blue][color=green]
> >Let's not forget that the memory must be 8-byte aligned.
>>I would guess it allocates 8 bytes for the length prefix, and
>>ignores the first four.[/color]
>
> OK.
>
> ... so why does this matter to Egbert?[/color]

The prefix was concernig my question, the alignment not.

CoTaskMemAlloc nowhere states in the docs that you must 'align' memory. I
think that RtlHeap* etc do align the allocations by themselves but I'm not
that guru to know this.

x64 and BSTR allocation, what has changed?

x64 and BSTR allocation, what has changed?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment