x64 and BSTR allocation, what has changed?

[Egbert Nierop \(MVP for IIS\)]

Re: x64 and BSTR allocation, what has changed?


"Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in message
news:%23f8t05XT GHA.4340@tk2msf tngp13.phx.gbl. ..[color=blue]
> Hi Egbert!
>[color=green][color=darkred]
>>>> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
>>>> length portion rather than a 4 byte length? I'd be surprised if it
>>>> was.
>>>
>>> A BSTR does not have a "length portion"... it is just a pointer...
>>>
>>> Or did I miss something?
>>>
>>> BSTR is just "wchar_t*". ..[/color]
>>
>> You missed a lot by thinking this :)[/color]
>
> No problem for me... I never rely on undecomented features...[/color]

Sure, it's your party, but BSTR is _not_ just a wchar_t* is documented!

[color=blue]
> (As "Alexander said: "And also let's not forget that the
> allocation strategy for BSTR is officially undocumented... ")[/color]

This is not rocket science :)
If you need an 11 wchar length string, what else must you store but the
length _and_ the compatibility with wchar_t* ie a zero terminator?

The other thing you need to know, is that all oleautomation memory allocs,
should go through CoTaskMemAlloc. Now I'm a rocket scientist :)

I started to hesitate on the x64 because of GPs but now I fixed it.

[David Lowndes]

Re: x64 and BSTR allocation, what has changed?

>> ... so why does this matter to Egbert?[color=blue]
>
>The prefix was concernig my question, the alignment not.[/color]

I still don't see why any of this was of any significance to you. What
are you doing that you need to know this for?

Dave

[Egbert Nierop \(MVP for IIS\)]

Re: x64 and BSTR allocation, what has changed?


"David Lowndes" <DavidL@example .invalid> wrote in message
news:0pv1229d0n up611o95hcct7uv fiajt0c75@4ax.c om...[color=blue][color=green][color=darkred]
>>> ... so why does this matter to Egbert?[/color]
>>
>>The prefix was concernig my question, the alignment not.[/color]
>
> I still don't see why any of this was of any significance to you. What
> are you doing that you need to know this for?[/color]

Well, in fact, in this thread I explained it. What I do? I'm a rocket
scientist :)

This explains it all...

http://technolog.nl/eprogrammer/

[David Lowndes]

Re: x64 and BSTR allocation, what has changed?

>>>> ... so why does this matter to Egbert?[color=blue][color=green][color=darkred]
>>>
>>>The prefix was concernig my question, the alignment not.[/color]
>>
>> I still don't see why any of this was of any significance to you. What
>> are you doing that you need to know this for?[/color]
>
>Well, in fact, in this thread I explained it. What I do? I'm a rocket
>scientist :)[/color]

I read all the thread and *I* couldn't work out why you needed to
know.
[color=blue]
>This explains it all...
>http://technolog.nl/eprogrammer/[/color]

Aha, the key bit being:

"I was managing my own BSTR allocation replacements"

I'd not sussed that from this thread.

Dave

[Eberhard Schefold]

Re: x64 and BSTR allocation, what has changed?

Jochen Kalmbach [MVP] wrote:
[color=blue][color=green][color=darkred]
>>> BSTR is just "wchar_t*". ..[/color]
>>
>> You missed a lot by thinking this :)[/color]
>
> No problem for me... I never rely on undecomented features...[/color]

But you do realize that you must create, manipulate and destroy a BSTR
only in the documented ways, essentially using the Sysxxx functions? You
can provide a BSTR in instances where a LPCWSTR is expected, but not the
other way round. The preceding length value is an implementation detail,
and therefore we should not make any assumptions -- agreed. But if you
fail to remember that a BSTR is /not/ just a wchar_t array, you'll get
into trouble.

[Jochen Kalmbach [MVP]]

Re: x64 and BSTR allocation, what has changed?

Hi Eberhard![color=blue][color=green][color=darkred]
>>> You missed a lot by thinking this :)[/color]
>>
>> No problem for me... I never rely on undecomented features...[/color]
>
> But you do realize that you must create, manipulate and destroy a BSTR
> only in the documented ways, essentially using the Sysxxx functions?[/color]

Yes. That´s why I do not care about the internals of BSTR...
[color=blue]
> The preceding length value is an implementation detail,
> and therefore we should not make any assumptions -- agreed. But if you
> fail to remember that a BSTR is /not/ just a wchar_t array, you'll get
> into trouble.[/color]

Yes. You are right. For BSTR you should only use the SysXxx functions.

Greetings
Jochen

[Scherbina Vladimir]

Re: x64 and BSTR allocation, what has changed?

The internals are always good to know, especially for the OP case - if you
need to control allocations ...

--
Vladimir

http://spaces.msn.com/vladimir-scherbina/

"Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in message
news:eVG$MkaTGH A.4900@TK2MSFTN GP12.phx.gbl...[color=blue]
> Hi Eberhard![color=green][color=darkred]
>>>> You missed a lot by thinking this :)
>>>
>>> No problem for me... I never rely on undecomented features...[/color]
>>
>> But you do realize that you must create, manipulate and destroy a BSTR
>> only in the documented ways, essentially using the Sysxxx functions?[/color]
>
> Yes. That´s why I do not care about the internals of BSTR...
>[color=green]
>> The preceding length value is an implementation detail, and therefore we
>> should not make any assumptions -- agreed. But if you fail to remember
>> that a BSTR is /not/ just a wchar_t array, you'll get into trouble.[/color]
>
> Yes. You are right. For BSTR you should only use the SysXxx functions.
>
> Greetings
> Jochen[/color]

[Skywing]

Re: x64 and BSTR allocation, what has changed?

The fact that BSTRs have a length prefix is fully and publicly documented <
see
http://msdn.microsoft.com/library/de...663a57d2b5.asp >

This is a very important thing that you need to know and understand if you
ever manage BSTRs yourself. I don't know where you got the idea that this
is an undocumented, secret implementation detail - but it's absolutely not.
If you've done nontrivial code that works with BSTRs, I'm not sure how you
could get by without knowing these sorts of things.

Additionally, even the allocation strategy used for BSTRs by oleaut32 is
also documented (see the above link).

"Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in message
news:%23f8t05XT GHA.4340@tk2msf tngp13.phx.gbl. ..[color=blue]
> Hi Egbert!
>[color=green][color=darkred]
>>>> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
>>>> length portion rather than a 4 byte length? I'd be surprised if it
>>>> was.
>>>
>>> A BSTR does not have a "length portion"... it is just a pointer...
>>>
>>> Or did I miss something?
>>>
>>> BSTR is just "wchar_t*". ..[/color]
>>
>> You missed a lot by thinking this :)[/color]
>
> No problem for me... I never rely on undecomented features...
>
> (As "Alexander said: "And also let's not forget that the
> allocation strategy for BSTR is officially undocumented... ")
>
> Greetings
> Jochen[/color]

[Jochen Kalmbach [MVP]]

Re: x64 and BSTR allocation, what has changed?

Hi Skywing![color=blue]
> The fact that BSTRs have a length prefix is fully and publicly documented <
> see
> http://msdn.microsoft.com/library/de...663a57d2b5.asp >[/color]

Great!
[color=blue]
> This is a very important thing that you need to know and understand if you
> ever manage BSTRs yourself.[/color]

I don't see the need to know the internals of this BSTR...
[color=blue]
> I don't know where you got the idea that this
> is an undocumented, secret implementation detail - but it's absolutely not.[/color]

Maybe I was not yet in the situation to handle "nontrivial " BSTR code...


But in general: It is always good to known the the underlying system is
doing...

--
Greetings
Jochen

My blog about Win32 and .NET

http://blog.kalmbachnet.de/

[Egbert Nierop \(MVP for IIS\)]

Re: x64 and BSTR allocation, what has changed?


"Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in message
news:u2d5dmdTGH A.6048@TK2MSFTN GP11.phx.gbl...[color=blue]
> Hi Skywing![color=green]
>> The fact that BSTRs have a length prefix is fully and publicly documented
>> < see
>> http://msdn.microsoft.com/library/de...663a57d2b5.asp >[/color]
>
> Great!
>[color=green]
>> This is a very important thing that you need to know and understand if
>> you ever manage BSTRs yourself.[/color]
>
> I don't see the need to know the internals of this BSTR...[/color]

Who told you that you need to?

When I ask a question, you just can assume that I see need and if you don't
know to answer, please refrain from vain comments then :)

In fact, I wanted a mechanisme, that has 'disabled' BSTR caching, without
affecting the whole process or using a sneaky API to disable caching.

Caching of strings in a process, is bad for performance for a Service.

You don't see a need, so don't waste time about that.

[Alexander Nickolov]

Re: x64 and BSTR allocation, what has changed?

Since I started it, let me clarify. The layout is documented
to the extent that the 4 bytes preceding the pointer are
its length (in bytes I might add). It is also documented
that you should use SysAllocString and friends to allocate
BSTRs. What is not documented, however, is how SysAlloc
functions allocate the raw memory. It was common knowledge
that on Win32 the allocated pointer is 4 bytes before the returned
pointer, but this does not make it documented! (It is also public
knowledge that CoTaskMemAlloc is used for the memory
allocation of course.) Apparently, this allowed Microsoft to
change the allocation strategy to reflect the alignment requirement
for Win64 and return a 64-bit aligned pointer from SysAlloc
functions. Of course this breaks the aforementioned public
knowledge (and we extended it obviously...). There's nothing
wrong with knowing undocumented aspects. What's wrong is
to expect that these assumptions will continue to be valid
going forward.

--
=============== =============== =======
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: agnickolov@mvps .org
MVP VC FAQ: http://www.mvps.org/vcfaq
=============== =============== =======

"Skywing" <skywing_NO_SPA M_@valhallalege nds.com> wrote in message
news:OhdmmUdTGH A.4600@TK2MSFTN GP11.phx.gbl...[color=blue]
> The fact that BSTRs have a length prefix is fully and publicly documented
> < see
> http://msdn.microsoft.com/library/de...663a57d2b5.asp >
>
> This is a very important thing that you need to know and understand if you
> ever manage BSTRs yourself. I don't know where you got the idea that this
> is an undocumented, secret implementation detail - but it's absolutely
> not. If you've done nontrivial code that works with BSTRs, I'm not sure
> how you could get by without knowing these sorts of things.
>
> Additionally, even the allocation strategy used for BSTRs by oleaut32 is
> also documented (see the above link).
>
> "Jochen Kalmbach [MVP]" <nospam-Jochen.Kalmbach @holzma.de> wrote in
> message news:%23f8t05XT GHA.4340@tk2msf tngp13.phx.gbl. ..[color=green]
>> Hi Egbert!
>>[color=darkred]
>>>>> So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
>>>>> length portion rather than a 4 byte length? I'd be surprised if it
>>>>> was.
>>>>
>>>> A BSTR does not have a "length portion"... it is just a pointer...
>>>>
>>>> Or did I miss something?
>>>>
>>>> BSTR is just "wchar_t*". ..
>>>
>>> You missed a lot by thinking this :)[/color]
>>
>> No problem for me... I never rely on undecomented features...
>>
>> (As "Alexander said: "And also let's not forget that the
>> allocation strategy for BSTR is officially undocumented... ")
>>
>> Greetings
>> Jochen[/color]
>
>[/color]

[Pavel Lebedinsky [MSFT]]

Re: x64 and BSTR allocation, what has changed?

"Egbert Nierop (MVP for IIS)" wrote:
[color=blue]
> In fact, I wanted a mechanisme, that has 'disabled' BSTR caching, without
> affecting the whole process or using a sneaky API to disable caching.
>
> Caching of strings in a process, is bad for performance for a Service.[/color]

What are the symptoms you're seeing? High contention/cpu usage/
memory usage/something else? Does setting OA_NOCACHE=1 help?

What OS is this?

--
This posting is provided "AS IS" with no warranties, and confers no
rights.

[Pavel Lebedinsky [MSFT]]

Re: x64 and BSTR allocation, what has changed?

BTW, I'm not suggesting OA_NOCACHE as a permanent workaround.
But it would be useful to know if it helps alleviate whatever symptoms
you are seeing.

--
This posting is provided "AS IS" with no warranties, and confers no
rights.

"Pavel Lebedinsky [MSFT]" wrote:
[color=blue][color=green]
>> In fact, I wanted a mechanisme, that has 'disabled' BSTR caching, without
>> affecting the whole process or using a sneaky API to disable caching.
>>
>> Caching of strings in a process, is bad for performance for a Service.[/color]
>
> What are the symptoms you're seeing? High contention/cpu usage/
> memory usage/something else? Does setting OA_NOCACHE=1 help?
>
> What OS is this?[/color]

[Egbert Nierop \(MVP for IIS\)]

Re: x64 and BSTR allocation, what has changed?


"Pavel Lebedinsky [MSFT]" <pavel@online.m icrosoft.com> wrote in message
news:e46$8CgTGH A.2656@TK2MSFTN GP10.phx.gbl...[color=blue]
> "Egbert Nierop (MVP for IIS)" wrote:
>[color=green]
>> In fact, I wanted a mechanisme, that has 'disabled' BSTR caching, without
>> affecting the whole process or using a sneaky API to disable caching.
>>
>> Caching of strings in a process, is bad for performance for a Service.[/color]
>
> What are the symptoms you're seeing? High contention/cpu usage/
> memory usage/something else? Does setting OA_NOCACHE=1 help?
>
> What OS is this?[/color]

Any server version of windows (starting with windows 2000).

I very simple program showed me that BSTR caching, only works within -the
same- thread. So, in IIS, when you use a lot of oleautomation (COM &
scripting) and each thread creates its own garbage etc, it is really useless
to create a caching mechanism. In terms of scalability, using thread local
storage, is evil :)

And OA_NOCACHE only helps as I understood, for a debug DLL as the
documentation says.

And there is also a new API call, starting from XP Sp1 which disables
caching. But my component, should not disable caching in a process that I do
not host.

Thanks anyway for your comments.

[Egbert Nierop \(MVP for IIS\)]

Re: x64 and BSTR allocation, what has changed?


"Alexander Nickolov" <agnickolov@mvp s.org> wrote in message
news:e$t5OeeTGH A.4300@TK2MSFTN GP14.phx.gbl...[color=blue]
> Since I started it, let me clarify. The layout is documented
> to the extent that the 4 bytes preceding the pointer are
> its length (in bytes I might add). It is also documented
> that you should use SysAllocString and friends to allocate
> BSTRs. What is not documented, however, is how SysAlloc
> functions allocate the raw memory. It was common knowledge
> that on Win32 the allocated pointer is 4 bytes before the returned
> pointer, but this does not make it documented! (It is also public
> knowledge that CoTaskMemAlloc is used for the memory
> allocation of course.) Apparently, this allowed Microsoft to
> change the allocation strategy to reflect the alignment requirement
> for Win64 and return a 64-bit aligned pointer from SysAlloc
> functions. Of course this breaks the aforementioned public
> knowledge (and we extended it obviously...). There's nothing
> wrong with knowing undocumented aspects. What's wrong is
> to expect that these assumptions will continue to be valid
> going forward.[/color]

I don't quite understand this.
Imagine, that the total prefix, still was 4 bytes, instead of 8, this has
nothing to do with alignment imho.
CoTaskMemAlloc, just returns a 64-bit pointer and in 32-bit windows, a
32-bit pointer. If alignment _plays_ a role, then not the SysAlloc*
functions deal with that, but CoTask* deals with it! I dare to bet my shoes
on this...

:)