Passing jagged array byte[][] to unmanaged code

[=?Utf-8?B?U2hhcm9u?=]

Hi Gurus,

I need to transfer a jagged array of byte[][] by reference to unmanaged
function, The unmanaged code should changed the values of the array, and when
the unmanaged function returns I need to show the array data to the end user.

Can I do that?
How?


-------
Thanks
Sharon

[Nicholas Paldino [.NET/C# MVP]]

Re: Passing jagged array byte[][] to unmanaged code

Sharon,

You are going to have to marshal this manually, as I don't think there
is a mechanism in the framework to handle this on your own.

You would have to iterate through each array of arrays, dynamically
allocating the memory in unmanaged memory, then copying the values. On
return, you would have to copy the values back.

Unsafe code would help a little bit here, as you wouldn't have to worry
about allocating and deallocating the memory (you could use stackalloc to
allocate the memory for you).


--
- Nicholas Paldino [.NET/C# MVP]
- mvp@spam.guard. caspershouse.co m

"Sharon" <SharonG@newsgr oups.nospamwrot e in message
news:9EBDD349-2CBE-4105-B31C-4A43C90865C4@mi crosoft.com...

Hi Gurus,
>
I need to transfer a jagged array of byte[][] by reference to unmanaged
function, The unmanaged code should changed the values of the array, and
when
the unmanaged function returns I need to show the array data to the end
user.
>
Can I do that?
How?
>
>
-------
Thanks
Sharon

[=?Utf-8?B?U2hhcm9u?=]

Re: Passing jagged array byte[][] to unmanaged code

I'm afraid this kind of work will undermine the performance as there is a lot
of data to copy.
I thought there is a way to use the managed array by reference on the
unmanaged side and avoiding the memory copy.
If there is no solution for that, I think I will write some more piece of
code in native C++ (unmanaged) to do that.

--------
Thanks
Sharon

[Willy Denoyette [MVP]]

Re: Passing jagged array byte[][] to unmanaged code

"Sharon" <SharonG@newsgr oups.nospamwrot e in message
news:68411523-3EE9-41F8-B8CF-051716FEEB4A@mi crosoft.com...

I'm afraid this kind of work will undermine the performance as there is a
lot
of data to copy.
I thought there is a way to use the managed array by reference on the
unmanaged side and avoiding the memory copy.
If there is no solution for that, I think I will write some more piece of
code in native C++ (unmanaged) to do that.
>
--------
Thanks
Sharon

What make you think that copying will be an issue? How large are these
array's? And why are you using an jagged array, while this kind of type
can't be handled directly in C or C++ in the first place?

Willy.

[Ben Voigt [C++ MVP]]

Re: Passing jagged array byte[][] to unmanaged code


"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:eyqlTlerHH A.5028@TK2MSFTN GP05.phx.gbl...

"Sharon" <SharonG@newsgr oups.nospamwrot e in message
news:68411523-3EE9-41F8-B8CF-051716FEEB4A@mi crosoft.com...

>I'm afraid this kind of work will undermine the performance as there is a
>lot
>of data to copy.
>I thought there is a way to use the managed array by reference on the
>unmanaged side and avoiding the memory copy.
>If there is no solution for that, I think I will write some more piece of
>code in native C++ (unmanaged) to do that.
>>
>--------
>Thanks
>Sharon

>
>
>
What make you think that copying will be an issue? How large are these
array's? And why are you using an jagged array, while this kind of type
can't be handled directly in C or C++ in the first place?

C++, native as well as managed, handle jagged arrays just fine. But
unmanaged C++ can't directly handle .NET arrays of anything but fundamental
types, which includes .NET jagged arrays.

Why not change the C++ code in question to C++/CLI, then you can use the
jagged .NET array directly?

This poster also previously asked the same question on the C++ list (or
maybe it was interop), and got the same answers there.

>
Willy.
>

[=?Utf-8?B?U2hhcm9u?=]

Re: Passing jagged array byte[][] to unmanaged code

The data maybe be large as several giga bytes. and the data copy will be done
when moving it to the unmanaged code and once more when getting it back.

I'm getting a lot of System.Array that encapsulates a byte[] from a COM
component, so I thought to put this arrays (without copying) in a jagged
array of byte[][], and then letting another native C++ (unmanaged) DLL change
this jagged array for me for later use. This native C++ DLL is doing some
heavy algorithm and therefore is written in C++ and not .NET.

The original data that is read from a COM component to a managed code is
used for display and other stuff before it's moved to the C++ DLL (unmanaged
algorithm component). So I have the data in a managed memory, and should be
changed by an unmanaged code and back to the managed code.

How can I do it effectively regarding performance and development time.
Simple and effective ???


---------
Thanks
Sharon

[=?Utf-8?B?U2hhcm9u?=]

Re: Passing jagged array byte[][] to unmanaged code

I also believe that the best solution will be something like that.
I think that I will write an additional CLI/C++ component that will be used
as a bridge between the unmanaged code and the managed code.


---------
Thanks
Sharon

[Willy Denoyette [MVP]]

Re: Passing jagged array byte[][] to unmanaged code

"Sharon" <SharonG@newsgr oups.nospamwrot e in message
news:94A57FB8-24E5-429C-961F-757B410243CC@mi crosoft.com...

The data maybe be large as several giga bytes. and the data copy will be
done
when moving it to the unmanaged code and once more when getting it back.
>

Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?

I'm getting a lot of System.Array that encapsulates a byte[] from a COM
component, so I thought to put this arrays (without copying) in a jagged
array of byte[][], and then letting another native C++ (unmanaged) DLL
change
this jagged array for me for later use. This native C++ DLL is doing some
heavy algorithm and therefore is written in C++ and not .NET.
>

Why a jagged array?

The original data that is read from a COM component to a managed code is
used for display and other stuff before it's moved to the C++ DLL
(unmanaged
algorithm component). So I have the data in a managed memory, and should
be
changed by an unmanaged code and back to the managed code.
>
How can I do it effectively regarding performance and development time.
Simple and effective ???

So, you are getting arrays of bytes from COM, which means that you are
already copying from native memory to managed memory. Also, the COM array is
not jagged, so why are you putting this data in a jagged array? Why not pass
the CLR byte array to C++ and let the C++ function change the array contents
without copying?

Willy.

[Willy Denoyette [MVP]]

Re: Passing jagged array byte[][] to unmanaged code

"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
news:52CD899F-DD62-45B3-9B48-617091F25886@mi crosoft.com...

>
"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:eyqlTlerHH A.5028@TK2MSFTN GP05.phx.gbl...

>"Sharon" <SharonG@newsgr oups.nospamwrot e in message
>news:6841152 3-3EE9-41F8-B8CF-051716FEEB4A@mi crosoft.com...

>>I'm afraid this kind of work will undermine the performance as there is
>>a lot
>>of data to copy.
>>I thought there is a way to use the managed array by reference on the
>>unmanaged side and avoiding the memory copy.
>>If there is no solution for that, I think I will write some more piece
>>of
>>code in native C++ (unmanaged) to do that.
>>>
>>--------
>>Thanks
>>Sharon

>>
>>
>>
>What make you think that copying will be an issue? How large are these
>array's? And why are you using an jagged array, while this kind of type
>can't be handled directly in C or C++ in the first place?

>
C++, native as well as managed, handle jagged arrays just fine. But
unmanaged C++ can't directly handle .NET arrays of anything but
fundamental types, which includes .NET jagged arrays.
>

Note that I said "this kind of type can't be handled directly in C and C++",
we are talking about C# here which implies "managed array types".

Why not change the C++ code in question to C++/CLI, then you can use the
jagged .NET array directly?
>

Maybe the OP can change the C++ code to C++/CLI, maybe he doesn't own the
source, or is not willing to introduce yet another language in his code
base.

This poster also previously asked the same question on the C++ list (or
maybe it was interop), and got the same answers there.
>

He also asked the same question several times before, and clearly said that
the "algorithm" was written by another group and had to be in unmanaged
code.

Willy.

[Ben Voigt [C++ MVP]]

Re: Passing jagged array byte[][] to unmanaged code


"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:e3kLHhmrHH A.3448@TK2MSFTN GP05.phx.gbl...

"Sharon" <SharonG@newsgr oups.nospamwrot e in message
news:94A57FB8-24E5-429C-961F-757B410243CC@mi crosoft.com...

>The data maybe be large as several giga bytes. and the data copy will be
>done
>when moving it to the unmanaged code and once more when getting it back.
>>

>
Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?
>

>I'm getting a lot of System.Array that encapsulates a byte[] from a COM
>component, so I thought to put this arrays (without copying) in a jagged
>array of byte[][], and then letting another native C++ (unmanaged) DLL
>change
>this jagged array for me for later use. This native C++ DLL is doing some
>heavy algorithm and therefore is written in C++ and not .NET.
>>

>
Why a jagged array?

Simply an array of arrays. Jagged, because each member array can have a
different size.

>

>The original data that is read from a COM component to a managed code is
>used for display and other stuff before it's moved to the C++ DLL
>(unmanaged
>algorithm component). So I have the data in a managed memory, and should
>be
>changed by an unmanaged code and back to the managed code.
>>
>How can I do it effectively regarding performance and development time.
>Simple and effective ???

Stop receiving the COM array into a .NET array. Store each SAFEARRAY* as an
element in an array of IntPtr instead. Then give that IntPtr array to the
unmanaged code.

>
>
So, you are getting arrays of bytes from COM, which means that you are
already copying from native memory to managed memory. Also, the COM array
is not jagged, so why are you putting this data in a jagged array? Why not
pass

Each incoming array is one-dimensional, apparently. Knowing what jagged
means helps here.

the CLR byte array to C++ and let the C++ function change the array
contents without copying?

If the arrays coming from COM are allocated scattered throughout memory,
then it can't be passed as a single native array. It will have to be an
array of pointers.

>
Willy.
>

[Willy Denoyette [MVP]]

Re: Passing jagged array byte[][] to unmanaged code

"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
news:uaYvIdurHH A.1268@TK2MSFTN GP03.phx.gbl...

>
"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:e3kLHhmrHH A.3448@TK2MSFTN GP05.phx.gbl...

>"Sharon" <SharonG@newsgr oups.nospamwrot e in message
>news:94A57FB 8-24E5-429C-961F-757B410243CC@mi crosoft.com...

>>The data maybe be large as several giga bytes. and the data copy will be
>>done
>>when moving it to the unmanaged code and once more when getting it back.
>>>

>>
>Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?
>>

>>I'm getting a lot of System.Array that encapsulates a byte[] from a COM
>>component, so I thought to put this arrays (without copying) in a jagged
>>array of byte[][], and then letting another native C++ (unmanaged) DLL
>>change
>>this jagged array for me for later use. This native C++ DLL is doing
>>some
>>heavy algorithm and therefore is written in C++ and not .NET.
>>>

>>
>Why a jagged array?

>
Simply an array of arrays. Jagged, because each member array can have a
different size.
>

I know what a managed jagged array is, you can not pass such type from C#
(or any other managed code) to C++ , even if you could, C++ cannot directly
access *managed* jagged arrays, it has no idea how they are laid out in
memory.

>>

>>The original data that is read from a COM component to a managed code is
>>used for display and other stuff before it's moved to the C++ DLL
>>(unmanaged
>>algorithm component). So I have the data in a managed memory, and should
>>be
>>changed by an unmanaged code and back to the managed code.
>>>
>>How can I do it effectively regarding performance and development time.
>>Simple and effective ???

>
Stop receiving the COM array into a .NET array. Store each SAFEARRAY* as
an element in an array of IntPtr instead. Then give that IntPtr array to
the unmanaged code.
>

>>
>>
>So, you are getting arrays of bytes from COM, which means that you are
>already copying from native memory to managed memory. Also, the COM array
>is not jagged, so why are you putting this data in a jagged array? Why
>not pass

>
Each incoming array is one-dimensional, apparently. Knowing what jagged
means helps here.
>

Again, I know what a *managed* jagged array is, it's a type that is not know
by C++.

>the CLR byte array to C++ and let the C++ function change the array
>contents without copying?

>
If the arrays coming from COM are allocated scattered throughout memory,
then it can't be passed as a single native array. It will have to be an
array of pointers.
>

In the OP's case, the array that comes from COM is a one-dimentional byte
array, as such it gets marshaled as a one-dimentional managed array, so COM
is a non issue here.

What the OP can do is store the address of each individual array in an array
of IntPtr and pass that array to C++ together with an array that defines the
size of each array pointed to by the pointers in the array of pointers, and
a third parameter that defines the size of the array of pointers, but that
's not the same as passing a jagged array.

The signature could look something like this:

[DllImport("blab ala")]
static extern Foo(IntPtr[] ptr, int[] sizeOfElem, int elem);

however, this is not possible:

[DllImport("blab ala")]
static extern Foo(byte[][] ptr);

See what I mean?

Willy.

[Ben Voigt [C++ MVP]]

Re: Passing jagged array byte[][] to unmanaged code


"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:u$CNDv2rHH A.3448@TK2MSFTN GP05.phx.gbl...

"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
news:uaYvIdurHH A.1268@TK2MSFTN GP03.phx.gbl...

>>
>"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
>news:e3kLHhmrH HA.3448@TK2MSFT NGP05.phx.gbl.. .

>>"Sharon" <SharonG@newsgr oups.nospamwrot e in message
>>news:94A57F B8-24E5-429C-961F-757B410243CC@mi crosoft.com...
>>>The data maybe be large as several giga bytes. and the data copy will
>>>be done
>>>when moving it to the unmanaged code and once more when getting it
>>>back.
>>>>
>>>
>>Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?
>>>
>>>I'm getting a lot of System.Array that encapsulates a byte[] from a COM
>>>component, so I thought to put this arrays (without copying) in a
>>>jagged
>>>array of byte[][], and then letting another native C++ (unmanaged) DLL
>>>change
>>>this jagged array for me for later use. This native C++ DLL is doing
>>>some
>>>heavy algorithm and therefore is written in C++ and not .NET.
>>>>
>>>
>>Why a jagged array?

>>
>Simply an array of arrays. Jagged, because each member array can have a
>different size.
>>

>
I know what a managed jagged array is, you can not pass such type from C#
(or any other managed code) to C++ , even if you could, C++ cannot
directly access *managed* jagged arrays, it has no idea how they are laid
out in memory.
>
>

>>>
>>>The original data that is read from a COM component to a managed code
>>>is
>>>used for display and other stuff before it's moved to the C++ DLL
>>>(unmanaged
>>>algorithm component). So I have the data in a managed memory, and
>>>should be
>>>changed by an unmanaged code and back to the managed code.
>>>>
>>>How can I do it effectively regarding performance and development time.
>>>Simple and effective ???

>>
>Stop receiving the COM array into a .NET array. Store each SAFEARRAY* as
>an element in an array of IntPtr instead. Then give that IntPtr array to
>the unmanaged code.
>>

>>>
>>>
>>So, you are getting arrays of bytes from COM, which means that you are
>>already copying from native memory to managed memory. Also, the COM
>>array is not jagged, so why are you putting this data in a jagged array?
>>Why not pass

>>
>Each incoming array is one-dimensional, apparently. Knowing what jagged
>means helps here.
>>

Again, I know what a *managed* jagged array is, it's a type that is not
know by C++.
>

>>the CLR byte array to C++ and let the C++ function change the array
>>contents without copying?

>>
>If the arrays coming from COM are allocated scattered throughout memory,
>then it can't be passed as a single native array. It will have to be an
>array of pointers.
>>

>
In the OP's case, the array that comes from COM is a one-dimentional byte
array, as such it gets marshaled as a one-dimentional managed array, so
COM is a non issue here.
>
What the OP can do is store the address of each individual array in an
array of IntPtr and pass that array to C++ together with an array that
defines the size of each array pointed to by the pointers in the array of
pointers, and a third parameter that defines the size of the array of
pointers, but that 's not the same as passing a jagged array.

That *is* a jagged array. Any array of arrays is a jagged array, all
languages support the concept. P/Invoke cannot pass an array of
non-fundamental types and a .NET array is not a fundamental type. So native
C++ can't use a managed jagged array.

>
The signature could look something like this:
>
[DllImport("blab ala")]
static extern Foo(IntPtr[] ptr, int[] sizeOfElem, int elem);

That is a native jagged array being passed. Now, instead of unmarshalling
the data from SAFEARRAY to a .NET array, and using Marshal to make a copy in
HGLOBAL memory, why not just store the SAFEARRAY* in the array, if the data
is just going to be passed back to unmanaged code?

>
however, this is not possible:
>
[DllImport("blab ala")]
static extern Foo(byte[][] ptr);
>
See what I mean?
>
Willy.
>
>
>
>

[Willy Denoyette [MVP]]

Re: Passing jagged array byte[][] to unmanaged code

"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
news:789FB231-66E3-4A4D-870F-C1451CC815E3@mi crosoft.com...

>
"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:u$CNDv2rHH A.3448@TK2MSFTN GP05.phx.gbl...

>"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
>news:uaYvIdurH HA.1268@TK2MSFT NGP03.phx.gbl.. .

>>>
>>"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
>>news:e3kLHhmr HHA.3448@TK2MSF TNGP05.phx.gbl. ..
>>>"Sharon" <SharonG@newsgr oups.nospamwrot e in message
>>>news:94A57FB 8-24E5-429C-961F-757B410243CC@mi crosoft.com...
>>>>The data maybe be large as several giga bytes. and the data copy will
>>>>be done
>>>>when moving it to the unmanaged code and once more when getting it
>>>>back.
>>>>>
>>>>
>>>Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?
>>>>
>>>>I'm getting a lot of System.Array that encapsulates a byte[] from a
>>>>COM
>>>>component , so I thought to put this arrays (without copying) in a
>>>>jagged
>>>>array of byte[][], and then letting another native C++ (unmanaged) DLL
>>>>change
>>>>this jagged array for me for later use. This native C++ DLL is doing
>>>>some
>>>>heavy algorithm and therefore is written in C++ and not .NET.
>>>>>
>>>>
>>>Why a jagged array?
>>>
>>Simply an array of arrays. Jagged, because each member array can have a
>>different size.
>>>

>>
>I know what a managed jagged array is, you can not pass such type from C#
>(or any other managed code) to C++ , even if you could, C++ cannot
>directly access *managed* jagged arrays, it has no idea how they are laid
>out in memory.
>>
>>

>>>>
>>>>The original data that is read from a COM component to a managed code
>>>>is
>>>>used for display and other stuff before it's moved to the C++ DLL
>>>>(unmanage d
>>>>algorithm component). So I have the data in a managed memory, and
>>>>should be
>>>>changed by an unmanaged code and back to the managed code.
>>>>>
>>>>How can I do it effectively regarding performance and development
>>>>time.
>>>>Simple and effective ???
>>>
>>Stop receiving the COM array into a .NET array. Store each SAFEARRAY*
>>as an element in an array of IntPtr instead. Then give that IntPtr
>>array to the unmanaged code.
>>>
>>>>
>>>>
>>>So, you are getting arrays of bytes from COM, which means that you are
>>>already copying from native memory to managed memory. Also, the COM
>>>array is not jagged, so why are you putting this data in a jagged
>>>array? Why not pass
>>>
>>Each incoming array is one-dimensional, apparently. Knowing what jagged
>>means helps here.
>>>

>Again, I know what a *managed* jagged array is, it's a type that is not
>know by C++.
>>

>>>the CLR byte array to C++ and let the C++ function change the array
>>>contents without copying?
>>>
>>If the arrays coming from COM are allocated scattered throughout memory,
>>then it can't be passed as a single native array. It will have to be an
>>array of pointers.
>>>

>>
>In the OP's case, the array that comes from COM is a one-dimentional byte
>array, as such it gets marshaled as a one-dimentional managed array, so
>COM is a non issue here.
>>
>What the OP can do is store the address of each individual array in an
>array of IntPtr and pass that array to C++ together with an array that
>defines the size of each array pointed to by the pointers in the array of
>pointers, and a third parameter that defines the size of the array of
>pointers, but that 's not the same as passing a jagged array.

>
That *is* a jagged array. Any array of arrays is a jagged array, all
languages support the concept. P/Invoke cannot pass an array of
non-fundamental types and a .NET array is not a fundamental type. So
native C++ can't use a managed jagged array.
>

When I'm talking of jagged arrays I'm referring to an array type defined in
the CLI (CLR), I don't know if the C or C++ standard has a definition for
"jagged arrays", but "jagged arrays" on the CLR, are arrays of arrays,
right, but the constituent arrays don't need to be of the same size.

For instance the following illustrates how you can define and access a
jagged array of int C#:
int jagged [][] = new int[][]{
new int[]{1, 2, 3, 4},
new int[]{4, 5, 6}
};
jagged[0][2] = 100;
jagged[1][2] = 5;
...
how you can you define and access such an array type in C/C++, note I'm not
asking for an array of pointers.

>>
>The signature could look something like this:
>>
>[DllImport("blab ala")]
>static extern Foo(IntPtr[] ptr, int[] sizeOfElem, int elem);

>
That is a native jagged array being passed.

How that?, the first argument is an array of IntPtr, marshaled as an array
of pointers, or do you mean that what is passed collectively defines an
array of arrays.

Now, instead of unmarshalling

the data from SAFEARRAY to a .NET array, and using Marshal to make a copy
in HGLOBAL memory, why not just store the SAFEARRAY* in the array, if the
data is just going to be passed back to unmanaged code?
>

I didn't say he should make a copy to "unmanaged" memory, I said he could
take the address of each individualy array store it in an array of IntPtr
(or void*) and pass that array to C++, something like this:

[DllImport("blab la.dll"), SuppressUnmanag edCodeSecurity] extern static
void Foo(IntPtr[] ia, int[] elems, int size);
...
// say jagged is an array that holds the marshaled COM arrays (be it
conforming or SAFEARRAY's is not the issue)
byte[][] jagged = new byte[][];
jagged[0] = CallCOM(); // returns an array of bytes.
jagged[1] = .......
.....
unsafe
{
IntPtr[] ptrArray = new IntPtr[jagged.Length];
int[] elems = new int[jagged.Length];
for(int n = 0; n < jagged.Length; n++)
{
fixed(byte* p = &jagged[n][0])
{
ptrArray[n] = new IntPtr(p);
elems[n] = jagged[n].Length;
}
}
Foo(ptrArray, elems, jagged.Length); // call "native code" that
changes the contents of jagged ... inline...
}
....

Willy.

[Ben Voigt [C++ MVP]]

Re: Passing jagged array byte[][] to unmanaged code


"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:elfG244rHH A.5032@TK2MSFTN GP04.phx.gbl...

"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
news:789FB231-66E3-4A4D-870F-C1451CC815E3@mi crosoft.com...

>>
>"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
>news:u$CNDv2rH HA.3448@TK2MSFT NGP05.phx.gbl.. .

>>"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
>>news:uaYvIdur HHA.1268@TK2MSF TNGP03.phx.gbl. ..
>>>>
>>>"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
>>>news:e3kLHhm rHHA.3448@TK2MS FTNGP05.phx.gbl ...
>>>>"Sharon" <SharonG@newsgr oups.nospamwrot e in message
>>>>news:94A57F B8-24E5-429C-961F-757B410243CC@mi crosoft.com...
>>>>>The data maybe be large as several giga bytes. and the data copy will
>>>>>be done
>>>>>when moving it to the unmanaged code and once more when getting it
>>>>>back.
>>>>>>
>>>>>
>>>>Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?
>>>>>
>>>>>I'm getting a lot of System.Array that encapsulates a byte[] from a
>>>>>COM
>>>>>componen t, so I thought to put this arrays (without copying) in a
>>>>>jagged
>>>>>array of byte[][], and then letting another native C++ (unmanaged)
>>>>>DLL change
>>>>>this jagged array for me for later use. This native C++ DLL is doing
>>>>>some
>>>>>heavy algorithm and therefore is written in C++ and not .NET.
>>>>>>
>>>>>
>>>>Why a jagged array?
>>>>
>>>Simply an array of arrays. Jagged, because each member array can have
>>>a different size.
>>>>
>>>
>>I know what a managed jagged array is, you can not pass such type from
>>C# (or any other managed code) to C++ , even if you could, C++ cannot
>>directly access *managed* jagged arrays, it has no idea how they are
>>laid out in memory.
>>>
>>>
>>>>>
>>>>>The original data that is read from a COM component to a managed code
>>>>>is
>>>>>used for display and other stuff before it's moved to the C++ DLL
>>>>>(unmanag ed
>>>>>algorith m component). So I have the data in a managed memory, and
>>>>>should be
>>>>>changed by an unmanaged code and back to the managed code.
>>>>>>
>>>>>How can I do it effectively regarding performance and development
>>>>>time.
>>>>>Simple and effective ???
>>>>
>>>Stop receiving the COM array into a .NET array. Store each SAFEARRAY*
>>>as an element in an array of IntPtr instead. Then give that IntPtr
>>>array to the unmanaged code.
>>>>
>>>>>
>>>>>
>>>>So, you are getting arrays of bytes from COM, which means that you are
>>>>already copying from native memory to managed memory. Also, the COM
>>>>array is not jagged, so why are you putting this data in a jagged
>>>>array? Why not pass
>>>>
>>>Each incoming array is one-dimensional, apparently. Knowing what
>>>jagged means helps here.
>>>>
>>Again, I know what a *managed* jagged array is, it's a type that is not
>>know by C++.
>>>
>>>>the CLR byte array to C++ and let the C++ function change the array
>>>>contents without copying?
>>>>
>>>If the arrays coming from COM are allocated scattered throughout
>>>memory, then it can't be passed as a single native array. It will have
>>>to be an array of pointers.
>>>>
>>>
>>In the OP's case, the array that comes from COM is a one-dimentional
>>byte array, as such it gets marshaled as a one-dimentional managed
>>array, so COM is a non issue here.
>>>
>>What the OP can do is store the address of each individual array in an
>>array of IntPtr and pass that array to C++ together with an array that
>>defines the size of each array pointed to by the pointers in the array
>>of pointers, and a third parameter that defines the size of the array
>>of pointers, but that 's not the same as passing a jagged array.

>>
>That *is* a jagged array. Any array of arrays is a jagged array, all
>languages support the concept. P/Invoke cannot pass an array of
>non-fundamental types and a .NET array is not a fundamental type. So
>native C++ can't use a managed jagged array.
>>

>
When I'm talking of jagged arrays I'm referring to an array type defined
in the CLI (CLR), I don't know if the C or C++ standard has a definition
for "jagged arrays", but "jagged arrays" on the CLR, are arrays of arrays,
right, but the constituent arrays don't need to be of the same size.
>
For instance the following illustrates how you can define and access a
jagged array of int C#:
int jagged [][] = new int[][]{
new int[]{1, 2, 3, 4},
new int[]{4, 5, 6}
};
jagged[0][2] = 100;
jagged[1][2] = 5;
...
how you can you define and access such an array type in C/C++, note I'm
not asking for an array of pointers.

In C++, arrays and pointers are interchangable (except for sizeof), so an
array of pointers is an array of arrays. You can of course have an array
object which combines the pointer and the length information, so a C++
jagged array could also look like: std::vector<std ::vector<int>*>

>
>

>>>
>>The signature could look something like this:
>>>
>>[DllImport("blab ala")]
>>static extern Foo(IntPtr[] ptr, int[] sizeOfElem, int elem);

>>
>That is a native jagged array being passed.

How that?, the first argument is an array of IntPtr, marshaled as an array
of pointers, or do you mean that what is passed collectively defines an
array of arrays.
>
Now, instead of unmarshalling

>the data from SAFEARRAY to a .NET array, and using Marshal to make a copy
>in HGLOBAL memory, why not just store the SAFEARRAY* in the array, if the
>data is just going to be passed back to unmanaged code?
>>

>
I didn't say he should make a copy to "unmanaged" memory, I said he could
take the address of each individualy array store it in an array of IntPtr
(or void*) and pass that array to C++, something like this:
>
[DllImport("blab la.dll"), SuppressUnmanag edCodeSecurity] extern static
void Foo(IntPtr[] ia, int[] elems, int size);
...
// say jagged is an array that holds the marshaled COM arrays (be it
conforming or SAFEARRAY's is not the issue)
byte[][] jagged = new byte[][];
jagged[0] = CallCOM(); // returns an array of bytes.
jagged[1] = .......
.....
unsafe
{
IntPtr[] ptrArray = new IntPtr[jagged.Length];
int[] elems = new int[jagged.Length];
for(int n = 0; n < jagged.Length; n++)
{
fixed(byte* p = &jagged[n][0])
{
ptrArray[n] = new IntPtr(p);
elems[n] = jagged[n].Length;
}
}
Foo(ptrArray, elems, jagged.Length); // call "native code" that
changes the contents of jagged ... inline...
}
...
>
Willy.
>
>
>
>

[Willy Denoyette [MVP]]

Re: Passing jagged array byte[][] to unmanaged code

"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
news:365F78DC-E9F7-4FE3-ADEB-498CD50692B4@mi crosoft.com...

>
"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
news:elfG244rHH A.5032@TK2MSFTN GP04.phx.gbl...

>"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
>news:789FB23 1-66E3-4A4D-870F-C1451CC815E3@mi crosoft.com...

>>>
>>"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
>>news:u$CNDv2r HHA.3448@TK2MSF TNGP05.phx.gbl. ..
>>>"Ben Voigt [C++ MVP]" <rbv@nospam.nos pamwrote in message
>>>news:uaYvIdu rHHA.1268@TK2MS FTNGP03.phx.gbl ...
>>>>>
>>>>"Willy Denoyette [MVP]" <willy.denoyett e@telenet.bewro te in message
>>>>news:e3kLHh mrHHA.3448@TK2M SFTNGP05.phx.gb l...
>>>>>"Sharon" <SharonG@newsgr oups.nospamwrot e in message
>>>>>news:94A57 FB8-24E5-429C-961F-757B410243CC@mi crosoft.com...
>>>>>>The data maybe be large as several giga bytes. and the data copy
>>>>>>will be done
>>>>>>when moving it to the unmanaged code and once more when getting it
>>>>>>back.
>>>>>>>
>>>>>>
>>>>>Hmmm.... Several Gigabytes? Is this running on a 64-bit OS?
>>>>>>
>>>>>>I'm getting a lot of System.Array that encapsulates a byte[] from a
>>>>>>COM
>>>>>>component , so I thought to put this arrays (without copying) in a
>>>>>>jagged
>>>>>>array of byte[][], and then letting another native C++ (unmanaged)
>>>>>>DLL change
>>>>>>this jagged array for me for later use. This native C++ DLL is doing
>>>>>>some
>>>>>>heavy algorithm and therefore is written in C++ and not .NET.
>>>>>>>
>>>>>>
>>>>>Why a jagged array?
>>>>>
>>>>Simply an array of arrays. Jagged, because each member array can have
>>>>a different size.
>>>>>
>>>>
>>>I know what a managed jagged array is, you can not pass such type from
>>>C# (or any other managed code) to C++ , even if you could, C++ cannot
>>>directly access *managed* jagged arrays, it has no idea how they are
>>>laid out in memory.
>>>>
>>>>
>>>>>>
>>>>>>The original data that is read from a COM component to a managed
>>>>>>code is
>>>>>>used for display and other stuff before it's moved to the C++ DLL
>>>>>>(unmanage d
>>>>>>algorit hm component). So I have the data in a managed memory, and
>>>>>>should be
>>>>>>changed by an unmanaged code and back to the managed code.
>>>>>>>
>>>>>>How can I do it effectively regarding performance and development
>>>>>>time.
>>>>>>Simple and effective ???
>>>>>
>>>>Stop receiving the COM array into a .NET array. Store each SAFEARRAY*
>>>>as an element in an array of IntPtr instead. Then give that IntPtr
>>>>array to the unmanaged code.
>>>>>
>>>>>>
>>>>>>
>>>>>So, you are getting arrays of bytes from COM, which means that you
>>>>>are already copying from native memory to managed memory. Also, the
>>>>>COM array is not jagged, so why are you putting this data in a jagged
>>>>>array? Why not pass
>>>>>
>>>>Each incoming array is one-dimensional, apparently. Knowing what
>>>>jagged means helps here.
>>>>>
>>>Again, I know what a *managed* jagged array is, it's a type that is not
>>>know by C++.
>>>>
>>>>>the CLR byte array to C++ and let the C++ function change the array
>>>>>contents without copying?
>>>>>
>>>>If the arrays coming from COM are allocated scattered throughout
>>>>memory, then it can't be passed as a single native array. It will
>>>>have to be an array of pointers.
>>>>>
>>>>
>>>In the OP's case, the array that comes from COM is a one-dimentional
>>>byte array, as such it gets marshaled as a one-dimentional managed
>>>array, so COM is a non issue here.
>>>>
>>>What the OP can do is store the address of each individual array in an
>>>array of IntPtr and pass that array to C++ together with an array that
>>>defines the size of each array pointed to by the pointers in the array
>>>of pointers, and a third parameter that defines the size of the array
>>>of pointers, but that 's not the same as passing a jagged array.
>>>
>>That *is* a jagged array. Any array of arrays is a jagged array, all
>>languages support the concept. P/Invoke cannot pass an array of
>>non-fundamental types and a .NET array is not a fundamental type. So
>>native C++ can't use a managed jagged array.
>>>

>>
>When I'm talking of jagged arrays I'm referring to an array type defined
>in the CLI (CLR), I don't know if the C or C++ standard has a definition
>for "jagged arrays", but "jagged arrays" on the CLR, are arrays of
>arrays, right, but the constituent arrays don't need to be of the same
>size.
>>
>For instance the following illustrates how you can define and access a
>jagged array of int C#:
>int jagged [][] = new int[][]{
> new int[]{1, 2, 3, 4},
> new int[]{4, 5, 6}
>};
> jagged[0][2] = 100;
> jagged[1][2] = 5;
> ...
>how you can you define and access such an array type in C/C++, note I'm
>not asking for an array of pointers.

>
In C++, arrays and pointers are interchangable (except for sizeof), so an
array of pointers is an array of arrays. You can of course have an array
object which combines the pointer and the length information, so a C++
jagged array could also look like: std::vector<std ::vector<int>*>

I expected this :-).
It's not because you can "construct" something that looks and behaves like a
"jagged" array in C++ that they are the same as managed "jagged arrays", a
"jagged array" is a well defined language feature in .NET (all managed
languages including C++/CLI) with well defined language syntax rules and
based on a CLS compliant type from the CTS, and this is not true in C++.
in C#
int[3][] ja;
ja is a jagged array.

which is not possible in C++
int ja[3][];
will not compile.....
and
int ja[3][5];

is a two dimentional rectangular array, an array of arrays, but it's not a
jagged array!

Anyway, what's important here and what the OP is after, is that they can be
passed and accessed *directly* across the managed/unmanaged language (here
C# and C++) boundaries *without* the need to resort to some kind of custom
marshaling (like I've illustrated), and we both know that this is not the
case, right?

Willy.