Determining the size of allocated memory from a pointer

[Hackles]

Hello,
Is it possible to retrieve the size of allocated memory from a pointer (more specifically, a pointer returned my malloc/realloc). This question may have already been asked countless times. However, please consider the following:

1. The program's heap stores the size of an element
2. When you use realloc to allocate more memory, a new address may sometimes be issued in order to accommodate the space demand. In these cases, the existing data is copied to the new location, which seems to indicate that realloc can determine the allocated size

I realise that common reactions to such questions include "maintain the size with an int/long/struct" or "use a sentinel", but these approaches would be inappropriate for overloading the assignment constructor (if this was to be used in a class), or efficiency (as decoding escape characters can result in unnecessary overhead) respectively.
If there is a platform-dependent method that requires assembly, please tell me.
Thank you.

[JosAH]

Hello,
Is it possible to retrieve the size of allocated memory from a pointer (more specifically, a pointer returned my malloc/realloc). This question may have already been asked countless times. However, please consider the following:

1. The program's heap stores the size of an element
2. When you use realloc to allocate more memory, a new address may sometimes be issued in order to accommodate the space demand. In these cases, the existing data is copied to the new location, which seems to indicate that realloc can determine the allocated size

I realise that common reactions to such questions include "maintain the size with an int/long/struct" or "use a sentinel", but these approaches would be inappropriate for overloading the assignment constructor (if this was to be used in a class), or efficiency (as decoding escape characters can result in unnecessary overhead) respectively.
If there is a platform-dependent method that requires assembly, please tell me.
Thank you.

If you're talking about malloc, it oftenly stores the size of the allocated block somewhere
'before' the pointer it returns (read: at a lower address); this all happens in an
implementation dependent way. The C++ 'new' operator stores a bit more there,
e.g. a pointer to a class description.

kind regards,

Jos

[Hackles]

If you're talking about malloc, it oftenly stores the size of the allocated block somewhere
'before' the pointer it returns (read: at a lower address); this all happens in an
implementation dependent way. The C++ 'new' operator stores a bit more there,
e.g. a pointer to a class description.

kind regards,

Jos

Thank you for your response Jos,
Are there any functions defined in the standard C/C++ libraries that can be used to determine the size for either malloc or new?
Thank you,
Xiao

[JosAH]

Thank you for your response Jos,
Are there any functions defined in the standard C/C++ libraries that can be used to determine the size for either malloc or new?
Thank you,
Xiao

No there aren't. Every system is free to organize and keep track of allocated blocks
of memory and none of that is supposed to be exposed to the programmer, nor
defined in the Standards for those languages.

kind regards,

Jos

[smalpani]

No there aren't. Every system is free to organize and keep track of allocated blocks
of memory and none of that is supposed to be exposed to the programmer, nor
defined in the Standards for those languages.

kind regards,

Jos

I will xplain my understanding.

1. In one word the answer is "yes" on unix. That is you, as a application writer should be able to retrive the size of the memory allocated from the pointer. But there is more.

I will tell you how all this happens in linux, I have some intution on windows but not sure.

When you make a call to malloc, it allocates a block of memory from heap to your program. And it will keep some house keeping information ie the size of the block allocated to this pointer. Generally this information will be kept at a constant offset from the returned pointer.

Thus the answer is totally dependent on how your C/C++ runtime is implemented. If you know the offset you should be able to retrieve the information. But be very careful if you do anything fancy with the location then you will end up crashing.


Indepth, when a process is created OS (read kernel) will give it some memory as heap (how much is a result of negotiation between OS and the programmer)

so if you malloc a small amount of memory, then your process size neither grows nor shrinks. If you alloc large memory, then your C/C++ runtime will internally request the OS to increase the heap size. Programmer stays opaque to all this. You can also voluntary shrink your size but that generally does not happen. The system calls of importance on linux/unix is brk() and sbrk() calls.


Windows story (I am not pretty sure about it):

In windows malloc is just like a wrapper function that talks to OS (read kernel32.dll) every time it is called. So the house keeping is not kept by C/C++ runtime n windows but by the kernel32.dll. I have not got any idea if you be able to retrieve the information even if you knew how it is done on linux you might be restricted.


On other OS say some embedded OS.. It will be there own story. So you should not at all bank on it if you are writing portable code, rather you should keep your info in your structures.

[smalpani]

Just to add,

C++ standard container define a method capacity, though not related to your question you might want to know about them. If you know already please ignore

[Hackles]

Thank you smalpani for your explanation. It just seems surprising to me that the C/C++ specifications would miss such a useful function, considering the malloc/realloc specifications imply that size information would be stored. In any case, I will attempt to find a solution to my problem (I am developing for the Windows platform) and post back once successful.
Thank you all for your help,
Xiao

[Hackles]

I've got it!

[code=cpp]
unsigned char *test = malloc(42);
cout << "The size allocated is " << _msize(test) << "bytes!";
free(test);
[/code]

Obviously, this is Visual C++/Windows-specific, but still I felt I should tell you (source ).

Anyway, thank you to all that contributed!

Have fun!

[JosAH]

I've got it!

[code=cpp]
unsigned char *test = malloc(42);
cout << "The size allocated is " << _msize(test) << "bytes!";
free(test);
[/code]

Obviously, this is Visual C++/Windows-specific, but still I felt I should tell you (source ).

Anyway, thank you to all that contributed!

Have fun!

Please realize that _msize() is not part of the Standard and is highly implementation
defined. _msize() might even not be there on unix boxes.

kind regards,

Jos