In-place memory manager, mmap (was: Fastest way to store ints andfloats on disk)

[castironpi]

Hi,

I've got an "in-place" memory manager that uses a disk-backed memory-
mapped buffer. Among its possibilities are: storing variable-length
strings and structures for persistence and interprocess communication
with mmap.

It allocates segments of a generic buffer by length and returns an
offset to the reserved block, which can then be used with struct to
pack values to store. The data structure is adapted from the GNU PAVL
binary tree.

Allocated blocks can be cast to ctypes.Structur e instances using some
monkey patching, which is optional.

Want to open-source it. Any interest?

[Kris Kennaway]

Re: In-place memory manager, mmap (was: Fastest way to store intsand floats on disk)

castironpi wrote:

Hi,
>
I've got an "in-place" memory manager that uses a disk-backed memory-
mapped buffer. Among its possibilities are: storing variable-length
strings and structures for persistence and interprocess communication
with mmap.
>
It allocates segments of a generic buffer by length and returns an
offset to the reserved block, which can then be used with struct to
pack values to store. The data structure is adapted from the GNU PAVL
binary tree.
>
Allocated blocks can be cast to ctypes.Structur e instances using some
monkey patching, which is optional.
>
Want to open-source it. Any interest?

Just do it. That way users can come along later.

Kris

[castironpi]

Re: In-place memory manager, mmap (was: Fastest way to store ints andfloats on disk)

On Aug 24, 9:52 am, Kris Kennaway <k...@FreeBSD.o rgwrote:

castironpi wrote:

Hi,

>

I've got an "in-place" memory manager that uses a disk-backed memory-
mapped buffer.  Among its possibilities are: storing variable-length
strings and structures for persistence and interprocess communication
with mmap.

>

It allocates segments of a generic buffer by length and returns an
offset to the reserved block, which can then be used with struct to
pack values to store.  The data structure is adapted from the GNU PAVL
binary tree.

>

Allocated blocks can be cast to ctypes.Structur e instances using some
monkey patching, which is optional.

>

Want to open-source it.  Any interest?

>
Just do it.  That way users can come along later.
>
Kris

How? My website? Google Code? Too small for source forge, I think.

[Mohamed Yousef]

Re: In-place memory manager,

source-forge places no limits and is a very good place (specific
project site / releases / usage counters)

[Kris Kennaway]

Re: In-place memory manager, mmap

castironpi wrote:

On Aug 24, 9:52 am, Kris Kennaway <k...@FreeBSD.o rgwrote:

>castironpi wrote:

>>Hi,
>>I've got an "in-place" memory manager that uses a disk-backed memory-
>>mapped buffer. Among its possibilities are: storing variable-length
>>strings and structures for persistence and interprocess communication
>>with mmap.
>>It allocates segments of a generic buffer by length and returns an
>>offset to the reserved block, which can then be used with struct to
>>pack values to store. The data structure is adapted from the GNU PAVL
>>binary tree.
>>Allocated blocks can be cast to ctypes.Structur e instances using some
>>monkey patching, which is optional.
>>Want to open-source it. Any interest?

>Just do it. That way users can come along later.
>>
>Kris

>
How? My website? Google Code? Too small for source forge, I think.
--

http://mail.python.org/mailman/listinfo/python-list

>
>

Any of those 3 would work fine, but the last two are probably better
(sourceforge hosts plenty of tiny projects) if you don't want to have to
manage your server and related infrastructure yourself.

Kris

[castironpi]

Re: In-place memory manager, mmap

On Aug 24, 12:19 pm, Kris Kennaway <k...@FreeBSD.o rgwrote:

castironpi wrote:

On Aug 24, 9:52 am, Kris Kennaway <k...@FreeBSD.o rgwrote:

castironpi wrote:
>Hi,
>I've got an "in-place" memory manager that uses a disk-backed memory-
>mapped buffer.  Among its possibilities are: storing variable-length
>strings and structures for persistence and interprocess communication
>with mmap.
>It allocates segments of a generic buffer by length and returns an
>offset to the reserved block, which can then be used with struct to
>pack values to store.  The data structure is adapted from the GNU PAVL
>binary tree.
>Allocated blocks can be cast to ctypes.Structur e instances using some
>monkey patching, which is optional.
>Want to open-source it.  Any interest?
Just do it.  That way users can come along later.

>

Kris

>

How?  My website?  Google Code?  Too small for source forge, I think.
--
http://mail.python.org/mailman/listinfo/python-list

>
Any of those 3 would work fine, but the last two are probably better
(sourceforge hosts plenty of tiny projects) if you don't want to have to
manage your server and related infrastructure yourself.
>
Kris

I decided on Google Code. The demo creates 'mappedtree.dat ' at 3000
bytes, and allocates or frees memory blocks in it at random. There is
a insert-stress test, and a concurrent read-write test too. Tested on
WinXP with Python 2.5. Have a look.

http://code.google.com/p/pymmapstruct/source/browse/trunk/allocbuf.py