Simple Single-Threaded Region Allocator...

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Chris Thomasson

    Simple Single-Threaded Region Allocator...

    Here is the code:


    Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.



    One advantage to using a region allocator is that you can usually skip most
    calls to free. Instead you can merge multiple free calls into a single reset
    call. Here is simple example:
    _______________ _______________ _______________ _______________ _______
    int main(void) {
    allocator this_allocator;

    /* create allocator instance with initial region size of 8192 and
    a low-watermark region count of 2
    */
    if (! allocator_creat e(&this_allocat or, 8192, 2)) {
    for (;;) {
    int i;
    for (i = 1; i < 1024; ++i) {
    allocator_reque st(&this_alloca tor, i);
    }
    allocator_reset (&this_allocato r);
    }
    allocator_destr oy(&this_alloca tor);
    }
    return 0;
    }
    _______________ _______________ _______________ _______________ _______




    The above program will never run out of memory... However, this algorihtm
    does not prohibit you from using free. For example:
    _______________ _______________ _______________ _______________ _______
    int main(void) {
    allocator this_allocator;
    if (! allocator_creat e(&this_allocat or, 8192, 2)) {
    for (;;) {
    int i;
    for (i = 1; i < 1024; ++i) {
    void* const ptr = allocator_reque st(&this_alloca tor, i);
    if (ptr) {
    allocator_relea se(&this_alloca tor, ptr);
    }
    }
    }
    allocator_destr oy(&this_alloca tor);
    }
    return 0;
    }
    _______________ _______________ _______________ _______________ _______



    Do you have any helpful comments/suggestions/critiques?


    Thanks.


    --
    Chris M. Thomasson


  • Chris Thomasson

    #2
    Re: Simple Single-Threaded Region Allocator...

    "Chris Thomasson" <cristom@comcas t.netwrote in message
    news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
    Here is the code:
    >
    >
    http://pastebin.com/m4a405e67
    [...]

    WHOOPS! I posted wrong crappy version!


    Here is the good one:


    Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.



    I am VERY sorry for that NON-SENSE!!!

    ;^(

    Comment

    • Chris Thomasson

      #3
      Re: Simple Single-Threaded Region Allocator...

      "Chris Thomasson" <cristom@comcas t.netwrote in message
      news:t7GdnTQrxb SL_4fVnZ2dnUVZ_ sGvnZ2d@comcast .com...
      "Chris Thomasson" <cristom@comcas t.netwrote in message
      news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
      >Here is the code:
      >>
      >>
      >http://pastebin.com/m4a405e67
      [...]
      >
      WHOOPS! I posted wrong crappy version!
      >
      >
      Here is the good one:
      >
      >
      http://pastebin.com/m31a4ad41
      ^^^^^^^^^^^^^^^ ^^
      has error; but still seems to compile! :^/


      Here is version that gets rid of error:

      typedef struct region_s region;

      typedef struct region_s {
      unsigned char* buf;
      size_t size;
      size_t offset;
      unsigned count;
      dlnode node;
      };


      The leading typedef on 'struct region_s' is bad. Fix:

      Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.


      It seems to compile successfully. Even on Comeau... ;^)



      I am VERY sorry for that NON-SENSE!!!
      >
      ;^(

      Comment

      • Chris Thomasson

        #4
        Re: Simple Single-Threaded Region Allocator...

        "Chris Thomasson" <cristom@comcas t.netwrote in message
        news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
        Here is the code:
        [...]

        Has anybody tried this allocator yet:

        Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.


        ?

        Comment

        • Dann Corbit

          #5
          Re: Simple Single-Threaded Region Allocator...

          "Chris Thomasson" <cristom@comcas t.netwrote in message
          news:R7idnRpzap M9UobVnZ2dnUVZ_ tGonZ2d@comcast .com...
          "Chris Thomasson" <cristom@comcas t.netwrote in message
          news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
          >Here is the code:
          [...]
          >
          Has anybody tried this allocator yet:
          >
          http://pastebin.com/m52ba914
          What does it do that other allocators fail to do?
          Is it faster?
          <OT>
          Is it robust and efficient under threading conditions?
          </OT>
          Does it track errors?

          I have efficient memory allocators and debugging memory allocators and
          clever sub-allocators etc. but I am always ready to try something new if I
          see a good reason for it.
          Does it have documentation?


          ** Posted from http://www.teranews.com **

          Comment

          • Chris Thomasson

            #6
            Re: Simple Single-Threaded Region Allocator...

            "Dann Corbit" <dcorbit@connx. comwrote in message
            news:1e977$481b f01e$21886@news .teranews.com.. .
            "Chris Thomasson" <cristom@comcas t.netwrote in message
            news:R7idnRpzap M9UobVnZ2dnUVZ_ tGonZ2d@comcast .com...
            >"Chris Thomasson" <cristom@comcas t.netwrote in message
            >news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
            >>Here is the code:
            >[...]
            >>
            >Has anybody tried this allocator yet:
            >>
            >http://pastebin.com/m52ba914
            >
            What does it do that other allocators fail to do?
            Nothing.

            Is it faster?
            Mabey.

            <OT>
            Is it robust and efficient under threading conditions?
            </OT>
            This C version is currently single-threaded, however, one could create an
            instance per-thread, however, you could not perform any remote
            deallocations. That is, thread_a cannot deallocate objects allocated by
            thread_b. You can look on 'comp.programmi ng.threads' for a crude
            multi-threaded region allocator prototype:





            Does it track errors?
            Na.



            I have efficient memory allocators and debugging memory allocators and
            clever sub-allocators etc. but I am always ready to try something new if I
            see a good reason for it.
            I am doing some experimentation with reducing the granularity of
            multi-threaded region allocators. I have a commercial "closed-source"
            distributed slab allocator shown here:


            (the first link has primitive C pseudo-code...)



            Does it have documentation?
            No. You can Goolge for other region-based memory allocation algorithms.

            Comment

            • Chris Thomasson

              #7
              Re: Simple Single-Threaded Region Allocator...


              "Dann Corbit" <dcorbit@connx. comwrote in message
              news:1e977$481b f01e$21886@news .teranews.com.. .
              "Chris Thomasson" <cristom@comcas t.netwrote in message
              news:R7idnRpzap M9UobVnZ2dnUVZ_ tGonZ2d@comcast .com...
              >"Chris Thomasson" <cristom@comcas t.netwrote in message
              >news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
              >>Here is the code:
              >[...]
              >>
              >Has anybody tried this allocator yet:
              >>
              >http://pastebin.com/m52ba914
              >
              What does it do that other allocators fail to do?
              [...]

              Well, it has a reset procedure. Most "traditiona l" allocators usually expect
              that any successful call to malloc will eventually result in a successful
              call to free. This is not true wrt region allocation in general. You can
              merge several calls to free into a single call to a reset procedure. So, in
              certain scenarios, region allocation can make reclaiming dynamic memory more
              "simplistic " indeed...

              Comment

              • user923005

                #8
                Re: Simple Single-Threaded Region Allocator...

                On May 4, 3:49 pm, "Chris Thomasson" <cris...@comcas t.netwrote:
                "Dann Corbit" <dcor...@connx. comwrote in message
                >
                news:1e977$481b f01e$21886@news .teranews.com.. ."Chris Thomasson" <cris...@comcas t.netwrote in message
                news:R7idnRpzap M9UobVnZ2dnUVZ_ tGonZ2d@comcast .com...
                "Chris Thomasson" <cris...@comcas t.netwrote in message
                >news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
                >Here is the code:
                [...]
                >
                Has anybody tried this allocator yet:
                >>
                What does it do that other allocators fail to do?
                >
                [...]
                >
                Well, it has a reset procedure. Most "traditiona l" allocators usually expect
                that any successful call to malloc will eventually result in a successful
                call to free. This is not true wrt region allocation in general. You can
                merge several calls to free into a single call to a reset procedure. So, in
                certain scenarios, region allocation can make reclaiming dynamic memory more
                "simplistic " indeed...
                It sounds like "partially manual GC" in that you can run a free() on
                command of whatever is outstanding. Is that right?

                Comment

                • Chris Thomasson

                  #9
                  Re: Simple Single-Threaded Region Allocator...

                  "user923005 " <dcorbit@connx. comwrote in message
                  news:f80e8289-3c5e-4b09-a490-4b78f4cee0ab@s5 0g2000hsb.googl egroups.com...
                  On May 4, 3:49 pm, "Chris Thomasson" <cris...@comcas t.netwrote:
                  "Dann Corbit" <dcor...@connx. comwrote in message

                  news:1e977$481b f01e$21886@news .teranews.com.. ."Chris Thomasson"
                  <cris...@comcas t.net wrote in message
                  >news:R7idnRpza pM9UobVnZ2dnUVZ _tGonZ2d@comcas t.com...
                  >"Chris Thomasson" <cris...@comcas t.netwrote in message
                  >>news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
                  >>Here is the code:
                  >[...]
                  >Has anybody tried this allocator yet:
                  What does it do that other allocators fail to do?
                  [...]

                  Well, it has a reset procedure. Most "traditiona l" allocators usually
                  expect
                  that any successful call to malloc will eventually result in a
                  successful
                  call to free. This is not true wrt region allocation in general. You can
                  merge several calls to free into a single call to a reset procedure. So,
                  in
                  certain scenarios, region allocation can make reclaiming dynamic memory
                  more
                  "simplistic " indeed...
                  It sounds like "partially manual GC" in that you can run a free() on
                  command of whatever is outstanding. Is that right?
                  Humm. I guess you could say that its roughly similar to a per-thread
                  semi-automatic GC. However, there are some caveats. It nice to keep in mind
                  that calling 'allocator_rese t()' on an allocator object completely
                  invalidates and reclaims _all_ of its outstanding allocations. You could
                  have multiple allocators per-thread, so the reset procedure can be made
                  "granular". ..

                  Comment

                  • Chris Thomasson

                    #10
                    Re: Simple Single-Threaded Region Allocator...

                    "Dann Corbit" <dcorbit@connx. comwrote in message
                    news:1e977$481b f01e$21886@news .teranews.com.. .
                    "Chris Thomasson" <cristom@comcas t.netwrote in message
                    news:R7idnRpzap M9UobVnZ2dnUVZ_ tGonZ2d@comcast .com...
                    >"Chris Thomasson" <cristom@comcas t.netwrote in message
                    >news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
                    >>Here is the code:
                    >[...]
                    >>
                    >Has anybody tried this allocator yet:
                    >>
                    >http://pastebin.com/m52ba914
                    >
                    What does it do that other allocators fail to do?
                    [...]

                    It can be based on local thread stack memory... Think:


                    void ThreadEntry() {
                    unsigned char ThreadLocalBuff er[32767];
                    {
                    DynamicRegionAl locator DRAlloc(ThreadL ocalBuffer);
                    pthread_setspec ific(..., &RAlloc);
                    {
                    [...];
                    }
                    pthread_setspec ific(..., NULL);
                    }
                    }


                    The initial buffer can be based on the calling threads stack...

                    Comment

                    • Chris Thomasson

                      #11
                      Re: Simple Single-Threaded Region Allocator...

                      "Chris Thomasson" <cristom@comcas t.netwrote in message
                      news:rd-dnabMvbeTlb3VnZ 2dnUVZ_gqdnZ2d@ comcast.com...
                      "Dann Corbit" <dcorbit@connx. comwrote in message
                      news:1e977$481b f01e$21886@news .teranews.com.. .
                      >"Chris Thomasson" <cristom@comcas t.netwrote in message
                      >news:R7idnRpza pM9UobVnZ2dnUVZ _tGonZ2d@comcas t.com...
                      >>"Chris Thomasson" <cristom@comcas t.netwrote in message
                      >>news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
                      >>>Here is the code:
                      >>[...]
                      >>>
                      >>Has anybody tried this allocator yet:
                      >>>
                      >>http://pastebin.com/m52ba914
                      >>
                      >What does it do that other allocators fail to do?
                      [...]
                      >
                      It can be based on local thread stack memory... Think:
                      >
                      >
                      void ThreadEntry() {
                      unsigned char ThreadLocalBuff er[32767];
                      {
                      DynamicRegionAl locator DRAlloc(ThreadL ocalBuffer);
                      pthread_setspec ific(..., &RAlloc);
                      {
                      [...];
                      }
                      pthread_setspec ific(..., NULL);
                      }
                      }
                      >
                      >
                      The initial buffer can be based on the calling threads stack...
                      Nothing new. I had to put vzoom on a Quadros OS and implement the global
                      heap out of Per-Quadros Task space... It scaled very well:



                      ;^)

                      Comment

                      • Chris Thomasson

                        #12
                        Re: Simple Single-Threaded Region Allocator...

                        "Chris Thomasson" <cristom@comcas t.netwrote in message
                        news:R7idnRpzap M9UobVnZ2dnUVZ_ tGonZ2d@comcast .com...
                        "Chris Thomasson" <cristom@comcas t.netwrote in message
                        news:aL-dnXc1kID9_IfVnZ 2dnUVZ_gudnZ2d@ comcast.com...
                        >Here is the code:
                        [...]
                        >
                        Has anybody tried this allocator yet:
                        >
                        http://pastebin.com/m52ba914
                        There can be a serious performance problem. It has to do with the
                        region_aligner union. The code is attempting to calculate a "pseudo" maximum
                        alignment value from sizeof(region_a ligner). Big problem... Here is how to
                        help alleviate some of the problem:



                        The max alignment can potentially be much larger than it needs to be; sorry
                        about that...

                        ;^(

                        Comment

                        Working...