Sorting records using sort()

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Francis Glassborow

    #16
    Re: Sorting records using sort()

    In article <pan.2004.01.02 .12.56.21.62577 3@remove.this.p art.rtij.nl>,
    Martijn Lievaart <m@remove.this. part.rtij.nl> writes[color=blue]
    >On Fri, 02 Jan 2004 03:13:26 -0800, Elijah Bailey wrote:
    >
    >[ Please don't top post, thank you. M4 ]
    >
    >[ Crossposted to csc++, is this standard complient? ]
    >
    >[ Short description of the problem ]
    >
    >We have an array of n*m bytes. It holds n objects of size m. Both are only
    >known at runtime. For some strange reason we want to sort this using
    >std::sort. We know there are other solutions, the question is, can it be
    >done using std::sort.
    >
    >Constraints: Not clear, but memory seems to be an issue, creating an array
    >of pointers and sort that is out.[/color]

    But if n and m are only known at execution time the array must be
    created dynamically which immediately makes me think of using a vector.
    The following trivial program demonstrates that.

    #include <iostream>
    #include <vector>
    #include <algorithm>
    using namespace std;

    int main(){
    int n, m;
    cout << "N and m";
    cin >> n >> m;
    vector< vector<char> > a(n, m);
    for(int i(0); i != n; ++i){
    for(int j(0); j != m; ++j){
    a[i][j] = 'a' - i - j;
    }
    }

    for(int i(0); i != n; ++i){
    for(int j(0); j != m; ++j){
    cout << a[i][j] << " ";
    }
    cout << '\n';
    }
    sort(a.begin(), a.end());
    cout << "\n\n";
    for(int i(0); i != n; ++i){
    for(int j(0); j != m; ++j){
    cout << a[i][j] << " ";
    }
    cout << '\n';
    }
    }






    --
    Francis Glassborow ACCU
    Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
    or http://www.robinton.demon.co.uk
    Happy Xmas, Hanukkah, Yuletide, Winter/Summer Solstice to all.

    ---
    [ comp.std.c++ is moderated. To submit articles, try just posting with ]
    [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.e du ]
    [ --- Please see the FAQ before posting. --- ]
    [ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]

    Comment

    • Elijah Bailey

      #17
      Re: Sorting records using sort()

      m@remove.this.p art.rtij.nl (Martijn Lievaart) wrote in message news:<pan.2004. 01.02.12.56.21. 625773@remove.t his.part.rtij.n l>...[color=blue]
      > On Fri, 02 Jan 2004 03:13:26 -0800, Elijah Bailey wrote:
      >
      > [ Please don't top post, thank you. M4 ]
      >
      > [ Crossposted to csc++, is this standard complient? ]
      >
      > [ Short description of the problem ]
      >
      > We have an array of n*m bytes. It holds n objects of size m. Both are only
      > known at runtime. For some strange reason we want to sort this using
      > std::sort. We know there are other solutions, the question is, can it be
      > done using std::sort.
      >[/color]

      Thanks Martijn. This is a much better description than i came up with.
      [color=blue]
      > Constraints: Not clear, but memory seems to be an issue, creating an array
      > of pointers and sort that is out.[/color]

      Constraints: O(n) space is not allowed. So you can not create a
      pointer
      to each object and sort them. O(log(n)) space is perfectly ok.
      O(log(n)*m) is also allowed if that simplifies things.

      My data has n >> m, i.e. n is much greater than m.
      [color=blue]
      >[color=green]
      > > "Ron Natalie" <ron@sensor.com > wrote in message
      > > news:<3ff4297b$ 0$31857$9a6e19e a@news.newshost ing.com>...[color=darkred]
      > >> "Jeff Schwab" <jeffplus@comca st.net> wrote in message
      > >> news:WoidnaAaas 0i8mmiRVn-hA@comcast.com. ..
      > >>
      > >> > Writing your own iterators would not be trivial, but probably would
      > >> > be a educational. Having such iterators available would make your
      > >> > data structure compatible with most of the standard library, and
      > >> > might avoid countless future headaches.
      > >>
      > >> The trouble with writing iterators for this problem is that he needs it
      > >> to be variable size at run time and that the iterators have to be
      > >> derferenceable and the derefenced types assignable. Add his
      > >> requirement to not allocate even enough memory to hold pointers to each
      > >> object, I can't figure out how you could do it.[/color]
      > >
      > > Thanks Ron, I think that is definitely a problem. But I still "think" it
      > > can be done, dont know how to thou...:)[/color]
      >
      > I also think it can be done. However, it requires so much trickery that it
      > is not interesting to do so. Any of the other options quickly become very
      > appealing. My approach uses some dynamic memory, but probably less than an
      > array of pointers would take. The actual amount depends on the number of
      > copies std::sort makes internally, there is no way around this. I guess
      > for a typical implementation this would be around log(n) copies, unless
      > the implementation takes special care to dispose of temporaries. You
      > cannot count on any of this though.
      >[/color]

      Hopefully, by the end of this thread, we'll have something simple
      enough
      to implement...:)
      [color=blue]
      > OK, how can we do it? Obviously every iterator must know the size of the
      > object it is supposed to handle. The main problems we are facing:
      >
      > - We cannot create real objects (directly), the size of the real object is
      > only known at runtime.
      > - std::sort is allowed to (and most frequently does) make extra copies of
      > objects.
      >
      > Asusmptions:
      > - There are functions to copy these objects and to compare these objects.
      > - The objects to be sorted don't need special construction or destruction
      > and can be copied by the previously mentioned routine. They can be
      > constructed by copying into a new memory area.
      > - std::sort never uses pointers to objects, only iterators. I think the
      > standard mandates this, but couldn't find it.
      >[/color]

      Would be interesting to know, what std::sort() is required to have
      from
      the standard. I think you are perfectly right that sort doesnt use
      pointers
      to the objects, only iterators and derefernces(*) the iterators...
      [color=blue]
      > We create a proxy class and let the iterator return proxy objects. These
      > proxy objects store a pointer to some master descriptor that stores the
      > location and size of the array and the size of the individual objects. We
      > could store this in the proxy object itself, but that seems wasteful.
      > Also, the proxy object stores a void* to store the data for the real
      > object.
      >
      > This proxy class is what our iterators value_type is.
      >
      > Now the proxy object implements some intelligent constructors. I doubt we
      > need a default constructor, the value type needs to be assignable only
      > (this follows from the iterator requirements). So we create a constructor
      > that is used only inside the iterator, it sets the data pointer to point
      > inside the array at the correct location. If the copy constructor is
      > called (which would be by std::sort) we dynamically allocate memory to
      > hold the object and use the copying routine to fill it.
      >
      > The assignment operator must use the copying routine if the destination is
      > inside the array. If not, it can either use copy-construct-and-swap or the
      > copying routine.
      >
      > The destructor would check if the pointer points inside the array and if
      > not deletes the object.
      >
      > Obviously, we need to define an operator< for the proxy objects, this
      > calls the comparison function above. If we template the proxy objects, we
      > can use a functor for the comparison, potentially inlining the comaparison
      > (this would be the only reason to use std::sort anyhow, so it makes
      > sense).
      >
      > Looking at the standard, this would fullfill all requirements for
      > std::sort I could find. Somehow I think I did overlook something here, so
      > scrutiny by others is appreciated. I also left obvious details out,
      > obviously :-)
      >
      > A note on exceptions. This would give the basic guarentee, if the copying
      > and comparison routines give at least the basic guarentee.
      >
      > A note on memory usage. Assuming that std::sort holds log(n) copies of
      > objects internally, this aproach uses less memory than the
      > sort-array-of-pointers aproach if log(n)*m < n*sizeof(void*) . So this
      > greatly depends on these variables, but quickly holds for large n and gets
      > less interesting for large m. Assume sizeof(void*)== 4, we get m <
      > 4n/log(n). For n=100, m should be less than 60. For n=1000, m should be
      > less than 400. These are gross approximations but should give you a feel.
      >[/color]

      Usually this is certainly the case for my data.
      [color=blue]
      > I hope anyone sees that you must have good reasons to do this. It is ugly,
      > errorprone, prone to assumptions your implementation of std::sort makes
      > (in this implementation &*it does not return a pointer to the real object,
      > any pointer arithmetic will fail), difficult to maintain. There is no real
      > reason why you should not call qsort in the first place, except as an
      > accademic exercise.
      >[/color]

      First problem to qsort: It's slower than using sort()
      Second problem to qsort: Later I might want to use other STL
      algorithms on this data!
      like for_each()? find()? ...?

      Thanks a lot for your comments,
      --Elijah

      ---
      [ comp.std.c++ is moderated. To submit articles, try just posting with ]
      [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.e du ]
      [ --- Please see the FAQ before posting. --- ]
      [ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]

      Comment

      • Martijn Lievaart

        #18
        Re: Sorting records using sort()

        On Sun, 04 Jan 2004 04:15:33 +0000, Elijah Bailey wrote:

        [ csc++ removed, not relevant for what I want to say ]
        [color=blue]
        > First problem to qsort: It's slower than using sort()[/color]

        Qsort is slower only if std::sort can inline the comparison. In this case
        this seems to me that with all the other overhead qsort will actually be
        faster (let alone easier).

        M4

        Comment

        • Elijah Bailey

          #19
          Re: Sorting records using sort()

          Martijn Lievaart <m@remove.this. part.rtij.nl> wrote in message news:<pan.2004. 01.04.18.19.43. 722196@remove.t his.part.rtij.n l>...[color=blue]
          > On Sun, 04 Jan 2004 04:15:33 +0000, Elijah Bailey wrote:
          >
          > [ csc++ removed, not relevant for what I want to say ]
          >[color=green]
          > > First problem to qsort: It's slower than using sort()[/color]
          >
          > Qsort is slower only if std::sort can inline the comparison. In this case
          > this seems to me that with all the other overhead qsort will actually be
          > faster (let alone easier).
          >
          > M4[/color]

          Yes, IF there is no 'simple' way to make std::sort() work.

          And IF that is the case, then shouldnt there be another version
          of std::sort() to do what I want to do. Doesnt it seem natural to
          have an interface in c++ that does the same thing that the c
          function qsort CAN do, without actually going the "c" way??

          Thanks,
          --Elijah

          [ See http://www.gotw.ca/resources/clcm.htm for info about ]
          [ comp.lang.c++.m oderated. First time posters: Do this! ]

          Comment

          • david m-

            #20
            Re: Sorting records using sort()

            Sounds like some sort of intermediate class is required in order to spoof a
            RandomAccessIte rator as required by the STL sort() function.



            i.e. you can create these to adapt the dynamic size of the elements to be
            sorted to the static requirements of an iterator.



            Though whether this is technically possible I'm not sure.



            david m.



            "Elijah Bailey" <geomrock@hotma il.com> wrote in message
            news:e008fef8.0 401031936.312ff a9a@posting.goo gle.com...[color=blue]
            > m@remove.this.p art.rtij.nl (Martijn Lievaart) wrote in message[/color]
            news:<pan.2004. 01.02.12.56.21. 625773@remove.t his.part.rtij.n l>...[color=blue][color=green]
            > > On Fri, 02 Jan 2004 03:13:26 -0800, Elijah Bailey wrote:
            > >
            > > [ Please don't top post, thank you. M4 ]
            > >
            > > [ Crossposted to csc++, is this standard complient? ]
            > >
            > > [ Short description of the problem ]
            > >
            > > We have an array of n*m bytes. It holds n objects of size m. Both are[/color][/color]
            only[color=blue][color=green]
            > > known at runtime. For some strange reason we want to sort this using
            > > std::sort. We know there are other solutions, the question is, can it be
            > > done using std::sort.
            > >[/color]
            >
            > Thanks Martijn. This is a much better description than i came up with.
            >[color=green]
            > > Constraints: Not clear, but memory seems to be an issue, creating an[/color][/color]
            array[color=blue][color=green]
            > > of pointers and sort that is out.[/color]
            >
            > Constraints: O(n) space is not allowed. So you can not create a
            > pointer
            > to each object and sort them. O(log(n)) space is perfectly ok.
            > O(log(n)*m) is also allowed if that simplifies things.
            >
            > My data has n >> m, i.e. n is much greater than m.
            >[color=green]
            > >[color=darkred]
            > > > "Ron Natalie" <ron@sensor.com > wrote in message
            > > > news:<3ff4297b$ 0$31857$9a6e19e a@news.newshost ing.com>...
            > > >> "Jeff Schwab" <jeffplus@comca st.net> wrote in message
            > > >> news:WoidnaAaas 0i8mmiRVn-hA@comcast.com. ..
            > > >>
            > > >> > Writing your own iterators would not be trivial, but probably would
            > > >> > be a educational. Having such iterators available would make your
            > > >> > data structure compatible with most of the standard library, and
            > > >> > might avoid countless future headaches.
            > > >>
            > > >> The trouble with writing iterators for this problem is that he needs[/color][/color][/color]
            it[color=blue][color=green][color=darkred]
            > > >> to be variable size at run time and that the iterators have to be
            > > >> derferenceable and the derefenced types assignable. Add his
            > > >> requirement to not allocate even enough memory to hold pointers to[/color][/color][/color]
            each[color=blue][color=green][color=darkred]
            > > >> object, I can't figure out how you could do it.
            > > >
            > > > Thanks Ron, I think that is definitely a problem. But I still "think"[/color][/color][/color]
            it[color=blue][color=green][color=darkred]
            > > > can be done, dont know how to thou...:)[/color]
            > >
            > > I also think it can be done. However, it requires so much trickery that[/color][/color]
            it[color=blue][color=green]
            > > is not interesting to do so. Any of the other options quickly become[/color][/color]
            very[color=blue][color=green]
            > > appealing. My approach uses some dynamic memory, but probably less than[/color][/color]
            an[color=blue][color=green]
            > > array of pointers would take. The actual amount depends on the number of
            > > copies std::sort makes internally, there is no way around this. I guess
            > > for a typical implementation this would be around log(n) copies, unless
            > > the implementation takes special care to dispose of temporaries. You
            > > cannot count on any of this though.
            > >[/color]
            >
            > Hopefully, by the end of this thread, we'll have something simple
            > enough
            > to implement...:)
            >[color=green]
            > > OK, how can we do it? Obviously every iterator must know the size of the
            > > object it is supposed to handle. The main problems we are facing:
            > >
            > > - We cannot create real objects (directly), the size of the real object[/color][/color]
            is[color=blue][color=green]
            > > only known at runtime.
            > > - std::sort is allowed to (and most frequently does) make extra copies[/color][/color]
            of[color=blue][color=green]
            > > objects.
            > >
            > > Asusmptions:
            > > - There are functions to copy these objects and to compare these[/color][/color]
            objects.[color=blue][color=green]
            > > - The objects to be sorted don't need special construction or[/color][/color]
            destruction[color=blue][color=green]
            > > and can be copied by the previously mentioned routine. They can be
            > > constructed by copying into a new memory area.
            > > - std::sort never uses pointers to objects, only iterators. I think the
            > > standard mandates this, but couldn't find it.
            > >[/color]
            >
            > Would be interesting to know, what std::sort() is required to have
            > from
            > the standard. I think you are perfectly right that sort doesnt use
            > pointers
            > to the objects, only iterators and derefernces(*) the iterators...
            >[color=green]
            > > We create a proxy class and let the iterator return proxy objects. These
            > > proxy objects store a pointer to some master descriptor that stores the
            > > location and size of the array and the size of the individual objects.[/color][/color]
            We[color=blue][color=green]
            > > could store this in the proxy object itself, but that seems wasteful.
            > > Also, the proxy object stores a void* to store the data for the real
            > > object.
            > >
            > > This proxy class is what our iterators value_type is.
            > >
            > > Now the proxy object implements some intelligent constructors. I doubt[/color][/color]
            we[color=blue][color=green]
            > > need a default constructor, the value type needs to be assignable only
            > > (this follows from the iterator requirements). So we create a[/color][/color]
            constructor[color=blue][color=green]
            > > that is used only inside the iterator, it sets the data pointer to point
            > > inside the array at the correct location. If the copy constructor is
            > > called (which would be by std::sort) we dynamically allocate memory to
            > > hold the object and use the copying routine to fill it.
            > >
            > > The assignment operator must use the copying routine if the destination[/color][/color]
            is[color=blue][color=green]
            > > inside the array. If not, it can either use copy-construct-and-swap or[/color][/color]
            the[color=blue][color=green]
            > > copying routine.
            > >
            > > The destructor would check if the pointer points inside the array and if
            > > not deletes the object.
            > >
            > > Obviously, we need to define an operator< for the proxy objects, this
            > > calls the comparison function above. If we template the proxy objects,[/color][/color]
            we[color=blue][color=green]
            > > can use a functor for the comparison, potentially inlining the[/color][/color]
            comaparison[color=blue][color=green]
            > > (this would be the only reason to use std::sort anyhow, so it makes
            > > sense).
            > >
            > > Looking at the standard, this would fullfill all requirements for
            > > std::sort I could find. Somehow I think I did overlook something here,[/color][/color]
            so[color=blue][color=green]
            > > scrutiny by others is appreciated. I also left obvious details out,
            > > obviously :-)
            > >
            > > A note on exceptions. This would give the basic guarentee, if the[/color][/color]
            copying[color=blue][color=green]
            > > and comparison routines give at least the basic guarentee.
            > >
            > > A note on memory usage. Assuming that std::sort holds log(n) copies of
            > > objects internally, this aproach uses less memory than the
            > > sort-array-of-pointers aproach if log(n)*m < n*sizeof(void*) . So this
            > > greatly depends on these variables, but quickly holds for large n and[/color][/color]
            gets[color=blue][color=green]
            > > less interesting for large m. Assume sizeof(void*)== 4, we get m <
            > > 4n/log(n). For n=100, m should be less than 60. For n=1000, m should be
            > > less than 400. These are gross approximations but should give you a[/color][/color]
            feel.[color=blue][color=green]
            > >[/color]
            >
            > Usually this is certainly the case for my data.
            >[color=green]
            > > I hope anyone sees that you must have good reasons to do this. It is[/color][/color]
            ugly,[color=blue][color=green]
            > > errorprone, prone to assumptions your implementation of std::sort makes
            > > (in this implementation &*it does not return a pointer to the real[/color][/color]
            object,[color=blue][color=green]
            > > any pointer arithmetic will fail), difficult to maintain. There is no[/color][/color]
            real[color=blue][color=green]
            > > reason why you should not call qsort in the first place, except as an
            > > accademic exercise.
            > >[/color]
            >
            > First problem to qsort: It's slower than using sort()
            > Second problem to qsort: Later I might want to use other STL
            > algorithms on this data!
            > like for_each()? find()? ...?
            >
            > Thanks a lot for your comments,
            > --Elijah
            >
            > ---
            > [ comp.std.c++ is moderated. To submit articles, try just posting with ]
            > [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.e du ]
            > [ --- Please see the FAQ before posting. --- ]
            > [ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]
            >[/color]

            ---
            [ comp.std.c++ is moderated. To submit articles, try just posting with ]
            [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.e du ]
            [ --- Please see the FAQ before posting. --- ]
            [ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]

            Comment

            • Martijn Lievaart

              #21
              Re: Sorting records using sort()

              On Sun, 04 Jan 2004 18:42:58 -0500, Elijah Bailey wrote:
              [color=blue]
              > Martijn Lievaart <m@remove.this. part.rtij.nl> wrote in message news:<pan.2004. 01.04.18.19.43. 722196@remove.t his.part.rtij.n l>...[color=green]
              >> On Sun, 04 Jan 2004 04:15:33 +0000, Elijah Bailey wrote:[color=darkred]
              >> > First problem to qsort: It's slower than using sort()[/color]
              >>
              >> Qsort is slower only if std::sort can inline the comparison. In this case
              >> this seems to me that with all the other overhead qsort will actually be
              >> faster (let alone easier).
              >>
              >> M4[/color]
              >
              > Yes, IF there is no 'simple' way to make std::sort() work.
              >
              > And IF that is the case, then shouldnt there be another version
              > of std::sort() to do what I want to do. Doesnt it seem natural to
              > have an interface in c++ that does the same thing that the c
              > function qsort CAN do, without actually going the "c" way??[/color]

              I don't think so. Qsort is not type safe, there is no way to do so in C.
              Std::sort improves on this, but you don't have a type known compile time.
              Your problem is exotic in my eyes, I don't think std::sort should take it
              into account.

              HTH,
              M4


              [ See http://www.gotw.ca/resources/clcm.htm for info about ]
              [ comp.lang.c++.m oderated. First time posters: Do this! ]

              Comment

              • P.J. Plauger

                #22
                Re: Sorting records using sort()

                "Elijah Bailey" <geomrock@hotma il.com> wrote in message
                news:e008fef8.0 401041455.19aca fb2@posting.goo gle.com...
                [color=blue]
                > And IF that is the case, then shouldnt there be another version
                > of std::sort() to do what I want to do. Doesnt it seem natural to
                > have an interface in c++ that does the same thing that the c
                > function qsort CAN do, without actually going the "c" way??[/color]

                Yes, it's called qsort. Why is it so hard for you to accept that
                it's a part of Standard C++, not to mention the best fit to the
                problem you describe?

                P.J. Plauger
                Dinkumware, Ltd.



                [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                [ comp.lang.c++.m oderated. First time posters: Do this! ]

                Comment

                • Francis Glassborow

                  #23
                  Re: Sorting records using sort()

                  In message <e008fef8.04010 41455.19acafb2@ posting.google. com>, Elijah
                  Bailey <geomrock@hotma il.com> writes[color=blue]
                  >Martijn Lievaart <m@remove.this. part.rtij.nl> wrote in message news:<pan.2004. 01.04.18.19.43. 722196@remove.t his.part.rtij.n l>...[color=green]
                  >> On Sun, 04 Jan 2004 04:15:33 +0000, Elijah Bailey wrote:
                  >>
                  >> [ csc++ removed, not relevant for what I want to say ]
                  >>[color=darkred]
                  >> > First problem to qsort: It's slower than using sort()[/color]
                  >>
                  >> Qsort is slower only if std::sort can inline the comparison. In this case
                  >> this seems to me that with all the other overhead qsort will actually be
                  >> faster (let alone easier).
                  >>
                  >> M4[/color]
                  >
                  >Yes, IF there is no 'simple' way to make std::sort() work.
                  >
                  >And IF that is the case, then shouldnt there be another version
                  >of std::sort() to do what I want to do. Doesnt it seem natural to
                  >have an interface in c++ that does the same thing that the c
                  >function qsort CAN do, without actually going the "c" way??[/color]

                  Do you think that the OP's requirements are particularly common? Rightly
                  or wrongly, I do not. We have a lot of work to do on the Standard C++
                  Library without spending time on something such as this. If someone
                  really needs it they can write it for themselves (std::sort() does not
                  rely on some special magic) If they think it likely to be of more
                  general use they can submit their work either to Boost or directly to
                  the Library Working Group of WG21.

                  Actually we have a perfectly good interface that does what C's qsort
                  does, it is called qsort. What you are asking for is something else
                  which seems to be of very limited utility.


                  --
                  Francis Glassborow ACCU
                  Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
                  or http://www.robinton.demon.co.uk
                  Happy Xmas, Hanukkah, Yuletide, Winter/Summer Solstice to all.


                  [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                  [ comp.lang.c++.m oderated. First time posters: Do this! ]

                  Comment

                  • kanze@gabi-soft.fr

                    #24
                    Re: Sorting records using sort()

                    geomrock@hotmai l.com (Elijah Bailey) wrote in message
                    news:<e008fef8. 0401041455.19ac afb2@posting.go ogle.com>...[color=blue]
                    > Martijn Lievaart <m@remove.this. part.rtij.nl> wrote in message
                    > news:<pan.2004. 01.04.18.19.43. 722196@remove.t his.part.rtij.n l>...[color=green]
                    > > On Sun, 04 Jan 2004 04:15:33 +0000, Elijah Bailey wrote:[/color][/color]
                    [color=blue][color=green][color=darkred]
                    > > > First problem to qsort: It's slower than using sort()[/color][/color][/color]
                    [color=blue][color=green]
                    > > Qsort is slower only if std::sort can inline the comparison. In this
                    > > case this seems to me that with all the other overhead qsort will
                    > > actually be faster (let alone easier).[/color][/color]
                    [color=blue]
                    > Yes, IF there is no 'simple' way to make std::sort() work.[/color]

                    I've not followed the discussions completely, but if I understand
                    correctly, you want to invoke *one* sort function for different types of
                    data. (This isn't the way you present it, but it comes out to the same
                    thing. You have data in raw memory, represented by a char[]; sometimes,
                    it is one type, with one size, and sometimes it is another type, with
                    another size.)

                    This is not the way std::sort works, or for that matter, not the way the
                    standard library in general works. The whole point of templating is
                    that you have a separate instance of the function for each type of data.
                    [color=blue]
                    > And IF that is the case, then shouldnt there be another version of
                    > std::sort() to do what I want to do. Doesnt it seem natural to have
                    > an interface in c++ that does the same thing that the c function qsort
                    > CAN do, without actually going the "c" way??[/color]

                    Well, the interface in C++ would be pretty close to the interface in C
                    anyway -- about the only difference might be to allow a functional
                    object to be used instead of only a pointer to a function.

                    --
                    James Kanze GABI Software mailto:kanze@ga bi-soft.fr
                    Conseils en informatique orientée objet/ http://www.gabi-soft.fr
                    Beratung in objektorientier ter Datenverarbeitu ng
                    11 rue de Rambouillet, 78460 Chevreuse, France, +33 (0)1 30 23 45 16

                    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                    [ comp.lang.c++.m oderated. First time posters: Do this! ]

                    Comment

                    • Elijah Bailey

                      #25
                      Re: Sorting records using sort()

                      "P.J. Plauger" <pjp@dinkumware .com> wrote in message
                      news:<3N2Kb.490 28$E17.21339@nw rddc02.gnilink. net>...[color=blue]
                      > "Elijah Bailey" <geomrock@hotma il.com> wrote in message
                      > news:e008fef8.0 401041455.19aca fb2@posting.goo gle.com...
                      >[color=green]
                      > > And IF that is the case, then shouldnt there be another version
                      > > of std::sort() to do what I want to do. Doesnt it seem natural to
                      > > have an interface in c++ that does the same thing that the c
                      > > function qsort CAN do, without actually going the "c" way??[/color]
                      >
                      > Yes, it's called qsort. Why is it so hard for you to accept that
                      > it's a part of Standard C++, not to mention the best fit to the
                      > problem you describe?
                      >[/color]

                      Here's the real problem that I need it for (Which incidentally I never
                      mentioned)
                      Consider a database of fixed record size (Because doing variable record size
                      is already pain, even if you had everything in memory). Now if you had
                      a "STL" way of doing this, then you could
                      run all "STL" functions on this database compared to reading the file, doing
                      the same thing and writing it. (Is that faster? Yes, indeed it is. Benchmark
                      an mmap compared to fopen/read on ur machine, and u wud hopefully agree)

                      Yes, its true that maybe STL was only written for In-Memory data,
                      and STL implementations did not care about out of memory data.
                      But I believe that both for inmemory and outmemory data, now or
                      later, locality will be imporatant. And if STL takes into account that
                      fact, then not only could we use it for data in memory, but also on
                      data that resides on disk.

                      I dont want the Standard people plagued with the burden of doing it
                      for just my case. But I believe that handling fixed size record files is
                      fairly common, if not as common as using sort() for instance...

                      And anyway, I have an implementation right now which uses qsort
                      to do it. I just was amazed that it was so tough to modify sort() to
                      do the same thing...

                      Thanks for your time and comments,
                      --Elijah

                      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                      [ comp.lang.c++.m oderated. First time posters: Do this! ]

                      Comment

                      • kanze@gabi-soft.fr

                        #26
                        Re: Sorting records using sort()

                        geomrock@hotmai l.com (Elijah Bailey) wrote in message
                        news:<e008fef8. 0401051505.654c a42d@posting.go ogle.com>...[color=blue]
                        > "P.J. Plauger" <pjp@dinkumware .com> wrote in message
                        > news:<3N2Kb.490 28$E17.21339@nw rddc02.gnilink. net>...[color=green]
                        > > "Elijah Bailey" <geomrock@hotma il.com> wrote in message
                        > > news:e008fef8.0 401041455.19aca fb2@posting.goo gle.com...[/color][/color]
                        [color=blue][color=green][color=darkred]
                        > > > And IF that is the case, then shouldnt there be another version
                        > > > of std::sort() to do what I want to do. Doesnt it seem natural to
                        > > > have an interface in c++ that does the same thing that the c
                        > > > function qsort CAN do, without actually going the "c" way??[/color][/color][/color]
                        [color=blue][color=green]
                        > > Yes, it's called qsort. Why is it so hard for you to accept that
                        > > it's a part of Standard C++, not to mention the best fit to the
                        > > problem you describe?[/color][/color]
                        [color=blue]
                        > Here's the real problem that I need it for (Which incidentally I never
                        > mentioned) Consider a database of fixed record size (Because doing
                        > variable record size is already pain, even if you had everything in
                        > memory). Now if you had a "STL" way of doing this, then you could run
                        > all "STL" functions on this database compared to reading the file,
                        > doing the same thing and writing it. (Is that faster? Yes, indeed it
                        > is. Benchmark an mmap compared to fopen/read on ur machine, and u wud
                        > hopefully agree)[/color]

                        Do you know the structure of this data at compile time? If so, there is
                        no problem creating the necessary PODs and iterators, and mapping them
                        to the data. Theoretically, you can't guarantee the lack of padding,
                        but it is a technique I've used once or twice, it generally works, and
                        if you are using mmap, you aren't very portable anyway.

                        My understanding was that the size and structure of the elements was not
                        known until runtime. If this is the case, I really don't see any
                        general answer -- the STL is based on compile time type resolution.

                        --
                        James Kanze GABI Software mailto:kanze@ga bi-soft.fr
                        Conseils en informatique orientée objet/ http://www.gabi-soft.fr
                        Beratung in objektorientier ter Datenverarbeitu ng
                        11 rue de Rambouillet, 78460 Chevreuse, France, +33 (0)1 30 23 45 16

                        [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                        [ comp.lang.c++.m oderated. First time posters: Do this! ]

                        Comment

                        • kanze@gabi-soft.fr

                          #27
                          Re: Sorting records using sort()

                          Francis Glassborow <francis@robint on.demon.co.uk> wrote in message
                          news:<DmBRfmCZy K+$EwWS@robinto n.demon.co.uk>. ..[color=blue]
                          > In message <e008fef8.04010 41455.19acafb2@ posting.google. com>, Elijah
                          > Bailey <geomrock@hotma il.com> writes[color=green]
                          > >Martijn Lievaart <m@remove.this. part.rtij.nl> wrote in message
                          > >news:<pan.2004 .01.04.18.19.43 .722196@remove. this.part.rtij. nl>...[color=darkred]
                          > >> On Sun, 04 Jan 2004 04:15:33 +0000, Elijah Bailey wrote:[/color][/color][/color]
                          [color=blue][color=green][color=darkred]
                          > >> > First problem to qsort: It's slower than using sort()[/color][/color][/color]
                          [color=blue][color=green][color=darkred]
                          > >> Qsort is slower only if std::sort can inline the comparison. In
                          > >> this case this seems to me that with all the other overhead qsort
                          > >> will actually be faster (let alone easier).[/color][/color][/color]
                          [color=blue][color=green]
                          > >Yes, IF there is no 'simple' way to make std::sort() work.[/color][/color]
                          [color=blue][color=green]
                          > >And IF that is the case, then shouldnt there be another version of
                          > >std::sort() to do what I want to do. Doesnt it seem natural to have
                          > >an interface in c++ that does the same thing that the c function
                          > >qsort CAN do, without actually going the "c" way??[/color][/color]
                          [color=blue]
                          > Do you think that the OP's requirements are particularly common?[/color]

                          Yes and no. For the moment, I'm not completely sure what his
                          requirements are.

                          There are certainly various reasons why one might want to overlay a data
                          structure over pure memory. To date, I've had very good success in
                          doing this, provided that the struct was a POD type, and that all
                          alignment requirements were met in the actual data set. (Most of the
                          time, the struct will only contain arrays of char, so there are no
                          alignment requirements.)

                          Technically, the standard doesn't make any guarantees, but in practice,
                          I've had no trouble with doing things like:

                          struct Record
                          {
                          char state ;
                          char id[ 20 ] ;
                          char value[ 11 ] ;
                          // ...
                          } ;

                          std::vector< char > buffer ;
                          buffer.resize( recordCount * sizeof( Record ) ) ;
                          Record* record = reinterpret_cas t< Record* >( &buffer[ 0 ] ) ;

                          // ...

                          char* buffer = mmap( ... ) ;
                          Record* record = reinterpret_cas t< Record* >( buffer ) ;

                          etc. Obviously, if this works, it is trivial to create an STL
                          compatible iterator for the buffer, and use it with the STL algorithms.

                          As I said, this is useful, and it is not guaranteed by the standard. On
                          the other hand, I'm not convinced that it is worth the effort that might
                          be necessary -- while the intent seems rather straightforward , I'd hate
                          to have to come up with the words to express it rigorously without
                          limiting implementations too much otherwise.

                          --
                          James Kanze GABI Software mailto:kanze@ga bi-soft.fr
                          Conseils en informatique orientée objet/ http://www.gabi-soft.fr
                          Beratung in objektorientier ter Datenverarbeitu ng
                          11 rue de Rambouillet, 78460 Chevreuse, France, +33 (0)1 30 23 45 16

                          [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                          [ comp.lang.c++.m oderated. First time posters: Do this! ]

                          Comment

                          • Jeff Schwab

                            #28
                            Re: Sorting records using sort()

                            Elijah Bailey wrote:[color=blue]
                            > I want to sort a set of records using STL's sort() function,
                            > but dont see an easy way to do it.
                            >
                            > I have a
                            >
                            > char *data;
                            >
                            >
                            > which has size mn bytes where m is size of the record and
                            > n is the number of records. Both these numbers are known
                            > only dynamically. I have a function less_than that can compare
                            > two records of size m given the pointers to the two records.
                            >
                            > Is there an easy way to call STL sort() on this data and sort it.
                            > The data is big and I do NOT want to allocate a list of pointers
                            > of size n or anything linear in size. Assume that except the data,
                            > we do not have much space...
                            >
                            > I thought of tricking sort() using a dummy Record class that is
                            > templated using the size of the record...But since m can change
                            > dynamically this doesnt work.
                            >
                            >
                            > Thanks in advance for your comments,
                            > --Elijah[/color]




                            Comment

                            • Elijah Bailey

                              #29
                              Re: Sorting records using sort()

                              >[color=blue]
                              > http://home.comcast.net/~jeffrey.schwab/code/records/[/color]

                              I tried compiling it with g++ 2.96 and 3.1...both dont compile the code...
                              Thanks for the code thou...wud be nice if I could test it on my linux box.
                              --Elijah

                              Comment

                              • Stephen Howe

                                #30
                                Re: Sorting records using sort()

                                > Here's the real problem that I need it for (Which incidentally I never[color=blue]
                                > mentioned)[/color]

                                You should have mentioned it. Nobody likes putting sweat and effort into
                                solving artificial problems and later being confronted with the "real"
                                problem.
                                [color=blue]
                                > I dont want the Standard people plagued with the burden of doing it
                                > for just my case. But I believe that handling fixed size record files is
                                > fairly common, if not as common as using sort() for instance...
                                >
                                > And anyway, I have an implementation right now which uses qsort
                                > to do it. I just was amazed that it was so tough to modify sort() to
                                > do the same thing...[/color]

                                It is tough. sort() is templatised which means it _has_ to determine the
                                size of its elements at compile-time rather than run-time.
                                And there is no way that is likely to ever change.
                                Anyone who proposes adding additional dynamic run-time ideas to C++ will
                                normally be shot down.
                                Anyone who proposes adding additional static compile-time ideas to C++ will
                                be welcome with open arms.

                                To get sort() working, one of your constraints has to go.

                                I have sort() working with arbitrary size elements. But the price I paid was
                                in your original message as I have a vector of pointers to the elements. And
                                in the enviroment I am in, pointers are 4-bytes, the records are
                                considerably larger, sorting the pointers seems a good option. But with the
                                messages posted here, I am wondering whether I would have been better off
                                with qsort(). The code would be simpler. If the element size is 4 or less
                                bytes, I can make the additional optimisation of not using a vectors of
                                pointer at all but using qsort() directly on the records.

                                [color=blue]
                                >Yes, its true that maybe STL was only written for In-Memory data,
                                >and STL implementations did not care about out of memory data.
                                >But I believe that both for inmemory and outmemory data, now or
                                >later, locality will be imporatant. And if STL takes into account that
                                >fact, then not only could we use it for data in memory, but also on
                                >data that resides on disk.[/color]

                                If the idea is to sort more data than fits in memory, it has been done. In
                                such a situation, you read as much as possible in chunks, sort, and write to
                                temporary files.
                                Later you perform an n-way merge of the temporary files, writing the final
                                result o some output file. All in Knuth.

                                Stephen Howe



                                [ See http://www.gotw.ca/resources/clcm.htm for info about ]
                                [ comp.lang.c++.m oderated. First time posters: Do this! ]

                                Comment

                                Working...