Array size limits

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Carol Depore

    Array size limits

    How do I determine the maximum array size?

    For example, int a[10000] works, but a[10000000] does not (run time
    error).

    Thank you.

  • Jack Klein

    #2
    Re: Array size limits

    On Tue, 31 Aug 2004 02:08:08 GMT, Carol Depore <nobody@nowhere .com>
    wrote in comp.lang.c:
    [color=blue]
    > How do I determine the maximum array size?[/color]

    You can't
    [color=blue]
    > For example, int a[10000] works, but a[10000000] does not (run time
    > error).
    >
    > Thank you.[/color]

    The original C standard (ANSI 1989/ISO 1990) required that a compiler
    successfully translate at least one program containing at least one
    example of a set of environmental limits. One of those limits was
    being able to create an object of at least 32,767 bytes.

    This minimum limit was raised in the 1999 update to the C standard to
    be at least 65,535 bytes.

    No C implementation is required to provide for objects greater than
    that size, which means that they don't need to allow for an array of
    ints greater than (int)(65535 / sizeof(int)).

    In very practical terms, on modern computers, it is not possible to
    say in advance how large an array can be created. It can depend on
    things like the amount of physical memory installed in the computer,
    the amount of virtual memory provided by the OS, the number of other
    tasks, drivers, and programs already running and how much memory that
    are using. So your program may be able to use more or less memory
    running today than it could use yesterday or it will be able to use
    tomorrow.

    Many platforms place their strictest limits on automatic objects, that
    is those defined inside of a function without the use of the 'static'
    keyword. On some platforms you can create larger arrays if they are
    static or by dynamic allocation.

    --
    Jack Klein
    Home: http://JK-Technology.Com
    FAQs for
    comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html
    comp.lang.c++ http://www.parashift.com/c++-faq-lite/
    alt.comp.lang.l earn.c-c++

    Comment

    • Ben Pfaff

      #3
      Re: Array size limits

      Carol Depore <nobody@nowhere .com> writes:
      [color=blue]
      > How do I determine the maximum array size?[/color]

      There is no portable way. You are better off using dynamic
      allocation, because then you can try different sizes at runtime.
      --
      "Am I missing something?"
      --Dan Pop

      Comment

      • Carol Depore

        #4
        Re: Array size limits

        On Mon, 30 Aug 2004 21:48:07 -0500, Jack Klein <jackklein@spam cop.net>
        wrote:
        [color=blue]
        >On Tue, 31 Aug 2004 02:08:08 GMT, Carol Depore <nobody@nowhere .com>
        >wrote in comp.lang.c:
        >[color=green]
        >> How do I determine the maximum array size?[/color]
        >
        >You can't
        >[color=green]
        >> For example, int a[10000] works, but a[10000000] does not (run time
        >> error).
        >>
        >> Thank you.[/color]
        >
        >The original C standard (ANSI 1989/ISO 1990) required that a compiler
        >successfully translate at least one program containing at least one
        >example of a set of environmental limits. One of those limits was
        >being able to create an object of at least 32,767 bytes.
        >
        >This minimum limit was raised in the 1999 update to the C standard to
        >be at least 65,535 bytes.
        >
        >No C implementation is required to provide for objects greater than
        >that size, which means that they don't need to allow for an array of
        >ints greater than (int)(65535 / sizeof(int)).[/color]

        So...are you saying that I am guaranteed at least int a[65535], but no
        guarantees beyond that?




        [color=blue]
        >
        >In very practical terms, on modern computers, it is not possible to
        >say in advance how large an array can be created. It can depend on
        >things like the amount of physical memory installed in the computer,
        >the amount of virtual memory provided by the OS, the number of other
        >tasks, drivers, and programs already running and how much memory that
        >are using. So your program may be able to use more or less memory
        >running today than it could use yesterday or it will be able to use
        >tomorrow.
        >
        >Many platforms place their strictest limits on automatic objects, that
        >is those defined inside of a function without the use of the 'static'
        >keyword. On some platforms you can create larger arrays if they are
        >static or by dynamic allocation.[/color]

        Comment

        • Ben Pfaff

          #5
          Re: Array size limits

          Carol Depore <nobody@nowhere .com> writes:
          [color=blue]
          > On Mon, 30 Aug 2004 21:48:07 -0500, Jack Klein <jackklein@spam cop.net>
          > wrote:[color=green]
          >>No C implementation is required to provide for objects greater than
          >>that size, which means that they don't need to allow for an array of
          >>ints greater than (int)(65535 / sizeof(int)).[/color]
          >
          > So...are you saying that I am guaranteed at least int a[65535], but no
          > guarantees beyond that?[/color]

          No. To start with, sizeof(int) may be, and usually is, greater
          than 1.
          --
          "...what folly I commit, I dedicate to you."
          --William Shakespeare, _Troilus and Cressida_

          Comment

          • Carol Depore

            #6
            Re: Array size limits

            On Mon, 30 Aug 2004 20:48:45 -0700, Ben Pfaff <blp@cs.stanfor d.edu>
            wrote:
            [color=blue]
            >Carol Depore <nobody@nowhere .com> writes:
            >[color=green]
            >> On Mon, 30 Aug 2004 21:48:07 -0500, Jack Klein <jackklein@spam cop.net>
            >> wrote:[color=darkred]
            >>>No C implementation is required to provide for objects greater than
            >>>that size, which means that they don't need to allow for an array of
            >>>ints greater than (int)(65535 / sizeof(int)).[/color]
            >>
            >> So...are you saying that I am guaranteed at least int a[65535], but no
            >> guarantees beyond that?[/color]
            >
            >No. To start with, sizeof(int) may be, and usually is, greater
            >than 1.[/color]


            Oh, I see...so, if sizeof(int) is 2 bytes, then I am guaranteed at
            least int a[32767], but no larger sized arrays are guaranteed.

            Thanks for the help!

            Comment

            • Ravi Uday

              #7
              Re: Array size limits


              "Carol Depore" <nobody@nowhere .com> wrote in message
              news:iu08j0top6 p5gui8rophsm4ih ih5n5qql0@4ax.c om...[color=blue]
              > On Mon, 30 Aug 2004 20:48:45 -0700, Ben Pfaff <blp@cs.stanfor d.edu>
              > wrote:
              >[color=green]
              > >Carol Depore <nobody@nowhere .com> writes:
              > >[color=darkred]
              > >> On Mon, 30 Aug 2004 21:48:07 -0500, Jack Klein <jackklein@spam cop.net>
              > >> wrote:
              > >>>No C implementation is required to provide for objects greater than
              > >>>that size, which means that they don't need to allow for an array of
              > >>>ints greater than (int)(65535 / sizeof(int)).
              > >>
              > >> So...are you saying that I am guaranteed at least int a[65535], but no
              > >> guarantees beyond that?[/color]
              > >
              > >No. To start with, sizeof(int) may be, and usually is, greater
              > >than 1.[/color]
              >
              >
              > Oh, I see...so, if sizeof(int) is 2 bytes, then I am guaranteed at
              > least int a[32767], but no larger sized arrays are guaranteed.
              >[/color]
              No, You have to check INT_MAX in your limits.h file to know the exact
              maximum
              value that an integer can hold on your implementation.

              [color=blue]
              > Thanks for the help!
              >[/color]


              Comment

              • Keith Thompson

                #8
                Re: Array size limits

                "Ravi Uday" <raviuday@gmail .com> writes:[color=blue]
                > "Carol Depore" <nobody@nowhere .com> wrote in message
                > news:iu08j0top6 p5gui8rophsm4ih ih5n5qql0@4ax.c om...[/color]
                [...][color=blue][color=green]
                > > Oh, I see...so, if sizeof(int) is 2 bytes, then I am guaranteed at
                > > least int a[32767], but no larger sized arrays are guaranteed.
                > >[/color]
                > No, You have to check INT_MAX in your limits.h file to know the exact
                > maximum
                > value that an integer can hold on your implementation.[/color]

                INT_MAX isn't relevant. The limit in question is the maximum size of
                an object, which is at least 65535 bytes (in a hosted environment
                only). If sizeof(int) is 2, you're guaranteed at least int a[32767];
                if sizeof(int) is 4; you're only guaranteed at least int a[16383].

                But actually the guarantee is even weaker than that. The standard
                only requires that the implementation must be able to translate and
                execute at least one program containing an object of 65535 bytes
                (along with a number of other limits, such as 127 arguments in one
                function call). It's not required to handle *your* program containing
                an object of 65535 bytes.

                Practically speaking, though, the easiest way to satisfy the
                translation limits is generally to impose no explicit limits, but to
                support whatever will fit in memory, either at compile time or at run
                time. A (non-binding) footnote in C99 5.2.4.1 says, "Implementation s
                should avoid imposing fixed translation limits whenever possible."

                --
                Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
                San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
                We must do something. This is something. Therefore, we must do this.

                Comment

                • pete

                  #9
                  Re: Array size limits

                  Keith Thompson wrote:[color=blue]
                  >
                  > "Ravi Uday" <raviuday@gmail .com> writes:[color=green]
                  > > "Carol Depore" <nobody@nowhere .com> wrote in message
                  > > news:iu08j0top6 p5gui8rophsm4ih ih5n5qql0@4ax.c om...[/color]
                  > [...][color=green][color=darkred]
                  > > > Oh, I see...so, if sizeof(int) is 2 bytes, then I am guaranteed at
                  > > > least int a[32767], but no larger sized arrays are guaranteed.
                  > > >[/color]
                  > > No, You have to check INT_MAX in your limits.h file to know the exact
                  > > maximum
                  > > value that an integer can hold on your implementation.[/color]
                  >
                  > INT_MAX isn't relevant. The limit in question is the maximum size of
                  > an object, which is at least 65535 bytes (in a hosted environment
                  > only). If sizeof(int) is 2, you're guaranteed at least int a[32767];
                  > if sizeof(int) is 4; you're only guaranteed at least int a[16383].
                  >
                  > But actually the guarantee is even weaker than that. The standard
                  > only requires that the implementation must be able to translate and
                  > execute at least one program containing an object of 65535 bytes
                  > (along with a number of other limits, such as 127 arguments in one
                  > function call). It's not required to handle *your* program containing
                  > an object of 65535 bytes.[/color]

                  I think what they meant by the "at least one" part,
                  is closer to saying
                  "A C implementation is something which can translate and execute
                  a C program, and anything that can't translate and execute
                  a C program, isn't a C implementation"

                  I don't think that they meant to suggest that an implementation
                  which doesn't self destruct after program translation and execution,
                  "exceeds ANSI standards".

                  --
                  pete

                  Comment

                  • Carol Depore

                    #10
                    Re: Array size limits


                    Everyone, thank you for your help. I feel a little like I'm asking
                    Einstein to explain relativity to me. I'm way out of my league.
                    So, thanks for your patience.

                    Anyway, I'm still confused about how large a simple int array can be.
                    I understand what Jack and Ben said about the Standard guaranteeing
                    at least int a[32767], but I don't understand why I can't have an
                    array of int a[600000], especially since I have 311MB of unused memory
                    on my machine, and I thought the machine would allow programs up to
                    4GB.

                    Here's my little test program, which works for 500000, but fails for
                    600000.

                    All help to relieve my confusion is appreciated. I think I'm not
                    understanding something very fundamental.


                    #include <stdio.h>

                    //#define NNN 600000
                    #define NNN 500000

                    int main() {
                    long i;
                    int a[NNN];
                    for (i=0;i<NNN;i++) {
                    printf("%i\n",i );
                    a[i] = 1;
                    }
                    }

                    Comment

                    • Keith Thompson

                      #11
                      Re: Array size limits

                      pete <pfiland@mindsp ring.com> writes:[color=blue]
                      > Keith Thompson wrote:[/color]
                      [...][color=blue][color=green]
                      > > But actually the guarantee is even weaker than that. The standard
                      > > only requires that the implementation must be able to translate and
                      > > execute at least one program containing an object of 65535 bytes
                      > > (along with a number of other limits, such as 127 arguments in one
                      > > function call). It's not required to handle *your* program containing
                      > > an object of 65535 bytes.[/color]
                      >
                      > I think what they meant by the "at least one" part,
                      > is closer to saying
                      > "A C implementation is something which can translate and execute
                      > a C program, and anything that can't translate and execute
                      > a C program, isn't a C implementation"
                      >
                      > I don't think that they meant to suggest that an implementation
                      > which doesn't self destruct after program translation and execution,
                      > "exceeds ANSI standards".[/color]

                      Actually, I think that is what they meant. For example, if an
                      implementation can handle a single carefully written program that
                      meets each of the translation limits, including a single object of
                      exactly 65535 bytes, but falls over and dies if you add a 1-byte
                      object declaration to that same program, that implementation is
                      conforming (assuming it doesn't have any other problems).

                      That doesn't imply that such an implementation is *useful*; that's a
                      QoI (Quality of Implementation) issue.

                      In real life, nobody bothers to implement a conforming C compiler
                      that's totally useless (or if anybody does, it rapidly vanishes).

                      --
                      Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
                      San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
                      We must do something. This is something. Therefore, we must do this.

                      Comment

                      • Eric Sosman

                        #12
                        Re: Array size limits

                        Carol Depore wrote:[color=blue]
                        > Everyone, thank you for your help. I feel a little like I'm asking
                        > Einstein to explain relativity to me. I'm way out of my league.
                        > So, thanks for your patience.
                        >
                        > Anyway, I'm still confused about how large a simple int array can be.
                        > I understand what Jack and Ben said about the Standard guaranteeing
                        > at least int a[32767], but I don't understand why I can't have an
                        > array of int a[600000], especially since I have 311MB of unused memory
                        > on my machine, and I thought the machine would allow programs up to
                        > 4GB.[/color]

                        Your machine (like many) apparently exceeds the minimum
                        requirements imposed by the C language Standard. It's sort
                        of like the Federal definition of the minimum wage: Employers
                        are required to pay at least thus-and-such much per hour of
                        labor, but many laborers nonetheless demand and receive more.
                        Be happy; you're rich!
                        [color=blue]
                        > Here's my little test program, which works for 500000, but fails for
                        > 600000.[/color]

                        Your C implementation is giving you more than the minimum,
                        but does in fact have a limit on how big the array can be.
                        You're rich, but you're not Bill Gates.
                        [color=blue]
                        > All help to relieve my confusion is appreciated. I think I'm not
                        > understanding something very fundamental.
                        >
                        >
                        > #include <stdio.h>
                        >
                        > //#define NNN 600000
                        > #define NNN 500000
                        >
                        > int main() {
                        > long i;
                        > int a[NNN];[/color]

                        Here's another issue. C data objects can have various
                        "storage classes:" automatic, static, and dynamic. The amount
                        of memory available for an object can be different for the
                        different storage classes. Your a[] array occupies automatic
                        storage, which is typically (although not necessarily) subject
                        to the tightest space restrictions. You may see dramatically
                        different results if you change the above to

                        int main() {
                        static int a[NNN};
                        ...

                        or

                        int main() {
                        int *a = malloc(NNN * sizeof *a);
                        if (a != NULL) {
                        ...

                        The first of these uses static storage for a[], and the
                        second replaces the array with a pointer to dynamic storage.
                        There will be limits on how much static or dynamic memory
                        you can devote to your data, but they're likely to be looser
                        than the limits on automatic storage.

                        You're rich, but if you want to augment your wealth it's
                        better to rob big banks than little ones.

                        --
                        Eric.Sosman@sun .com

                        Comment

                        • Keith Thompson

                          #13
                          Re: Array size limits

                          Carol Depore <nobody@nowhere .com> writes:
                          [...][color=blue]
                          > Anyway, I'm still confused about how large a simple int array can be.
                          > I understand what Jack and Ben said about the Standard guaranteeing
                          > at least int a[32767], but I don't understand why I can't have an
                          > array of int a[600000], especially since I have 311MB of unused memory
                          > on my machine, and I thought the machine would allow programs up to
                          > 4GB.
                          >
                          > Here's my little test program, which works for 500000, but fails for
                          > 600000.
                          >
                          > All help to relieve my confusion is appreciated. I think I'm not
                          > understanding something very fundamental.
                          >
                          > #include <stdio.h>
                          >
                          > //#define NNN 600000
                          > #define NNN 500000
                          >
                          > int main() {
                          > long i;
                          > int a[NNN];
                          > for (i=0;i<NNN;i++) {
                          > printf("%i\n",i );
                          > a[i] = 1;
                          > }
                          > }[/color]

                          The standard doesn't guarantee at least int a[32767] unless
                          sizeof(int) happens to be 1 or 2. The specific guarantee is an object
                          of at least 65535 bytes. But almost all implementations exceed that
                          guarantee.

                          You say the program fails for 600000, but you don't say *how* it
                          fails. It probably doesn't matter much in this case, but in general
                          knowing *how* something fails can be critical to figuring out what the
                          problem is.

                          The maximum size allowed for int a[NNN] in your program, and what's
                          going to happen if you exceed it, is going to depend on any of a
                          number of things, most of which we can't help you with here. The
                          total amount of memory available on the system is only one possible
                          factor. Some systems impose specific limits on stack size (the array
                          is probably going to be allocated on "the stack", though the C
                          standard doesn't define such a thing), but you probably don't know
                          what else is allocate there. Some systems might allow you to adjust
                          the limits (on Unix-like systems, see the "limit" or "ulimit" command;
                          on other systems, I have no clue). Some systems may support larger
                          chunks of memory in different contexts; for example, declaring a as a
                          global variable might put it in the data section rather than on the
                          stack, or you might try allocating it via malloc().

                          If you want to ask about the details, you should try a newsgroup
                          that's specific to your system.

                          --
                          Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
                          San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
                          We must do something. This is something. Therefore, we must do this.

                          Comment

                          • CBFalconer

                            #14
                            Re: Array size limits

                            Carol Depore wrote:[color=blue]
                            >[/color]
                            .... snip ...[color=blue]
                            >
                            > Anyway, I'm still confused about how large a simple int array
                            > can be. I understand what Jack and Ben said about the Standard
                            > guaranteeing at least int a[32767], but I don't understand why
                            > I can't have an array of int a[600000], especially since I have
                            > 311MB of unused memory on my machine, and I thought the machine
                            > would allow programs up to 4GB.
                            >
                            > Here's my little test program, which works for 500000, but
                            > fails for 600000.
                            >
                            > #include <stdio.h>
                            >
                            > //#define NNN 600000
                            > #define NNN 500000
                            >
                            > int main() {
                            > long i;
                            > int a[NNN];
                            > for (i=0;i<NNN;i++) {
                            > printf("%i\n",i );
                            > a[i] = 1;
                            > }
                            > }[/color]

                            This tells me that your system assigns a default stack size
                            (assuming it has a stack) of between 500,000 * sizeof int and
                            600,000 * sizeof int. The most likely values are 1 meg and 2 meg.

                            --
                            "Churchill and Bush can both be considered wartime leaders, just
                            as Secretariat and Mr Ed were both horses." - James Rhodes.
                            "We have always known that heedless self-interest was bad
                            morals. We now know that it is bad economics" - FDR

                            Comment

                            • Wei Li

                              #15
                              Re: Array size limits

                              I don't know how ANSI C says. But the stacksize of a program in runtime is
                              limited by Opterating System. So that C compiler always doesn't give out an
                              compile time error.

                              If you are on Linux, you can use "limit" command to show the limits and use
                              "limit stacksize 1000000" to change them. I think your program will work OK
                              after change the limit.

                              Thanks!
                              Wei


                              "Carol Depore" <nobody@nowhere .com> wrote in message
                              news:lmn7j09msk s2q00nihpps23sn o0olk1hl6@4ax.c om...[color=blue]
                              > How do I determine the maximum array size?
                              >
                              > For example, int a[10000] works, but a[10000000] does not (run time
                              > error).
                              >
                              > Thank you.
                              >[/color]


                              Comment

                              Working...