#include optimization

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Michael Mair

    #16
    Re: #include optimization

    Hi there,
    [color=blue][color=green][color=darkred]
    >>>>I am currently maintaining a legacy code with a very very large code
    >>>>base. I am facing problems with C/C++ files having a lot of
    >>>>un-necessary #includes. On an average every C/C++ file has around
    >>>>150+ .h files included. I find 75% of the files unnecessary and
    >>>>could be removed. Considering the fact that I have a huge code base,
    >>>>I can't manually fix it.
    >>>
    >>>If it ain't broken, don't fix it.[/color]
    >>
    >>Unless fixing it will make future maintenance easier, which is
    >>probably the case here.[/color]
    >
    > Why?[/color]

    I cannot look into Keith's head but one possible reason is that
    from looking at which file includes which header you get information
    where to look when changing something.

    Especially in huge codes with many man years of work in them and
    too few man days of documentation work and me not being an expert,
    this (e.g. by find . -type f -exec grep obscureheader.h \{} -print)
    is -- after browsing around with tags -- one step I might
    take to get a feeling for how this connects with that and whether some
    people took "shortcuts" somewhere that need to be fixed.
    If you basically have people including about fifty header files for
    no reason in every translation unit they get into their hands so
    that they do not have to know about where which functions, definitions
    and so on come from, then this approach obviously is not feasible.

    Just a guess.


    Cheers
    Michael

    Comment

    • John Bode

      #17
      Re: #include optimization

      "E. Robert Tisdale" <E.Robert.Tisda le@jpl.nasa.gov > wrote in message news:<cjcev5$1e t$1@nntp1.jpl.n asa.gov>...[color=blue]
      > Ramesh wrote:
      >[color=green]
      > > I am currently maintaining a legacy code with a very very large code base.
      > > I am facing problems with C/C++ files having a lot of un-necessary
      > > #includes.[/color]
      >
      > You are probably mistaken.
      > Good programmers don't include header files that aren't necessary.
      >[/color]

      Yes they do, on occasion. It's possible that the original programmers
      weren't very good. It's also possible that the headers were necessary
      at one time, but over time the code was hacked beyond recognition and
      the symbols in those headers are no longer being used in that source
      file. Or it could be the result of a cut-and-paste fest stemming from
      unclear requirements and a laughably unrealistic schedule.

      I've seen this movie more than once. Hell, I'm *in* the movie.
      [color=blue][color=green]
      > > On an average every C/C++ file has around 150+ .h files included.
      > > I find 75% of the files unnecessary and could be removed.[/color]
      >
      > How did you determine that?
      >[/color]

      By checking to see if the source file actually *uses* any of the
      symbols defined in the header file maybe?
      [color=blue][color=green]
      > > Considering the fact that I have a huge code base,
      > > I can't manually fix it.[/color]
      >
      > What, exactly, are you trying to fix?
      >[/color]

      He is trying to remove unnecessary #include directives, thereby making
      the code easier to read and maintain (and maybe save some cycles
      during builds).
      [color=blue][color=green]
      > > Are there any tools that would report un wanted .h files?[/color]
      >
      > Of course not. How would such a tool know
      > what should and shouldn't be in any given header file?
      >[/color]

      The tool could scan each header file for symbols (macros, typedefs,
      enums, function declarations, external variable declarations) and then
      check the source file to see if any of those symbols are present. If
      none of the symbols in the header are present in the source file, then
      the tool can mark that header as (probably) superfluous. I don't
      personally know of any tool that does that particular job, and I won't
      claim that it would be easy to write one (handling nested includes
      would be "fun"), but it can be done (lex/flex and yacc/bison would
      probably be the best way to go about it).
      [color=blue]
      >
      > Try this:
      >[color=green]
      > > cat file.h[/color]
      > #ifndef GUARD_FILE_H
      > #define GUARD_FILE_H 1
      > #include <stdlib.h>
      > #include <stdio.h>
      > #include <stdint.h>
      > #include <math.h>
      > #include <string.h>
      > #include <values.h>
      > #include <time.h>
      > #include <signal.h>
      > #endif//GUARD_FILE_H
      >[color=green]
      > > cat file1.c[/color]
      > #include "file.h"
      >[color=green]
      > > time gcc -Wall -std=c99 -pedantic -c file1.c[/color]
      > 0.133u 0.049s 0:00.18 94.4% 0+0k 0+0io 0pf+0w[color=green]
      > > cat file128.c[/color]
      > #include "file.h"
      > #include "file.h"
      > .
      > .
      > .
      > #include "file.h"
      >[color=green]
      > > time gcc -Wall -std=c99 -pedantic -c file128.c[/color]
      > 0.144u 0.039s 0:00.19 89.4% 0+0k 0+0io 0pf+0w
      >
      > which shows that it doesn't take any more time to process
      > a header file 128 time than it takes to process it once!
      > Once the C preprocessor has read an idempotent file,
      > it doesn't read it again no matter how many times it is included.[/color]

      But that's not really the OP's problem, is it?

      Comment

      • Keith Thompson

        #18
        Re: #include optimization

        Dan.Pop@cern.ch (Dan Pop) writes:[color=blue]
        > In <lnwtye1bb6.fsf @nuthaus.mib.or g> Keith Thompson <kst-u@mib.org> writes:[color=green]
        >>Dan.Pop@cern. ch (Dan Pop) writes:[color=darkred]
        >>> In <11e6710.040928 0435.731bebd1@p osting.google.c om>
        >>> ramesh25@gmail. com (Ramesh) writes:
        >>>>I am currently maintaining a legacy code with a very very large code
        >>>>base. I am facing problems with C/C++ files having a lot of
        >>>>un-necessary #includes. On an average every C/C++ file has around
        >>>>150+ .h files included. I find 75% of the files unnecessary and
        >>>>could be removed. Considering the fact that I have a huge code base,
        >>>>I can't manually fix it.
        >>>
        >>> If it ain't broken, don't fix it.[/color]
        >>
        >>Unless fixing it will make future maintenance easier, which is
        >>probably the case here.[/color]
        >
        > Why?[/color]

        Because.

        Take a look at what's been said so far. The source files have an
        average of about 150 #include directives, most of which are
        unnecessary. It's a "very very large code base". It seems fairly
        obvious to me that the whole thing is a mess, and that cleaning it up
        would make it easier to maintain. (I'm tempted to suggest the
        possibility of throwing it away and starting from scratch, but that's
        probably not feasible.)

        Maybe it isn't really a problem, or maybe the unnecessary #includes
        are such a small part of the problem that eliminating them wouldn't
        really help, which is why I qualified my statement with the word
        "probably". But since Ramesh specifically said that it's a problem,
        and he's asking for ways to fix it, I'm not going to assume that he's
        mistaken about the premise for his question.

        --
        Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
        San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
        We must do something. This is something. Therefore, we must do this.

        Comment

        • CBFalconer

          #19
          Re: #include optimization

          Keith Thompson wrote:[color=blue]
          > Dan.Pop@cern.ch (Dan Pop) writes:[color=green]
          >> Keith Thompson <kst-u@mib.org> writes:[color=darkred]
          >>> Dan.Pop@cern.ch (Dan Pop) writes:
          >>>> ramesh25@gmail. com (Ramesh) writes:[/color][/color]
          >[color=green][color=darkred]
          >>>>> I am currently maintaining a legacy code with a very very large
          >>>>> code base. I am facing problems with C/C++ files having a lot
          >>>>> of un-necessary #includes. On an average every C/C++ file has
          >>>>> around 150+ .h files included. I find 75% of the files
          >>>>> unnecessary and could be removed. Considering the fact that I
          >>>>> have a huge code base, I can't manually fix it.
          >>>>
          >>>> If it ain't broken, don't fix it.
          >>>
          >>> Unless fixing it will make future maintenance easier, which is
          >>> probably the case here.[/color]
          >>
          >> Why?[/color]
          >
          > Because.
          >
          > Take a look at what's been said so far. The source files have an
          > average of about 150 #include directives, most of which are
          > unnecessary. It's a "very very large code base". It seems fairly
          > obvious to me that the whole thing is a mess, and that cleaning it
          > up would make it easier to maintain. (I'm tempted to suggest the
          > possibility of throwing it away and starting from scratch, but
          > that's probably not feasible.)
          >
          > Maybe it isn't really a problem, or maybe the unnecessary #includes
          > are such a small part of the problem that eliminating them wouldn't
          > really help, which is why I qualified my statement with the word
          > "probably". But since Ramesh specifically said that it's a
          > problem, and he's asking for ways to fix it, I'm not going to
          > assume that he's mistaken about the premise for his question.[/color]

          Actually it is a job that can be attacked and checked piecemeal.
          Remove the (presumably) useless includes in one file by commenting
          out (thus retaining line numbers) and compile it to an object
          file. Do a binary compare against the original object file. If
          identical, all is well. If not, investigate the causes (which may
          include a compilation datestamp). This assumes that the system
          will not generate external linkages on seeing only a prototype.

          Another approach revolves around a cross referance, or the
          interactive equivalent supplied by cscope.

          --
          A: Because it fouls the order in which people normally read text.
          Q: Why is top-posting such a bad thing?
          A: Top-posting.
          Q: What is the most annoying thing on usenet and in e-mail?


          Comment

          • Peter van Merkerk

            #20
            Re: #include optimization

            CBFalconer wrote:[color=blue]
            > Keith Thompson wrote:
            >[color=green]
            >>Dan.Pop@cern. ch (Dan Pop) writes:
            >>[color=darkred]
            >>>Keith Thompson <kst-u@mib.org> writes:
            >>>
            >>>>Dan.Pop@cer n.ch (Dan Pop) writes:
            >>>>
            >>>>>ramesh25@g mail.com (Ramesh) writes:[/color]
            >>[color=darkred]
            >>>>>>I am currently maintaining a legacy code with a very very large
            >>>>>>code base. I am facing problems with C/C++ files having a lot
            >>>>>>of un-necessary #includes. On an average every C/C++ file has
            >>>>>>around 150+ .h files included. I find 75% of the files
            >>>>>>unnecessa ry and could be removed. Considering the fact that I
            >>>>>>have a huge code base, I can't manually fix it.
            >>>>>
            >>>>>If it ain't broken, don't fix it.
            >>>>
            >>>>Unless fixing it will make future maintenance easier, which is
            >>>>probably the case here.
            >>>
            >>>Why?[/color]
            >>
            >>Because.
            >>
            >>Take a look at what's been said so far. The source files have an
            >>average of about 150 #include directives, most of which are
            >>unnecessary . It's a "very very large code base". It seems fairly
            >>obvious to me that the whole thing is a mess, and that cleaning it
            >>up would make it easier to maintain. (I'm tempted to suggest the
            >>possibility of throwing it away and starting from scratch, but
            >>that's probably not feasible.)
            >>
            >>Maybe it isn't really a problem, or maybe the unnecessary #includes
            >>are such a small part of the problem that eliminating them wouldn't
            >>really help, which is why I qualified my statement with the word
            >>"probably". But since Ramesh specifically said that it's a
            >>problem, and he's asking for ways to fix it, I'm not going to
            >>assume that he's mistaken about the premise for his question.[/color]
            >
            >
            > Actually it is a job that can be attacked and checked piecemeal.
            > Remove the (presumably) useless includes in one file by commenting
            > out (thus retaining line numbers) and compile it to an object
            > file. Do a binary compare against the original object file. If
            > identical, all is well.[/color]

            Even if binary compare shows that there are differences all may be well
            too. I learned this the hard way; compiling the same code twice with a
            certain compiler yielded different binaries. Apparently the compiler
            wrote the build date or something like that into the binary. When debug
            information is stored in object files this test may also fail for the
            wrong reasons.

            --
            Peter van Merkerk
            peter.van.merke rk(at)dse.nl

            Comment

            • CBFalconer

              #21
              Re: #include optimization

              Peter van Merkerk wrote:[color=blue]
              > CBFalconer wrote:[color=green]
              >>[/color]
              >... snip ...[color=green]
              >>
              >> Actually it is a job that can be attacked and checked piecemeal.
              >> Remove the (presumably) useless includes in one file by commenting
              >> out (thus retaining line numbers) and compile it to an object
              >> file. Do a binary compare against the original object file. If
              >> identical, all is well.[/color]
              >
              > Even if binary compare shows that there are differences all may be well
              > too. I learned this the hard way; compiling the same code twice with a
              > certain compiler yielded different binaries. Apparently the compiler
              > wrote the build date or something like that into the binary. When debug
              > information is stored in object files this test may also fail for the
              > wrong reasons.[/color]

              Why did you snip the last sentence or two from my paragraph, which
              mentioned precisely this problem?

              --
              A: Because it fouls the order in which people normally read text.
              Q: Why is top-posting such a bad thing?
              A: Top-posting.
              Q: What is the most annoying thing on usenet and in e-mail?

              Comment

              • Michael Mair

                #22
                Re: #include optimization

                >>>Actually it is a job that can be attacked and checked piecemeal.[color=blue][color=green][color=darkred]
                >>>Remove the (presumably) useless includes in one file by commenting
                >>>out (thus retaining line numbers) and compile it to an object
                >>>file. Do a binary compare against the original object file. If
                >>>identical, all is well.[/color]
                >>
                >>Even if binary compare shows that there are differences all may be well
                >>too. I learned this the hard way; compiling the same code twice with a
                >>certain compiler yielded different binaries. Apparently the compiler
                >>wrote the build date or something like that into the binary. When debug
                >>information is stored in object files this test may also fail for the
                >>wrong reasons.[/color]
                >
                > Why did you snip the last sentence or two from my paragraph, which
                > mentioned precisely this problem?[/color]

                Probably because you gave a solution to the problem without stating
                the problem...
                To be honest, I did not really get what you were aiming at, either.

                For completeness, here is the left-out part:
                [color=blue][color=green][color=darkred]
                >>> Another approach revolves around a cross referance, or the
                >>> interactive equivalent supplied by cscope.[/color][/color][/color]

                Cheers,
                Michael

                Comment

                • Dan Pop

                  #23
                  Re: #include optimization

                  In <lnekkkg9po.fsf @nuthaus.mib.or g> Keith Thompson <kst-u@mib.org> writes:
                  [color=blue]
                  >Dan.Pop@cern.c h (Dan Pop) writes:[color=green]
                  >> In <lnwtye1bb6.fsf @nuthaus.mib.or g> Keith Thompson <kst-u@mib.org> writes:[color=darkred]
                  >>>Dan.Pop@cern .ch (Dan Pop) writes:
                  >>>> In <11e6710.040928 0435.731bebd1@p osting.google.c om>
                  >>>> ramesh25@gmail. com (Ramesh) writes:
                  >>>>>I am currently maintaining a legacy code with a very very large code
                  >>>>>base. I am facing problems with C/C++ files having a lot of
                  >>>>>un-necessary #includes. On an average every C/C++ file has around
                  >>>>>150+ .h files included. I find 75% of the files unnecessary and
                  >>>>>could be removed. Considering the fact that I have a huge code base,
                  >>>>>I can't manually fix it.
                  >>>>
                  >>>> If it ain't broken, don't fix it.
                  >>>
                  >>>Unless fixing it will make future maintenance easier, which is
                  >>>probably the case here.[/color]
                  >>
                  >> Why?[/color]
                  >
                  >Because.
                  >
                  >Take a look at what's been said so far. The source files have an
                  >average of about 150 #include directives, most of which are
                  >unnecessary. It's a "very very large code base". It seems fairly
                  >obvious to me that the whole thing is a mess, and that cleaning it up
                  >would make it easier to maintain. (I'm tempted to suggest the
                  >possibility of throwing it away and starting from scratch, but that's
                  >probably not feasible.)[/color]

                  One could merge all the 150 headers in a single header and you'll see
                  a single header being included instead of 150. Including that header
                  will also provide some unnecessary definitions and declarations, but this
                  is no different from including <stdlib.h> and getting more declarations
                  than your application actually needs (when was the last time you included
                  <stdlib.h> and used everything declared within?).

                  So, the fact that the source files include 150 headers, not all of them
                  necessary to each source file, is not a problem in itself. If the
                  maintainer is annoyed by seeing such a bunch of includes everywhere,
                  he can trivially write a new header, that includes all the application
                  headers, and include only this header in each source file.
                  [color=blue]
                  >Maybe it isn't really a problem, or maybe the unnecessary #includes
                  >are such a small part of the problem that eliminating them wouldn't
                  >really help, which is why I qualified my statement with the word
                  >"probably". But since Ramesh specifically said that it's a problem,
                  >and he's asking for ways to fix it, I'm not going to assume that he's
                  >mistaken about the premise for his question.[/color]

                  And I'm not going to believe him until provided with *concrete* examples.

                  I'm tempted to believe that the mistake was creating so many application
                  specific headers in the first place and the *right* fix would be to
                  drastically reduce their number. Without knowing the specifics, there
                  is no way of telling whether merging all of them in a single header
                  (e.g. by using another header that includes all of them) is the right
                  fix or if there is a real need for more than one application specific
                  header. But I'm reasonably convinced that there is no need for 150 header
                  files.

                  OTOH, I'm reasonably convinced that their existence is not causing
                  any maintenance problems, either. It's just that the source files
                  containing so many include directives look mildly annoying.

                  Dan
                  --
                  Dan Pop
                  DESY Zeuthen, RZ group
                  Email: Dan.Pop@ifh.de
                  Currently looking for a job in the European Union

                  Comment

                  • CBFalconer

                    #24
                    Re: #include optimization

                    Michael Mair wrote: *** And removed attributions ***[color=blue]
                    >[color=green][color=darkred]
                    >>>> Actually it is a job that can be attacked and checked piecemeal.
                    >>>> Remove the (presumably) useless includes in one file by commenting
                    >>>> out (thus retaining line numbers) and compile it to an object
                    >>>> file. Do a binary compare against the original object file. If
                    >>>> identical, all is well.
                    >>>
                    >>> Even if binary compare shows that there are differences all may be
                    >>> well too. I learned this the hard way; compiling the same code
                    >>> twice with a certain compiler yielded different binaries.
                    >>> Apparently the compiler wrote the build date or something like
                    >>> that into the binary. When debug information is stored in object
                    >>> files this test may also fail for the wrong reasons.[/color]
                    >>
                    >> Why did you snip the last sentence or two from my paragraph, which
                    >> mentioned precisely this problem?[/color]
                    >
                    > Probably because you gave a solution to the problem without stating
                    > the problem...
                    > To be honest, I did not really get what you were aiming at, either.
                    >
                    > For completeness, here is the left-out part:
                    >[color=green][color=darkred]
                    >>> Another approach revolves around a cross referance, or the
                    >>> interactive equivalent supplied by cscope.[/color][/color][/color]

                    Wrong left-out part, which was as follows, and was an integral
                    portion of the quoted paragraph:
                    [color=blue][color=green][color=darkred]
                    >>> identical, all is well. If not, investigate the causes (which may
                    >>> include a compilation datestamp). This assumes that the system
                    >>> will not generate external linkages on seeing only a prototype.[/color][/color][/color]

                    Your elimination of attributions leaves the (mistaken) impression
                    that I was complaining about your actions, while the actual
                    offender was Peter van Merkerk.

                    --
                    A: Because it fouls the order in which people normally read text.
                    Q: Why is top-posting such a bad thing?
                    A: Top-posting.
                    Q: What is the most annoying thing on usenet and in e-mail?


                    Comment

                    • Michael Mair

                      #25
                      Re: #include optimization

                      [Me messing up a discussion I did not take part in][color=blue][color=green]
                      >>
                      >>For completeness, here is the left-out part:
                      >>
                      >>[color=darkred]
                      >>>>Another approach revolves around a cross referance, or the
                      >>>>interacti ve equivalent supplied by cscope.[/color][/color]
                      >
                      >
                      > Wrong left-out part, which was as follows, and was an integral
                      > portion of the quoted paragraph:[/color]

                      Argh, sorry, I did so not get it :-([color=blue]
                      >
                      >[color=green][color=darkred]
                      >>>>identical , all is well. If not, investigate the causes (which may
                      >>>>include a compilation datestamp). This assumes that the system
                      >>>>will not generate external linkages on seeing only a prototype.[/color][/color]
                      >
                      > Your elimination of attributions leaves the (mistaken) impression
                      > that I was complaining about your actions, while the actual
                      > offender was Peter van Merkerk.[/color]

                      Er, right. Was a quick one when compiling my crap...
                      More thinking before sending next time.
                      Once again, sorry for messing it up!

                      --Michael

                      Comment

                      • Michael Wojcik

                        #26
                        Re: #include optimization


                        In article <2s2033F1bi389U 1@uni-berlin.de>, Peter van Merkerk <merkerk@deadsp am.com> writes:[color=blue]
                        >
                        > Even if binary compare shows that there are differences all may be well
                        > too. I learned this the hard way; compiling the same code twice with a
                        > certain compiler yielded different binaries. Apparently the compiler
                        > wrote the build date or something like that into the binary.[/color]

                        All it takes is one reference to __DATE__ or __TIME__.

                        I have projects where many of the TUs explicitly include build
                        timestamps using __DATE__ and __TIME__, since that information can be
                        useful in identifying precisely which build is being used. (Yes, the
                        projects also have explicit version information, and the modules have
                        version information that's updated automatically by the SCM system,
                        but it doesn't hurt to have confirmation.)

                        With that sort of project, binary comparison is basically useless.
                        Chuck's method certainly can work in some projects, but for something
                        the size that the OP is describing I suspect you'd spend a great deal
                        of time investigating false positives.

                        --
                        Michael Wojcik michael.wojcik@ microfocus.com

                        You brung in them two expert birdwatchers ... sayin' it was to keep us from
                        makin' dern fools of ourselfs ... whereas it's the inherent right of all to
                        make dern fools of theirselfs ... it ain't a right held by you official types
                        alone. -- Walt Kelly

                        Comment

                        • CBFalconer

                          #27
                          Re: #include optimization

                          Michael Wojcik wrote:[color=blue]
                          > Peter van Merkerk <merkerk@deadsp am.com> writes:[color=green]
                          >>
                          >> Even if binary compare shows that there are differences all may be well
                          >> too. I learned this the hard way; compiling the same code twice with a
                          >> certain compiler yielded different binaries. Apparently the compiler
                          >> wrote the build date or something like that into the binary.[/color]
                          >
                          > All it takes is one reference to __DATE__ or __TIME__.
                          >
                          > I have projects where many of the TUs explicitly include build
                          > timestamps using __DATE__ and __TIME__, since that information can be
                          > useful in identifying precisely which build is being used. (Yes, the
                          > projects also have explicit version information, and the modules have
                          > version information that's updated automatically by the SCM system,
                          > but it doesn't hurt to have confirmation.)
                          >
                          > With that sort of project, binary comparison is basically useless.
                          > Chuck's method certainly can work in some projects, but for something
                          > the size that the OP is describing I suspect you'd spend a great deal
                          > of time investigating false positives.[/color]

                          No it isn't useless. The differing areas will generally be of the
                          same size, if you have taken the precautions I mentioned. A
                          decent binary difference utility will show the differing bytes and
                          carry on. Mine shows the offset, the numerical difference, the
                          xor, the hex values from each file, and if printable the char
                          values. Something with the only difference a datestamp will
                          normally show 5 to 10 bytes of difference, and all else is not
                          present in the output.

                          c:>fdiff
                          usage: FDIFF [/q] [/nnn] file1 file2
                          options /q prevents page pauses
                          /nnn (hex digits) sets addr display.
                          binary comparison between file1 and file2

                          Ex: FDIFF /q/100 fdiff.com fdiff.bin
                          no page pauses, addr starts as 0100h

                          --
                          Chuck F (cbfalconer@yah oo.com) (cbfalconer@wor ldnet.att.net)
                          Available for consulting/temporary embedded and systems.
                          <http://cbfalconer.home .att.net> USE worldnet address!


                          Comment

                          • Dave Thompson

                            #28
                            Re: #include optimization

                            On 29 Sep 2004 05:34:35 -0700, "Ramesh Natarajan"
                            <ramesh25@gmail .com> wrote in comp.lang.c; comp.sys.tandem added as
                            more appropriate for most of this and may have better ideas:

                            <snip>[color=blue]
                            > The problem with these lot of un wanted #includes is that my
                            > development
                            > platform is Tandem and as I understand, the file open and close are the
                            > most
                            > expensive operations on the platform. Unfortunately we dont use a
                            > cross compiler
                            > and depend on a native compiler that needs to be run on the Tandem!!
                            >
                            > In general the compilation is pretty slow and with these header
                            > problems it takes
                            > forever to compile!!
                            >[/color]
                            From your example names I guess you are using the POSIX "personalit y"
                            or "subsystem" OSS. It's been a while since I've done so but as I
                            recall OSS file opens (or more precisely lookups) are unusually
                            expensive because the Unix-like filesystem must be emulated on the
                            real (Guardian) filesystem. Plus on Tandem almost all I/O is somewhat
                            more expensive because the OS is (and must be) message-based.

                            However, I would still expect opening and reading a few hundred files
                            (as long as none of them are absurdly large) to take only a few
                            seconds, maybe 5-10 at worst. If you have many -I directories that
                            need to be checked, maybe a few times that. If you are seeing worse,
                            it might be the system is not well configured for what you're doing --
                            there are a _lot_ of tuning "knobs" on Tandem and very few of them
                            automatic. You might ask your system manager if s/he has measured and
                            tuned for your type of workload. (Unless you are compiling on a
                            production system; then the response will probably be that the system
                            is tuned for production and if development suffers tough noogies.)

                            In fact if you have a large number of -I directories and can just
                            reduce them significantly, it will probably help. Maybe just create
                            one directory that contains a link or (I believe now) symlink to each
                            real file; you can do that with a few shell commands.

                            If a substantial number perhaps most of your included files are (or
                            can be) in one (single-level) directory or a few such, with names of
                            only up to 7 alphanumeric plus the .h, you might try putting them in a
                            Guardian subvolume and putting /G/somevol/mysubvol early in your
                            include path -- that _may_ bypass the emulation and go direct to
                            discprocess, I'm not sure.

                            Even more kludgily, the Tandem compilers have a nonstandard option to
                            #include only named sections of a file. You could combine all or at
                            least many of your current files into a single file with sections, and
                            change from a list of #include's to a single #include with a list of
                            sections; then you only have one open. But this won't help if one
                            "file" (section) #include's another, so you have to do the "all
                            includes at top level" style to really benefit. Also, regardless of
                            the order you specify the section names in the #include, they are
                            included in the order they appear in the file; you must make sure that
                            is consistent with any dependencies -- and if there are circular
                            dependencies that may be impossible. And of course this isn't
                            portable, although you could easily write a few lines of awk or
                            similar to convert it back when needed, or #if it.

                            All of these address only the symptom; if many of the #include's are
                            in fact unneeded as you said, it's obviously preferable to eliminate
                            them, for other systems and human readers as well. In addition to the
                            generic suggestions from others, I have one Tandem idea that _might_
                            help. The Tandem compilers used to have options to generate
                            cross-reference listings, although I can't find it in current (AFAICT)
                            manuals. If that option still exists, and possibly only if it includes
                            macros (#define's) which I don't recall, you could write a simple
                            program to go through such a listing, tally used symbols by file id,
                            and report any file ids not having any such symbol.

                            - David.Thompson1 at worldnet.att.ne t

                            Comment

                            Working...