c program binary/image in memory

[tele-commuter]

Hi folks,

I want to understand how exactly is an image(compiled
c code and loaded into memory) stored in memory.

What exactly is a linker script?

I work with a lot of c code on a daily basis but I really
don't understand :

How exactly the sections
like "text,bss,d ata etc."
work?

What exactly are they?

I believe the linker script describes this layout.

What exactly is a 'bundled image'?

What exactly is 'microcode'?
What exactly is 'firmware'?

I admit I use these terms myself pretty often,but
without much clarity.

Anyways any information or pointers(intern et urls) in
this regard would be very helpful.

If it helps,my work involves a lot of driver code
in C on systems using MIPS processors.

Thanks a bunch..

[santosh]

Re: c program binary/image in memory

tele-commuter wrote:[color=blue]
> Hi folks,
>
> I want to understand how exactly is an image(compiled
> c code and loaded into memory) stored in memory.[/color]

None of your questions are topical to this newsgroup, which deals only
with ISO standard C.

A good starting link:

http://www.iecc.com/linker/

[jacob navia]

Re: c program binary/image in memory

tele-commuter a écrit :[color=blue]
> Hi folks,
>
> I want to understand how exactly is an image(compiled
> c code and loaded into memory) stored in memory.
>
> What exactly is a linker script?
>[/color]

This is an extension of the GNU linkers that allows
GNU people to configure their linker. Normal users
are not supposed to use that, at least I didn't understand
anything about it. This is GNU specific, so off topic
here.
[color=blue]
> I work with a lot of c code on a daily basis but I really
> don't understand :
>
> How exactly the sections
> like "text,bss,d ata etc."[/color]

This sections map following parts of the language:

1) The "text" section is the translated output of
the compiler, the program text. For instance

int fn(void) { /* ... */

the sequence of instructions in this function will be stored
in the "text" section. This section is normally not
writable.

2) The bss section implements the zero initialized data. For instance
when you write

int b[23];

This array will be mapped into the bss section. This section will
be zeroed by the operating system when the program is loaded.
The standard specifies that this must be zeroed before the program
starts. Note that the bss sections and the data sections are
writable, i.e. can be modified.

3) The data section corresponds to the initialized data of your program.

For instance when you write:

int c[] = { 1,2,3};

The sequence of integers 1,2,3 will be stored in the data section.

In short, a C program consists of program instructions, uninitialized
data and initialized data. This logical sections of the program
are mapped into text,bss,and data, respectively.

Other sections may exist, for instance sections containing debug
information, but they are optional.

It is legal to write a program with only the text section, without any
bss/data sections. For instance this program:

int main(int argc,char *argv[])
{
return (argc*argc);
}

has only a text section, and no data/bss section.

Obviously this program could end up having a data/bss section
anyway, since the code that runs before main (the startup code
in a hosted implementation) could need those sections.


jacob
[color=blue]
> What exactly is a 'bundled image'?
>[/color]

No idea
[color=blue]
> What exactly is 'microcode'?[/color]

Microcode is the software that runs in the CPU itself. Modern
CPUs present a user interface with registers, ALU, etc. Internally
the high level instructions are decomposed into smaller and more
basic instructions. A sequence of them implements the user visible
instructions. This is more hardware related and it is off topic here.
[color=blue]
> What exactly is 'firmware'?
>[/color]

It is the software supplied by the hardware manufacturer. Again,
this is hardware related and off topic here.

[tele-commuter]

Re: c program binary/image in memory

Thanks a lot for the information.

Now let me provide the context.

I work with a network device (a distributed system).

Its a multiple CPU system.The main CPU runs a single monolithic image
(i referred to this as a "bundled image" in my post).

Portions of it get downloaded to other cards in the chassis during
system initialization/bootup.

What I wanted to understand in some detail was how the main CPU gets
this work done
of downloading portions of code onto other cards.I presume this is
related to way the code
is written,how makefiles are written,and how the binary is laid out in
memory to achieve this
functionality etc.

I do understand portions of my queries are off topic but I am trying to
get some clarity on
how much of all this really language/and hence compiler/linker/loader
dependant and also
how these tools come together to achieve some complicated ability like
the context I have
described.

Thanks anyways for all the help.



All the questions I have asked were keeping this situation in mind.

[Flash Gordon]

Re: c program binary/image in memory

jacob navia wrote:[color=blue]
> tele-commuter a écrit :[color=green]
>> Hi folks,
>>
>> I want to understand how exactly is an image(compiled
>> c code and loaded into memory) stored in memory.
>>
>> What exactly is a linker script?[/color]
>
> This is an extension of the GNU linkers that allows
> GNU people to configure their linker. Normal users
> are not supposed to use that, at least I didn't understand
> anything about it. This is GNU specific, so off topic
> here.[/color]

<OT>
I've used plenty of systems that had absolutely nothing to do with GNU
that used things called linker scripts where the user was *meant* to
write a linker script.

So you are correct that it is off topic, and may well be correct with
reference to GNU (or maybe not, I'm not going to verify either way) but
you are definitely *not* correct in general.
</OT>
[color=blue][color=green]
>> I work with a lot of c code on a daily basis but I really
>> don't understand :
>>
>> How exactly the sections
>> like "text,bss,d ata etc."[/color]
>
> This sections map following parts of the language:[/color]

<snip>

This is also all highly system specific. I've worked on plenty of
systems that did not have this combination of sections.

You should also have flagged this as off topic.
--
Flash Gordon
Living in interesting times.
Although my email address says spam, it is real and I read it.

[Flash Gordon]

Re: c program binary/image in memory

tele-commuter wrote:[color=blue]
> Thanks a lot for the information.[/color]

What information? Please provide context so people can see what you are
replying to. For information on how to do this read

http://cfaj.freeshell.org/google/

[color=blue]
> Now let me provide the context.
>
> I work with a network device (a distributed system).
>
> Its a multiple CPU system.The main CPU runs a single monolithic image
> (i referred to this as a "bundled image" in my post).[/color]

<snip>
[color=blue]
> All the questions I have asked were keeping this situation in mind.[/color]

Add all the questions you asked were, and still are, OFF TOPIC.

Now, since Google which you are using shows a post before Jacob's
telling you that it is off topic but you are asking again, I'll explain
what that means.

It means you should ask somewhere else. So please find a group dedicated
to your system and ask there, because it is not what is discussed here.
I'm giving you this advice for your own good, because posting here is
unlikely to get you good answers to your questions.

FYI Jacob only usex x86 systems, and mainly Windows at that, so I doubt
that he knows a lot about how MIPS based systems work. Which is not to
say what he said was wrong for MIPS, it might be entirely correct for
all I know, but the MIPS experts won't be correcting any errors he might
have made.
--
Flash Gordon
Living in interesting times.
Although my email address says spam, it is real and I read it.

[jacob navia]

Re: c program binary/image in memory

Flash Gordon a écrit :[color=blue]
>[color=green][color=darkred]
>>> I work with a lot of c code on a daily basis but I really
>>> don't understand :
>>>
>>> How exactly the sections
>>> like "text,bss,d ata etc."[/color]
>>
>>
>> This sections map following parts of the language:[/color]
>
>
> <snip>
>
> This is also all highly system specific. I've worked on plenty of
> systems that did not have this combination of sections.
>
> You should also have flagged this as off topic.[/color]

No, I do not agree.

C is a low level systems language, and as such, it is important to
be clear of the usage and the works of this mysterious tool called
THE LINKER.

This part of the system is as important as the compiler actually,
but will never explained and never mentioned anywhere. This is not
correct. C programs DO have the distinction between uninitialized
data areas and initialized ones, and that is even specified in the
standard. (The uninitialized ones should be set to zero).

There *is* always a section of the program that is reserved for
the machine instructions the circuit understands. There is almost always
a section for the initialized data, and the uninitialized data.

I think explaining this pertains to the C language, actually I think it
is an essential part that will often be neglected, treating the
compilation system as a magic black box that must be used
without any further understanding.

But of course we could agree on our disagreement ... :-)

jacob

[Richard Bos]

Re: c program binary/image in memory

jacob navia <jacob@jacob.re mcomp.fr> wrote:
[color=blue]
> Flash Gordon a écrit :[color=green]
> >[color=darkred]
> >>> How exactly the sections
> >>> like "text,bss,d ata etc."
> >>
> >> This sections map following parts of the language:[/color]
> >
> > This is also all highly system specific. I've worked on plenty of
> > systems that did not have this combination of sections.
> >
> > You should also have flagged this as off topic.[/color]
>
> No, I do not agree.
>
> C is a low level systems language, and as such, it is important to
> be clear of the usage and the works of this mysterious tool called
> THE LINKER.[/color]

IOW, you do not understend the difference between "how a toy compiler
works on a toy OS" and "how a compiler _could_ work anywhere".

A vague understanding of some irrelevant details of a Windows
implementation are of no help when you're trying to compile for, say,
the Mac. On a real system, the linker and the compiler can[1] be
different systems; one could use a variety of compilers with a single
linker, or a single compiler with a variety of linkers. And be none the
wiser about BSS, and no worse off for all that.

Richard

[1] Note: _can_, not _need_ - the opposite may also be true

[Ingo Menger]

Re: c program binary/image in memory


jacob navia schrieb:
[color=blue]
> THE LINKER.
>
> This part of the system is as important as the compiler actually,
> but will never explained and never mentioned anywhere.[/color]


Not true.
On my system
man 1 ld
tells me a lot about the linker. Likewise,
man 4 a.out
tells me about the executable format and how the OS's loader treats it.

But it is absolutely CERTAIN, that on other systems, the manuals will
tell different things. On some platforms, there is even more than one
executable format.

So, this is absolutely off topic.

[jacob navia]

Re: c program binary/image in memory

Richard Bos a écrit :[color=blue]
> jacob navia <jacob@jacob.re mcomp.fr> wrote:
>
>[color=green]
>>Flash Gordon a écrit :
>>[color=darkred]
>>>>> How exactly the sections
>>>>> like "text,bss,d ata etc."
>>>>
>>>>This sections map following parts of the language:
>>>
>>>This is also all highly system specific. I've worked on plenty of
>>>systems that did not have this combination of sections.
>>>
>>>You should also have flagged this as off topic.[/color]
>>
>>No, I do not agree.
>>
>>C is a low level systems language, and as such, it is important to
>>be clear of the usage and the works of this mysterious tool called
>>THE LINKER.[/color]
>
>
> IOW, you do not understend the difference between "how a toy compiler
> works on a toy OS" and "how a compiler _could_ work anywhere".
>[/color]
[color=blue]
> A vague understanding of some irrelevant details of a Windows
> implementation are of no help when you're trying to compile for, say,
> the Mac.[/color]

The gcc compiler is the compiler under MAC,and I would be surprised
that it wouldn't use the bss/data/and text sections. Really surprised.


On a real system, the linker and the compiler can[1] be[color=blue]
> different systems; one could use a variety of compilers with a single
> linker, or a single compiler with a variety of linkers. And be none the
> wiser about BSS, and no worse off for all that.
>
> Richard
>
> [1] Note: _can_, not _need_ - the opposite may also be true[/color]

I am speaking about logical sections, that are implicit in
the semantic of the C language.

int a = 488776;

This implies that somewhere in the executable, the data
488776 is stored in the binary representation specific
to the system where the translaion being done

This is the data section.

I will ignore your remarks about "toy compiler" or "toy system".
The system you are working in is obviously THE ONLY TRUE
OPERATING SYSTEM, and surely THE BEST ONE.

[jacob navia]

Re: c program binary/image in memory

Ingo Menger a écrit :[color=blue]
> jacob navia schrieb:
>
>[color=green]
>>THE LINKER.
>>
>>This part of the system is as important as the compiler actually,
>>but will never explained and never mentioned anywhere.[/color]
>
>
>
> Not true.
> On my system
> man 1 ld
> tells me a lot about the linker. Likewise,
> man 4 a.out
> tells me about the executable format and how the OS's loader treats it.
>
> But it is absolutely CERTAIN, that on other systems, the manuals will
> tell different things. On some platforms, there is even more than one
> executable format.
>
> So, this is absolutely off topic.
>[/color]
I am not speaking about linker options, I am speaking about what the
linker DOES.

Obviously the bss/text/data sections can be named differently. Microsoft
uses the terminology "segment" instead of "section", for instance.

The semantics are the same.

And I am not speaking about executable formats but about
executable parts in a logical, abstract way.

jacob

[Ingo Menger]

Re: c program binary/image in memory


jacob navia schrieb:
[color=blue]
> int a = 488776;
>
> This implies that somewhere in the executable, the data
> 488776 is stored in the binary representation specific
> to the system where the translaion being done[/color]

Not at all. It could also be, that the following code is generated:
clr
inc
inc
inc
.... and so forth 488776 times
store a

[Ingo Menger]

Re: c program binary/image in memory


jacob navia schrieb:
[color=blue]
> Ingo Menger a écrit :[/color]
[color=blue]
> I am not speaking about linker options, I am speaking about what the
> linker DOES.[/color]

ld takes one or more object files or libraries as input and combines
them to produce a single (usually executable) file. In doing so it
resolves references to external symbols, assigns final addresses to
procedures and variables, revises code and data to reflect new
addresses (a process called "relocation "), and updates symbolic debug
information when present in the file.
[color=blue]
> And I am not speaking about executable formats but about
> executable parts in a logical, abstract way.[/color]

There is no such thing as an "abstract executable part".

[jacob navia]

Re: c program binary/image in memory

Ingo Menger a écrit :[color=blue]
> jacob navia schrieb:
>
>[color=green]
>>int a = 488776;
>>
>>This implies that somewhere in the executable, the data
>>488776 is stored in the binary representation specific
>>to the system where the translaion being done[/color]
>
>
> Not at all. It could also be, that the following code is generated:
> clr
> inc
> inc
> inc
> .... and so forth 488776 times
> store a
>[/color]

And it does that each time you access that variable... How clever

And what does it when you ask for the address of that integer like
int *p=&a;
???


It is funny the contortions people will propose just to say:

"Jacob is always wrong"...

Imagine 488776 increment intstructions!! !

That system would store each integer in 400MB!

Well...

You are right Ingo :-)

jacob

[Ingo Menger]

Re: c program binary/image in memory


jacob navia schrieb:
[color=blue]
> Ingo Menger a écrit :[color=green]
> > jacob navia schrieb:
> >
> >[color=darkred]
> >>int a = 488776;
> >>
> >>This implies that somewhere in the executable, the data
> >>488776 is stored in the binary representation specific
> >>to the system where the translaion being done[/color]
> >
> >
> > Not at all. It could also be, that the following code is generated:
> > clr
> > inc
> > inc
> > inc
> > .... and so forth 488776 times
> > store a
> >[/color]
>
> And it does that each time you access that variable... How clever[/color]

Nobody said that.

[color=blue]
> And what does it when you ask for the address of that integer like
> int *p=&a;[/color]

How about
lda a
[color=blue]
> It is funny the contortions people will propose just to say:
>
> "Jacob is always wrong"...[/color]

No, I did not say that.
You said, that code like
int i = 488776;
would imply, that the number 488776 must be stored somehow in the
object file. And I showed you, that this is /not/ implied.
[color=blue]
> Imagine 488776 increment intstructions!! !
> That system would store each integer in 400MB![/color]

This is not the point. The point is that there is an unlimited number
of ways to create a given number in a memory cell. Loading a constant
that is somehow stored in the obejct file is /one possibility/, but the
C standard certainly does not require to implement it this way.
Likewise, as said before, the C standard does not require text or data
or bss segments and it also tells us nothing about "abstract executable
segments".