splitting a string and put it into an array

[Default User]

Re: splitting a string and put it into an array

Paul Hsieh wrote:[color=blue]
>
> The C library has a totally worthless complement of string parsing
> functions.[/color]


Oh yes. It's completely impossible to create any application using the
string-parsing utilities in C.

Unless you are a competent programmer.

You know, there are *other* things besides strtok() available. It's
really not that hard to use strchr() and your own state machine for
parsing if strtok() doesn't fit your needs.

Don't get wrong, std::string has a lot of nice features, but to poo-poo
the C string capabilities portrays either a lack of experience, or lack
of skill.




Brian Rodenborn

[Peter Nilsson]

Re: splitting a string and put it into an array

Sidney Cadot <sidney@jigsaw. nl> wrote in message news:<bu4ir5$ls 1$1@news.tudelf t.nl>...[color=blue]
> Mark McIntyre wrote:
>[color=green]
> > On Wed, 14 Jan 2004 11:15:50 +0100, in comp.lang.c , Dario
> > <dario@despamme d.com> wrote:
> >
> >[color=darkred]
> >>Sean Kenwrick wrote:
> >>
> >>
> >>>Which part of his code is C++?
> >>
> >>Every part that can be successfully compiled by a C++ compiler.[/color]
> >
> > Thats so obviously wrong, it scarcely merits comment.[/color]
>
> How's that?
>
> Assume a (correct) C++ compiler. Consider any text. Assume that this
> text, when fed to the aforementioned C++ compiler, leads to "succesful
> compilation". Why is it wrong to conclude that the text is C++ code?[/color]

Nothing, so long as you're willing to accept binary streams of random
noise as valid C++ code.

I don't know about C++, but the only C construct where a translation
must fail is the #error directive. [And Dan Pop has argued that this
is still compilable since, in such circumstances, it can be considered
'unsuccessful' translation.]

--
Peter

[Jeremy Yallop]

Re: splitting a string and put it into an array

Peter Nilsson wrote:[color=blue]
> I don't know about C++, but the only C construct where a translation
> must fail is the #error directive.[/color]

In C89 an #error directive causes the issuance of a diagnostic
message; it isn't required to cause translation failure. In fact, a
conforming implementation isn't even generally /allowed/ to stop
translation on encountering #error, although there are a couple of
loopholes that can be used to justify doing so. There is at least one
implementation where an #error directive doesn't cause translation to
fail.

In C99 the implementation is not allowed to ("successfully" ) translate
programs containing active #error directives.

Jeremy.

[Sidney Cadot]

Re: splitting a string and put it into an array

Peter Nilsson wrote:[color=blue]
> Sidney Cadot <sidney@jigsaw. nl> wrote in message news:<bu4ir5$ls 1$1@news.tudelf t.nl>...
>[color=green]
>>Mark McIntyre wrote:
>>
>>[color=darkred]
>>>On Wed, 14 Jan 2004 11:15:50 +0100, in comp.lang.c , Dario
>>><dario@despa mmed.com> wrote:
>>>
>>>
>>>
>>>>Sean Kenwrick wrote:
>>>>
>>>>
>>>>
>>>>>Which part of his code is C++?
>>>>
>>>>Every part that can be successfully compiled by a C++ compiler.
>>>
>>>Thats so obviously wrong, it scarcely merits comment.[/color]
>>
>>How's that?
>>
>>Assume a (correct) C++ compiler. Consider any text. Assume that this
>>text, when fed to the aforementioned C++ compiler, leads to "succesful
>>compilation ". Why is it wrong to conclude that the text is C++ code?[/color]
>
> Nothing, so long as you're willing to accept binary streams of random
> noise as valid C++ code.[/color]

Hang on, I've lost you there... Here's a bunch of random characters:

!&^fgfctq9786h0 %(*&%h2owyp[1238n

Are you saying that a correct C++ compiler may compile this succesfully?
Same question for a correct C compiler?
[color=blue]
> I don't know about C++, but the only C construct where a translation
> must fail is the #error directive. [And Dan Pop has argued that this
> is still compilable since, in such circumstances, it can be considered
> 'unsuccessful' translation.][/color]

Ok, under the assumption that feeding

!&^fgfctq9786h0 %(*&%h2owyp[1238n

....to a correct C compiler does not have to fail, what /does/ the
standard have to say on what can be expected of a compiler on such input?


Best regards,

Sidney

[Jeremy Yallop]

Re: splitting a string and put it into an array

Sidney Cadot wrote:[color=blue]
> Hang on, I've lost you there... Here's a bunch of random characters:
>
> !&^fgfctq9786h0 %(*&%h2owyp[1238n
>
> Are you saying that a correct C++ compiler may compile this succesfully?
> Same question for a correct C compiler?[/color]

Yes (for C, at least). Also, if it is "accepted" (whatever that may
mean) by a conforming implementation then it's a conforming program.
[color=blue]
> Ok, under the assumption that feeding
>
> !&^fgfctq9786h0 %(*&%h2owyp[1238n
>
> ...to a correct C compiler does not have to fail, what /does/ the
> standard have to say on what can be expected of a compiler on such input?[/color]

The compiler must issue a diagnostic about the syntax error.

Jeremy.

[CBFalconer]

Re: splitting a string and put it into an array

Sidney Cadot wrote:[color=blue]
>[/color]
.... snip ...[color=blue]
>
> Hang on, I've lost you there... Here's a bunch of random characters:
>
> !&^fgfctq9786h0 %(*&%h2owyp[1238n
>
> Are you saying that a correct C++ compiler may compile this
> succesfully? Same question for a correct C compiler?[/color]

Offhand the !&^ and &% sequences may be a problem, but with
patience and a few macros I believe we could translate and parse
the rest :-)

--
Chuck F (cbfalconer@yah oo.com) (cbfalconer@wor ldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home .att.net> USE worldnet address!

[Paul Hsieh]

Re: splitting a string and put it into an array

Daniel Haude <haude@physnet. uni-hamburg.de> wrote:[color=blue]
> ["Followup-To:" header set to comp.lang.c.]
> On 15 Jan 2004 03:58:59 -0800,
> Paul Hsieh <qed@pobox.co m> wrote
> in Msg. <796f488f.04011 50358.68aed839@ posting.google. com>
>[color=green]
> > The C library has a totally worthless complement of string parsing
> > functions.[/color]
>
> As far as I'm aware of it only has one function that deserves the name
> "parsing function", which is sscanf().[/color]

An interesting point of view -- considering that sscanf is probably
the least generic of all the parsing mechanisms. Actually strtok
*would* be a good function, if only it were more like strtok_r as is
implemented in Linux.
[color=blue]
> Anyway, it's simple to roll your own.[/color]

And even simpler to get it wrong, or create an inadequate solution as
we shall see.
[color=blue]
> [...] For delimiter-separated text data I always use this function:
>
> char **dh_splitstrin g(char *line, int maxfields, int sep)
> {
> char *a, *e, *s;
> int i;
> char **arr;
>
> if (NULL == (arr = malloc((maxfiel ds+1) * sizeof *arr))) return NULL;
>
> e = line;
> for (i = 0; i < maxfields; i++) {
> /* skip leading WS */
> for (s = e; *s && isspace(*s) && *s != sep; s++) ;
> if (!*s) break;
> for (a = s; *a && *a != sep; a++) ;
> e = *a == sep ? a+1 : a;
> /* chop off trailing WS */
> for (a--; isspace(*a) && a > s; a--) ;
> if (i < maxfields-1) *++a = 0;
> arr[i] = s;
> }
> for ( ; i < maxfields+1; i++) {
> arr[i] = NULL;
> }
> return arr;
> }[/color]

A reasonably effort, but it betrays the limited scope that is very
typical of C programmers.

1) This code is complicated. You have several loops, and state
machines, and I have difficulty knowing how to know that this code is
correct. In the middle there you've hidden a nice "a > s" comparison
.... I didn't know you could compare pointers like that in a portable
way.

2) The code only shows the inner loop -- the original request asks for
filling a two dimensional array.

3) It encodes the classic C worthlessness of requiring that you
specify the size of your containers (array) up front, before you know
the size of the data you require. The original poster also posted to
C++ -- using an STL vector to hold the result is probably the right
answer.
[color=blue]
> line: The string to be parsed[/color]

With or without the '\n' on the end? And do you compensate for a
potential '\r' in there? Are you doing the typical C thing of pushing
issues and complexities upward?
[color=blue]
> maxfields: max number of fields[/color]

Its also the minimum malloc size. BTW, what if you decide to pass a
negative value for maxfields?
[color=blue]
> sep: field separator char (i.e., '\t')[/color]

tabs but not spaces? Tabs/spaces are chosen because of their human
readability/editability features. I means if you use them as
seperators you use them interchangeably , in arbitrary numbers also
with the possibility that one or the other might not be present. A
quick look at your algorithm makes it look like it will fail if one
pair of entries it only separated by spaces.
[color=blue]
> Returns: A newly-allocated, NULL-terminated array of pointers to strings
> with maxfields+1 elements. All fields have leading and trailing whitespace
> removed.
>
> Warning: Modifies line. If this is not what you want, pass a copy.[/color]

Which makes it only barely better than strtok ...
[color=blue]
> The returned pointers point into different positions of 'line', so 'line'
> must not be freed or modified as long as you want to use the result.[/color]

Ok ... this is another classic case of pushing up complexity upwards.
[color=blue]
> Multiple separator tokens are not lumped together, but result in empty
> tokens ("\0"). This makes using tab-separated data files dangerous because
> there may be different numbers of tabs between columns. I always use
> semicolons.[/color]

Ok, I'm not sure the OP was asking that you change the policy they
have decided upon. Using arbitrary space or tab delimiters allows for
easy human editability, which might be a fairly high concern. For
humans its easy to miss a ";" and its also easy to confuse one for a
":".

--
Paul Hsieh

http://www.pobox.com/~qed/

http://bstring.sf.net/

[Paul Hsieh]

Re: splitting a string and put it into an array

Default User <first.last@boe ing.com.invalid > wrote:[color=blue]
> Paul Hsieh wrote:[color=green]
> > The C library has a totally worthless complement of string parsing
> > functions.[/color]
>
> Oh yes. It's completely impossible to create any application using the
> string-parsing utilities in C.[/color]

I never said that. However, attempting to do so ends up contributing
to the coffers of Security Focus, CERT, Reasoning, and other late/post
production software failure analysis companies.
[color=blue]
> Unless you are a competent programmer.[/color]

With a masochistic bent, of course. For this particular task its
actually best to completely leave the entire C string library behind
-- it just doesn't offer anything useful that can't be done better
using some other way.
[color=blue]
> You know, there are *other* things besides strtok() available. It's
> really not that hard to use strchr() and your own state machine for
> parsing if strtok() doesn't fit your needs.[/color]

Actually, strcspn(), and strspn() would probably be the better choices
if you really wanted to push it. But the point is that if you are
forced to implement your own state machines to hand hold the parsing
anyway, then you might as well roll your own right down to the raw
characters. A big for-switch statement with a few counters and flags
-- it will be just as readable.

Compare this to my Bstrlib based solution -- can you find a state
machine of any kind in there? If anywhere, it would be hidden away in
the thoroughly tested Bstrlib functions and is implemented in a
generically usable manner.
[color=blue]
> Don't get wrong, std::string has a lot of nice features, but to poo-poo
> the C string capabilities portrays either a lack of experience, or lack
> of skill.[/color]

As demonstrated by your impressive proposal to solve the OP's problem.

--
Paul Hsieh

http://www.pobox.com/~qed/

http://bstring.sf.net/

[pete]

Re: splitting a string and put it into an array

Paul Hsieh wrote:[color=blue]
>
> Default User <first.last@boe ing.com.invalid > wrote:[color=green]
> > Paul Hsieh wrote:[color=darkred]
> > > The C library has a totally worthless complement of string parsing
> > > functions.[/color]
> >
> > Oh yes. It's completely impossible to
> > create any application using the
> > string-parsing utilities in C.[/color]
>
> I never said that. However, attempting to do so ends up contributing
> to the coffers of Security Focus, CERT, Reasoning, and other late/post
> production software failure analysis companies.
>[color=green]
> > Unless you are a competent programmer.[/color]
>
> With a masochistic bent, of course. For this particular task its
> actually best to completely leave the entire C string library behind
> -- it just doesn't offer anything useful that can't be done better
> using some other way.
>[color=green]
> > You know, there are *other* things besides strtok() available. It's
> > really not that hard to use strchr() and your own state machine for
> > parsing if strtok() doesn't fit your needs.[/color]
>
> Actually, strcspn(), and strspn() would probably be the better choices
> if you really wanted to push it.[/color]

There's
my head.

There's a beat in my head.

/* BEGIN str_tok_r.c */

#include <stdio.h>

char *str_tok_r (char *, const char *, char **);
size_t str_spn (const char *, const char *);
size_t str_cspn (const char *, const char *);
char *str_chr (const char *, int);
char *str_cpy(char *, const char *);
char *squeeze (char *, const char *);

#define STRING "\tThere's\ n a\r beat in \r\tmy head. \n"

int main(void)
{
const char *const original = STRING;
char s1[sizeof STRING];

puts(original);
puts(squeeze(st r_cpy(s1, original), "\n\r\t"));
return 0;
}

char *str_tok_r(char *s1, const char *s2, char **p1)
{
if (s1 != NULL) {
*p1 = s1;
}
s1 = *p1 + str_spn(*p1, s2);
if (*s1 == '\0') {
return NULL;
}
*p1 = s1 + str_cspn(s1, s2);
if (**p1 != '\0') {
*(*p1)++ = '\0';
}
return s1;
}

size_t str_spn(const char *s1, const char *s2)
{
const char *const p1 = s1;

while (*s1 != '\0' && str_chr(s2, *s1) != NULL) {
++s1;
}
return s1 - p1;
}

size_t str_cspn(const char *s1, const char *s2)
{
const char *const p1 = s1;

while (str_chr(s2, *s1) == NULL) {
++s1;
}
return s1 - p1;
}

char *str_chr(const char *s, int c)
{
while (*s != (char)c) {
if (!*s) {
return NULL;
}
++s;
}
return (char *)s;
}

char *str_cpy(char *s1, const char *s2)
{
char *const p1 = s1;

do {
*s1 = *s2++;
} while (*s1++ != '\0');
return p1;
}

char *squeeze(char *s1, const char *s2)
{
char *p3;
char const *p2;
char *const p1 = s1;

p2 = str_tok_r(s1, s2, &p3);
while (p2 != NULL) {
while (*p2 != '\0') {
*s1++ = *p2++;
}
p2 = str_tok_r(NULL, s2, &p3);
}
*s1 = '\0';
return p1;
}

/* END str_tok_r.c */

--
pete

[Daniel Haude]

Re: splitting a string and put it into an array

["Followup-To:" header set to comp.lang.c.]
On 16 Jan 2004 02:50:32 -0800,
Paul Hsieh <qed@pobox.co m> wrote
in Msg. <796f488f.04011 60250.7481c22@p osting.google.c om>
[color=blue]
> And even simpler to get it wrong, or create an inadequate solution as
> we shall see.[/color]

I've never claimed that my solution was adequate in that it solved the
OP's problem by 100%.
[color=blue]
> 1) This code is complicated.[/color]

It's a mere 23 lines. You want to see complicated code?
[color=blue]
> You have several loops,[/color]

Just two, non-nested, and only one of them non-trivial.
[color=blue]
> and state machines,[/color]

zero state machines
[color=blue]
> and I have difficulty knowing how to know that this code is
> correct.[/color]

Like with any third-party function you've got to either trust the
documentation or write your own.
[color=blue]
> In the middle there you've hidden a nice "a > s" comparison
> ... I didn't know you could compare pointers like that in a portable
> way.[/color]

You can as long as they point into the same array.
[color=blue]
> 2) The code only shows the inner loop -- the original request asks for
> filling a two dimensional array.[/color]

Again, I never claimed to've been trying to solve the OP's problem. I was
merely giving an example of how to parse a string in C.
[color=blue]
> 3) It encodes the classic C worthlessness of requiring that you
> specify the size of your containers (array) up front, before you know
> the size of the data you require.[/color]

Easily fixed by a reallocing mechanism that I didn't bother with. The
function I gave is specifically geared to parsing csv tables where the
number of columns is usually known.
[color=blue]
> The original poster also posted to
> C++ -- using an STL vector to hold the result is probably the right
> answer.[/color]

In C++ it would be.
[color=blue][color=green]
>> line: The string to be parsed[/color]
>
> With or without the '\n' on the end? And do you compensate for a
> potential '\r' in there?[/color]

The "doc" (my comments) states that leading and trailing whitespace gets
chopped off all tokens.
[color=blue]
> Are you doing the typical C thing of pushing
> issues and complexities upward?[/color]

No, but you're doing the typical thing of not reading the documentation.
[color=blue]
> Its also the minimum malloc size. BTW, what if you decide to pass a
> negative value for maxfields?[/color]

UB, obviously. Trivially fixed with a single line. A bug.
[color=blue][color=green]
>> sep: field separator char (i.e., '\t')[/color]
>
> tabs but not spaces? Tabs/spaces are chosen because of their human
> readability/editability features.[/color]

Yes, but it's difficult to parse tables that contain empty cells or
elements with whitespace in them.
[color=blue]
> I means if you use them as
> seperators you use them interchangeably , in arbitrary numbers also
> with the possibility that one or the other might not be present.[/color]
[color=blue]
> A
> quick look at your algorithm makes it look like it will fail if one
> pair of entries it only separated by spaces.[/color]

You're right: My algorithm gets tripped up when anything that's
isspace() is used a s separator.
[color=blue][color=green]
>> Warning: Modifies line. If this is not what you want, pass a copy.[/color]
>
> Which makes it only barely better than strtok ...[/color]

It's re-entrant, and in the usual file-parsing situation (reading csv
data) each line is typically only used once.

I specifically didn't want to allocate additional memory for the field's
contents in the function because it's 1) an unnecessary waste of memory
and performance most of the time, and 2) it's easily provided by 2 extra
standard function calls outside my routine.
[color=blue]
> Ok ... this is another classic case of pushing up complexity upwards.[/color]

It's better to push complexity upwards than implementing it downstairs
where it, although unneeded in most cases, may lead to resource and
performance penalties. Especially when the "complexity " involves nothing
but one call to strdup() and another one to free().
[color=blue]
> Ok, I'm not sure the OP was asking that you change the policy they
> have decided upon.[/color]

Hey, I have better things to do than writing code for other people. All I
did was give a simple example of how to do efficient string parsing with a
few lines in C.

--Daniel

--
"With me is nothing wrong! And with you?" (from r.a.m.p)

[Default User]

Re: splitting a string and put it into an array

Paul Hsieh wrote:[color=blue]
>
> Default User <first.last@boe ing.com.invalid > wrote:[color=green]
> > Paul Hsieh wrote:[color=darkred]
> > > The C library has a totally worthless complement of string parsing
> > > functions.[/color]
> >
> > Oh yes. It's completely impossible to create any application using the
> > string-parsing utilities in C.[/color]
>
> I never said that.[/color]

Yes, you did, you said the string parsing functions were worthless. You
can't use worthless things to create things of worth. I would by
"tricky" or "able to blow your foot off", but not "worthless.
[color=blue]
> However, attempting to do so ends up contributing
> to the coffers of Security Focus, CERT, Reasoning, and other late/post
> production software failure analysis companies.[/color]

Nonsense. You are saying that any attempt to use them, no matter the
dedication and skill level of the programmer, will fail. That is
manifestly untrue. While there have been products with problems, there
are products without problems as well.
[color=blue][color=green]
> > Unless you are a competent programmer.[/color]
>
> With a masochistic bent, of course.[/color]

If you are working in C, then the choices are limited. While using these
functions takes some time to learn, once you become comfortable it's not
particularly onerous.
[color=blue]
> For this particular task its
> actually best to completely leave the entire C string library behind
> -- it just doesn't offer anything useful that can't be done better
> using some other way.[/color]

That's your opinion, not one that I share. I've been using these
functions for a long time, and find them to be useful. For C++
programming, I would (and do) use std::string.
[color=blue][color=green]
> > You know, there are *other* things besides strtok() available. It's
> > really not that hard to use strchr() and your own state machine for
> > parsing if strtok() doesn't fit your needs.[/color]
>
> Actually, strcspn(), and strspn() would probably be the better choices
> if you really wanted to push it. But the point is that if you are
> forced to implement your own state machines to hand hold the parsing
> anyway, then you might as well roll your own right down to the raw
> characters. A big for-switch statement with a few counters and flags
> -- it will be just as readable.[/color]

No, go the other way. Encapsulate this and make your own version of
strtok() that is safe. Then you have that in your personal library. Or
find one of the many ones already available.
[color=blue][color=green]
> > Don't get wrong, std::string has a lot of nice features, but to poo-poo
> > the C string capabilities portrays either a lack of experience, or lack
> > of skill.[/color]
>
> As demonstrated by your impressive proposal to solve the OP's problem.[/color]

I made no attempt to solve said problem. The "problem" I was addressing
was your post.




Brian Rodenborn

[Jerry Coffin]

Re: splitting a string and put it into an array

In article <bu2tdu$cumc3$1 @ID-189932.news.uni-berlin.de>,
kai.jaensch@gmx .de says...[color=blue]
> Hello,
>
> i am an newbie and i have to to solve this problem as fast as i can. But
> at this time i don´t have a lot of success.
> Can anybody help me (and understand my english :-))?
>
> I have a .txt-file in which the data is structured in that way:
> Project-Nr. ID name lastname
> 33 9 Lars Lundel
> 33 12 Emil Korla
> 34 19 Lara Keuler
> 33 13 Thorsten Lammert
>
> These data have to be read out row by row.
> Every row has to be splitted (delimiter is TAB) and has to be saved in[/color]

Under the circumstances, I would NOT use a 2D array -- instead, I'd use
a struct (or in C++ a class). I'd then create an array of those structs
(or in C++, a map or perhaps a set).

I'm not going to post code since you've cross-posted to c.l.c and
c.l.c++, and any code that's well-written for one will be off-topic in
the other.

--
Later,
Jerry.

The universe is a figment of its own imagination.