Regex needed for splitting on commas (not inside quotes)

[conspireagainst]

I'm having quite a time with this particular problem:

I have users that enter tag words as form input, let's say for a photo or a topic of discussion. They are allowed to delimit tags with spaces and commas, and can use quotes to encapsulate multiple words.

An example:

tag1, tag2 tag3, "tag4 tag4, tag4" tag5, "tag6 tag6"

So, as we can see here anything is allowed, but the problem is that splitting on commas obviously destroys tag4 (the tag inside quotes but with a comma).

I've tried tons of regex, using preg_split, preg_match, and more but cannot figure out a solution. If this cannot be done, then it's also completely okay to tell the user they are not allowed to use commas inside of quoted strings via error message - which I tried doing using preg_match but my regex fails.

My regex for preg_match to alert the user of the problem:

/".*,.*"/

This works fine, but if you put any other quotes after the first quoted string (referring to "tag6 tag6") then it fails when it should not, since there are no commas inside tag6.

[bergy]

I'm no regex guru by any means, but here is something to try in case you haven't thought of thise already. Change your example to this:

tag1, tag2 tag3, \"tag4 tag4, tag4\" tag5, \"tag6 tag6\"

And see if that works in your expression. If so just do a addslashes() function on the tag string before you pass it to your regex.

I'm having quite a time with this particular problem:

I have users that enter tag words as form input, let's say for a photo or a topic of discussion. They are allowed to delimit tags with spaces and commas, and can use quotes to encapsulate multiple words.

An example:

tag1, tag2 tag3, "tag4 tag4, tag4" tag5, "tag6 tag6"

So, as we can see here anything is allowed, but the problem is that splitting on commas obviously destroys tag4 (the tag inside quotes but with a comma).

I've tried tons of regex, using preg_split, preg_match, and more but cannot figure out a solution. If this cannot be done, then it's also completely okay to tell the user they are not allowed to use commas inside of quoted strings via error message - which I tried doing using preg_match but my regex fails.

My regex for preg_match to alert the user of the problem:

/".*,.*"/

This works fine, but if you put any other quotes after the first quoted string (referring to "tag6 tag6") then it fails when it should not, since there are no commas inside tag6.

[conspireagainst]

That doesn't seem to make a difference in my particular case. Thanks for the reply, though! Any other thoughts?

I'm no regex guru by any means, but here is something to try in case you haven't thought of thise already. Change your example to this:

tag1, tag2 tag3, \"tag4 tag4, tag4\" tag5, \"tag6 tag6\"

And see if that works in your expression. If so just do a addslashes() function on the tag string before you pass it to your regex.

[bergy]

That doesn't seem to make a difference in my particular case. Thanks for the reply, though! Any other thoughts?

Well this is really dirty, but... what if you do a simple str_replace on quotes in the string (before regex) something like replacing the " with ~ or even ___QUOTE___, do your regex and then re-replace the ~ or ___QUOTE___ with " when you're done. Definitely not the best or fastest solution, but it should work.

[pbmods]

Here's what I came up with:

[code=php]
$str = 'tag1, tag2 tag3, "tag4 tag4, tag4" tag5, "tag6 tag6"';
print("$str<br /><br />\n\n");

$matches = preg_split('/(?<!\\\\)"/', $str);

$tags = array();

foreach($matche s as $match) {
$innerTags = preg_split('/(?<!\\\\),/', $match);
foreach($innerT ags as $tag)
if(preg_match('/\w/', $tag))
$tags[] = trim($tag);
}

var_dump($tags) ;
[/code]

First, we split the string by '"', unless it's escaped (preceded by '\'). Now we know where the '"'s are.

Next, we run through each set of tags in between '"'s and split those by unescaped ',' (similarly to '\"', '\,' won't get split). Now we know where the individual tags are; all we have to do now is clean up.

We run through each potential tag and check to see if it has at least one alphanumeric character. If it does, we add it to $tags, sans leading/trailing whitespace.

This might not be the best way to do it; I whipped it up in about 10 minutes. But it works.

[EDIT: If you have magic_quotes turned on, you'll need to run stripslashes on your input first.]

[conspireagainst]

I have finally found a way to do what is needed, based on what everyone has said so far. It is complicated, that much is true, but it works. The last solution posted did not work still, so I am forgetting Regex for now and simply using PHP's string functions.

Test data: (string)$tag_li st = tag1, tag2, "tag3 tag3, tag3" tag4, "tag5, tag5", tag6 tag7

[PHP]
// ensure there is at least one beginning and one ending quote
if(strpos($tag_ list, '"') !== false && strrpos($tag_li st, '"', 1) !== false) {
$quotes_found = true; // initialize the while loop
// find the position of the last quote
$last_quote_pos = strrpos($tag_li st, '"', 1);
// initialize so that the first time we try and find the beginning quote we start from position 0
$end_quote_pos = 0;
// we go through the whole string until we reach the last quote
while($quotes_f ound) {
// find the position of the beginning quote for this tag string
$begin_quote_po s = strpos($tag_lis t, '"', ($end_quote_pos + 1));
// find the position of the end quote for this tag string
$end_quote_pos = strpos($tag_lis t, '"', ($begin_quote_p os + 1));
// have we reached the last quote?
if($begin_quote _pos == $last_quote_pos ||
$end_quote_pos == $last_quote_pos ) {
$quotes_found = false; // set the while loop to stop after this
}
// split out the quoted tag string
$original_tag = substr($tag_lis t, $begin_quote_po s, ($end_quote_pos - $begin_quote_po s + 1));
// remove commas from the inside of the quoted string (this is the whole point of this entire if/while combo)
$fixed_tag = strtr($original _tag, ',', ' ');
// replace the quoted string tags with the fixed ones (sans commas)
$tag_list = strtr($tag_list , array($original _tag=>$fixed_ta g));
} // end while going through the string with quotes
} // end making sure there is at least one beginning and end quote
[/PHP]

End result: (string)$tag_li st = tag1, tag2, "tag3 tag3 tag3" tag4, "tag5 tag5", tag6 tag7

I can now do what was originally sought out, which is to use explode() on $tag_list commas and/or spaces, giving me what I originally needed - which is a list of the tags in an array!

[pbmods]

Heh. Oops. My code doesn't check to see if the comma is inside of quotes (which was the whole point).

Well at any rate, I'm glad you found a solution for your problem.

[pbmods]

I couldn't deal with the fact that I couldn't figure this out, so I gave it one more go, and I think I've got it this time:

[code=php]
<?php
$str = 'tag1, tag2 tag3, "tag4 tag4, tag4" tag5, "tag6 tag6"';
print("$str<br /><br />\n\n");

$matches = preg_split('/(?<!\\\\)"/', $str);

$tags = array();

foreach($matche s as $idx => $match) {
if($idx % 2)
$tags[] = stripslashes(tr im($match));
else {
$innerTags = preg_split('/(?<!\\\\),/', $match);
foreach($innerT ags as $tag)
if(preg_match('/\w/', $tag))
$tags[] = stripslashes(tr im($tag));
}
}

var_dump($tags) ;
?>
[/code]

The only difference is that we directly copy odd-index matches instead of parsing them for commas.

Think about it. When you split by quote, matches 0, 2, 4, etc. are outside of the quotes, but matches 1, 3, 5 are inside. So matches 1, 3 and 5 should not be parsed, but matches 0, 2 and 4 should. If you don't believe me, var_dump $matches.

Also, I added stripslashes so that if you escape a quote or comma, it doesn't print the slash.

Here's what I get for output:

tag1, tag2 tag3, "tag4 tag4, tag4" tag5, "tag6 tag6"

array(5) {
  [0]=>
  string(4) "tag1"
  [1]=>
  string(9) "tag2 tag3"
  [2]=>
  string(15) "tag4 tag4, tag4"
  [3]=>
  string(4) "tag5"
  [4]=>
  string(9) "tag6 tag6"
}

[EDIT: If you want to split tags by space in addition to comma, use this regex instead:

[code=php]$innerTags = preg_split('/(?<!\\\\)((,\s* )|((?<!,)\s+))/', $match);[/code]

This splits by either comma (with optional following whitespace character[s]) or else just a whitespace character[s] (with a negative lookbehind so that we don't end up catching '\,' properly only to then split at the following whitespace character).

This will result in the following output:

tag1, tag2 tag3, "tag4 tag4, tag4" tag5, "tag6 tag6"

array(6) {
  [0]=>
  string(4) "tag1"
  [1]=>
  string(4) "tag2"
  [2]=>
  string(4) "tag3"
  [3]=>
  string(15) "tag4 tag4, tag4"
  [4]=>
  string(4) "tag5"
  [5]=>
  string(9) "tag6 tag6"
}

Or try this on for size:

tag1, tag2\ tag3, "tag4 tag4, tag4" tag5\, \"tag6 tag6\"

array(5) {
  [0]=>
  string(4) "tag1"
  [1]=>
  string(9) "tag2 tag3"
  [2]=>
  string(15) "tag4 tag4, tag4"
  [3]=>
  string(11) "tag5, "tag6"
  [4]=>
  string(5) "tag6""
}

]

[EDIT EDIT: The one big flaw in these regexes is that they don't check for escaped slashes (e.g., \\"). But you know what? My brain hurts. You go figure it out. o_O]

[EDIT EDIT EDIT: Nevermind. I got that working, too:

[code=php]
<?php
$str = 'tag1, tag2\ tag3, "tag4 tag4, tag4" tag5\\\\, "tag6 tag6\"';
print("$str<br /><br />\n\n");

$matches = preg_split('/((?<!\\\\)|(?<= \\\\\\\\))"/', $str);

$tags = array();

foreach($matche s as $idx => $match) {
if($idx % 2)
$tags[] = stripslashes(tr im($match));
else {
$innerTags = preg_split('/((?<!\\\\)|(?<= \\\\\\\\))((,\s *)|((?<!,)\s))/', $match);
foreach($innerT ags as $tag)
if(preg_match('/\w/', $tag))
$tags[] = stripslashes(tr im($tag));
}
}

var_dump($tags) ;
?>
[/code]

The major difference is that we will split if the quote or comma and/or/xor space is either not preceded by a slash or is preceded by two slashes.

Sample output (note that the comma after tag5 gets parsed properly):

tag1, tag2\ tag3, "tag4 tag4, tag4" tag5\\, "tag6 tag6\"

array(5) {
  [0]=>
  string(4) "tag1"
  [1]=>
  string(9) "tag2 tag3"
  [2]=>
  string(15) "tag4 tag4, tag4"
  [3]=>
  string(5) "tag5\"
  [4]=>
  string(10) "tag6 tag6""
}

This is why pbmods should not be stuck in his apartment on a national holiday.]

[conspireagainst]

This is why pbmods should not be stuck in his apartment on a national holiday.

Wow, that's a lot of work you did! That code is probably better than what I have and accounts for things that I hope users won't do but I'm sure they will.

Thanks!

[pbmods]

Thanks!

You are welcome.

And don't worry about me; my templating engine needed an overhaul anyway ~_^