CSV file manipulation

[keithl]

OK... the important bit here is that I am not able to compile in an external module right now.. long story but I can't !

I have a csv file, comma delimited. Problem is that some fields which are provided as quote delimited also have commas in.

I want to parse the data line to remove commas that fall between quotes, before then splitting the line as per normal.

Could anyone give me or point me in the direction of some native perl code that will do this ?

Many thanks

Keith

[miller]

Hello Keith,

Even if you're not able to install an external module, I would still advise you to use an external package, and just install it in your local user.

cpan Text::CSV 0.01

Download the source from the above package, and put it in the subdirectory "Text" from whenever you are creating your script. Then simply use the package exactly like the examples provided.

It is generally not a good habit to get into to do this, but if you have no other choice, it still saves you from reinventing the wheel. And at the very least, you can simply look at the source and learn how you might accomplish this functionality on your own.

- Miller

[keithl]

Hi Miller,

Thanks for that - I did not realise that the modules could be used in such a way.
Does that appy for any of the generally available perl modules - specifically DBI ?

Many thanks

Keith

[miller]

Some modules are not pure perl, and require compiling in some way. Some modules are dependent on other modules, and so you need to "install" all dependencies.

In general it is a bad idea to start downloading and using modules in the way that I described, but if you have no choice, then it's an option. I would not advice you to do it for DBI, or any larger more complex modules. But something like Test::CSV is extremely simple and isn't going to be changing ever.

In my own case, I never do this, but I have root permissions on all machines that I use so I use cpan to install packages. However, I have looked at the source of certain modules to determine how they work when they did not directly solve the problems that I wanted. It's a great way to learn.

- Miller

[keithl]

Hi Miller,

One more question ! lol

When I install and run my script as you suggested I get the following message:

$ ./process_inerl01 .pl
Can't locate auto/Text/CSV/autosplit.ix in @INC (@INC contains: /usr/opt/perl5/
ib/5.8.0/aix-thread-multi /usr/opt/perl5/lib/5.8.0 /usr/opt/perl5/lib/site_perl
5.8.0/aix-thread-multi /usr/opt/perl5/lib/site_perl/5.8.0 /usr/opt/perl5/lib/si
e_perl .) at /usr/opt/perl5/lib/5.8.0/AutoLoader.pm line 158.
at Text/CSV.pm line 23

But the script still appears to run. Is this something I should be worried about at all ?

Many thanks for your help again

Keith

[miller]

Well, these are the types of issues that you run into when you do not install a module in the standard way. I personally would probably edit the module to remove the messages, but I have more experience doing that type of thing than you probably.

I'll go ahead and tell you how to do that, but if there are any further messages, I would advise you to just try to convince the sysadmin to install Text::CSV, or just ignore the messages.

Move these lines (32-34) to the end of the file

[CODE=perl]
1;

__END__
[/CODE]

Edit line 26 to equal. I removed the AutoLoader

[CODE=perl]
@ISA = qw(Exporter);
[/CODE]

And remove line 23 entirely.

[CODE=perl]
use AutoLoader qw(AUTOLOAD);
[/CODE]

Now, this should remove the warning message and still have everything work perfectly. However, I haven't tested this, so I suggest that you make a backup of the file before you start editting it.

Anyway, good luck with your project,
- Miller