How To Handle Out Of Memory Error In Perl
Contents |
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss out of memory error while running perl script the workings and policies of this site About Us Learn more about perl out of memory windows Stack Overflow the company Business Learn more about hiring developers or posting ads with us Stack Overflow Questions
Perl Out Of Memory Reading Large File
Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of 4.7 million programmers, just like you, helping each other.
How To Solve Out Of Memory Error In Perl
Join them; it only takes a minute: Sign up Why does my Perl script die with an “out of memory” exception? up vote 1 down vote favorite I need to read a 200mb "space"-separated file line-by-line and collect its contents into an array. Every time I run the script, Perl throws an "out of memory" exception, but I perl ulimit don't understand why! Some advice please? #!/usr/bin/perl -w use strict; use warnings; open my $fh, "<", "../cnai_all.csd"; my @parse = (); while (<$fh>) { my @words = split(/\s/,$_); push (@parse, \@words); } print scalar @parse; the cnai file looks like this: it contains 11000 rows and 4200 values, seperated by "space", per line. VALUE_GROUP_A VALUE_GROUP_B VALUE_GROUP_C VALUE_GROUP_A VALUE_GROUP_B VALUE_GROUP_C VALUE_GROUP_A VALUE_GROUP_B VALUE_GROUP_C VALUE_GROUP_A VALUE_GROUP_B VALUE_GROUP_C The code above is just a stripped down sample.The final script will store all values in a hash and write it to a database later . But first, I have to solve that memory problem! database perl memory share|improve this question edited Aug 17 '10 at 2:21 Greg Bacon 75.6k17147197 asked Feb 4 '10 at 16:45 Floopy-Doo 63117 Code said while (<$fh>) but it was not displayed in the markup. –mob Feb 4 '10 at 16:50 @floppy-doo Please edit your question to give us an idea what the contents of cnai_all.csd look like? –Greg Bacon Feb 4 '10 at 16:55 See also: stackoverflow.com/qu
Q&A Tutorials Poetry RecentThreads NewestNodes Donate What'sNew on Feb 07, 2007 at 04:23UTC ( #598685=perlquestion: print w/replies, xml ) Need Help?? jesuashok has asked for the wisdom of the
Perl Memory Usage
Perl Monks concerning the following question: Dear monks, I have a perl script which is trying to read some files and storing those lines into a hash. But each file size comes around 3MB or more. when I run the script I got the following error [belief] /apps/inst2/metrica/analysis_ericsson/schema_analysis> perl s +chema_analysis.pl -r rename_columns -u /apps/inst2/metrica/anthony/Er http://stackoverflow.com/questions/2201432/why-does-my-perl-script-die-with-an-out-of-memory-exception +icson_R10_Onsite/VFOZ_BACKUP/summaryspr/ -y /apps/inst2/metrica/anth +ony/Ericson_R10_Onsite/VFOZ_BACKUP/metalayer/ \ > -o only_in_old_schema -p /apps/inst2/metrica/anthony/Ericson_R10_Ons +ite/VFOZ_BACKUP/reportspr/ Out of memory! [download] Is there any way, can I resolve this Issue. Is it possible to control the memory usage. Comment on Out of memory!Download Code Replies are listed 'Best First'. Re: Out of memory! by GrandFather (Sage) on Feb 07, 2007 at http://www.perlmonks.org/?node_id=598685 04:29UTC Is there any way you can post sample code that demonstrates the issue and indicate how many files are being manipulated? We need a little more information than "I have a problem with large hashes. How do I solve it?" if you want a better answer than "Use a tied hash" or "Install more memory". DWIM is Perl's answer to Gödel [reply] Re: Out of memory! by Tanktalus (Canon) on Feb 07, 2007 at 04:55UTC Generally speaking, when I've had that problem, it was that there was a memory limit via ulimit - removing that limit 'solved' the problem. At least, insofaras letting me use way more memory. Perhaps you have an underlying memory problem - wasting memory or leaking it, we can't be sure from your description. Assuming that not to be the case, though, it's probably a ulimit on memory.[reply] Re: Out of memory! by chargrill (Parson) on Feb 07, 2007 at 04:55UTC "Doctor, it hurts when I do this!" "T
PERL Beginners I wrote a small script that uses message ID's as unique values and extracts recipient address info. The goal is to count 1019 events per message ID. It also gets the sum of recipients per message ID. http://www.justskins.com/forums/out-of-memory-error-115248.html The script works fine but when it runs against a very large file (2GB+) I receive an out of memory error. Is there a more efficient way of handling the hash portion that is less memory intense and preferably faster? --Paul # Tracking log pr use strict; my $recips; my %event_id; my $counter; my $total_recips; my $count; # Get log file die "You ... Thread Tools Show Printable Version Email this Page… Subscribe to this out of Thread… Display Linear Mode Switch to Hybrid Mode Switch to Threaded Mode December 16th,07:15 PM #1 Out of memory error problem I wrote a small script that uses message ID's as unique values and extracts recipient address info. The goal is to count 1019 events per message ID. It also gets the sum of recipients per message ID. The script works fine but when it runs against a very large file (2GB+) I receive an out out of memory of memory error. Is there a more efficient way of handling the hash portion that is less memory intense and preferably faster? --Paul # Tracking log pr use strict; my $recips; my %event_id; my $counter; my $total_recips; my $count; # Get log file die "You must enter a tracking log. \n" if $#ARGV <0; my $logfile = shift; open (LOGFILE, $logfile) || die "Unable to open $logfile because\n $!\n"; foreach (