Error Reading Data. Unable To Set Up The Load Function
Contents |
here for a quick overview of the site Help Center Detailed answers to any questions you error 2081: unable to setup the load function. might have Meta Discuss the workings and policies of this error 1002: unable to store alias site About Us Learn more about Stack Overflow the company Business Learn more about hiring developers org apache pig backend hadoop executionengine mapreducelayer mapreducelauncher failed or posting ads with us Stack Overflow Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community
Unable To Open Iterator For Alias
of 4.7 million programmers, just like you, helping each other. Join them; it only takes a minute: Sign up PIG/Hadoop issue: ERROR 2081: Unable to setup the load function up vote 0 down vote favorite I'm running Pig 0.13.0 and Hadoop 2.5.1, both installed from the Apache distros, they're not packages from Horton or Cloudera or anything. I'm working with a tutorial and can get it to work fine when running Pig locally ($> ./pig -x local), but when trying to run it on the Hadoop instance I get an error that I'm having a hard time researching on the internet. This command: movies = LOAD '/home/hduser/pig-tutorial-master/movies_data.csv' USING PigStorage(',') as (id,name,year,rating,duration); DUMP movies; Works fine running locally. When I run it in Hadoop/MR mode, it seems to work fine when I run the first line of code: grunt> movies = LOAD '/home/hduser/pig-tutorial-master/movies_data.csv' USING PigStorage(',') as (id,name,year,rating,duration); 2014-10-29 18:16:26,281 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS 2014-10-29 18:16:26,281 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address But when I try to $> DUMP movies it gives me this trace: grunt> dump movies 2014-10-29 18:17:15,419 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2014-10-29 18:17:15,420 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionF
Hadoop Administration for Big Data Certificate in NoSQL Databases for Big Data Advanced MS Excel with Macro, VBA and Dashboards Blog Tutorials Online Hackathons Create personalized offers for customers base... Analysis of Community Interactions using Spar... Modelling and Thinking in Graphs (Neo4J) Student http://stackoverflow.com/questions/26637863/pig-hadoop-issue-error-2081-unable-to-setup-the-load-function Portfolios Sign In Build Projects, Learn Skills, Get Hired Request Info Learn how you can build Big Data Projects I tried to load the file through PIG and getting error as below. 0 KeerthiKanth grunt> nydailyprices = LOAD '/home/cloudera/keerthi/pig_sample_data/nasdaq-sample/NASDAQ/NASDAQ_daily_prices_A.csv' https://www.dezyre.com/questions/2689/i-tried-to-load-the-file-through-pig-and-getting-error-as-below- using PigStorage(',') AS (exchange:chararray,stock_symbol:chararray,date:chararray,stock_price_open:float,stock_price_high:float,stock_price_low:float,stock_price_close:float,stock_volume:float,stock_price_adj_close:float); grunt> dump nydailyprices 2014-06-26 23:09:18,780 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN 2014-06-26 23:09:18,780 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - pig.usenewlogicalplan is set to true. New logical plan will be used. 2014-06-26 23:09:18,934 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: nydailyprices: Store(hdfs://localhost/tmp/temp490928225/tmp-869449159:org.apache.pig.impl.io.InterStorage) - scope-29 Operator Key: scope-29) 2014-06-26 23:09:18,944 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false 2014-06-26 23:09:18,977 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1 2014-06-26 23:09:18,977 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1 2014-06-26 23:09:19,041 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job 2014-06-26 23:09:19,054 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to defau
loadfunction1. This seems to require I manually make sure my code is in the localclasspath (doing REGISTER in the pig script only http://grokbase.com/t/pig/user/09acbt6vw2/illustrate-with-custom-loadfunc makes it availableremotely). Is this by design?2. I have implemented custom LoadFunc (implements SamplableLoader infact) and am getting errors when trying to use ILLUSTRATE with it.The error is:2009-10-12 11:54:08,057 [main] https://www.mathworks.com/matlabcentral/answers/41679-problem-with-load-function-for-ascii-files ERRORorg.apache.pig.pen.ExampleGenerator - Error reading data. Unable tosetup the load function.2009-10-12 11:54:08,059 [main] ERROR org.apache.pig.tools.grunt.Grunt- ERROR 2999: Unexpected internal error. Unable to setup the loadfunction.stack trace from the log file:Pig unable to Stack Trace---------------ERROR 2999: Unexpected internal error. Unable to setup the loadfunction.java.lang.RuntimeException: Unable to setup the load function.atorg.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:96)at org.apache.pig.PigServer.getExamples(PigServer.java:723)atorg.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:545)atorg.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)atorg.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)atorg.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)at org.apache.pig.Main.main(Main.java:320)It works fine when I run the job normally. Is there anything special Ineed to do in order to get it to work with ILLUSTRATE?thx,-srSam Rashsamr@ning.com reply Tweet Search Discussions Search All Groups user 4 responses Oldest Nested Alan unable to set Gates Looking at the code, the two cases I can see when it should get this error is if it fails to open the file, or the call to bindTo fails (see POLoad.setUp()). When you say it works when you run it normally does normally mean in local mode or hadoop mode? Illustrate is trying to run your function in local mode. Alan. Alan Gates at Oct 12, 2009 at 9:30 pm ⇧ Looking at the code, the two cases I can see when it should get thiserror is if it fails to open the file, or the call to bindTo fails(see POLoad.setUp()). When you say it works when you run it normallydoes normally mean in local mode or hadoop mode? Illustrate is tryingto run your function in local mode.Alan.On Oct 12, 2009, at 11:55 AM, Sam Rash wrote:Hi,I have a couple questions about using ILLUSTRATE with a custom loadfunction1. This seems to require I manually make sure my code is in thelocal classpath (doing REGISTER in the pig script only makes itavailable rem
Support Answers MathWorks Search MathWorks.com MathWorks Answers Support MATLAB Answers™ MATLAB Central Community Home MATLAB Answers File Exchange Cody Blogs Newsreader Link Exchange ThingSpeak Anniversary Home Ask Answer Browse More Contributors Recent Activity Flagged Content Flagged as Spam Help MATLAB Central Community Home MATLAB Answers File Exchange Cody Blogs Newsreader Link Exchange ThingSpeak Anniversary Home Ask Answer Browse More Contributors Recent Activity Flagged Content Flagged as Spam Help Trial software Norm Nelson (view profile) 1 question 1 answer 1 accepted answer Reputation: 0 Vote0 Problem with load function for ASCII files Asked by Norm Nelson Norm Nelson (view profile) 1 question 1 answer 1 accepted answer Reputation: 0 on 20 Jun 2012 Accepted Answer by Norm Nelson Norm Nelson (view profile) 1 question 1 answer 1 accepted answer Reputation: 0 39 views (last 30 days) 39 views (last 30 days) Hi,I'm unable to load some ASCII files in R2012a. I can open the file in the editor and confirm it's formatted properly (rectangular array of floating point numbers, comments at top of file begun with '%'), so it's not a problem with permissions. When I try to load() the files it returns empty values.the files look like this, roughly:% temp.txt 81 0.3 2.364483884 0.075 0.045 0 0 2 81 0.3 2.389637968 0.075 0.045 0 0 2 81 0.3 2.739698969 0.075 0.045 0 0 2 81 0.3 2.662140543 0.075 0.045 0 0 2 ... This is a transcript of me trying to load the file and confirm it's loaded.>> load temp.txt >> whos Name Size Bytes Class Attributes temp 0x0 0 double >> There are no error messages, as you can see, just returning an empty matrix. No idea how to proceed from here. 5 Comments Show 2 older comments Walter Roberson Walter Roberson (view profile) 27 questions 27,540 answers 9,618 accepted answers Reputation: 49,765 on 20 Jun 2012 Direct link to this comment: https://www.mathworks.com/matlabcentral/answers/41679#comment_85346 I copied that exact data; it worked for me in R2008b. per isakson per isakson (view profile) 33 questions 1,476 answers 669 accepted answers Reputation: 3,899 on 20 Jun 2012 Direct link to this comment: https://www.mathworks.com/matlabcentral/answers/41679#comment_85347 Is "% temp.txt" the first line of the file? Norm Nelson Norm Nelson (view profile) 1