Issues with the json serde. #1
Description
Hi,
I'm using the json serde in hive for parsing another set of json files I have from valentines day. I noticed that there is no option to ignore malformed json, and there seems to be some problems with deserializing all json.
This tweet is causing the error:
{"text":"@KimKardashian happy valentines day, hope it's a good one","retweet_count":0,"geo":{"type":"Point","coordinates":[38.7313358,-108.05278695]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":25365536,"source":"\u003Ca href="http://twitter.com/download/android" rel="nofollow"\u003ETwitter for Android\u003C/a\u003E","in_reply_to_user_id_str":"25365536","id_str":"169483808003989505","entities":{"user_mentions":[{"indices":[0,14],"screen_name":"KimKardashian","id_str":"25365536","name":"Kim Kardashian","id":25365536}],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/6a7e7dbf9d6c7ac4.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Delta, CO","bounding_box":{"type":"Polygon","coordinates":[[[-108.104644,38.71503],[-108.021863,38.71503],[-108.021863,38.769794],[-108.104644,38.769794]]]},"name":"Delta","id":"6a7e7dbf9d6c7ac4","country":"United States"},"in_reply_to_screen_name":"Ki{"text":"@bbrandivirgo too bad I dont have the number. Happy valentines day tho :)","retweet_count":0,"geo":{"type":"Point","coordinates":[33.77406404,-84.39270512]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"source":"\u003Ca href="http://mobile.twitter.com" rel="nofollow"\u003EMobile Web\u003C/a\u003E","in_reply_to_user_id_str":null,"id_str":"169497701241716736","entities":{"user_mentions":[],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/8173485c72e78ca5.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Atlanta, GA","bounding_box":{"type":"Polygon","coordinates":[[[-84.54674,33.647908],[-84.289389,33.647908],[-84.289389,33.887618],[-84.54674,33.887618]]]},"name":"Atlanta","id":"8173485c72e78ca5","country":"United States"},"in_reply_to_screen_name":null,"favorited":false,"truncated":false,"created_at":"Tue Feb 14 19:06:15 +0000 2012","contributors":null,"user":{"contributors_enabled":false,"profile_background_image_url":"http://a3.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","url":"http://facebook.com/cperk3","profile_link_color":"0084B4","followers_count":773,"profile_image_url":"http://a3.twimg.com/profile_images/1792490671/000011110000_normal.jpg","default_profile_image":false,"show_all_inline_media":true,"statuses_count":3271,"profile_background_color":"C0DEED","description":"Ga Tech Athlete-Student.. Black&Samoan...Follow me as I follow Jesus-","location":"Atlanta, GA","profile_background_tile":true,"favourites_count":1,"profile_background_image_url_https":"https://si0.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","time_zone":"Quito","profile_sidebar_fill_color":"DDEEF6","screen_name":"Cpeezy21","id_str":"312682111","lang":"en","geo_enabled":true,"profile_image_url_https":"https://si0.twimg.com/profile_images/1792490671/000011110000_normal.jpg","verified":false,"notifications":null,"profile_sidebar_border_color":"04080a","protected":false,"listed_count":5,"created_at":"Tue Jun 07 14:14:34 +0000 2011","name":"Charles Perkins III","is_translator":false,"follow_request_sent":null,"following":null,"profile_use_background_image":true,"friends_count":223,"id":312682111,"default_profile":false,"utc_offset":-18000,"profile_text_color":"333333"},"retweeted":false,"id":169497701241716736,"coordinates":{"type":"Point","coordinates":[-84.39270512,33.77406404]}}
I'm getting this error when processing some sample twitter data:
2012-09-26 15:15:39,059 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2012-09-26 15:15:39,215 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
2012-09-26 15:15:39,372 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: /mapred/local/taskTracker/distcache/-624804405132306423_-2027207125_45603557/hadoop1.domain.com/tmp/hive-root/hive_2012-09-26_15-15-33_715_8669028640552125101/-mr-10004/af319f96-99f0-4f06-8fba-3fbf5b880148 <- /mapred/local/taskTracker/root/jobcache/job_201209252321_0010/attempt_201209252321_0010_m_000000_0/work/HIVE_PLANaf319f96-99f0-4f06-8fba-3fbf5b880148
2012-09-26 15:15:39,380 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /mapred/local/taskTracker/root/jobcache/job_201209252321_0010/jars/job.jar <- /mapred/local/taskTracker/root/jobcache/job_201209252321_0010/attempt_201209252321_0010_m_000000_0/work/job.jar
2012-09-26 15:15:39,388 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /mapred/local/taskTracker/root/jobcache/job_201209252321_0010/jars/.job.jar.crc <- /mapred/local/taskTracker/root/jobcache/job_201209252321_0010/attempt_201209252321_0010_m_000000_0/work/.job.jar.crc
2012-09-26 15:15:39,451 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
2012-09-26 15:15:39,452 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId=
2012-09-26 15:15:39,767 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0
2012-09-26 15:15:39,773 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@484845aa
2012-09-26 15:15:40,065 WARN org.apache.hadoop.hive.conf.HiveConf: hive-site.xml not found on CLASSPATH
2012-09-26 15:15:40,222 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library is available
2012-09-26 15:15:40,222 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library loaded
2012-09-26 15:15:40,232 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead
2012-09-26 15:15:40,236 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0
2012-09-26 15:15:40,242 INFO ExecMapper: maximum memory = 119341056
2012-09-26 15:15:40,243 INFO ExecMapper: conf classpath = [file:/var/run/cloudera-scm-agent/process/93-mapreduce-TASKTRACKER/, file:/usr/java/jdk1.6.0_31/lib/tools.jar, file:/usr/lib/hadoop-0.20-mapreduce/, file:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.0.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.5.4.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.5.4.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-api-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/core-3.1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.0.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jdiff-1.0.9.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/json-simple-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.16.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/oro-2.0.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.3.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar, file:/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar, file:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar, file:/usr/lib/hadoop-hdfs/lib/avro-1.5.4.jar, file:/usr/lib/hadoop-hdfs/lib/paranamer-2.3.jar, file:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar, file:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar, file:/usr/lib/hadoop-hdfs/lib/slf4j-api-1.6.1.jar, file:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar, file:/usr/lib/hadoop-hdfs/lib/snappy-java-1.0.3.2.jar, file:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar, file:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar, file:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar, file:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.0.1.jar, file:/usr/lib/hadoop-hdfs/lib/log4j-1.2.15.jar, file:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.1-tests.jar, file:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar, file:/usr/lib/hadoop/lib/commons-codec-1.4.jar, file:/usr/lib/hadoop/lib/jets3t-0.6.1.jar, file:/usr/lib/hadoop/lib/json-simple-1.1.jar, file:/usr/lib/hadoop/lib/guava-11.0.2.jar, file:/usr/lib/hadoop/lib/avro-1.5.4.jar, file:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar, file:/usr/lib/hadoop/lib/commons-configuration-1.6.jar, file:/usr/lib/hadoop/lib/asm-3.2.jar, file:/usr/lib/hadoop/lib/paranamer-2.3.jar, file:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar, file:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar, file:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar, file:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar, file:/usr/lib/hadoop/lib/commons-cli-1.2.jar, file:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.1.jar, file:/usr/lib/hadoop/lib/commons-lang-2.5.jar, file:/usr/lib/hadoop/lib/kfs-0.3.jar, file:/usr/lib/hadoop/lib/hue-plugins-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar, file:/usr/lib/hadoop/lib/jettison-1.1.jar, file:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar, file:/usr/lib/hadoop/lib/jsch-0.1.42.jar, file:/usr/lib/hadoop/lib/stax-api-1.0.1.jar, file:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar, file:/usr/lib/hadoop/lib/jsr305-1.3.9.jar, file:/usr/lib/hadoop/lib/snappy-java-1.0.3.2.jar, file:/usr/lib/hadoop/lib/jsp-api-2.1.jar, file:/usr/lib/hadoop/lib/oro-2.0.8.jar, file:/usr/lib/hadoop/lib/jersey-server-1.8.jar, file:/usr/lib/hadoop/lib/commons-digester-1.8.jar, file:/usr/lib/hadoop/lib/commons-math-2.1.jar, file:/usr/lib/hadoop/lib/jline-0.9.94.jar, file:/usr/lib/hadoop/lib/core-3.1.1.jar, file:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar, file:/usr/lib/hadoop/lib/commons-el-1.0.jar, file:/usr/lib/hadoop/lib/jersey-core-1.8.jar, file:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar, file:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar, file:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.1.jar, file:/usr/lib/zookeeper/zookeeper-3.4.3-cdh4.0.1.jar, file:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar, file:/usr/lib/hadoop/lib/commons-net-3.1.jar, file:/usr/lib/hadoop/lib/servlet-api-2.5.jar, file:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar, file:/usr/lib/hadoop/lib/commons-io-2.1.jar, file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar, file:/usr/lib/hadoop/lib/commons-logging-api-1.1.jar, file:/usr/lib/hadoop/lib/xmlenc-0.52.jar, file:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar, file:/usr/lib/hadoop/lib/activation-1.1.jar, file:/usr/lib/hadoop/lib/jersey-json-1.8.jar, file:/usr/lib/hadoop/lib/aspectjrt-1.6.5.jar, file:/usr/lib/hadoop/lib/log4j-1.2.15.jar, file:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1-tests.jar, file:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.0.1.jar, file:/mapred/local/taskTracker/root/jobcache/job_201209252321_0010/jars/classes, file:/mapred/local/taskTracker/root/jobcache/job_201209252321_0010/jars/job.jar, file:/mapred/local/taskTracker/root/distcache/4260026189093522549_-70309741_45603944/hadoop1.domain.com/user/root/.staging/job_201209252321_0010/libjars/hive-builtins-0.8.1-cdh4.0.1.jar, file:/mapred/local/taskTracker/root/distcache/-6339710882011042599_2132445101_45603979/hadoop1.domain.com/user/root/.staging/job_201209252321_0010/libjars/hive-serdes-1.0-SNAPSHOT.jar, file:/mapred/local/taskTracker/root/distcache/7269667103068590023_-978189584_45604014/hadoop1.domain.com/user/root/.staging/job_201209252321_0010/libjars/hive-contrib-0.8.1-cdh4.0.1.jar, file:/mapred/local/taskTracker/root/jobcache/job_201209252321_0010/attempt_201209252321_0010_m_000000_0/work/]
2012-09-26 15:15:40,243 INFO ExecMapper: thread classpath = [file:/var/run/cloudera-scm-agent/process/93-mapreduce-TASKTRACKER/, file:/usr/java/jdk1.6.0_31/lib/tools.jar, file:/usr/lib/hadoop-0.20-mapreduce/, file:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.0.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.5.4.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.5.4.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-api-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/core-3.1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.0.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jdiff-1.0.9.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/json-simple-1.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.16.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/oro-2.0.8.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.3.2.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar, file:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar, file:/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar, file:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar, file:/usr/lib/hadoop-hdfs/lib/avro-1.5.4.jar, file:/usr/lib/hadoop-hdfs/lib/paranamer-2.3.jar, file:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar, file:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar, file:/usr/lib/hadoop-hdfs/lib/slf4j-api-1.6.1.jar, file:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar, file:/usr/lib/hadoop-hdfs/lib/snappy-java-1.0.3.2.jar, file:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar, file:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar, file:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar, file:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.0.1.jar, file:/usr/lib/hadoop-hdfs/lib/log4j-1.2.15.jar, file:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.1-tests.jar, file:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar, file:/usr/lib/hadoop/lib/commons-codec-1.4.jar, file:/usr/lib/hadoop/lib/jets3t-0.6.1.jar, file:/usr/lib/hadoop/lib/json-simple-1.1.jar, file:/usr/lib/hadoop/lib/guava-11.0.2.jar, file:/usr/lib/hadoop/lib/avro-1.5.4.jar, file:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar, file:/usr/lib/hadoop/lib/commons-configuration-1.6.jar, file:/usr/lib/hadoop/lib/asm-3.2.jar, file:/usr/lib/hadoop/lib/paranamer-2.3.jar, file:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar, file:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar, file:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar, file:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar, file:/usr/lib/hadoop/lib/commons-cli-1.2.jar, file:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.1.jar, file:/usr/lib/hadoop/lib/commons-lang-2.5.jar, file:/usr/lib/hadoop/lib/kfs-0.3.jar, file:/usr/lib/hadoop/lib/hue-plugins-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar, file:/usr/lib/hadoop/lib/jettison-1.1.jar, file:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar, file:/usr/lib/hadoop/lib/jsch-0.1.42.jar, file:/usr/lib/hadoop/lib/stax-api-1.0.1.jar, file:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar, file:/usr/lib/hadoop/lib/jsr305-1.3.9.jar, file:/usr/lib/hadoop/lib/snappy-java-1.0.3.2.jar, file:/usr/lib/hadoop/lib/jsp-api-2.1.jar, file:/usr/lib/hadoop/lib/oro-2.0.8.jar, file:/usr/lib/hadoop/lib/jersey-server-1.8.jar, file:/usr/lib/hadoop/lib/commons-digester-1.8.jar, file:/usr/lib/hadoop/lib/commons-math-2.1.jar, file:/usr/lib/hadoop/lib/jline-0.9.94.jar, file:/usr/lib/hadoop/lib/core-3.1.1.jar, file:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar, file:/usr/lib/hadoop/lib/commons-el-1.0.jar, file:/usr/lib/hadoop/lib/jersey-core-1.8.jar, file:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar, file:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar, file:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.1.jar, file:/usr/lib/zookeeper/zookeeper-3.4.3-cdh4.0.1.jar, file:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar, file:/usr/lib/hadoop/lib/commons-net-3.1.jar, file:/usr/lib/hadoop/lib/servlet-api-2.5.jar, file:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar, file:/usr/lib/hadoop/lib/commons-io-2.1.jar, file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar, file:/usr/lib/hadoop/lib/commons-logging-api-1.1.jar, file:/usr/lib/hadoop/lib/xmlenc-0.52.jar, file:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar, file:/usr/lib/hadoop/lib/activation-1.1.jar, file:/usr/lib/hadoop/lib/jersey-json-1.8.jar, file:/usr/lib/hadoop/lib/aspectjrt-1.6.5.jar, file:/usr/lib/hadoop/lib/log4j-1.2.15.jar, file:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1-tests.jar, file:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.0.1.jar, file:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.0.1.jar, file:/mapred/local/taskTracker/root/jobcache/job_201209252321_0010/jars/classes, file:/mapred/local/taskTracker/root/jobcache/job_201209252321_0010/jars/job.jar, file:/mapred/local/taskTracker/root/distcache/4260026189093522549_-70309741_45603944/hadoop1.domain.com/user/root/.staging/job_201209252321_0010/libjars/hive-builtins-0.8.1-cdh4.0.1.jar, file:/mapred/local/taskTracker/root/distcache/-6339710882011042599_2132445101_45603979/hadoop1.domain.com/user/root/.staging/job_201209252321_0010/libjars/hive-serdes-1.0-SNAPSHOT.jar, file:/mapred/local/taskTracker/root/distcache/7269667103068590023_-978189584_45604014/hadoop1.domain.com/user/root/.staging/job_201209252321_0010/libjars/hive-contrib-0.8.1-cdh4.0.1.jar, file:/mapred/local/taskTracker/root/jobcache/job_201209252321_0010/attempt_201209252321_0010_m_000000_0/work/]
2012-09-26 15:15:40,253 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Adding alias tweets to work list for file hdfs://hadoop1.domain.com:8020/uploads
2012-09-26 15:15:40,256 INFO org.apache.hadoop.hive.ql.exec.MapOperator: dump TS structtext:string,user:struct<screen_name:string>
2012-09-26 15:15:40,256 INFO ExecMapper:
Id =3
Id =0
Id =1
Id =2
Id = 1 null<\Parent>
<\FS>
<\Children>
Id = 0 null<\Parent>
<\SEL>
<\Children>
Id = 3 null<\Parent>
<\TS>
<\Children>
<\MAP>
2012-09-26 15:15:40,257 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initializing Self 3 MAP
2012-09-26 15:15:40,257 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing Self 0 TS
2012-09-26 15:15:40,257 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Operator 0 TS initialized
2012-09-26 15:15:40,257 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing children of 0 TS
2012-09-26 15:15:40,257 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing child 1 SEL
2012-09-26 15:15:40,257 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing Self 1 SEL
2012-09-26 15:15:40,262 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT structtext:string,user:struct<screen_name:string>
2012-09-26 15:15:40,262 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Operator 1 SEL initialized
2012-09-26 15:15:40,262 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing children of 1 SEL
2012-09-26 15:15:40,262 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Initializing child 2 FS
2012-09-26 15:15:40,262 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Initializing Self 2 FS
2012-09-26 15:15:40,293 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Operator 2 FS initialized
2012-09-26 15:15:40,293 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Initialization Done 2 FS
2012-09-26 15:15:40,293 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initialization Done 1 SEL
2012-09-26 15:15:40,293 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Initialization Done 0 TS
2012-09-26 15:15:40,293 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 3 MAP
2012-09-26 15:15:40,298 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing path hdfs://hadoop1.domain.com:8020/uploads/twitter.txt
2012-09-26 15:15:40,298 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tweets for file hdfs://hadoop1.domain.com:8020/uploads
2012-09-26 15:15:40,497 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 forwarding 1 rows
2012-09-26 15:15:40,497 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
2012-09-26 15:15:40,497 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows
2012-09-26 15:15:40,497 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS hdfs://hadoop1.domain.com:8020/tmp/hive-root/hive_2012-09-26_15-15-33_715_8669028640552125101/_tmp.-ext-10002/000000_0
2012-09-26 15:15:40,498 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS hdfs://hadoop1.domain.com:8020/tmp/hive-root/hive_2012-09-26_15-15-33_715_8669028640552125101/_task_tmp.-ext-10002/_tmp.000000_0
2012-09-26 15:15:40,498 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS hdfs://hadoop1.domain.com:8020/tmp/hive-root/hive_2012-09-26_15-15-33_715_8669028640552125101/_tmp.-ext-10002/000000_0
2012-09-26 15:15:40,560 INFO ExecMapper: ExecMapper: processing 1 rows: used memory = 24284800
2012-09-26 15:15:40,577 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 forwarding 10 rows
2012-09-26 15:15:40,577 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 10 rows
2012-09-26 15:15:40,577 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 10 rows
2012-09-26 15:15:40,577 INFO ExecMapper: ExecMapper: processing 10 rows: used memory = 24860552
2012-09-26 15:15:40,705 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 forwarding 100 rows
2012-09-26 15:15:40,705 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 100 rows
2012-09-26 15:15:40,705 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 100 rows
2012-09-26 15:15:40,705 INFO ExecMapper: ExecMapper: processing 100 rows: used memory = 28885000
2012-09-26 15:15:41,499 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 forwarding 1000 rows
2012-09-26 15:15:41,499 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1000 rows
2012-09-26 15:15:41,499 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1000 rows
2012-09-26 15:15:41,499 INFO ExecMapper: ExecMapper: processing 1000 rows: used memory = 7598072
2012-09-26 15:15:42,992 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"text":"@KimKardashian happy valentines day, hope it's a good one","retweet_count":0,"geo":{"type":"Point","coordinates":[38.7313358,-108.05278695]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":25365536,"source":"\u003Ca href="http://twitter.com/download/android" rel="nofollow"\u003ETwitter for Android\u003C/a\u003E","in_reply_to_user_id_str":"25365536","id_str":"169483808003989505","entities":{"user_mentions":[{"indices":[0,14],"screen_name":"KimKardashian","id_str":"25365536","name":"Kim Kardashian","id":25365536}],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/6a7e7dbf9d6c7ac4.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Delta, CO","bounding_box":{"type":"Polygon","coordinates":[[[-108.104644,38.71503],[-108.021863,38.71503],[-108.021863,38.769794],[-108.104644,38.769794]]]},"name":"Delta","id":"6a7e7dbf9d6c7ac4","country":"United States"},"in_reply_to_screen_name":"Ki{"text":"@bbrandivirgo too bad I dont have the number. Happy valentines day tho :)","retweet_count":0,"geo":{"type":"Point","coordinates":[33.77406404,-84.39270512]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"source":"\u003Ca href="http://mobile.twitter.com" rel="nofollow"\u003EMobile Web\u003C/a\u003E","in_reply_to_user_id_str":null,"id_str":"169497701241716736","entities":{"user_mentions":[],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/8173485c72e78ca5.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Atlanta, GA","bounding_box":{"type":"Polygon","coordinates":[[[-84.54674,33.647908],[-84.289389,33.647908],[-84.289389,33.887618],[-84.54674,33.887618]]]},"name":"Atlanta","id":"8173485c72e78ca5","country":"United States"},"in_reply_to_screen_name":null,"favorited":false,"truncated":false,"created_at":"Tue Feb 14 19:06:15 +0000 2012","contributors":null,"user":{"contributors_enabled":false,"profile_background_image_url":"http://a3.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","url":"http://facebook.com/cperk3","profile_link_color":"0084B4","followers_count":773,"profile_image_url":"http://a3.twimg.com/profile_images/1792490671/000011110000_normal.jpg","default_profile_image":false,"show_all_inline_media":true,"statuses_count":3271,"profile_background_color":"C0DEED","description":"Ga Tech Athlete-Student.. Black&Samoan...Follow me as I follow Jesus-","location":"Atlanta, GA","profile_background_tile":true,"favourites_count":1,"profile_background_image_url_https":"https://si0.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","time_zone":"Quito","profile_sidebar_fill_color":"DDEEF6","screen_name":"Cpeezy21","id_str":"312682111","lang":"en","geo_enabled":true,"profile_image_url_https":"https://si0.twimg.com/profile_images/1792490671/000011110000_normal.jpg","verified":false,"notifications":null,"profile_sidebar_border_color":"04080a","protected":false,"listed_count":5,"created_at":"Tue Jun 07 14:14:34 +0000 2011","name":"Charles Perkins III","is_translator":false,"follow_request_sent":null,"following":null,"profile_use_background_image":true,"friends_count":223,"id":312682111,"default_profile":false,"utc_offset":-18000,"profile_text_color":"333333"},"retweeted":false,"id":169497701241716736,"coordinates":{"type":"Point","coordinates":[-84.39270512,33.77406404]}}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:393)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Unexpected character ('t' (code 116)): was expecting comma to separate OBJECT entries
at [Source: java.io.StringReader@366ef7ba; line: 1, column: 999]
at com.cloudera.hive.serde.JSONSerDe.deserialize(JSONSerDe.java:128)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
... 9 more
Caused by: org.codehaus.jackson.JsonParseException: Unexpected character ('t' (code 116)): was expecting comma to separate OBJECT entries
at [Source: java.io.StringReader@366ef7ba; line: 1, column: 999]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:306)
at org.codehaus.jackson.impl.ReaderBasedParser.nextToken(ReaderBasedParser.java:285)
at org.codehaus.jackson.map.deser.MapDeserializer._readAndBind(MapDeserializer.java:220)
at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:165)
at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:25)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602)
at com.cloudera.hive.serde.JSONSerDe.deserialize(JSONSerDe.java:126)
... 10 more
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 finished. closing...
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 forwarded 4551 rows
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:1
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished. closing...
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 4551 rows
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 finished. closing...
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarded 4551 rows
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: 2 finished. closing...
2012-09-26 15:15:42,993 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: 2 forwarded 0 rows
2012-09-26 15:15:43,066 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: TABLE_ID_1_ROWCOUNT:4551
2012-09-26 15:15:43,066 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 Close done
2012-09-26 15:15:43,066 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 Close done
2012-09-26 15:15:43,066 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 3 Close done
2012-09-26 15:15:43,066 INFO ExecMapper: ExecMapper: processed 4551 rows: used memory = 17571376
2012-09-26 15:15:43,074 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2012-09-26 15:15:43,077 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"text":"@KimKardashian happy valentines day, hope it's a good one","retweet_count":0,"geo":{"type":"Point","coordinates":[38.7313358,-108.05278695]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":25365536,"source":"\u003Ca href="http://twitter.com/download/android" rel="nofollow"\u003ETwitter for Android\u003C/a\u003E","in_reply_to_user_id_str":"25365536","id_str":"169483808003989505","entities":{"user_mentions":[{"indices":[0,14],"screen_name":"KimKardashian","id_str":"25365536","name":"Kim Kardashian","id":25365536}],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/6a7e7dbf9d6c7ac4.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Delta, CO","bounding_box":{"type":"Polygon","coordinates":[[[-108.104644,38.71503],[-108.021863,38.71503],[-108.021863,38.769794],[-108.104644,38.769794]]]},"name":"Delta","id":"6a7e7dbf9d6c7ac4","country":"United States"},"in_reply_to_screen_name":"Ki{"text":"@bbrandivirgo too bad I dont have the number. Happy valentines day tho :)","retweet_count":0,"geo":{"type":"Point","coordinates":[33.77406404,-84.39270512]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"source":"\u003Ca href="http://mobile.twitter.com" rel="nofollow"\u003EMobile Web\u003C/a\u003E","in_reply_to_user_id_str":null,"id_str":"169497701241716736","entities":{"user_mentions":[],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/8173485c72e78ca5.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Atlanta, GA","bounding_box":{"type":"Polygon","coordinates":[[[-84.54674,33.647908],[-84.289389,33.647908],[-84.289389,33.887618],[-84.54674,33.887618]]]},"name":"Atlanta","id":"8173485c72e78ca5","country":"United States"},"in_reply_to_screen_name":null,"favorited":false,"truncated":false,"created_at":"Tue Feb 14 19:06:15 +0000 2012","contributors":null,"user":{"contributors_enabled":false,"profile_background_image_url":"http://a3.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","url":"http://facebook.com/cperk3","profile_link_color":"0084B4","followers_count":773,"profile_image_url":"http://a3.twimg.com/profile_images/1792490671/000011110000_normal.jpg","default_profile_image":false,"show_all_inline_media":true,"statuses_count":3271,"profile_background_color":"C0DEED","description":"Ga Tech Athlete-Student.. Black&Samoan...Follow me as I follow Jesus-","location":"Atlanta, GA","profile_background_tile":true,"favourites_count":1,"profile_background_image_url_https":"https://si0.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","time_zone":"Quito","profile_sidebar_fill_color":"DDEEF6","screen_name":"Cpeezy21","id_str":"312682111","lang":"en","geo_enabled":true,"profile_image_url_https":"https://si0.twimg.com/profile_images/1792490671/000011110000_normal.jpg","verified":false,"notifications":null,"profile_sidebar_border_color":"04080a","protected":false,"listed_count":5,"created_at":"Tue Jun 07 14:14:34 +0000 2011","name":"Charles Perkins III","is_translator":false,"follow_request_sent":null,"following":null,"profile_use_background_image":true,"friends_count":223,"id":312682111,"default_profile":false,"utc_offset":-18000,"profile_text_color":"333333"},"retweeted":false,"id":169497701241716736,"coordinates":{"type":"Point","coordinates":[-84.39270512,33.77406404]}}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:393)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable {"text":"@KimKardashian happy valentines day, hope it's a good one","retweet_count":0,"geo":{"type":"Point","coordinates":[38.7313358,-108.05278695]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":25365536,"source":"\u003Ca href="http://twitter.com/download/android" rel="nofollow"\u003ETwitter for Android\u003C/a\u003E","in_reply_to_user_id_str":"25365536","id_str":"169483808003989505","entities":{"user_mentions":[{"indices":[0,14],"screen_name":"KimKardashian","id_str":"25365536","name":"Kim Kardashian","id":25365536}],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/6a7e7dbf9d6c7ac4.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Delta, CO","bounding_box":{"type":"Polygon","coordinates":[[[-108.104644,38.71503],[-108.021863,38.71503],[-108.021863,38.769794],[-108.104644,38.769794]]]},"name":"Delta","id":"6a7e7dbf9d6c7ac4","country":"United States"},"in_reply_to_screen_name":"Ki{"text":"@bbrandivirgo too bad I dont have the number. Happy valentines day tho :)","retweet_count":0,"geo":{"type":"Point","coordinates":[33.77406404,-84.39270512]},"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"source":"\u003Ca href="http://mobile.twitter.com" rel="nofollow"\u003EMobile Web\u003C/a\u003E","in_reply_to_user_id_str":null,"id_str":"169497701241716736","entities":{"user_mentions":[],"urls":[],"hashtags":[]},"in_reply_to_status_id":null,"place":{"url":"http://api.twitter.com/1/geo/id/8173485c72e78ca5.json","place_type":"city","country_code":"US","attributes":{},"full_name":"Atlanta, GA","bounding_box":{"type":"Polygon","coordinates":[[[-84.54674,33.647908],[-84.289389,33.647908],[-84.289389,33.887618],[-84.54674,33.887618]]]},"name":"Atlanta","id":"8173485c72e78ca5","country":"United States"},"in_reply_to_screen_name":null,"favorited":false,"truncated":false,"created_at":"Tue Feb 14 19:06:15 +0000 2012","contributors":null,"user":{"contributors_enabled":false,"profile_background_image_url":"http://a3.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","url":"http://facebook.com/cperk3","profile_link_color":"0084B4","followers_count":773,"profile_image_url":"http://a3.twimg.com/profile_images/1792490671/000011110000_normal.jpg","default_profile_image":false,"show_all_inline_media":true,"statuses_count":3271,"profile_background_color":"C0DEED","description":"Ga Tech Athlete-Student.. Black&Samoan...Follow me as I follow Jesus-","location":"Atlanta, GA","profile_background_tile":true,"favourites_count":1,"profile_background_image_url_https":"https://si0.twimg.com/profile_background_images/376284279/yyyyyyyyyyyyyyyyyyyy.jpg","time_zone":"Quito","profile_sidebar_fill_color":"DDEEF6","screen_name":"Cpeezy21","id_str":"312682111","lang":"en","geo_enabled":true,"profile_image_url_https":"https://si0.twimg.com/profile_images/1792490671/000011110000_normal.jpg","verified":false,"notifications":null,"profile_sidebar_border_color":"04080a","protected":false,"listed_count":5,"created_at":"Tue Jun 07 14:14:34 +0000 2011","name":"Charles Perkins III","is_translator":false,"follow_request_sent":null,"following":null,"profile_use_background_image":true,"friends_count":223,"id":312682111,"default_profile":false,"utc_offset":-18000,"profile_text_color":"333333"},"retweeted":false,"id":169497701241716736,"coordinates":{"type":"Point","coordinates":[-84.39270512,33.77406404]}}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 8 more
Caused by: org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Unexpected character ('t' (code 116)): was expecting comma to separate OBJECT entries
at [Source: java.io.StringReader@366ef7ba; line: 1, column: 999]
at com.cloudera.hive.serde.JSONSerDe.deserialize(JSONSerDe.java:128)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
... 9 more
Caused by: org.codehaus.jackson.JsonParseException: Unexpected character ('t' (code 116)): was expecting comma to separate OBJECT entries
at [Source: java.io.StringReader@366ef7ba; line: 1, column: 999]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:306)
at org.codehaus.jackson.impl.ReaderBasedParser.nextToken(ReaderBasedParser.java:285)
at org.codehaus.jackson.map.deser.MapDeserializer._readAndBind(MapDeserializer.java:220)
at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:165)
at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:25)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602)
at com.cloudera.hive.serde.JSONSerDe.deserialize(JSONSerDe.java:126)
... 10 more
2012-09-26 15:15:43,081 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task