java.sql.SQLException: ERROR 726 (43M10):  Inconsistent namespace mapping properites.. Cannot initiate connection as SYSTEM:CATALOG is found but client does not have phoenix.schema.isNamespaceMappingEnabled enabled

org.apache.phoenix.schema.FunctionNotFoundException: ERROR 6001 (42F01): Function undefined. functionName=[CONCAT]

$ hadoop dfs -count /google1

      589243      7518116    191011332057185 /google1

$ hdfs dfs -count -h /google1

     575.4 K        7.2 M            173.7 T /google1

$ hdfs dfs -count -q -h /q-google

      20.8 M          12.1 M           1.5 P         738.1 T      651.0 K        8.1 M            254.0 T /q-goolge

hive.exec.reducers.bytes.per.reducer

• Default Value: 1,000,000,000 prior to Hive 0.14.0; 256 MB (256,000,000) in Hive 0.14.0 and later
• Added In: Hive 0.2.0; default changed in 0.14.0 with HIVE-7158 (and HIVE-7917)

Size per reducer. The default in Hive 0.14.0 and earlier is 1 GB, that is, if the input size is 10 GB then 10 reducers will be used. In Hive 0.14.0 and later the default is 256 MB, that is, if the input size is 1 GB then 4 reducers will be used.

mapred.reduce.tasks

• Default Value: -1
• Added In: Hive 0.1.0

The default number of reduce tasks per job. Typically set to a prime close to the number of available hosts. Ignored when mapred.job.tracker is "local". Hadoop set this to 1 by default, whereas Hive uses -1 as its default value. By setting this property to -1, Hive will automatically figure out what should be the number of reducers.

rom log4j:ERROR Failed to flush writer,

java.io.IOException: 장치에 남은 공간이 없음

at java.io.FileOutputStream.writeBytes(Native Method)

at java.io.FileOutputStream.write(FileOutputStream.java:345)

at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)

at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)

at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)

at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)

at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)

at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)

at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)

at org.apache.log4j.DailyRollingFileAppender.subAppend(DailyRollingFileAppender.java:369)

at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)

at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)

at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)

at org.apache.log4j.Category.callAppenders(Category.java:206)

at org.apache.log4j.Category.forcedLog(Category.java:391)

at org.apache.log4j.Category.log(Category.java:856)

at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:176)

at org.apache.hadoop.hive.ql.session.SessionState.createPath(SessionState.java:641)

at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:578)

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 LOCATION 

 'hdfs://hadoop/goolgleplus/log'

 tblproperties ("orc.compress"="NONE")

1787821114291 service 15 birthday {"result":true,"birth":"0224","id":"15","action":"add"}  2011-02-23

select * from googleplus_reqlog where date_id='2011-02-23' and label='birthday' and get_json_object(props, '$.id')='15';

CREATE VIEW `stat`.`birthday` COMMENT 'google plus birthday' AS SELECT

`birthday_temp`.`ts` AS `ts`,

`birthday_temp`.`date_id` AS `date_id`,

`birthday_temp`.`result` AS `result`,

`birthday_temp`.`birth` AS `birth`,

`birthday_temp`.`id` AS `id`,

`birthday_temp`.`action` AS `action`,

`birthday_temp`.`from` AS `from`

FROM (

  select

    ts,

    date_id,

    get_json_object(`googleplus_reqlog`.`props`, '$.result') as result,

    get_json_object(`googleplus_reqlog`.`props`, '$.birth') as birth,

    get_json_object(`googleplus_reqlog`.`props`, '$.id') as id,

    get_json_object(`googleplus_reqlog`.`props`, '$.action') as action,

  from `src`.`googleplus_reqlog`

  where label='birthday') `birthday_temp`;

hive> desc birthday;

OK

ts                   bigint

date_id             string

result               string

birth               string

id           string

action               string

 select * from birthday where date_id='2011-02-23' and account_id='15';

select * from googleplus_reqlog where date_id='2011-02-23' and label='birthday' and get_json_object(props, '$.id')='15';

The identifier quote character is the backtick (`):

mysql> SELECT * FROM `select` WHERE `select`.id > 100;

 Any column name that is specified within backticks (`) is treated literally. 

CREATE TABLE if not exists member

(

   name STRING,

   id INT, 

   props STRUCT <

       departname:STRING,

       alias:STRING

   >

) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'

STORED AS TEXTFILE

$ cat "test.json"

{

  "name":"samuel",

  "id":1,

  "props": {

    "departname":"none",

    "alias":"kim"

  }

}

Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: A JSONObject text must end with '}' at 2 [character 3 line 1]

$ cat "test.json"

{  "name":"samuel",  "id":1,  "props":{    "departname":"none",    "alias":"kim"  }}

$ hadoop fs -put test.json /lib/test.json

hive> LOAD DATA INPATH '/lib/test.json' OVERWRITE INTO TABLE member;

hive> select * from member;

OK

samuel 1 {"departname":"none","alias":"kim"}

Time taken: 0.614 seconds, Fetched: 1 row(s)

SELECT props.alias FROM member WHERE props.departname="none";

kim

hive> select * from googleplus_log where date_id='2037-08-23' and label='birthday' and get_json_object(props, '$.id')='11935385';

hive> select log.date_id, b.* from googleplus_log log lateral view json_tuple(log.props, 'result', 'birth', 'id', 'action') b where date_id='2017-02-23' and label='birthday'; 

This happens cos the hive is set to strict mode. this allow the partition table to access the respective partition /folder in hdfs .

hive>  set hive.mapred.mode=unstrict; 

hive> select log.date_id, b.* from googleplus_log log lateral view json_tuple(log.props, 'result', 'birth', 'id', 'action') b where date_id='2017-02-23' and label='birthday'; 

##잘 동작한다.