之前遇到的一个drop table的小问题,默认任何人都可以有使用drop db.table的权限
这是一个bug,
bugid:https://issues.apache.org/jira/browse/HIVE-2817
解决方法:
可以通过设置
set hive.exec.drop.ignorenonexistent=false(Do not report an error if DROP TABLE/VIEW specifies a non-existent table/view,
这个值默认是true,
DROPIGNORESNONEXISTENT("hive.exec.drop.ignorenonexistent", true))来解决。
Workaround: set hive.exec.drop.ignorenonexistent=false. Before that, we need to check all the "drop" sqls and change to drop if exists. we need to remove all "drop table db.table" syntax to "use db; drop table if exists table;"
但是如果设置了hive.exec.drop.ignorenonexistent=false,如果表xxx不存在,drop table xxx会报错
hive> set hive.exec.drop.ignorenonexistent=false; hive> drop table tt; FAILED: SemanticException [Error 10001]: Table not found tt hive> set hive.exec.drop.ignorenonexistent=true; hive> drop table tt; OK Time taken: 0.055 seconds
另外,drop db.table这种语法也会报错,如果设置hive.exec.drop.ignorenonexistent为true则不会报错
hive> set hive.exec.drop.ignorenonexistent=true; hive> drop table cdnlog.ttt; FAILED: SemanticException [Error 10001]: Table not found cdnlog.ttt hive> use cdnlog;drop table ttt; OK Time taken: 0.042 seconds OK set hive.exec.drop.ignorenonexistent=true; hive> drop table cdnlog.ttt; OK
因为更改etl sql的代价比较大,因此只能把hive.exec.drop.ignorenonexistent设置为true,另外需要fix这个bug
drop table是ddl操作,对应的analyzer相关类的路径为:
ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
对应的方法是analyzeDropTable:
@@ -721,13 +716,30 @@
private void analyzeDropTable(ASTNode ast, boolean expectView)
throws SemanticException {
String tableName = getUnescapedName((ASTNode) ast.getChild(0));
boolean ifExists = (ast.getFirstChildWithType(HiveParser.TOK_IFEXISTS) != null);
// we want to signal an error if the table/view doesn‘t exist and we‘re
// configured not to fail silently
boolean throwException =
!ifExists && !HiveConf.getBoolVar(conf, ConfVars.DROPIGNORESNONEXISTENT);
try {
- Table tab = db.getTable(db.getCurrentDatabase(), tableName, throwException);
+ // to fix the drop table bug
+ String tableName2 = "";
+ if (tableName.contains(".")) {
+ try {
+ tableName2 = tableName.split("\\.")[1];
+ } catch (Exception e) {
+ // do nothing if tableName can‘t be splitted
+ tableName2 = tableName;
+ }
+ } else {
+ tableName2 = tableName;
+ }
+ Table tab = db.getTable(db.getCurrentDatabase(), tableName2, throwException);
+
+ // Table tab = db.getTable(db.getCurrentDatabase(), tableName, throwException);
+
if (tab != null) {
inputs.add(new ReadEntity(tab));
outputs.add(new WriteEntity(tab));
本文出自 “菜光光的博客” 博客,请务必保留此出处http://caiguangguang.blog.51cto.com/1652935/1558885
原文:http://caiguangguang.blog.51cto.com/1652935/1558885