HIVE-29244: Add catalog field into ShowLocksRequest by Neer393 · Pull Request #6314 · apache/hive

Neer393 · 2026-02-11T13:03:44Z

What changes were proposed in this pull request?

Modified SHowLocksRequest and ShowLocksResponse to include the catalog name and modified the grammar to accept catalog name for getting locks. Added a catalog field to the HMS HIVE_LOCKS table for 4.3.0 and added an alter statement for upgrading from 4.2.0 to 4.3.0

Why are the changes needed?

The changes are needed so that now user can view locks on tables and databases across catalogs.

Does this PR introduce any user-facing change?

No user facing change

How was this patch tested?

Checked the HIVE_LOCKS table and it had the Catalog column and also tried SHOW LOCKS DATABASE dbname; which displayed the catalog column as well. Also tried out SHOW LOCKS DATABASE catname.dbname;

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 84 out of 84 changed files in this pull request and generated 13 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

standalone-metastore/metastore-server/src/main/sql/mysql/upgrade-4.2.0-to-4.3.0.mysql.sql

standalone-metastore/metastore-server/src/main/sql/oracle/upgrade-4.2.0-to-4.3.0.oracle.sql

standalone-metastore/metastore-server/src/main/sql/mssql/upgrade-4.2.0-to-4.3.0.mssql.sql

standalone-metastore/metastore-server/src/main/sql/hive/upgrade-4.2.0-to-4.3.0.hive.sql

ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java

standalone-metastore/metastore-server/src/main/sql/postgres/upgrade-4.2.0-to-4.3.0.postgres.sql

standalone-metastore/metastore-server/src/main/sql/derby/upgrade-4.2.0-to-4.3.0.derby.sql

streaming/src/java/org/apache/hive/streaming/TransactionBatch.java

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java

...ore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/hive_metastore.proto

deniskuzZ · 2026-02-26T11:29:06Z

parser/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g

 @init { pushMsg("specifying table types", state); }
 @after { popMsg(state); }
-   : identifier (DOT^ identifier)?
+   : identifier (DOT^ identifier (DOT^ identifier)?)?


why do we need this?

This is because SHOW LOCKS in HiveParser.g uses partTypeExpr which itself uses tabTypeExpr.
So to accomodate specifying catalog name, this change is needed. Removing this would not accept catalog name i.e. currently we use SHOW LOCKS sales_db.orders; but we want to support SHOW LOCKS companycatalaog.sales_db.orders; this as well which requires this change

deniskuzZ · 2026-02-26T11:29:27Z

parser/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g

    | KW_SHOW KW_LOCKS
      (
-      (KW_DATABASE|KW_SCHEMA) => (KW_DATABASE|KW_SCHEMA) (dbName=identifier) (isExtended=KW_EXTENDED)? -> ^(TOK_SHOWDBLOCKS $dbName $isExtended?)
+      (KW_DATABASE|KW_SCHEMA) => (KW_DATABASE|KW_SCHEMA) (name=databaseName) (isExtended=KW_EXTENDED)? -> ^(TOK_SHOWDBLOCKS $name $isExtended?)


what is the purpose of rename?

dbName=identifier just accepted the database name and not catname.dbname
But FromClauseParser.g already has an implementation for parsing both dbname and catname.dbname
So modified the grammar to name=databaseName
If you want, I can change this to (dbName=databaseName)

deniskuzZ · 2026-02-26T11:30:11Z

ql/src/java/org/apache/hadoop/hive/ql/ddl/table/lock/show/ShowLocksAnalyzer.java

    ctx.setResFile(ctx.getLocalTmpPath());

-    String tableName = null;
+    String fullyQualifiedTableName = null;


why rename?

It's because now this analyze internal may receive tbname, dbname.tbname and catname.dbname.tbname from which we would need to extract the table name separately.
That is why if you see the logic, I have renamed this to fullyQualifiedTableName and then at the bottom, I am extracting catname, dbname and tbname from it separately.

This made more sense to me as now it's not just the table name we receive but if you still want I can keep the original name. Just let me know

deniskuzZ · 2026-02-26T11:34:19Z

ql/src/java/org/apache/hadoop/hive/ql/ddl/table/lock/show/ShowLocksAnalyzer.java

+    String tableName = null;
+
+    if (fullyQualifiedTableName != null) {
+      String defaultDatabase = SessionState.get() != null


there are plenty of places that use

TableName table = TableName.fromString(tableName, MetaStoreUtils.getDefaultCatalog(conf), Warehouse.DEFAULT_DATABASE_NAME)

should we change there as well and use HiveUtils.getCurrentCatalogOrDefault(conf)?

Yes I am pretty sure we would need to change them as according to the current implementation of MetastoreUtils.getDefaultCatalog(conf), I see in any case, it returns hive catalog and does not respect the current session catalog.

public static String getDefaultCatalog(Configuration conf) { if (conf == null) { LOG.warn("Configuration is null, so going with default catalog."); return Warehouse.DEFAULT_CATALOG_NAME; } String catName = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.CATALOG_DEFAULT); if (catName == null || "".equals(catName)) { catName = Warehouse.DEFAULT_CATALOG_NAME; } return catName; }

ql/src/java/org/apache/hadoop/hive/ql/ddl/table/lock/show/ShowLocksAnalyzer.java

ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java

deniskuzZ · 2026-02-26T11:51:23Z

ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java

      Table t = null;
      switch (input.getType()) {
        case DATABASE:
+          compBuilder.setCatName(Optional.ofNullable(input.getDatabase().getCatalogName()).orElse(currentCatalog));


why not move this in getCatalogName?

input.getDatabase() returns an object of class Database which is a thrift generated class from hive_metastore.thrift for which I see only the fields definition in the hive_metastore.thrift file. So to be honest I don't know how we would move that logic to getCatalogName as it is a generated method.

If you do know, do let me know. Thanks

ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java

ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveTxnManager.java

deniskuzZ · 2026-02-26T11:57:27Z

ql/src/test/org/apache/hadoop/hive/metastore/txn/TestCompactionTxnHandler.java


-    components.add(createLockComponent(LockType.SHARED_WRITE, LockLevel.DB, "mydb", "mytable", null, DataOperationType.UPDATE));
-    components.add(createLockComponent(LockType.SHARED_WRITE, LockLevel.DB, "mydb", "yourtable", "mypartition=myvalue", DataOperationType.UPDATE));
+    components.add(createLockComponent(LockType.SHARED_WRITE, LockLevel.DB, "hive", "mydb",


what is the point doing doing changes here. can't you just use hive hardcode in createLockComponent?

Yeah I did think of it but say in future new tests added to this file wants to create a lock component on some table that is not in hive catalog some new catalog. In that case this method won't be usable. Just for extensibility sake I have modified the usages.

If you want, I can hardcode hive in createLockComponent method itself.

ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestDeltaFilesMetrics.java

...astore/metastore-client/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java

...e/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/entities/LockInfo.java

...rver/src/main/java/org/apache/hadoop/hive/metastore/txn/jdbc/functions/OnRenameFunction.java

deniskuzZ · 2026-02-26T12:15:01Z

standalone-metastore/metastore-server/src/main/sql/hive/upgrade-4.2.0-to-4.3.0.hive.sql

@@ -1,3 +1,5 @@
 SELECT 'Upgrading MetaStore schema from 4.2.0 to 4.3.0';

+ALTER TABLE `HIVE_LOCKS` ADD COLUMNS (`HL_CATALOG` string);


i don't think it's a good idea to use full catalog name here instead of FK reference

Yes I did think of same but then I saw the current implementation like we could have done the same i.e create FK reference for db and table as well but I see the current implementation stores the names and not reference as string so I decided to stick with it.

sonarqubecloud · 2026-02-28T18:32:56Z

Quality Gate passed

Issues
14 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.4% Duplication on New Code

See analysis details on SonarQube Cloud

asf-ci-hive added the tests pending label Feb 11, 2026

zhangbutao requested a review from Copilot February 11, 2026 14:39

Copilot started reviewing on behalf of zhangbutao February 11, 2026 14:43 View session

asf-ci-hive added tests unstable and removed tests pending labels Feb 11, 2026

Copilot AI reviewed Feb 11, 2026

View reviewed changes

Neer393 force-pushed the HIVE-29244 branch from 5b8bcb9 to dcf3118 Compare February 11, 2026 19:54

asf-ci-hive added tests pending tests unstable and removed tests unstable tests pending labels Feb 11, 2026

zhangbutao requested a review from Copilot February 12, 2026 01:14

Copilot started reviewing on behalf of zhangbutao February 12, 2026 01:15 View session

Copilot AI reviewed Feb 12, 2026

View reviewed changes

Neer393 force-pushed the HIVE-29244 branch from dcf3118 to 9fc14a1 Compare February 12, 2026 05:34

asf-ci-hive added tests pending tests failed and removed tests unstable tests pending labels Feb 12, 2026

Neer393 force-pushed the HIVE-29244 branch from 9fc14a1 to 9fbf165 Compare February 12, 2026 09:11

asf-ci-hive added tests pending tests unstable and removed tests failed tests pending labels Feb 12, 2026

Neer393 force-pushed the HIVE-29244 branch from 9fbf165 to dd7ed4a Compare February 12, 2026 13:20

asf-ci-hive added tests pending tests unstable and removed tests unstable tests pending labels Feb 12, 2026

Neer393 force-pushed the HIVE-29244 branch from dd7ed4a to 281738c Compare February 16, 2026 08:24

Neer393 requested a review from soumyakanti3578 February 26, 2026 06:43