Update from remote branch

.github/workflows/ci.yml

@@ -8,14 +8,18 @@ on:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        go: ['1.20', '1.21']
+        os: ['ubuntu-latest', 'windows-latest']
+    runs-on: ${{ matrix.os }}
     steps:
     - name: Set up Go 1.x
-      uses: actions/setup-go@v2
+      uses: actions/setup-go@v4
       with:
-        go-version: 1.16
-      id: go
+        go-version: ${{ matrix.go }}
     - name: Check out code
-      uses: actions/checkout@v2
+      uses: actions/checkout@v3
     - name: Runs tests
       run: make

CHANGELOG.md

@@ -4,11 +4,113 @@
 
 * ...
 
+
+## 6.2.2     2026-01-27
+
+* Upgrade to libpg_query 17-6.2.2
+  - Add IsUtilityStmt function to determine if query text contains utility statements
+    - This is a fast check for callers that don't actually need the parse tree itself
+  - Add Summary function to get summary information about a statement quickly
+    - This allows gathering certain information, for example which tables are referenced in a
+      statement, without requiring a Protobuf serialization step in a higher level library
+    - Additionally this can also be used to perform "smart truncation" of a query by
+      omitting deeply nested information (e.g. a CTE definition, or a target list) whilst
+        preserving more essential parts like the FROM clause
+  - Normalize:
+    - Fix handling of special strings in DefElem
+      - This avoids a crash when running the normalize function on certain utility statements
+    - Add support for CALL statements
+  - Deparser:
+    - Introduce pretty printing / formatting
+      - Introduces a new optional pretty print mode that emits a human readable
+        output. A detailed explanation of the mechanism can be found at the start
+        of the deparser file.
+    - Rework handling of expressions inside typecasts
+      - Prefer (..)::type syntax, unless we are already in a function call.
+    - Use lowercase keywords in xmlroot functions
+      - This matches other XML functions as well as the Postgres documentation,
+        since these are closer to function argument names than regular keywords.
+    - Fix deparse of ALTER TABLE a ALTER b SET STATISTICS DEFAULT
+    - Fix the null pointer dereference when handling identity columns
+  - Allow alternate definitions of NAMEDATALEN identifier limit
+    - This allows building libpg_query with an override of the built-time limit of
+      Postgres identifiers (typically 63 characters)
+  - Bump Postgres to 17.7 and switch back to release tarballs
+* Avoid conflicts with other cgo libraries utilizing xxhash by using XXH_NAMESPACE [#144](https://github.com/pganalyze/pg_query_go/pull/144)
+* Remove redundant copy of result in ParseToProtobuf [#139](https://github.com/pganalyze/pg_query_go/pull/139)
+
+
+## 6.1.0     2025-02-24
+
+* Upgrade to libpg_query 17-6.1.0
+  - Update to Postgres 17.4, and add recent patches scheduled for Postgres 17.5 (not yet released)
+    - Notably, this pulls in support for macOS 15.4 which defines strchrnul
+    in its standard library, fixing builds on up-to-date macOS versions.
+  - Deparser improvements
+    - Add parenthesis around AT LOCAL / AT TIMEZONE if needed
+    - Correctness improvements related to expressions and function calls
+* Allow vendoring pg_query_go with built-in "go mod vendor" command [(#131)](https://github.com/pganalyze/pg_query_go/pull/131)
+
+
+## 6.0.0     2024-11-26
+
+* Upgrade to libpg_query 17-6.0.0
+  - Updates to the Postgres 17 parser
+  - Deparser improvements:
+    - Add support for deparsing `JSON_TABLE`, `JSON_QUERY`, `JSON_EXISTS`, `JSON_VALUE`
+    - Add support for deparsing `JSON`, `JSON_SCALAR`, `JSON_SERIALIZE`
+    - Add support for deparsing `COPY ... FORCE_NULL(*)`
+    - Add support for deparsing `ALTER COLUMN ... SET EXPRESSION AS`
+    - Add support for deparsing `SET STATISTICS DEFAULT`
+    - Add support for deparsing `SET ACCESS METHOD DEFAULT`
+    - Add support for deparsing `... AT LOCAL`
+    - Add support for deparsing `merge_action()`
+    - Add support for deparsing `MERGE ... RETURNING`
+    - Add support for deparsing `NOT MATCHED [ BY TARGET ]`
+* Add function `TestNormalizeUtility` to normalize only utility statements [(#116)](https://github.com/pganalyze/pg_query_go/pull/116)
+
+## 5.1.0     2024-01-09
+
+* Update to libpg_query 16-5.1.0
+  - Add support for running on Windows
+  - Add support for compiling on 32-bit systems
+
+
+## 5.0.0     2023-12-23
+
+* Upgrade to libpg_query 16-5.0.0
+  - Updates to the Postgres 16 parser
+  - Multiple deparser improvements
+* Allow importing parse tree proto with cgo disabled
+* Add parser.SplitWithScanner() and parser.SplitWithParser()
+
+
+## 4.2.3     2023-08-04
+
+* Update to libpg_query 15-4.2.3, including changes from 15-4.2.2
+  - Fix builds when compiling with `glibc >=  2.38` [#203](https://github.com/pganalyze/libpg_query/pull/203)
+  - Deparser: Add support for COALESCE and other expressions in LIMIT clause [#199](https://github.com/pganalyze/libpg_query/pull/199)
+  - Deparser: Add support for multi-statement CREATE PROCEDURE definitions [#197](https://github.com/pganalyze/libpg_query/pull/197)
+  - Deparser: Correctly quote identifier in ALTER TABLE ... ADD CONSTRAINT [x] [#196](https://github.com/pganalyze/libpg_query/pull/196)
+  - Deparser: Add support for index fillfactor within CREATE TABLE, fix SHOW ALL [#193](https://github.com/pganalyze/libpg_query/pull/193)
+  - Deparser: Move to dedicated file for easier inclusion in third-party projects [#192](https://github.com/pganalyze/libpg_query/pull/192)
+
+
+## 4.2.1     2023-05-25
+
+* Update to libpg_query 15-4.2.1 ([#85](https://github.com/pganalyze/pg_query_go/pull/85))
+  - Deparser: Handle INTERVAL correctly when used in SET statements [#184](https://github.com/pganalyze/libpg_query/pull/184)
+  - Deparser: Ensure index names are quoted as identifiers [#182](https://github.com/pganalyze/libpg_query/pull/182)
+* Suppress -Wdeprecated-non-prototype warnings ([#83](https://github.com/pganalyze/pg_query_go/pull/83) and [#84](https://github.com/pganalyze/pg_query_go/pull/84))
+* Return full structured error info instead of just error message ([#76](https://github.com/pganalyze/pg_query_go/pull/76))
+
+
 ## 4.2.0     2023-02-08
 
 * Update to libpg_query 15-4.2.0
   - Update to PostgreSQL 15.1
 
+
 ## 2.2.0     2022-11-02
 
 * Update to libpg_query 13-2.2.0 ([#64](https://github.com/pganalyze/pg_query_go/pull/64))

LICENSE

@@ -1,4 +1,5 @@
 Copyright (c) 2015, Lukas Fittl <lukas@fittl.com>
+Copyright (c) 2016-2025, Duboce Labs, Inc. (pganalyze) <team@pganalyze.com>
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

Makefile

@@ -16,7 +16,7 @@ benchmark:
 
 # --- Below only needed for releasing new versions
 
-LIB_PG_QUERY_TAG = 15-4.2.0
+LIB_PG_QUERY_TAG = 17-6.2.2
 
 root_dir := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
 LIB_TMPDIR = $(root_dir)/tmp
@@ -31,21 +31,20 @@ $(LIBDIRGZ):
 	mkdir -p $(LIB_TMPDIR)
 	curl -o $(LIBDIRGZ) https://codeload.github.com/pganalyze/libpg_query/tar.gz/$(LIB_PG_QUERY_TAG)
 
-update_source: $(LIBDIR)
+update_source: clean $(LIBDIR)
 	rm -f parser/*.{c,h}
 	rm -fr parser/include
 	# Reduce everything down to one directory
 	cp -a $(LIBDIR)/src/* parser/
+	mv parser/postgres/include parser/include/postgres
 	rm parser/pg_query_outfuncs_protobuf_cpp.cc
 	mv parser/postgres/* parser/
 	rmdir parser/postgres
-	cp -a $(LIBDIR)/pg_query.h parser/include
-	# Make sure every .c in the top-level directory is its own translation unit
-	mv parser/*{_conds,_defs,_helper}.c parser/include
+	cp -a $(LIBDIR)/{pg_query.h,postgres_deparse.h} parser/include
 	# Protobuf definitions
 	mkdir -p $(PWD)/bin
-	GOBIN=$(PWD)/bin go install github.com/golang/protobuf/protoc-gen-go
-	PATH=$(PWD)/bin:$(PATH) protoc --proto_path=$(LIBDIR)/protobuf --go_out=. $(LIBDIR)/protobuf/pg_query.proto
+	GOBIN=$(PWD)/bin go install google.golang.org/protobuf/cmd/protoc-gen-go
+	PATH="$(PWD)/bin:$(PATH)" protoc --proto_path=$(LIBDIR)/protobuf --go_out=. --go_opt=Mpg_query.proto=/pg_query --go_opt=paths=source_relative $(LIBDIR)/protobuf/pg_query.proto
 	mkdir -p parser/include/protobuf
 	cp -a $(LIBDIR)/protobuf/*.h parser/include/protobuf
 	cp -a $(LIBDIR)/protobuf/*.c parser/
@@ -62,6 +61,7 @@ update_source: $(LIBDIR)
 	# Other support files
 	rm -fr testdata
 	cp -a $(LIBDIR)/testdata testdata
+	bash scripts/gokeep.sh
 
 clean:
 	-@ $(RM) -r $(LIB_TMPDIR)

README.md

@@ -1,4 +1,4 @@
-# pg_query_go [![GoDoc](https://godoc.org/github.com/pganalyze/pg_query_go/v4?status.svg)](https://godoc.org/github.com/pganalyze/pg_query_go/v4)
+# pg_query_go [![GoDoc](https://godoc.org/github.com/pganalyze/pg_query_go/v6?status.svg)](https://godoc.org/github.com/pganalyze/pg_query_go/v6)
 
 Go version of https://github.com/pganalyze/pg_query
 
@@ -10,28 +10,13 @@ You can find further background to why a query's parse tree is useful here: http
 ## Installation
 
 ```
-go get github.com/pganalyze/pg_query_go/v4@latest
+go get github.com/pganalyze/pg_query_go/v6@latest
 ```
 
 Due to compiling parts of PostgreSQL, the first time you build against this library it will take a bit longer.
 
 Expect up to 3 minutes. You can use `go build -x` to see the progress.
 
-## Usage with Go modules
-
-When integrating this library using Go modules, and using a vendor/ directory,
-you will need to explicitly copy over some of the C build files, since Go does
-not copy files in subfolders without .go files whilst vendoring.
-
-The best way to do so is to use [modvendor](https://github.com/goware/modvendor),
-and vendor your modules like this:
-
-```
-go mod vendor
-go get -u github.com/goware/modvendor
-modvendor -copy="**/*.c **/*.h **/*.proto" -v
-```
-
 ## Usage
 
 ### Parsing a query into JSON
@@ -42,23 +27,24 @@ Put the following in a new Go package, after having installed pg_query as above:
 package main
 
 import (
-  "fmt"
-  "github.com/pganalyze/pg_query_go"
+	"fmt"
+
+	pg_query "github.com/pganalyze/pg_query_go/v6"
 )
 
 func main() {
-  tree, err := pg_query.ParseToJSON("SELECT 1")
-  if err != nil {
-    panic(err);
-  }
-  fmt.Printf("%s\n", tree)
+	tree, err := pg_query.ParseToJSON("SELECT 1")
+	if err != nil {
+		panic(err)
+	}
+	fmt.Printf("%s\n", tree)
 }
 ```
 
 Running will output the query's parse tree as JSON:
 
 ```json
-{"version":130002,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget":{"val":{"A_Const":{"val":{"Integer":{"ival":1}},"location":7}},"location":7}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}]}
+{"version":170004,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget":{"val":{"A_Const":{"ival":{"ival":1},"location":7}},"location":7}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}]}
 ```
 
 ### Parsing a query into Go structs
@@ -71,7 +57,7 @@ package main
 import (
 	"fmt"
 
-	pg_query "github.com/pganalyze/pg_query_go/v2"
+	pg_query "github.com/pganalyze/pg_query_go/v6"
 )
 
 func main() {
@@ -81,13 +67,13 @@ func main() {
 	}
 
 	// This will output "42"
-	fmt.Printf("%d\n", result.Stmts[0].Stmt.GetSelectStmt().GetTargetList()[0].GetResTarget().GetVal().GetAConst().GetVal().GetInteger().Ival)
+	fmt.Printf("%d\n", result.Stmts[0].Stmt.GetSelectStmt().GetTargetList()[0].GetResTarget().GetVal().GetAConst().GetIval().Ival)
 }
 ```
 
 You can find all the node types in the `pg_query.pb.go` Protobuf definition.
 
-### Deparsing a parse tree back into a SQL statement (Experimental)
+### Deparsing a parse tree back into a SQL statement
 
 In order to go back from a parse tree to a SQL statement, you can use the deparsing functionality:
 
@@ -97,7 +83,7 @@ package main
 import (
 	"fmt"
 
-	pg_query "github.com/pganalyze/pg_query_go/v2"
+	pg_query "github.com/pganalyze/pg_query_go/v6"
 )
 
 func main() {
@@ -132,13 +118,14 @@ Put the following in a new Go package, after having installed pg_query as above:
 package main
 
 import (
-  "fmt"
-  "github.com/pganalyze/pg_query_go/v2"
+	"fmt"
+
+	pg_query "github.com/pganalyze/pg_query_go/v6"
 )
 
 func main() {
-  tree, err := pg_query.ParsePlPgSqlToJSON(
-  `CREATE OR REPLACE FUNCTION cs_fmt_browser_version(v_name varchar, v_version varchar)
+	tree, err := pg_query.ParsePlPgSqlToJSON(
+		`CREATE OR REPLACE FUNCTION cs_fmt_browser_version(v_name varchar, v_version varchar)
   			RETURNS varchar AS $$
   			BEGIN
   			    IF v_version IS NULL THEN
@@ -147,10 +134,10 @@ func main() {
   			    RETURN v_name || '/' || v_version;
   			END;
   			$$ LANGUAGE plpgsql;`)
-  if err != nil {
-    panic(err);
-  }
-  fmt.Printf("%s\n", tree)
+	if err != nil {
+		panic(err)
+	}
+	fmt.Printf("%s\n", tree)
 }
 ```
 
@@ -159,31 +146,40 @@ Running will output the functions's parse tree as JSON:
 ```json
 $ go run main.go
 [
-{"PLpgSQL_function":{"datums":[{"PLpgSQL_var":{"refname":"found","datatype":{"PLpgSQL_type":{"typname":"UNKNOWN"}}}}],"action":{"PLpgSQL_stmt_block":{"lineno":2,"body":[{"PLpgSQL_stmt_if":{"lineno":3,"cond":{"PLpgSQL_expr":{"query":"SELECT v_version IS NULL"}},"then_body":[{"PLpgSQL_stmt_return":{"lineno":4,"expr":{"PLpgSQL_expr":{"query":"SELECT v_name"}}}}]}},{"PLpgSQL_stmt_return":{"lineno":6,"expr":{"PLpgSQL_expr":{"query":"SELECT v_name || '/' || v_version"}}}}]}}}}
+{"PLpgSQL_function":{"datums":[{"PLpgSQL_var":{"refname":"v_name","datatype":{"PLpgSQL_type":{"typname":"UNKNOWN"}}}},{"PLpgSQL_var":{"refname":"v_version","datatype":{"PLpgSQL_type":{"typname":"UNKNOWN"}}}},{"PLpgSQL_var":{"refname":"found","datatype":{"PLpgSQL_type":{"typname":"UNKNOWN"}}}}],"action":{"PLpgSQL_stmt_block":{"lineno":2,"body":[{"PLpgSQL_stmt_if":{"lineno":3,"cond":{"PLpgSQL_expr":{"query":"v_version IS NULL"}},"then_body":[{"PLpgSQL_stmt_return":{"lineno":4,"expr":{"PLpgSQL_expr":{"query":"v_name"}}}}]}},{"PLpgSQL_stmt_return":{"lineno":6,"expr":{"PLpgSQL_expr":{"query":"v_name || '/' || v_version"}}}}]}}}}
 ]
 ```
 
 ## Benchmarks
 
 ```
-BenchmarkParseSelect1-4                  	 1542726	      7757 ns/op	    1168 B/op	      21 allocs/op
-BenchmarkParseSelect2-4                  	  496621	     24027 ns/op	    3184 B/op	      63 allocs/op
-BenchmarkParseCreateTable-4              	  231754	     51624 ns/op	    8832 B/op	     157 allocs/op
-BenchmarkParseSelect1Parallel-4          	 5451890	      2213 ns/op	    1168 B/op	      21 allocs/op
-BenchmarkParseSelect2Parallel-4          	 1711480	      7067 ns/op	    3184 B/op	      63 allocs/op
-BenchmarkParseCreateTableParallel-4      	  759412	     16157 ns/op	    8832 B/op	     157 allocs/op
-BenchmarkRawParseSelect1-4               	 2311986	      5158 ns/op	     192 B/op	       5 allocs/op
-BenchmarkRawParseSelect2-4               	  721333	     16517 ns/op	     384 B/op	       5 allocs/op
-BenchmarkRawParseCreateTable-4           	  328119	     35675 ns/op	    1248 B/op	       5 allocs/op
-BenchmarkRawParseSelect1Parallel-4       	 8378274	      1412 ns/op	     192 B/op	       5 allocs/op
-BenchmarkRawParseSelect2Parallel-4       	 2650692	      4553 ns/op	     384 B/op	       5 allocs/op
-BenchmarkRawParseCreateTableParallel-4   	 1000000	     10335 ns/op	    1248 B/op	       5 allocs/op
-BenchmarkFingerprintSelect1-4           	 5012028	      2388 ns/op	     112 B/op	       4 allocs/op
-BenchmarkFingerprintSelect2-4           	 2391704	      5026 ns/op	     112 B/op	       4 allocs/op
-BenchmarkFingerprintCreateTable-4       	 1304545	      9601 ns/op	     112 B/op	       4 allocs/op
-BenchmarkNormalizeSelect1-4              	 8767273	      1360 ns/op	      72 B/op	       4 allocs/op
-BenchmarkNormalizeSelect2-4              	 4465364	      2756 ns/op	     104 B/op	       4 allocs/op
-BenchmarkNormalizeCreateTable-4          	 2738284	      4345 ns/op	     184 B/op	       4 allocs/op
+$ make benchmark
+go build -a
+go test -test.bench=. -test.run=XXX -test.benchtime 10s -test.benchmem -test.cpu=4
+goos: darwin
+goarch: arm64
+pkg: github.com/pganalyze/pg_query_go/v6
+BenchmarkParseSelect1-4                          2874156              4186 ns/op            1040 B/op         18 allocs/op
+BenchmarkParseSelect2-4                           824781             14572 ns/op            2832 B/op         57 allocs/op
+BenchmarkParseCreateTable-4                       351037             34591 ns/op            8480 B/op        149 allocs/op
+BenchmarkParseSelect1Parallel-4                  9027080              1320 ns/op            1040 B/op         18 allocs/op
+BenchmarkParseSelect2Parallel-4                  2745390              4369 ns/op            2832 B/op         57 allocs/op
+BenchmarkParseCreateTableParallel-4              1000000             10487 ns/op            8480 B/op        149 allocs/op
+BenchmarkRawParseSelect1-4                       3778771              3183 ns/op             128 B/op          3 allocs/op
+BenchmarkRawParseSelect2-4                       1000000             10985 ns/op             288 B/op          3 allocs/op
+BenchmarkRawParseCreateTable-4                    460714             26397 ns/op            1056 B/op          3 allocs/op
+BenchmarkRawParseSelect1Parallel-4              13338790               902.7 ns/op           128 B/op          3 allocs/op
+BenchmarkRawParseSelect2Parallel-4               4060762              2956 ns/op             288 B/op          3 allocs/op
+BenchmarkRawParseCreateTableParallel-4           1709883              7001 ns/op            1056 B/op          3 allocs/op
+BenchmarkFingerprintSelect1-4                    6394882              1875 ns/op              48 B/op          2 allocs/op
+BenchmarkFingerprintSelect2-4                    2865390              4174 ns/op              48 B/op          2 allocs/op
+BenchmarkFingerprintCreateTable-4                1688920              7143 ns/op              48 B/op          2 allocs/op
+BenchmarkNormalizeSelect1-4                     10604962              1133 ns/op              32 B/op          2 allocs/op
+BenchmarkNormalizeSelect2-4                      6226136              1938 ns/op              64 B/op          2 allocs/op
+BenchmarkNormalizeCreateTable-4                  4542387              2635 ns/op             144 B/op          2 allocs/op
+PASS
+ok      github.com/pganalyze/pg_query_go/v6     258.376s
+
 ```
 
 Note that allocation counts exclude the cgo portion, so they are higher than shown here.
@@ -199,6 +195,7 @@ See `benchmark_test.go` for details on the benchmarks.
 ## License
 
 Copyright (c) 2015, Lukas Fittl <lukas@fittl.com><br>
+Copyright (c) 2016-2025, Duboce Labs, Inc. (pganalyze) <team@pganalyze.com>
 pg_query_go is licensed under the 3-clause BSD license, see LICENSE file for details.
 
 This project includes code derived from the [PostgreSQL project](http://www.postgresql.org/),