fix: robustly strip psql meta commands #4177
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
size:L
Author
|
@andrewmbenton please review |
This was referenced Nov 10, 2025
|
Thanks @ignat980 this looks a much more complete solution than I was expecting |
Collaborator
|
@ignat980 if you open this against main I can get this merged. Just make sure to includes Andrew's commits. |
kyleconroy
approved these changes
Dec 10, 2025
ignatremizov
force-pushed
the
fix/strip-psql-meta
branch
from
3436dde to
60585cd
Compare
Replace naive line-based removal with a single-pass state machine that correctly distinguishes psql meta-commands from backslashes in SQL code, literals, and comments.
The previous implementation would incorrectly strip any line starting with a backslash, breaking valid SQL containing:
- Backslashes in string literals (E'\\n', escape sequences)
- Meta-command text in comments or documentation
- Dollar-quoted function bodies with backslash content
Changes:
- Track parsing state for single quotes, dollar quotes, and block comments
- Only remove backslash commands at true line starts outside any literal context
- Properly handle escaped quotes (''), nested block comments (/* /* */ */)
- Support dollar-quoted tags with identifiers ($tag$...$tag$)
- Add comprehensive test suite covering:
* All documented psql meta-commands (\connect, \set, \d*, etc.)
* String literals with backslashes and nested quotes
* Dollar-quoted blocks with various tag formats
* Nested block comments containing meta-command text
* Edge cases: empty input, whitespace-only, missing newlines
Performance improvements:
- Pre-allocate output buffer with strings.Builder.Grow()
- Single pass eliminates redundant string operations
- Reduces allocations by avoiding intermediate line slice
ignatremizov
force-pushed
the
fix/strip-psql-meta
branch
from
60585cd to
2181f98
Compare
dosubot
bot
added
size:XL
dosubot
bot
added
size:L
Author
|
@kyleconroy Thanks! I rebased to latest sqlc/main and changed this PR's merge-into branch as sqlc/main. Just waiting on the test CI to finish |
Author
|
@kyleconroy bump for visibility - not sure why the CI tests haven't run, maybe you need to push something? |
Author
|
@kyleconroy bump again ⭐ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix(compiler): robustly strip psql meta commands without breaking SQL
Implements a single-pass state machine that correctly distinguishes psql meta-commands from backslashes in SQL code, literals, and comments.
This fixes schema parsing failures when files contain psql meta-commands like
\connect,\set,\d, etc., which are PostgreSQL client commands that aren't valid SQL.The Problem
Backslashes can appear in valid SQL:
E'\\n', escape sequences)A naive line-based approach would incorrectly strip these, breaking valid SQL.
Changes
''), nested block comments (/* /* */ */)$tag$...$tag$)\connect,\set,\d*, etc.) See PostgreSQLpsqldocsPerformance improvements
strings.Builder.Grow()Testing
go test ./internal/compilerremovePsqlMetaCommands()Credits
Co-authored-by: Andrew Benton [email protected]
Addresses gbarr's comment in #4082 which closes #4065