Skip to content

Conversation

@EvaSDK
Copy link

@EvaSDK EvaSDK commented Jul 11, 2025

Hello,

I tried to use this on a new project and had issues due to the code trying to tokenize whole files instead of yielding the content as it comes. Here is a proposal fixing the problem still making sure an exception will still behave properly.

EvaSDK added 3 commits July 11, 2025 14:10
The test will fail for now showing the shortcomings of trying to
tokenize the while at once.

Signed-off-by: Gilles Dartiguelongue <gilles.dartiguelongue@esiee.org>
Leave a chance to detect the headers even if the content of the file
does not validate against Python tokenizer.

Signed-off-by: Gilles Dartiguelongue <gilles.dartiguelongue@esiee.org>
Signed-off-by: Gilles Dartiguelongue <gilles.dartiguelongue@esiee.org>
@EvaSDK EvaSDK force-pushed the improve_exception_handling branch from c86a51c to 0e93b4c Compare July 11, 2025 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant