Fix syntax highlighting by locking middleman-syntax to 3.5.0

[huwd]

What’s changed

Lock middelman syntax to 3.5.0

Identifying a user need

Currently blocking tests preventing releases

Details (also in commit message)

Starting around July 29th 2025 tests started failing between ruby 3.1, 3.2 and 3.3 versions.

The two failures were:

1) GovukTechDocs::TechDocsHTMLRenderer#render a code block with syntax highlighting sets tab index to 0
     Failure/Error: fragment = Nokogiri::HTML::DocumentFragment.parse(super)

     NoMethodError:
       undefined method `each' for nil:NilClass

             opts.each { |k, v| @options[k.to_s] = v }
                 ^^^^^
     # ./vendor/bundle/ruby/3.1.0/gems/rouge-3.30.0/lib/rouge/lexer.rb:325:in `initialize'
     # ./vendor/bundle/ruby/3.1.0/gems/rouge-3.30.0/lib/rouge/lexer.rb:97:in `new'
     # ./vendor/bundle/ruby/3.1.0/gems/rouge-3.30.0/lib/rouge/lexer.rb:97:in `find_fancy'
     # ./vendor/bundle/ruby/3.1.0/gems/middleman-syntax-3.6.0/lib/middleman-syntax/highlighter.rb:13:in `highlight'
     # ./vendor/bundle/ruby/3.1.0/gems/middleman-syntax-3.6.0/lib/middleman-syntax/redcarpet_code_renderer.rb:10:in `block_code'
     # ./lib/govuk_tech_docs/tech_docs_html_renderer.rb:90:in `block_code'
     # ./spec/govuk_tech_docs/tech_docs_html_renderer_spec.rb:49:in `render'
     # ./spec/govuk_tech_docs/tech_docs_html_renderer_spec.rb:49:in `block (3 levels) in <top (required)>'
     # ./spec/govuk_tech_docs/tech_docs_html_renderer_spec.rb:85:in `block (4 levels) in <top (required)>'

   2) GovukTechDocs::TechDocsHTMLRenderer#render a code block with syntax highlighting renders the code with syntax highlighting
     Failure/Error: fragment = Nokogiri::HTML::DocumentFragment.parse(super)

     NoMethodError:
       undefined method `each' for nil:NilClass

             opts.each { |k, v| @options[k.to_s] = v }
                 ^^^^^
     # ./vendor/bundle/ruby/3.1.0/gems/rouge-3.30.0/lib/rouge/lexer.rb:325:in `initialize'
     # ./vendor/bundle/ruby/3.1.0/gems/rouge-3.30.0/lib/rouge/lexer.rb:97:in `new'
     # ./vendor/bundle/ruby/3.1.0/gems/rouge-3.30.0/lib/rouge/lexer.rb:97:in `find_fancy'
     # ./vendor/bundle/ruby/3.1.0/gems/middleman-syntax-3.6.0/lib/middleman-syntax/highlighter.rb:13:in `highlight'
     # ./vendor/bundle/ruby/3.1.0/gems/middleman-syntax-3.6.0/lib/middleman-syntax/redcarpet_code_renderer.rb:10:in `block_code'
     # ./lib/govuk_tech_docs/tech_docs_html_renderer.rb:90:in `block_code'
     # ./spec/govuk_tech_docs/tech_docs_html_renderer_spec.rb:49:in `render'
     # ./spec/govuk_tech_docs/tech_docs_html_renderer_spec.rb:49:in `block (3 levels) in <top (required)>'
     # ./spec/govuk_tech_docs/tech_docs_html_renderer_spec.rb:89:in `block (4 levels) in <top (required)>'

This seemed odd because we hadn't done anything to change this area in the intervening time.

After some investigations of if it could be related to a ruby change (and a slight adventure where a very old Gemfile.lock meant we couldn't reproduce this locally 🤦), it was pointed out that middleman-syntax had an update on Jule 23rd 2025 to v3.6.0: https://github.com/middleman/middleman-syntax/releases/tag/v3.6.0

This seems to be specifically about lexer, which draws attention: middleman/middleman-syntax#92

So what do I think is going on here?
Well, Lexer seems to be throwing the error, apparently because the opts parameter which should be a hash, is being passed as nil.

We can see why that would cause problems when calling the enumerator here:
https://github.com/rouge-ruby/rouge/blob/3b461b1ffe5fc6416373df8c3c35da83a283606d/lib/rouge/lexer.rb#L323

This gets called from lexer's find_fancy class, both nothing has changed there nor is there anything that I can see that mutates opts to nil in either lookup_fancy or find_fancy:
https://github.com/rouge-ruby/rouge/blob/3b461b1ffe5fc6416373df8c3c35da83a283606d/lib/rouge/lexer.rb#L46 https://github.com/rouge-ruby/rouge/blob/3b461b1ffe5fc6416373df8c3c35da83a283606d/lib/rouge/lexer.rb#L94

So we're back in our trace into middleman-syntax, here's where find_fancy gets called:
https://github.com/middleman/middleman-syntax/blob/d5042d6a583494aad3ceb0517685e945aa093d9b/lib/middleman-syntax/highlighter.rb#L13 Line 13 must make it through lexer without error before Rogue::Lexer::PLainText can be offered as a fallback. And at this point in a debugger i'm seeing that lexer_options has become nil!

So I think this line is sus:
https://github.com/middleman/middleman-syntax/blob/d5042d6a583494aad3ceb0517685e945aa093d9b/lib/middleman-syntax/highlighter.rb#L11 on line 11, the helper is attempting to delete lexer_options but when there is no key of lexer_options then it's returning nil, which is expected behaviour from delete.
https://ruby-doc.org/3.4.1/Hash.html#method-i-delete

You can test this with:

lexer_options = highlighter_options.delete(:lexer_options)
lexer_options

When I think what they want is:

lexer_options = {}.delete_if {|k,v| k == :lexer_options}

I'll open an issue over there, but for the time being for syntax highlighting to continue to work we want to stay locked to v3.5.0.

This change can be disregarded and the lock removed once we think this problem has been resolved and the existing test suite passes.

[natf-gds]

Sounds like a pragmatic fix.

I'm unsure of the standard behaviour at GDS, should we raise a ticket on the issues backlog to undo this change when is fixed on the dependency, for tracking?

We also have a ticket talking about uplifting the version of middleman to move to a Github version #386

What I have done is post a link to this issue on !386 #386 (comment) so that others have the context if they decide to uplift.

[levgorbunov1]

Do we want to add a comment on this line referencing the issue in the middleman repo/just a brief note, so we remember to unpin it later on?

[davidr-gds]

Woop, new middleman-syntax release! Maybe we can unpin it again now?

@huwd Just released a new version with this fix:
https://rubygems.org/gems/middleman-syntax/versions/3.6.1
https://github.com/middleman/middleman-syntax/blob/master/CHANGELOG.md#361

source

[huwd]

yay, they merged my PR:
middleman/middleman-syntax#94

[huwd]

cool, will close this down on behalf of either:
#427

or "do nothing"