Skip to content

Conversation

@marndt
Copy link

@marndt marndt commented Dec 23, 2025

lm-evaluation-harness (PyPI: lm-eval) is a framework for few-shot evaluation of language models.

  • Have you followed the guidelines for contributing?
  • Have you ensured that your commits follow the commit style guide?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

@github-actions github-actions bot added python Python use is a significant feature of the PR or issue new formula PR adds a new formula to Homebrew/homebrew-core rust Rust use is a significant feature of the PR or issue labels Dec 23, 2025
@github-actions
Copy link
Contributor

Thanks for contributing to Homebrew! 🎉 It looks like you're having trouble with a CI failure. See our contribution guide for help. You may be most interested in the section on dealing with CI failures. You can find the CI logs in the Checks tab of your pull request.

@p-linnane p-linnane added the CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. label Dec 23, 2025
@github-actions github-actions bot added the autosquash Automatically squash pull request commits according to Homebrew style. label Dec 24, 2025
@github-actions github-actions bot removed the autosquash Automatically squash pull request commits according to Homebrew style. label Dec 24, 2025
@chenrui333 chenrui333 marked this pull request as draft December 24, 2025 18:20
@daeho-ro
Copy link
Member

Test is too heavy, and safetensors build numpy and scipy again even though there are homebrew formula. There might be more efficient way to build this, but I have no idea yet.

lm-evaluation-harness (PyPI: lm-eval) is a framework for few-shot evaluation of language models.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. new formula PR adds a new formula to Homebrew/homebrew-core python Python use is a significant feature of the PR or issue rust Rust use is a significant feature of the PR or issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants