Skip to content

Comments

feat: regression cli tool#62

Open
huang-benny wants to merge 5 commits intomainfrom
behuang/regression_cli_tool
Open

feat: regression cli tool#62
huang-benny wants to merge 5 commits intomainfrom
behuang/regression_cli_tool

Conversation

@huang-benny
Copy link
Contributor

This PR introduces a CLI tool that uses linear regression to automatically determine optimal signal weights based on target scores for known packages. This tool helps calibrate the scoring algorithm by learning from examples of packages with desired scores.

The regression tool is a command-line tool that:

  • Takes a JSON file mapping package names to desired scores (0-100)
  • Collects signal data for each package using the existing analyzer
  • Uses non-negative least squares regression to find optimal signal weights
  • Shows predicted vs. target scores with mean absolute error
  • Generates a optimal-weights.json file with normalized weights (sum = 100)

Usage:

# Run with example targets
./bin/regression-cli src/regression-lib/example-targets.json

Example Input (src/regression-lib/example-targets.json)

{
  "aws-cdk-lib": 95,
  "cdk-ecr-deployment": 85,
  "@mrgrain/cdk-esbuild": 75,
  "cdk-stacksets": 65,
  "cdk-remote-stack": 55,
  "@aws-amplify/cdk-exported-backend": 35,
  "cdk-ssm-parameter-store": 25
}

Example Output
Screenshot 2025-12-02 at 4 23 14 PM

@huang-benny huang-benny changed the title Behuang/regression cli tool feat: regression cli tool Dec 2, 2025
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i thought we agreed not to publish this? or is this a different version or something

Comment on lines 42 to 44
/**
* Extract all signal names from the current config
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the docstrings here and below have messed up formatting

@@ -0,0 +1,16 @@
#!/usr/bin/env node

require('ts-node/register');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is ts-node/register for?

/**
* Load target scores from file or prompt user
*/
private async loadTargetScores(inputFile?: string): Promise<TargetScores> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you check why the colors are messed up from this point onwards? private is white whereas above this its colored blue. I think its because typescript generally expects all private functions to be defined above any public functions but honestly im not quite sure

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it would be good to have a new doc in the repo that explains the regression tool. like part of a CONTRIBUTING.md file --> where you explain how one might add a new metric to the list, re-calibrate weights, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants