Probes II

[disconcision]

Core features:

• Refractors: A variant on projectors that do not replace the underlying syntax, but instead decorate it, while allowing a custom offside display.
• Probes: Probes are now refractors, so you can edit a term even with a probe on, and you can have probes inside other probes.
• Autoprobes: These automatically add probes to each line of the autoprobed expression.
• Context menu: Right click on syntax to open. Replaces projector panel. Also has jump-to-definition.
• Sample dropdown: Hover over a probe on a function to access contextual actions (Pin and Step-Into; see below)
• Probe Sidebar: (camera lens icon) for probe extras and configuration

Core internals:

• Clarify separation internal dynamics (Sample collection) versus instrumentation UI (Probes). Probe construct removed from AST in favor of a map of sample targets which is passed to the evaluator

Non-probe Bug fixes:

• Fixed bug when click on code when caret is located offscreen would cause a sudden scroll to initial caret position

Non-probe supporting features:

• Better emoji and general Unicode support in strings
• Messages to/from worker are no longer serialized and hence much faster
• Projector system: Viewport culling for projector view rendering perf (for scratch and documentation mode only)
• Concave grout now the precedence that semicolons should (but don't) have. This is looser than any operator but tighter than things like function literals in order to get live feedback when inserting a new let expression inside an existing function literal
• Scroll to caret now keeps a buffer around the caret
• Underline decoration when hovering on references when command is held to indicate jump to def
• Script to generate a string of all builtins for documentation and prompting purposes

Context menu:

• Add context menu
• Add jump to definition
• Only show probe options for expressions/patterns
• Don't show probe option for fake expressions (e.g. labels)
• Restrict autoprobe to expressions
• Keyboard shortcut display

Adding/remove probes:

• New keyboard shortcut: Cmd/Ctrl-E to toggle a manual refractor probe on the indicated term
• Add via context menu
• Allow probes (refractors) to be placed on projectors
• Replace Probe projector in Panel with refractor
• Command to remove all probes within a selection (reuse Cmd-E)
• Make manual and auto probes mutually exclusive when toggling
• Remove a probe if another is added on the same line
• Smart probe placement for arrow types. If you'd be probing a term of arrow type, try to probe subterms containing or contained by that term instead. We currently consider 4 cases: (a) If probing a function literal, probe parameters and body instead. (b) If probing a variable in function position of an application, probe the application instead. (c) If probing a partially applied function, in function position wrt an application (typically but not limited to a reverse application), probe the application instead. (d) (combining (b) and (c)) Variable in fun pos of partial ap in fun pos of ap (typically reverse). In a separate case, probing a let definition just probes the definition term. The following expression cases do not support probing: type alias definition (no point), deferrals in partial aps (no point + breaks parse), and labels (same).
• Probe remove keyboard toggle works when a sample is focussed

Sample display:

• Only cap sample value length if there is actually more than one closure on that line
• Style Indet samples differently
• Add Sample legend in sidebar
• Add time-based coloring option
• Add duration-Interval-based coloring option
• Hide dyncursor-unrelated samples in Single sample mode
• Explanatory icons when no samples are shown; clicking these will try to show sample when possible

Internals:

• State: Trigger/deserialize probe projector invocation into refractor probes
• State: Serialize refractor probes into invocation
• Rebuilt all doc slides
• Sample collection: Eliminate Probe AST wrapping form in favor of a set of probed UUIDs
• Sample collections: Now collects times and other data

StepInto:

• StepInto jumps to function definition (function parameters) and adds an autoprobe
• StepInto shouldn't change probe status if already probed
• Add StepInto to sample dropdown
• StepInto pins the ap
• StepInto updates the DynCursor to include the source ap

Pinning:

• Adding: Add pin action to sample dropdown
• Adding: Add keyboard shortcut (p) when sample is focussed
• Adding: Disallow pinning constructor applications
• Adding: Disallow pinning builtins
• Adding: Only allow one pin at a time
• Removing: When removing (manual/auto) probe with pinned sample, remove pin
• Removing: When bulk-removing probes via selection, remove a pin on sample within selection
• View: Move pin icon to left instead of right so visible for large values
• View: When a probe would show samples, but doesn't because of pin, show an unpin affordance in lieu of samples

Autoprobes:

• Add via keyboard shortcut: Cmd/Ctrl-Shift-E
• Add via context menu, or convert to manual probe
• Indicate current probed state of term
• Basic placement logic: Autoprobing a term probes the 'last largest term' on each line of that term.
• Advanced logic: Modifying the above, favor probing useful things over less useful things.
• Less useful things determinable syntactically include: function literals (no value rep).
• Less useful things determinable statically include references of variables already seen through the patterns (either singly as a probe on a pattern variable, or in composite as in a tuple or list pattern, e.g. if we have a probe on the pattern (a, b) then we've seen and can tell the bindings a and b). UPDATE: This actually results in confusing omissions; disabled; was annoying when e.g. the ref is a branch of an a if expression
• Special case: For tuples and list literals where the literal and the last element but not the second last element end on the same line to put the probe on the last element. This is perhaps an awkward description. the idea is that when we want this behavior: (a, // show value of a here [linebreak] b) // show value of b here, NOT the value of (a,b)
• Special case: Technically in hazel tuples don't require parens around them, ie the parens counts as a separate term, the same as disambiguation parens. but if we have closing parens on a line following the last element's last line, we don't want to bother putting a probe on the parens (we are assuming that given that the last element will be probed, other elements of the tuple that the user wants to probe can or are being probed as well; not 100% sure about this logic, tbh)
• More special cases in AutoProbe.re
• Make autoprobe line change color on hover/indication to clarify which expression correspond to which value

Sidebar:

• Show indicated call if any as indication in call cursor (red outline)
• Show index in call cursor (brown outline), dim entries below index
• Show pinned call in call cursor
• Make sure indicated call is always in call cursor? i would think it would be but maybe not
• Combine manual and auto probe lists in sidebar
• Nest auto probes together in sidebar
• Partition probes in sidebar by top level definitions
• Toggle: Single/Many
• Toggle: Step/Call
• Toggles for below/above colors
• Toggles for caller/callee colors
• 1 versus Many toggles for below/above/caller/callee

---

Review:

Done:

• Context menu should say "add/remove projector"
• Allow both "P" and "Shift-P" for pinning
• Disable debug popup which causes a focus issue
• Don't show step into when application has anything other than a variable (e.g.function literals) in function position (For now I've just straight-up disabled both stepping and pinning for anything other than an ap with a variable in function position. We can revisit this later)
• Remove Option/Alt-V shortcut for probes everywhere in favor of Cmd/Ctrl-E
• Double click the sample to toggle back to single sample mode
• Replace bottom right corner-decoration with an indicator of whether there are any statics errors remaining. This should be already calculated in the statics cache. See what other global feedback we could offer that is already calculate and we could also have a status icon for no perf cost. Probably prioritize statics errors but if none could show if there are empty holes (but only if we can get this info cheap! don't want to iterate again over the syntax JUST for this)
• Exercise mode: Style for probe counts when there is an error on the probed term and not focused
• "Ad probe" typo in comments on doc slide for probe; remove option-v; general consistency pass for comments. Will need to be updated again in probes-III after sample coloring scheme changes
• Generalize mechanism that prevents two probes being on a line. Probably need to run a check after certain or all code edits. Iterate through probes (after measured has been recalculated; this may be subtle!) and see if two probes have ended up on the same line. If so, get rid of one of them. Find a reasonable way of deciding which
  Bugs:
• Bug: Clicking the "..." next to a sample toggles you to an arbitrary (?) sample
• Bug: Step into causes frankenstein state i.e. focus isn't kept in sync with browser notion of focus (Just transfers focus to parent editor for now)

[codecov]

Codecov Report

❌ Patch coverage is 22.12058% with 1873 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.83%. Comparing base (4653249) to head (29bb6c1).
⚠️ Report is 261 commits behind head on dev.

Files with missing lines	Patch %	Lines
.../haz3lcore/projectors/implementations/ProbeProj.re	0.33%	299 Missing ⚠️
src/web/app/editors/code/ContextMenu.re	0.90%	220 Missing ⚠️
src/haz3lcore/ProbePerform.re	0.00%	204 Missing ⚠️
src/util/SafeTriangle.re	0.58%	170 Missing ⚠️
src/language/dynamics/Sample.re	12.29%	157 Missing ⚠️
src/language/term/Abbreviate.re	31.73%	71 Missing ⚠️
src/web/app/editors/decoration/Arms.re	0.00%	60 Missing ⚠️
src/haz3lcore/projectors/ProbeText.re	0.00%	58 Missing ⚠️
src/web/app/editors/code/CodeEditable.re	0.00%	57 Missing ⚠️
src/haz3lcore/derived/AutoProbe.re	70.74%	55 Missing ⚠️
... and 55 more

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1879      +/-   ##
==========================================
- Coverage   49.83%   47.83%   -2.00%     
==========================================
  Files         218      229      +11     
  Lines       23103    24815    +1712     
==========================================
+ Hits        11513    11871     +358     
- Misses      11590    12944    +1354     

Files with missing lines	Coverage Δ
src/b2t2/Datasheet.re	100.00% <100.00%> (ø)
src/haz3lcore/CachedSyntax.re	47.05% <100.00%> (ø)
src/haz3lcore/derived/Indentation.re	93.97% <ø> (+1.11%)	⬆️
src/haz3lcore/derived/TermData.re	55.31% <100.00%> (+38.24%)	⬆️
src/haz3lcore/lang/Form.re	92.59% <100.00%> (+0.03%)	⬆️
src/haz3lcore/lang/Precedence.re	94.23% <ø> (ø)
src/haz3lcore/pretty/ExpToSegment.re	76.69% <100.00%> (+0.79%)	⬆️
src/haz3lcore/projectors/ProjectorCore.re	79.54% <100.00%> (ø)
src/haz3lcore/projectors/ProjectorInit.re	72.72% <100.00%> (ø)
src/haz3lcore/tiles/Segment.re	66.17% <100.00%> (-0.91%)	⬇️
... and 97 more

... and 10 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cyrus-]

most critical:

• clicking the "..." next to a sample toggles you to an arbitrary (?) sample
• cursor frankenstein state (focus isn't kept in sync with browser notion of focus)
• step into causes frankenstein state

other stuff (triage):

• Context menu should say "add/remove projector"
• On bottom right of chrome, have it be an indicator of whether there are any errors/warnings/holes remaining
• "Ad probe" typo in comments on doc slide for probe
• Do general consistency pass for comments
• Disable debug popup which causes a focus issue
• allow both "P" and "Shift-P" for pinning
• hide the "P" and "Enter" keyboard shortcut indicators when not focused
• double click the sample to toggle back to single sample mode
• remove Opt/Alt-V shortcut in favor of Cmd/Ctrl-E (and change text in doc slides)
• style for probe counts when there is an error on the probed term and not focused
• can still have two probes on the same line if it happens via an edit (e.g. backspace)
• include icon for samples not aligned with cursor in legend in many mode
• consistency of naming of "cursor" -> should be "dynamic cursor" whenever possible
• move Dynamic Cursor box in probearium to top or some other way to make clear that the legend reference to "Cursor" is the dynamic cursor
• think about being able to toggle off probes on a sub-expression when it is inside an autoprobe (lifetime as long as the autoprobe lives)
• step into doesn't work for function literals in function position
• be able to bulk probe via selection using same logic as auto probing
• when probes show up there appears to be some sort of animation/shimmer maybe involving the depth indicators?

[disconcision]

@cyrus- Addressed most of your comments; see the 'Review' section above for what is fixed here. Two things were defered to probes-III (search 'cyrus' on #2061) and four things were deferred to TBD (search 'cyrus' on #2058). This is good to merge imo