Type checking etc in string conversions (xstring, convert-property-key, etc)

[kchanqvq]

There are a number of problems currently. Let's discuss here!

1. convert-property-key currently doesn't support JS Symbol keys. We need to add internals.xpropkey for that.
2. clstring% and jsstring implementation should be symmetric, but currently they are quite inconsistent. clstring does type check on CL side with internals.make_lisp_string handling only one type (JS string); jsstring miss typecheck, while internals.xstring also does some type dispatch (on JS string and numbers, is this necessary?).
3. A bug:

Welcome to JSCL (version 0.9.0-alpha.0 built on 4 February 2026)

JSCL is a Common Lisp implementation on Javascript.
For more information, visit the project page at [GitHub](https://github.com/jscl-project/jscl).

CL-USER> (use-package :jscl/ffi)
T
CL-USER> (jsstring #(1 2))
TYPE-ERROR: #(1 2) is not a STRING.
    at https://jscl-project.github.io/jscl.js:42582:7
    at https://jscl-project.github.io/jscl.js:42585:3
    at JSCL_USER_ERROR (https://jscl-project.github.io/jscl.js:42586:3)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:2031:119
    at JSCL_USER_CLSTRING (https://jscl-project.github.io/jscl.js:2032:3)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at JSCL_USER_NIL (https://jscl-project.github.io/jscl.js:35436:36)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:35444:18
    at https://jscl-project.github.io/jscl.js:35445:3
    at JSCL_USER_CONVERTXSTRING (https://jscl-project.github.io/jscl.js:35446:3)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:35467:37
    at JSCL_USER_NIL (https://jscl-project.github.io/jscl.js:35468:3)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:36383:19
    at https://jscl-project.github.io/jscl.js:36384:3
    at https://jscl-project.github.io/jscl.js:36387:4
    at internals.bindSpecialBindings (https://jscl-project.github.io/jscl.js:536:12)
... 68 more frames, increase JSCL::*BACKTRACE-LIMIT* to view more
CL-USER> (funcall #'jsstring #(1 2))
#\SIMPLE-ERROR: ref vector out of range
    at https://jscl-project.github.io/jscl.js:42582:7
    at https://jscl-project.github.io/jscl.js:42585:3
    at internals.Symbol.JSCL_USER_ERROR [as fvalue] (https://jscl-project.github.io/jscl.js:42586:3)
    at internals.error (https://jscl-project.github.io/jscl.js:76:12)
    at https://jscl-project.github.io/jscl.js:9705:26
    at https://jscl-project.github.io/jscl.js:9707:3
    at JSCL_USER_AREF (https://jscl-project.github.io/jscl.js:9708:3)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:6695:27
    at https://jscl-project.github.io/jscl.js:6696:3
    at internals.Symbol.JSCL_USER_CHARNAME [as fvalue] (https://jscl-project.github.io/jscl.js:6697:3)
    at https://jscl-project.github.io/jscl.js:26341:205
    at JSCL_USER_WRITECHARAUX (https://jscl-project.github.io/jscl.js:26342:3)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:26460:54
    at https://jscl-project.github.io/jscl.js:26499:3
    at internals.Symbol.JSCL_USER_WRITEAUX [as fvalue] (https://jscl-project.github.io/jscl.js:26503:4)
    at JSCL_USER_NIL (https://jscl-project.github.io/jscl.js:26617:7)
    at internals.mvcall (https://jscl-project.github.io/jscl.js:66:10)
    at https://jscl-project.github.io/jscl.js:26626:18
... 23 more frames, increase JSCL::*BACKTRACE-LIMIT* to view more

and

JSCL> (funcall #'jsstring 3)
#\3

Originally posted by @davazp in #587

[kchanqvq]

2. internals.xstring also does some type dispatch (on JS string and numbers, is this necessary?).

The numbers case is not necessary. Legacy kludge.

[kchanqvq]

When looking into this I find we now have some hideous mix of JS/CL strings. Maybe the "ensure" behavior of jsstring/clstring isn't a good idea afterall.

E.g., most package now has CL string as packageName, but COMMON-LISP has JS string as packageName 🤦

[kchanqvq]

Maybe the "ensure" behavior of jsstring/clstring isn't a good idea afterall.

But we probably do want both JS string and CL string works as argument to stuff like oget, in, delete-property... right?

@davazp @vlad-km thoughts on this?

[kchanqvq]

But we probably do want both JS string and CL string works as argument to stuff like oget, in, delete-property

How about let's just do NO CONVERSION at all. Get rid of convert-property-key. User are required to pass in JS string. I think this is reasonable given they are working with FFI.

This would also make us support Symbol key without extra effort.

I'd also prefer just handling one type instead of the "ensure" behavior. The "ensure" convenience can go into standard STRING (i.e. we make jsstrings also valid "string designator"s).

[davazp]

I agree. No conversion at all. oget works with CL strings because of the .toString() method, no becuse of conversions (except constant folding).

(defun convert-property-key (form)
  (multiple-value-bind (value constantp) (constant-value form *environment*)
    (if constantp
      value
      ;; xstring handles strings, numbers, and Lisp strings at runtime
      `(call-internal |xstring| ,(convert form)))))

So we can just get rid of the xstring call..

I am not sure if I understand what is the problem with the ensure behaviour.

Can we solve the bug 3) and the symbol support first, and then discuss the clstring / jsstring ensure behaviour? I'm not against changing it to throw if it is needed but it's not very clear to me why yet.

[davazp]

I have a few changes of this in the bootstrap. Hold on with this a sec please. I want to merge the first part (infrastructure) for bootstrap, so we can run test to make sure we do not regress in that front and we consider the cases when changing the FFI.

[vlad-km]

How about let's just do NO CONVERSION at all.

My opinion:
What we have now is a working version.

CL-USER> (setf #j:o (object "name" (jsstring "Magdeburg")))
#<JS-OBJECT [object Object]>
CL-USER> (oget #j:o "name")
#j"Magdeburg"
CL-USER> ((jscl::oget (jscl::oget #j:o "name") "split") "d")
#(#j"Mag" #j"eburg")
;;
CL-USER> (clstring (oget #j:o "name"))
"Magdeburg"
CL-USER> (setq oname (oget #j:o "name"))
#j"Magdeburg"
CL-USER> ((oget oname "split") "e")
#(#j"Magd" #j"burg")
;;
CL-USER> (setf (oget #j:o "name") "Tallinn")
"Tallinn"
CL-USER> (setq o (oget #j:o "name"))
"Tallinn"
CL-USER> ((oget (jsstring o) "split") "a")
#(#\T #j"llinn")
;;
CL-USER> (jscl::in "name" #j:o)
T
;;
CL-USER> (jscl::delete-property "name" #j:o)
#j:undefined
CL-USER> (#j:Object:keys #j:o)
#()
;;
CL-USER> (setf (oget #j:o "name") "Tallinn")
"Tallinn"
CL-USER> (jsstring (oget #j:o "name"))
#j"Tallinn"
CL-USER> ((oget (jsstring (oget #j:o "name")) "split") "a")
#(#\T #j"llinn")

Now I'm not yet ready to say how complete and accurate it is.

(setf #j:o "name" "Magdeburg")
;;js console
;; o
;; {name: Array(7)}

Third party js code, when calls o.name.split('e') will get exception.

[vlad-km]

CL-USER> (funcall #'jsstring #(1 2))
#\SIMPLE-ERROR: ref vector out of range

CL-USER> (jsstring "aa")
#\SIMPLE-ERROR: ref vector out of range

[vlad-km]

Off topic:

Today I was playing around with Google's LLM.

Surprisingly, it quite reliably identified the code as JSCL.

It even remembers to use jsstring / clstring.

[davazp]

@vlad-km It's getting damn crazy. I asked it out of curiosity to make react bindings based on the FFI documentation, and to write an IDE example with source editor, repl and inspector.

react bindings and codemirror bindings for free. And its own source code. It does freeze when trying to edit it though. But to get something quick starting point... imagine how cool it'd be a emacs like extensible lisp ide. With cool graphics library, rendering objects to live interactive SVG... so many possibilities

jscl-react.mov

[kchanqvq]

react bindings and codemirror bindings for free. And its own source code. It does freeze when trying to edit it though. But to get something quick starting point... imagine how cool it'd be a emacs like extensible lisp ide. With cool graphics library, rendering objects to live interactive SVG... so many possibilities

Yeah, I've seen it doing such things, and I've passed the phase of awe :)

The problem is how much code it generates to do this, and how good is the code? Are these a good starting point or you would have to rewrite most of these anyway?

My biggest fear is the following phenomenon that I witnessed actually happened many many times: some project introduces LLM, LLM generates some shiny but large amount of code, after some time the project is impossible to comprehend or manage by human (given the human capacity of the project). Ok, the project can still live for quite long, but nobody understands the full picture and people must rely on LLM to develop. Then it's an exponential snowball, LLM generates more and more code that makes the project more and more inaccessible to human. Horrible! Fun fact: LLM generated documentation doesn't help, because LLM generates wrong docs, or docs get outdated, which confuse human but doesn't bother LLM too much (it will just read the code), so people end up trusting LLM as single source of truth. From first hand experience 😰

[davazp]

I replied #601 to keep this issue on the topic!

[vlad-km]

The problem is how much code it generates to do this, and how good is the code? Are these a good starting point or you would have to rewrite most of these anyway?

Knowing that they were also trained on my "monkey-style" prototype code, I have to check every statement.
Found some shitty code, kicked LLM - it apologized and gave propper result.

My biggest fear is the following phenomenon that I witnessed ...

Yeah, that's right. It scares me too. The price of progress. Progress?

[kchanqvq]

I am not sure if I understand what is the problem with the ensure behaviour.

My worry is it let incorrect use of JS/CL string slips under the radar. Like the packageName situation. If the conversions are not "ensure" style we'll likely notice it right away.

Or we can claim we intend JS/CL strings to be used interchangably. But I don't think that will work fully. Like if someday we decide to switch package implementation to Map, then mix of JS/CL string will probably cause problems.