Why not definterface, and why+how is deftype limited?

[edenworky]

The README mentions that definterface is not supported and that deftype has limited support with no further elaboration. The Babashka book details this a bit further https://book.babashka.org/#differences-with-clojure but is somewhat inconsistent with the README, saying there's "no deftype" at all. I take it this means sci has a limited deftype while Babashka doesn't have one at all?

definterface, being a Java-specific thing, makes sense insofar as not being implemented natively in sci, but is there any good reason not to be able to pass in definterface in SCI-on-JVM? Is this possible already, and thereby we'd be able to run Java-specific Clojure code on SCI-on-JVM by binding definterface to the session? If so that's great, and should likely be documented somewhere (I can see why enabling it by default on the JVM would be weird/undesired, although it would simplify the existing code, such as here https://github.com/babashka/sci/blob/master/src/sci/impl/records.cljc#L19 which errors out given an interface only on the JVM, which is the one place we have definterface anyway).

deftype to my understanding is mostly complicated because it fully relies on JVM constructs for its machinations, and there isn't really any way even in clojure.core to take a value and get a set of everything it satisfies?, which explains the one limitation (listed, again, in the Babashka book and not in the sci docs anywhere) is that reify only supports one type? Is this the case for sci in general? Is this the "limited support" described by the README? Are there other ways it's "limited?"

In a broader sense, is full/better deftype and/or definterface support planned? I can think of a few ways involving meta-in-reify and/or a macro generating a composition of deftypes to potentially address this, but the immediate issue is that the current state of these things is not apparent in the docs anywhere. If I understand it I'd be glad to send a PR for the README.

[borkdude]

The current state of affairs:

"supported" generally means: not supported from source, but when you insert your own definterface an a SCI configuration, it will work, e.g. like this:

user=> (def x (reify Dude)) (instance? Dude x)
#'user/x

user=> (sci/eval-string "(instance? Dude x)" {:classes {'Dude Dude} :namespaces {'user {'x x}}})}
true

Since the definterface doesn't run within SCI, it will work, like any other interface.

• definterface: not supported inside of SCI
• deftype: available, works with protocols but not with Java interfaces. The reason for this is that SCI was initially primarily designed to be used with babashka in GraalVM native-image and you can't create new Java types in native-image. deftype in SCI basically creates a built-in object that looks like a new type, to enable some of the common use cases in Clojure. The reason interfaces on the type aren't supported has to do with an explosion of combinations of types that have to be pre-created to be able match exactly what the users wants. We do this with reify in babashka for one interface at a time for a pre-selected amount of interfaces. We could do this for deftype too potentially, but it will still be limited to just one interface or a popular combination of multiple, but still: everything has to be pre-configured at build time. I have tried to pre-generate every conceivable combination of interfaces via macros, but this blew up the native-image size.

If things can be improved, I'm open to it of course.

If docs are behind, PRs are welcome to bring things up to date!

[edenworky]

Hi, thank you for the response, this answers some of my questions but not others.

My intention insofar as passing definterface into SCI isn't passing in an interface I define using definterface, but binding in the definterface macro itself and evaluating code in SCI which makes calls to definterface directly on the host system.
I'm asking primarily because, if so, this can massively simplify (and relax) related docs from "we don't support definterface" to "definterface is an interop detail, if you want it, bind it yourself like so: <CODE EXAMPLE HERE>".

Similarly for deftype, which (and thank you for elucidating on the complications involved) seems mostly complicated for GraalVM reasons, but SCI runs on many platforms (including as a native library), as well as custom GraalVM native images (i.e., not babashka). So why not be able to pass in interfaces and definterface if the host system has one? And honestly, while we're at it, is it possible to just bind in deftype?

Personally I'm much more interested in a solution to being able to reify on multiple types than Java interfaces in particular, is my intuition correct that this is solvable for non-babashka uses and is not currently solved? What the status on that?

Moreover, I don't entirely follow your explanation on the prior work in this direction. I understand ofc why it's too much too load all interfaces and then their permutations in the babashka native image, but I don't understand why we'd need all the permutations when we only really care about ones composed under deftype explicitly by name/reference. Put another way, why can't we just generate the permutations relevant to the specific explicit ones made in e.g. deftype, defprotocol, and reify calls? I can see how this design wouldn't work for SCI-in-native-image, but SCI doesn't just run in native images.

Moreover moreover, personally I use babashka for tasks and (working on) using it as a little repl session manager over JVM repls, and besides this there's the (probably much more common) use case of developing code in a JVM env and then compiling it down to a native image. In these situations, I have definterface available and the ability to make new Java types, which in any case would either be passed explicitly into SCI or else evaluated inside of it on a host which supports this. So even in this case I imagine it should theoretically be possible to wrap those explicit JVM bindings during the Java->native-image build and then at the very least we could generate all permutations of only explicitly binded (bound?) types/interfaces/compositions thereof.

This still ofc wouldn't let us do this stuff on runtime in a native image, but it would let us do this stuff not in native images, as well as in the process of building one, and thereby use "dynamic" SCI to build (even incrementally) better more bespoke "native" SCIs.

Edit: To clarify I don't mean to push for a solution specifically or even declare a problem necessarily (past the docs as mentioned which I'd be glad to patch as mentioned), moreso throwing some ideas and questions at you, some of which I imagine there are existing answers for and some which I imagine might not, and therefore proposing naive solutions naively. Mostly I'm just trying to delineate the state of affairs as mentioned. Thank you for your work and patience!