Refactor: everything is an object

[masak]

This is a work in progress.

The main thrust of this refactor is to finally rid 007 of all the Val:: types and all the Q:: types on the Perl 6 level.

Having the type structure in the host language mirror the type structure in the guest language has carried us unexpectedly far, but the idea is starting to crack at the seams as we introduce classes in 007, whose corresponding type in Perl 6 can't exist. Trying to uphold the illusion leads to contrived code like this:

        elsif $.type ~~ Val::Type {
            my $name = @properties[0].value;
            return $.type.new(:type(EVAL qq[class :: \{
                method attributes \{ () \}
                method ^name(\$) \{ "{$name}" \}
            \}]));
        }

MOP

The more direct beneficiary, though, is #144. Here's a quick note of the MOP (with a "not-quite-M") we're aiming for:

007 name	Perl 6 name	Function	Properties
Object	_007::Object	Most generic type; literally everything's base type; only type with None for a base	type, properties
Type	_007::Type	Represents a type/set/class of objects; handles object creation and creation of derived types; constrains the shape of an object to mirror that of the type	base, fields, methods

These two types circularly depend on each other:

• The type Type is a derived type/subclass of the type Object.
• Both Object and Type are instances of the type Type.

The cycles are established in host code, so the apparent chicken-and-egg problem is not a big issue in practice.

Fun fact: the fact that Type is an instance of type Type means that 007 is susceptible to Girard's paradox. Unfortunately, all explanations of this paradox that I've found are too abstract for me to realize in what way, if any, we'd end up being sorry for inviting this type-system equivalent of Russell's paradox into our language. Until further notice, I consider the above setup rather elegant.

The refactor is the biggest change to 007 since it was created. In order to even remotely manage complexity, I proceed by way of Fowler's asset capture: let the new object model handle one more type, and then remove the old type.

The files _007/Val.pm and _007/Q.pm will end up disappearing, because they represent an obsolete subdivision between "value types" and "Qtypes". Instead, we'll probably end up with _007/Type.pm and _007/Object.pm.

Wrapped types

The point of wrapped types is to borrow the semantics of types from the host language as an internal implementation detail in the guest language. It means that an instance of a wrapped type is an opaque thing with insufficient data on the 007 level to work the way it does. It'd mean that e.g. infix:<==>, infix:<+>, infix:<~> and infix:«<» are "axioms" from the point of view of 007, since they're opaque too and rooted in the host language.

So far the wrapped types are Int, Str, and Array. (Bool is not a wrapped type.)

TODO

Things that I need to finish up before merge:

• Turn TYPE_TYPE and TYPE_INT into a hash
• asset-capture Str and Array
• asset-capture NoneType and Bool
• asset-capture Exception
• create a Dict type, as simple as possible and asset-capture Object (note the pre-Introduce a dictionary type #184 confusion here)
• asset-capture Sub and Macro (this one will be tricky)
• asset-capture Regex
• asset-capture the whole Q type hierarchy
• remove _007/Q.pm
• asset-capture Type
• remove _007/Val.pm
• Rename sevenize into wrap once the latter becomes available
• Put back the type documentation
• Fix all the X::TypeCheck errors which got a meaningless :expected value of _007::Object during the refactor
• Make sure we have a real stringification protocol under Str and maybe repr
• Ditto boolification protocol and Bool
• Reinstate the test file t/features/objects.t that was removed in the heat of battle because it needed to be rewritten for mostly Dict anyway
• Make sure the property method in Runtime.pm goes away completely, fully replaced by method definitions on the classes themselves
• Provide a richer model for fields than just string names: field type, required optional, initializer.
• Go through all the types that used to have an initializer and stop needlessly passing in their default value
• Make sure that objects of type Type fully appear to subclass Object
• There's a pattern that started to show up involving type-chain.reverse.map — finding all the fields, including inherited ones, for example. Find all the uses of this pattern and extract one or more appropriate methods into Type
• Change Type.create from taking an array-of-arrays to taking a dict
• Go through the code base and make sure I didn't introduce temporary cheats somewhere
• Un-quarantine t/features/expr.t (after MoarVM/MoarVM@e86428d45e lands in Rakudo blead)

[masak]

Here's an interesting snag this refactor has run into:

So, we want to replace Val::Sub with a _007::Object of TYPE<Sub>. Fine. It has five properties which we can easily initialize using TYPE<Sub>.create. So far, so good. But it also has a method, named .call, which needs to be defined somehow. I suspect this is hard to think about because it has a chicken-and-egg circularity in it.

...Or maybe not. Maybe I can break the circularity by making the .call method be a NativeSub, providing a sort of base case for the infinite regress. Will try that.

But yeah, this is tortuous to think about. Even though in one sense 007 should've had the factoring we're heading to all along, I strongly doubt I would've been able to produce it from the start.

[masak]

I must say this is coming together really nicely.

[masak]

Yes, we did take a speed hit from lifting the whole Q hierarchy. Round numbers on my computer are: the test suite went from ~65 s to ~160 s.

Update: ~220 s. Still not sorry.

[masak]

This PR has now drifted far from master and might not merge, ever. This is the dreaded "long-running branch" scenario that happens now and then, despite good intentions.

I tried rebasing once already, a few weeks ago, and failed.

So here's a new-year's resolution for you: I want to merge all the things in this PR into master, sooner rather than later. I think the way to do it is in topical parts. Namely:

• Small tutorial fix
• The Object/Dict split
• The whole object system upgrade

The last step might be possible to re-do both as bigger steps, and in conceptual parts. Now we largely know where we're going.

In other words, I want to close this long-running PR, but on a happy note, knowing that all the advances going into this PR have made it in one way or another.

[masak]

Heads-up: I'm going on a two-week vacation starting today. Expect some 007 activity, especially focusing around picking the good stuff out of this PR. (But I also want to look closely at is parsed, regexes, and a bootstrapped parser.)

[masak]

In the design specified in this PR, the top Object type has two fields, always: type and properties.

I now believe that to be an error. Object, if it's to be a true top type in the system, should have 0 fields.

In master, type() is a function anyway. I'm fine with the system reaching into the object and examining some non-property aspect of it to find its type.

I don't immediately have a similarly nice solution for replacing .properties, but I'm still convinced it shouldn't be there. It's not a "regular" property. Maybe JS's Object.keys isn't so bad after all; let's use that one for now.

I can't quite motivate why I feel it's important for Object to be totally empty in this way. Call it a language design intuition.