[SUGGESTION] General plugin system improvement

[Technisha]

Heya! Just gonna list some improvements in the plugin system i'd personally like, in order of what i, personally, would think would be the easiest to implement, from easiest to hardest!

• The ability to add multiple authors. (Since i'd be developing as a team with friends)
• The ability to depend on other Python plugins and extend the functionality of them (like overriding the certain commands and calling the functions for it)
• The ability to depend on Java plugins
• The ability to actually call basic Java classes and extend the plugins (definitely the hardest, since Python-Java interop is super hard afaik, but could potentially be done via PY4J somehow or jpype)

[Thrameos]

The choice of py4j or jpype depends on how you intend to use java. Py4j is a pipe solution so best when you need to disconnect the jvm and restart it but with the downside you must wrap java objects to give them python interfaces and transfer speed can be an issue. Jpype is more for making existing java packages work as python modules with little wrapping. But jpype connects the virtual machines so it offers high speed transfers at the cost that the jvm will have to live for the duration of python and cant be restarted. So if you want an rpc like environment go with py4j. If you want to access java as python go with jpype.

[Technisha]

Honestly i think JPype would be the most ideal for a situation like this, it'd just need to be included into the main API so it doesn't need to start the JVM multiple times which will slow it down

[Technisha]

I think adding it as an optional part of the API would be nice so JPype isn't required but can be used for extra support and stuff

[benbaptist]

I love the idea of wrapping into Java directly somehow and getting deeper with the interactions! This would be amazing to make happen. The first two suggestions should be easy enough to add, so I'll make that a priority for my next commit. Definitely interested about the Java-Python interaction ideas

[Technisha]

Glad you think so! Thanks for working on such an awesome project btw, i'm really glad i found this! (I know basic Java but i prefer Python so much more)

[Thrameos]

Limitations to consider when using JPype:

Python is slower than Java and it shows. Very simple actions in Python can hit the object bottleneck. So if something is going to be called 1000 times but only do something meaningful 1% of the time, it is best to have a Java function that only calls the Python when the condition is met rather than checking the condition in Python. It is best to use Python for setup like code which is called once to install predefined hooks in Java rather than implement a lot of Python code that gets hit regularly.

JPype can implement Java interfaces in Python, but they operate using the Python threading model which requires a master lock. This can lead to deadlocks if weaving back and forth between Java and Python code.

JPype can not as of yet extend Java classes. It is a planned feature, but there are a lot of complexities.

Java can only call Python using interfaces currently. This is not Jython where Python objects have Java presence as anything other than foreign objects.

Python objects can be stored in Java structures but only if there is a wrapper as a Proxy for an interface. Circular references of Python and Java can create memory leaks. Python has hidden linked objects such as through global, so keeping things in modules with minimal references to objects outside scope helps prevent leaks. Thus try to keep containers simple (Java objects hold leaf Python objects or Python object holding leaf Java objects). Python container holding Java container holding Python objects is sure to leak.

Byte code unpacking and conversion is an option for simple things. You can disassemble certain Python functions into simple Python abstract syntax trees and then use Java ASM to create Java versions for critical sections. Though this requires a lot of Python and Java knowledge. I am working on this for Python extensions of Java objects. But you need a lot of decorations to pull this off as Python being weakly type translates poorly without some decorators to annotate the types.