JS: Set timeout dispatcher performance issues on react native

[trema96]

Summary

On React Native the default dispatcher is the SetTimeoutDispatcher, which schedules the queue processing as a js macrotask using setTimeout(0).

I think this does not align with the implementations of the other dispatchers, which schedule the queue processing using process.nextTick on node and use an already resolved promise (Promise.resolve(Unit).then({ process() }), therefore a microtask) on browser.

The observable effect is that something as simple as:

repeat(10_000) { t ->
	(js("Promise.resolve(t)") as Promise<Int>).await()
}

which is almost instantaneous on browser and node (~100ms and ~20ms on my machine), takes minutes on react native (2m45s on my machine, running a sample expo app on ios simulator).

Proposed solution

From what I read in the react native documentation I think setImmediate might be the best fit for a dispatcher specifically for react native.

NativeScript

I focused on the issue on react native as the issue is directly affecting me on that platform, but wouldn't NativeScript also have a similar issue? Is there a reason why the current SetTimeoutDispatcher can't use Promise.resolve(Unit).then({ process() })?

Yielding to macrotask

Implementing a dispatcher using setImmediate would prevent the code in my above example (awaiting already resolved promises) from yielding to the macrotask event loop until finished.

I think this is the appropriate behaviour, as it is the same behaviour as on the current dispatcher for browser, and is the same behaviour for equivalent javascript code:

    async function doProcess() {
        for (let i = 0; i <= 10_000; i++) {
            await Promise.resolve()
            // Will not actually show intermediate progress, rerender is not triggered until doProcess is finished.
            document.getElementById('progressText').innerText = "Progress: " + i
        }
    }

However this is different from the SetTimeoutDispatcher, which schedules the queue processing as a macrotask, and therefore the proposed change might be breaking for some applications.

[trema96]

Somewhat related to this issue, some days ago I opened an issue on youtrack with unintuitive behaviour of yield (https://youtrack.jetbrains.com/issue/KT-81765/KJS-Coroutines-Yield-doesnt-seem-to-let-event-loop-resolve).

When playing around with the dispatchers for this issue I actually encountered an opposite issue, where the coroutines are yielding to the macrotasks in unexpected moments... I'm wondering if more radical changes to the dispatchers are required to have the coroutines behave more naturally in js...

[dkhalanskyjb]

Hi! Thanks for the reports. There are many issues in one, let's split them.

1. Performance on React Native.

On the surface, it looks very reasonable, though unfortunately, none of those working on kotlinx.coroutines have had any experience with React Native (and in general, our knowledge of the JS ecosystem is quite limited), let alone using kotlinx.coroutines in a React Native project, which makes it challenging for us to evaluate your proposal.

It would help us if you could provide a project where the problem can be observed, as well as instructions on how to build and run it. Without such a project, we can also manage, though it will certainly take us longer.

2. yield not yielding to macrotasks.

yield() is not designed to be used for ensuring a specific ordering of work, its role is to prevent starvation of tasks. What you want, semantically, seems to be an "await next tick" function, which should be named accordingly. Your workaround looks good to me, and we may consider adding it to the library itself if this is a common need.

3. Macrotask starvation safeguards.

The behavior of coroutine dispatchers periodically yielding to the JS runtime was introduced near the beginning of the kotlinx.coroutines project: 6d54802 I doubt we could change it even if we wanted to, as people certainly rely on that by this point, but it's also unclear to me why we would.

kotlinx.coroutines has to maintain a balance between consistency with the target platform and consistency with itself on all platforms. I understand the behavior you are describing may be surprising to someone accustomed to the JS approach, yet in my view, it's completely predictable and desirable for a kotlinx.coroutines programmer coming from other platforms. kotlin.coroutines conventions dictate that if you don't want something to happen yet, then you shouldn't throw that piece of work into the queue and rely on it staying in the queue long enough that doing it becomes correct. Instead, there should be an explicit condition for when the work should actually start.

If you have a specific piece of code where the UI updating too soon is a problem, please share it. Then, we can restructure it in a kotlinx.coroutines-idiomatic way so that the order of events is as it should be!

[dkhalanskyjb]

@JSMonk, what's the Kotlin/JS policy regarding React Native in general, by the way? Is it tier 1, 2, 3, or unsupported at all, to borrow the terminology from Kotlin/Native (https://kotlinlang.org/docs/native-target-support.html)?

[trema96]

Performance on React Native

I had created a pull request for this, #4562

I have also created a minimal app for using kotlin from react native so you can try for yourself the performances https://github.com/trema96/react-native-kotlin-coroutines

Yield and macrotask starvation safeguard

For full disclosure I am not a JS expert either, I knew about the event loop but was not familiar with microtasks / macrotasks until recently when I came across these issues, so some things I say/said might be inaccurate.

Example use case - proof of work captcha

The use-case that sparked this issue is a kotlin-multiplatform implementation of a proof-of-work captcha.

In short a proof-of-work captcha is a form of captcha that does not require human interaction / behaviour check, but instead gives a challenge to the client and the client has to solve the challenge by performing some work. A common type of work requested in this operation is finding a value that has certain characteristics when hashed.

Since the captcha may take a few seconds we want to provide feedback to the end-user while the captcha is being calculated (e.g. a loading bar), so we include a callback for reporting progress as input to the challenge resolution method.

On most platform the captcha solution can be calculated on a thread separate from the UI thread, but as js is single threaded we can't do that (actually we could solve this issue by using web-workers on js, but there are some challenges wtih using non-trivial webworkers in a kotlin multiplatform library https://youtrack.jetbrains.com/issue/KT-45455/KJS-Worker-target-support)

So while computing the captcha on JS even though the progress callback is called, changes to DOM won't be rendered to the user unless we make sure we let the DOM rerendering task run after the callback.

Js crypto

For calculating the hashes on js we are using crypto.subtle. The api for digests returns promises. Until recently in the major browsers the methods were returning pending promises, but it seems recently there was a change and now in some browsers return already resolved promises.

The side-effect is that while before something like

        for (let i = 0; i < 100_000; i++) {
            await window.crypto.subtle.digest("SHA-256", new Int8Array([0]))
            if (i % 100 === 0) document.getElementById('s').innerText = "" + i
        }

would show the progressive updates to the end-user, on the browsers that have adopted the change the user only sees the last update.

For kotlin, if I understand correctly, this means that the resolvedPromise.await() still suspends the coroutine, but the coroutine will be resumed as part of a microtask, so the coroutine will resume execution without letting any macrotask resolve (unless we are using the SetTimeoutDispatcher).

Attempted solution with yield

My first attempt at solving the issue was to yield after invoking the progress callback in the challenge resolution, hoping this would let any pending DOM-rerender happen.

I don't think I was in the pitfalls of yield (e.g. trying to ensure a specific ordering of work), I was simply trying to have the UI be responsive.

However after re-reading the documentation of yield I understand that its intention to prevent other kotlin coroutines from starving.

The semantic of yield intentionally does not consider other processes that are depending on the same resources as the coroutines (in this case the other processes being the DOM rerendering and the resource being the js thread), and doesn't prevent them from starving: if this is needed then it is the job of the dispatcher.

Dispatcher yielding to macrotasks

My previous message was unclear, I don't think the dispatcher yielding to the macrotasks is a bad thing, on the contrary.

However, I feel like in my specific case I ended up in a situation where this safeguard is "failing" and displays some seemingly inconsistent behaviours.

I have made an example project for this https://github.com/trema96/kotlin-js-parallel-macrotask based on my proof-of-work captcha case.

The unexpected behaviour I'm observing is that if I have a single coroutine doing all the work, even if it suspends, as long as the coroutine resumes before macrotasks are checked, then the MessageQueue actually never yields to the macrotasks. This is the case for example if the coroutine was suspended for awaiting on an already resolved promise.

If I split the work across multiple coroutines (in my example 20, I think at least MessageQueue.yeldEvery + 1 is required) then there is the yield to the macrotask even if the coroutines would be allowed to resume before macrotasks are checked.

I would expect to have a yield to macrotasks also for a single coroutine then, since both the multi-coroutines and single-coroutines approaches are doing the same amount of work...

I think the reason for why we don't see the yield to the macrotask with a single coroutine is that:

1. We start the coroutine, it is processed by MessageQueue
2. The coroutine executes until the call the await on js digest's promise, and even though it is already resolved the coroutine (rightfully) suspends
3. We have finished processing the message queue, no need to do schedule or reschedule the queue processing
4. We proceed with the next js microtask, which is the then of the promise our coroutine was awaiting on and the coroutine can resume
5. We go back to step 2 without leaving time for the macrotask to execute (even if the coroutine already suspended MessageQueue.yeldEvery or more times).

Is this intended behaviour?

If not I think it could maybe be handled by using a counter of processable coroutines as a var at the level of the MessageQueue, and reset it only when a macrotask is executed (instead of the simple repeat(yieldEvery)).