Skip to content

fix(translator): always emit header declarations for native methods regardless of elimination status#4752

Closed
Copilot wants to merge 16 commits intosimd-revisitefrom
copilot/fix-ios-build-native-methods
Closed

fix(translator): always emit header declarations for native methods regardless of elimination status#4752
Copilot wants to merge 16 commits intosimd-revisitefrom
copilot/fix-ios-build-native-methods

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 15, 2026

  • Root cause analysis: Invoke.addDependencies() runs at parse time when resolveConcreteInvokeOwner() may return null (because the target class's @Concrete annotation hasn't been parsed yet). Later, appendInstruction() resolves it correctly — so the generated code calls IOSSimd methods, but SimdApiTest.m doesn't #include "IOSSimd.h".
  • Fix: In ByteCodeClass.updateAllDependencies() (which runs after ALL classes are parsed), re-resolve concrete invoke dependencies so that concrete class headers are properly included
  • Verify the fix doesn't break TreeMap super stubs (no change to appendMethodsToHeader)
  • Core unit tests pass (2347 tests)
  • Run validation (code review + CodeQL — both passed)
Original prompt

Problem

The iOS build fails with:

call to undeclared function 'com_codename1_impl_ios_IOSSimd_add___int_1ARRAY_int_1ARRAY_int_1ARRAY_int_int'
call to undeclared function 'com_codename1_impl_ios_IOSSimd_mul___float_1ARRAY_float_1ARRAY_float_1ARRAY_int_int'
call to undeclared function 'com_codename1_impl_ios_IOSSimd_add___byte_1ARRAY_byte_1ARRAY_byte_1ARRAY_int_int'

The translator's resolveConcreteInvokeOwner correctly devirtualizes calls to IOSSimd (because hasDeclaredNonAbstractMethod returns true for native overrides — native methods are never abstract). However, the generated com_codename1_impl_ios_IOSSimd.h is missing declarations for those three native methods.

Root Cause

In ByteCodeClass.appendMethodsToHeader() (line 1662 of ByteCodeClass.java), every method in methods is iterated and m.appendMethodHeader(b) is called. There is no elimination guard — but native methods have no instructions and the optimizer marks them as eliminated via the dead-code elimination pass (since they have no call graph entries pointing to them when the only callers go through Simd's virtual dispatch). Once eliminated, the vtable builder (fillVirtualMethodTable) skips them (if (bm.isEliminated()) continue), so they never enter the vtable and never trigger the header emission that way either.

The fix is simple: appendMethodsToHeader must always emit header declarations for native methods, even if they are marked as eliminated, because native methods are implemented in external .m/.c files — the translator cannot know whether they are truly unused without scanning all native sources, and the hasDeclaredNonAbstractMethod path can route devirtualized calls to them.

Fix

In vm/ByteCodeTranslator/src/com/codename1/tools/translator/ByteCodeClass.java, change appendMethodsToHeader from:

private void appendMethodsToHeader(StringBuilder b) {        
    for(BytecodeMethod m : methods) {
        m.appendMethodHeader(b);
    }
}

to:

private void appendMethodsToHeader(StringBuilder b) {        
    for(BytecodeMethod m : methods) {
        // Native methods must always be declared in the header even if the
        // optimizer marked them as eliminated, because their implementation
        // lives in external native source files (.m/.c) and the bytecode
        // translator can devirtualize calls directly to them via
        // resolveConcreteInvokeOwner / hasDeclaredNonAbstractMethod.
        if (m.isEliminated() && !m.isNative()) {
            continue;
        }
        m.appendMethodHeader(b);
    }
}

Note: isNative() is already a public method on BytecodeMethod (it returns nativeMethod).

This is a targeted, minimal fix. Do not change any other logic. Do not refactor. Only modify the appendMethodsToHeader method in ByteCodeClass.java.

The following is the prior conversation context from the user's chat exploration (may be truncated):

Assistant: Now let me look at the test to understand exactly what methods it's calling, and compare with what IOSSimd.java has vs what Simd.java has:

Assistant: Now I have a clear picture. Let me compare Simd.java vs IOSSimd.java carefully to find what's missing in IOSSimd.java.

Looking at Simd.java, these methods exist but are missing from IOSSimd.java:

  1. add(int[] srcA, int srcAOffset, int[] srcB, int srcBOffset, int[] dst, int dstOffset, int length) — present in Simd.java (line 228), but NOT in IOSSimd.java

Wait — let me re-check the error messages and compare both files more carefully.

The failing calls in the test are:

  • com_codename1_impl_ios_IOSSimd_add___int_1ARRAY_int_1ARRAY_int_1ARRAY_int_int — this is add(int[], int[], int[], int, int)
  • com_codename1_impl_ios_IOSSimd_mul___float_1ARRAY_float_1ARRAY_float_1ARRAY_int_int — this is mul(float[], float[], float[], int, int)
  • com_codename1_impl_ios_IOSSimd_add___byte_1ARRAY_byte_1ARRAY_byte_1ARRAY_int_int — this is add(byte[], byte[], byte[], int, int)

But IOSSimd.java does declare all three of those (lines 42, 105, 177). The implementations are also in IOSSimd.m. So the .m file is there, the .java is there — but the generated C code is calling them instead of the overridden method.

The key insight from the error is:

  • The compiler says com_codename1_impl_ios_IOSSimd_add___int_1ARRAY_... is undeclared (not just undefined)
  • But it points to com_codename1_util_Simd_add___int_1ARRAY_... being declared in com_codename1_util_Simd.h

This means the bytecode translator is generating a /* CustomInvoke */ call directly to the IOSSimd native peer — but it's using the wrong resolution. The translator is seeing IOSSimd.java methods as overrides and generating CustomInvoke calls to the IOSSimd version, but there is no IOSSimd.h header being generated/included for the test file.

The real issue is: **IOSSimd.java is missing the @Override annotation on som...

This pull request was created from Copilot chat.

Copilot AI changed the title [WIP] Fix iOS build by adding missing native method declarations fix(translator): always emit header declarations for native methods regardless of elimination status Apr 15, 2026
Copilot AI requested a review from shai-almog April 15, 2026 01:55
@shai-almog
Copy link
Copy Markdown
Collaborator

shai-almog commented Apr 15, 2026

Compared 37 screenshots: 37 matched.
✅ Native iOS screenshot tests passed.

Benchmark Results

  • VM Translation Time: 0 seconds
  • Compilation Time: 126 seconds

Detailed Performance Metrics

Metric Duration
Simulator Boot 0 ms
Simulator Boot (Run) 1000 ms
App Install 2000 ms
App Launch 4000 ms
Test Execution 180000 ms
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 native encode 1549.000 ms
Base64 CN1 encode 1772.000 ms
Base64 encode ratio (CN1/native) 1.144x (14.4% slower)
Base64 native decode 893.000 ms
Base64 CN1 decode 1322.000 ms
Base64 decode ratio (CN1/native) 1.480x (48.0% slower)
Base64 SIMD encode 3833.000 ms
Base64 encode ratio (SIMD/native) 2.474x (147.4% slower)
Base64 encode ratio (SIMD/CN1) 2.163x (116.3% slower)
Base64 SIMD decode 2129.000 ms
Base64 decode ratio (SIMD/native) 2.384x (138.4% slower)
Base64 decode ratio (SIMD/CN1) 1.610x (61.0% slower)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

✅ ByteCodeTranslator Quality Report

Test & Coverage

  • Tests: 381 total, 0 failed, 2 skipped

Benchmark Results

  • Execution Time: 10651 ms

  • Hotspots (Top 20 sampled methods):

    • 23.56% java.lang.String.indexOf (443 samples)
    • 17.98% com.codename1.tools.translator.Parser.isMethodUsed (338 samples)
    • 12.13% com.codename1.tools.translator.Parser.addToConstantPool (228 samples)
    • 10.05% java.util.ArrayList.indexOf (189 samples)
    • 5.00% java.lang.Object.hashCode (94 samples)
    • 2.98% java.lang.System.identityHashCode (56 samples)
    • 2.50% com.codename1.tools.translator.BytecodeMethod.optimize (47 samples)
    • 2.23% com.codename1.tools.translator.ByteCodeClass.updateAllDependencies (42 samples)
    • 1.70% com.codename1.tools.translator.ByteCodeClass.fillVirtualMethodTable (32 samples)
    • 1.49% com.codename1.tools.translator.Parser.getClassByName (28 samples)
    • 1.38% com.codename1.tools.translator.ByteCodeClass.markDependent (26 samples)
    • 1.33% com.codename1.tools.translator.ByteCodeClass.calcUsedByNative (25 samples)
    • 1.01% com.codename1.tools.translator.BytecodeMethod.appendMethodSignatureSuffixFromDesc (19 samples)
    • 1.01% java.lang.StringBuilder.append (19 samples)
    • 0.80% com.codename1.tools.translator.BytecodeMethod.equals (15 samples)
    • 0.74% java.io.UnixFileSystem.getBooleanAttributes0 (14 samples)
    • 0.69% com.codename1.tools.translator.Parser.cullMethods (13 samples)
    • 0.59% com.codename1.tools.translator.BytecodeMethod.isMethodUsedByNative (11 samples)
    • 0.53% sun.nio.fs.UnixNativeDispatcher.open0 (10 samples)
    • 0.53% com.codename1.tools.translator.Parser.generateClassAndMethodIndexHeader (10 samples)
  • ⚠️ Coverage report not generated.

Static Analysis

  • ✅ SpotBugs: no findings (report was not generated by the build).
  • ⚠️ PMD report not generated.
  • ⚠️ Checkstyle report not generated.

Generated automatically by the PR CI workflow.

shai-almog and others added 15 commits April 15, 2026 07:28
…e64 SIMD in Java (#4745)

* Add NEON-accelerated base64Encode/base64Decode to Simd API and wire into Base64 SIMD methods

Agent-Logs-Url: https://github.com/codenameone/CodenameOne/sessions/c218992e-943a-4ce5-8d63-f82c0792416f

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Address code review: clarify loop conditions and comment in NEON base64

Agent-Logs-Url: https://github.com/codenameone/CodenameOne/sessions/c218992e-943a-4ce5-8d63-f82c0792416f

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Add new SIMD generic primitive declarations and remove base64 methods from IOSSimd

- Added shl, shrLogical, addWrapping, subWrapping for byte arrays
- Added offset-based overloads for unpackUnsignedByteToInt, add, cmpEq, cmpLt, select
- Removed base64Encode and base64Decode declarations

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Replace base64 overrides with generic SIMD primitive validation wrappers in JavaSESimd

Remove base64Encode and base64Decode overrides. Add validation wrapper
overrides for new generic Simd primitives: shl, shrLogical, addWrapping,
subWrapping, unpackUnsignedByteToInt, add (int[]), cmpEq, cmpLt, and
select.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Replace base64 NEON section with generic SIMD primitives in IOSSimd.m

Remove NEON-accelerated Base64 encode/decode implementations and add
NEON implementations for new generic Simd primitives: shl, shrLogical,
addWrapping, subWrapping, unpackUnsignedByteToInt, add (int), cmpEq,
cmpLt, and select (with offset parameters).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Remove base64 methods from Simd; add generic byte/int primitives; rewrite Base64 SIMD in Java

- Remove base64Encode/base64Decode from Simd, IOSSimd, JavaSESimd, IOSSimd.m
- Add generic byte shift primitives: shl(byte[]), shrLogical(byte[])
- Add wrapping byte arithmetic: addWrapping(byte[]), subWrapping(byte[])
- Add offset-based int overloads: unpackUnsignedByteToInt, add, cmpLt, cmpEq, select
- Add NEON implementations for all new primitives in IOSSimd.m
- Add validation wrappers in JavaSESimd.java
- Rewrite encodeNoNewlineSimd in Java using int-domain SIMD compare/select
- Rewrite decodeNoWhitespaceSimd in Java using int-domain SIMD shift/or
- Update SimdTest with tests for new primitives
- All 2345 tests pass

Agent-Logs-Url: https://github.com/codenameone/CodenameOne/sessions/5fc6976c-f0ec-4638-b22b-2cbc9c9ca5dd

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Optimize SIMD Base64: replace slow scatter/gather + per-element SIMD calls with fast inline scalar Java

The previous approach called 15+ individual Simd operations per 48-byte chunk
(each a virtual dispatch + JNI transition on iOS), plus scalar scatter/gather
loops for byte↔int conversion. This added ~2500 JNI transitions per encode
of 8KB, making it 64-109% slower than the already-fast scalar code.

Replace with the same 4x-unrolled table-lookup approach used by
encodeNoNewline(), now with offset support. This matches the scalar
CN1 encode/decode performance while maintaining the same API contract.

Agent-Logs-Url: https://github.com/codenameone/CodenameOne/sessions/90b8c36e-8f20-47da-9fb4-56344f18a336

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>

* Revert "Optimize SIMD Base64: replace slow scatter/gather + per-element SIMD calls with fast inline scalar Java"

This reverts commit 00e5103.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>
… broke TreeMap compilation

Agent-Logs-Url: https://github.com/codenameone/CodenameOne/sessions/4469cb06-502d-4670-b0cf-45ae7c65fb0e

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>
…endencies

When @concrete devirtualizes calls (e.g. Simd -> IOSSimd), the concrete
class must be included as a dependency. Invoke.addDependencies runs at
parse time when the target's @concrete annotation may not be loaded yet.
Re-resolve in updateAllDependencies after all classes are parsed.

Agent-Logs-Url: https://github.com/codenameone/CodenameOne/sessions/c4936dfb-670f-4728-93e8-2ace2e806d0b

Co-authored-by: shai-almog <67850168+shai-almog@users.noreply.github.com>
@shai-almog shai-almog force-pushed the copilot/fix-ios-build-native-methods branch from bdaa1ae to c19e90e Compare April 15, 2026 04:28
@shai-almog shai-almog closed this Apr 15, 2026
@shai-almog
Copy link
Copy Markdown
Collaborator

shai-almog commented Apr 15, 2026

Compared 37 screenshots: 37 matched.

Native Android coverage

  • 📊 Line coverage: 7.83% (4132/52738 lines covered) [HTML preview] (artifact android-coverage-report, jacocoAndroidReport/html/index.html)
    • Other counters: instruction 6.15% (20445/332320), branch 2.98% (955/32062), complexity 3.65% (1121/30680), method 6.42% (919/14314), class 10.63% (202/1900)
    • Lowest covered classes
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysKt – 0.00% (0/6327 lines covered)
      • kotlin.collections.unsigned.kotlin.collections.unsigned.UArraysKt___UArraysKt – 0.00% (0/2384 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader – 0.00% (0/1519 lines covered)
      • kotlin.collections.kotlin.collections.CollectionsKt___CollectionsKt – 0.00% (0/1148 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.MethodWriter – 0.00% (0/923 lines covered)
      • kotlin.sequences.kotlin.sequences.SequencesKt___SequencesKt – 0.00% (0/730 lines covered)
      • kotlin.text.kotlin.text.StringsKt___StringsKt – 0.00% (0/623 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.Frame – 0.00% (0/564 lines covered)
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysJvmKt – 0.00% (0/495 lines covered)
      • kotlinx.coroutines.kotlinx.coroutines.JobSupport – 0.00% (0/423 lines covered)

✅ Native Android screenshot tests passed.

Native Android coverage

  • 📊 Line coverage: 7.83% (4132/52738 lines covered) [HTML preview] (artifact android-coverage-report, jacocoAndroidReport/html/index.html)
    • Other counters: instruction 6.15% (20445/332320), branch 2.98% (955/32062), complexity 3.65% (1121/30680), method 6.42% (919/14314), class 10.63% (202/1900)
    • Lowest covered classes
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysKt – 0.00% (0/6327 lines covered)
      • kotlin.collections.unsigned.kotlin.collections.unsigned.UArraysKt___UArraysKt – 0.00% (0/2384 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader – 0.00% (0/1519 lines covered)
      • kotlin.collections.kotlin.collections.CollectionsKt___CollectionsKt – 0.00% (0/1148 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.MethodWriter – 0.00% (0/923 lines covered)
      • kotlin.sequences.kotlin.sequences.SequencesKt___SequencesKt – 0.00% (0/730 lines covered)
      • kotlin.text.kotlin.text.StringsKt___StringsKt – 0.00% (0/623 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.Frame – 0.00% (0/564 lines covered)
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysJvmKt – 0.00% (0/495 lines covered)
      • kotlinx.coroutines.kotlinx.coroutines.JobSupport – 0.00% (0/423 lines covered)

Benchmark Results

Detailed Performance Metrics

Metric Duration
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 native encode 1040.000 ms
Base64 CN1 encode 157.000 ms
Base64 encode ratio (CN1/native) 0.151x (84.9% faster)
Base64 native decode 1037.000 ms
Base64 CN1 decode 201.000 ms
Base64 decode ratio (CN1/native) 0.194x (80.6% faster)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants