Fix snowflake error : "Lateral cannot be on the left side of join"

[amitaggarwal1]

For nested fields we were using lateral flatten. The issue with it is that if the

nested field is used in a join, snowflake complains that lateral flatten can't be on left side of join (https://stackoverflow.com/questions/63397022/snowflake-lateral-cannot-be-on-the-left-side-of-join)

fix is to always convert lateral flatten to a left join

Also simplify the array[scalar] vs array[record] treatment. earlier, at the unnesting step we were creating a object for array[scalar] case to make it consistent with the array[record] case. Now at the reference time, based on the case, use the right expression

• array[record] : parent.value:sql_field
• array[scalar] : parent.value

[lloydtabb]

I would imagine some tests are failing. The INDEX is the distinct key and necessary for deduping.

It would also be great to have a test that shows what this fixes..

[amitaggarwal1]

Will work on fixing the tests.

Re: INDEX, INDEX is already part of the flatten structure. This was taking something of the form {INDEX: i, value: v} and converting into {INDEX: i, value: {value: v}}

[amitaggarwal1]

@lloydtabb added test case. this test fails without the fix with the following error:

 query.run failed: SQL compilation error:
    Lateral View cannot be on the left side of join
    SQL: SELECT
       a_0."a_two" as "a_two"
    FROM (
          SELECT [2,4,6,8] AS "evens"
        ) as base
    ,LATERAL FLATTEN(INPUT => base."evens") AS evens_0_1, LATERAL (SELECT evens_0_1.INDEX, object_construct('value', evens_0_1.value) as value ) as "evens_0"
     LEFT JOIN ( SELECT 2 as "a_two" ) AS a_0
      ON a_0."a_two"="evens_0".value:"value"::DOUBLE

    OperationFailedError: SQL compilation error:
    Lateral View cannot be on the left side of join

[lloydtabb]

So this is a bigger (and long standing) problem. Malloy, generally, can't join on a nested field. I think I finally know now to fix it, but it is a pretty major rework of code generation. It's probably worth scheduling the work.

I'm curious to see how your test fares on the other dialects.

#1481

We're working on figuring out how to get tests running against external pull requests (that's the reason for the delay in merging).

[lloydtabb]

You can see that it fails in other dialects.

https://github.com/malloydata/malloy/actions/runs/12605890168/job/35135261018?pr=2074#step:4:223

I'll write up what I think the solution is in the #1481

[mtoy-googly-moogly]

        const nestedArrayJoin = `
           source: a is ${conName}.sql(""" SELECT 2 as ${quote('a_two')} """)
           source: b is ${evens} extend { join_one: a is a on a.a_two = evens.value}
           run: b -> {select: a_two is a.a_two}`;

[mtoy-googly-moogly]

I'm not 100% sure I know what is going because I don't have the un-nested state of the array exactly in my head at this moment, but something like this would be preferable, if this is the correct solution.

      const arrayRef = `"${parentAlias}".value` + (parentType === 'array[record]' ? `:${sqlName}`: '');

[mtoy-googly-moogly]

#2086 has these changes in it