Using flatc generated python funciton results in bad reads in flatcc generated code

[NReddyBase]

I have a struct:

struct Struct1 {
  field1: uint8;        // Target mode (0=Disabled, 1=PLL, 2=Sine Table, 3=Fixed Duty)
  field2: uint16;    // Frequency x 100 (for Sine Table mode, e.g., 6005 = 60.05 Hz)
  field3: uint8;          // Amplitude 0-100% (for Sine Table mode)
  field4: uint8;         // Duty cycle 0-100% (for Fixed Duty mode)
  field5: uint32;      // Bit-packed GPIO states (1 = ON, 0 = OFF)
  field6: uint8; // Dump burst capture
  field7: float;
  field8: float;
  field9: uint8;
}

Which compiles into this python function:

def CreateStruct1(builder, field1, field2, field3, field4, field5, field6, field7, field8, field9):
    builder.Prep(4, 28)
    builder.Pad(3)
    builder.PrependUint8(field9)
    builder.PrependFloat32(field8)
    builder.PrependFloat32(field7)
    builder.Pad(3)
    builder.PrependUint8(field6)
    builder.PrependUint32(field5)
    builder.Pad(2)
    builder.PrependUint8(field4)
    builder.PrependUint8(field3)
    builder.PrependUint16(field2)
    builder.Pad(1)
    builder.PrependUint8(field1)
    return builder.Offset()

Which when called like

builder = flatbuffers.Builder(0)
        off = CreateInverterControl(
            builder,
            3,  # target_mode
            6005,  # frequency_x100
            100,  # amplitude
            50,  # duty_cycle
            0xDEADBEEF,  # gpio_setting
            1,  # trigger_burst_dump
            1.234,  # idref
            4.567,  # iqref
            1,  # closed_loop_en
        )
        builder.Finish(off)
        # Create inverer control pre appends 4 unncessary bytes that prevent the inverter from processing the message
        payload = builder.Output()

gives me this message payload (hex)

 04 00 00 00 01 00 00 00 3C 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

But when I use the flatcc generated C code to try and read it results in a bad read:

static inline Inverter_CAN_Struct1_t *Inverter_CAN_Struct1_assign_to_pe(Inverter_CAN_Struct1_t *p,
  uint8_t v0, uint16_t v1, uint8_t v2, uint8_t v3,
  uint32_t v4, uint8_t v5, float v6, float v7, uint8_t v8)
{ p->field1 = v0; flatbuffers_uint16_assign_to_pe(&p->field2, v1); p->field3 = v2; p->field4 = v3;
  flatbuffers_uint32_assign_to_pe(&p->field5, v4); p->field6 = v5; flatbuffers_float_assign_to_pe(&p->field7, v6); flatbuffers_float_assign_to_pe(&p->field8, v7);
  p->field9 = v8;
  return p; }

Any advice or context here?

[mikkelfj]

Umm, that is a bit hard to conclude from. I need more context.

Officially it is not valid have a struct as a top-level buffer object in FlatBuffers.

FlatCC is the only language I know of that supports it. But even here, you have to create a struct buffer, not just directly use struct object to ensure the proper buffer header is written. That would usually be a create_as_root call of sorts - refer to the documentation for a struct buffer example.

In Python I believe you need a root table and store the struct in that, and obviously, C then needs a root table to read it. Struct buffers can be more time and space efficient for high speed transmission (often an 8 byte header and just a struct copy, though with guaranteed alignment).

The call you use in C: assign_to_pe is not what you would normally use, and if you did, you would typically use it to write a buffer, not read it (though if your host system is little endian it happens to valid anyway). _to_pe means towards the endianness of the protocl (flatbuffers, which is little endian) and implied: from the host endianness which may or may not be little endian.

The hexdump looks like a valid little endian struct and C can definitely read these. So if you are low-level hacking you could use _from_pe directly at the correct memory address, but not at the start of the buffer. (There is nothing wrong with that if you aim for performance and know what you are doing because structs are stable formats).

EDIT: the hexdump does not appear to match the data provided when read as a struct and it also does not look like a buffer header.

The hexdump should have contained at least a buffer header (8 bytes), and for general buffers a root table unless the dump is a local region of a larger buffer.

To read a buffer properly from C you would start with Mytype_as_root call, and it can indeed directly provide a struct, but only if you create a struct buffer to begin with.

If you go even more low-level, you can entirely skip the buffer header and just transport structs directly in your own protocol. FlatCC gives you the tools for that. It just isn't Flatbuffers anymore.