Weird behaviour of the strtok function

[JiriBilek]

Hi Juraj,
Recently, I found a strange behaviour of the strtok function in the Newlib library for Arm microcontrollers. However, I didn't see it in AVR.

There are two functions with similar behaviour: strtok and strtok_r. I noticed that the binary size of an application is reduced by more than 4K when using strtok_r instead of strtok. I believe this is a bug in the Newlib library because in the .map file you can see that an assertion routine and a large assertion text are linked to the binary when using strtok.

I share this with you because you use strtok heavily in this library.

You can try it with this simple code:

#include <string.h>

void setup() {
  char src[] = "this is a text";
  char *tok = strtok(src, " ");
  UNUSED(tok);
}

void loop() {
}

For stm32duino core and for example Generic STM32F1 Series, BluePill F103C6 (32K) and Newlib Nano library I get

Sketch uses 13832 bytes (42%) of program storage space. Maximum is 32768 bytes.
Global variables use 1224 bytes (11%) of dynamic memory, leaving 9016 bytes for local variables. Maximum is 10240 bytes.

and:

#include <string.h>

void setup() {
  char src[] = "this is a text";
  char *next;
  char *tok = strtok_r(src, " ", &next); 
  UNUSED(tok);
}

void loop() {
}

gives:

Sketch uses 7092 bytes (21%) of program storage space. Maximum is 32768 bytes.
Global variables use 1068 bytes (10%) of dynamic memory, leaving 9172 bytes for local variables. Maximum is 10240 bytes.