New Weighted Moving Regressor / Cyclic Error Filter porting

[Abasz]

I have been reviewing the new moving regressor algorithm and created the initial port. I will need to measure the performance but before I do I have some questions:

• could we cache the Gaussian Weights?

This is my most important question to you because it requires understanding of the Gaussain filter which I currently lack :)

Currently, you recalculate the Gaussian weight for each position on every push, for every point, and that is a lot of calculation. but after the flanklenght is full this will be static no? the relative positions don't change once the window is full. At least this is is what I deciphered from the filter description on the internet (but admittedly I do not full understand how this works in the ORM algorithm exactly apart from the fact that right now there is a lot of calculation ).

What I am looking for is whether I could calculate a lookup table at class construction or even at compile time. While I have yet to run benchmarks, unfortuantely I fear that the exponent calculation with a double will be expensive on the microcontroller becuase it has only single precision FPU...

Potential micro optimisations:

• I would cache the coefficients in a local variable to avoid function calls (this is micro optimisation probably on JS)
• I would provide a function which can calculate the local goodness of fit for an array of data, because that would avoid repeated calls to series.average() (it can be cached and called once per range instead of once per point)

These micro optimisations exists all around your code but as you said its not much worse it might not worth the hustle.

[Abasz]

One additiona thing:

in the MovingRegressior you are checking:

    while (i < aMatrix.length && quadraticTheilSenRegressor.reliable()) {

Where reliable is only the length of the array. But this also means that in case the globalGOF is zero but the array is large enough you will do calculation of 0 weights and include those in the weighetedAverages. I dont think this makes sense but I also acknowladge that ithese cases are limited :) any way I would check for Global GOF > 0 instead

But I am not sure if this is deliberately needed by the algorithm.

Furthermore, when local GOF is zero again weight is zero so I dont see much point in pushing these zero weighted points nor calculating the Gaussian weight (saving some execution time in some cases) into the WeightedAverage Series.

[JaapvanEkris]

Furthermore, when local GOF is zero again weight is zero so I dont see much point in pushing these zero weighted points nor calculating the Gaussian weight (saving some execution time in some cases) into the WeightedAverage Series.

GoF being 0 is quite an edge case, but I'll see if it even can happen with a filled array.

[JaapvanEkris]

I have been reviewing the new moving regressor algorithm and created the initial port. I will need to measure the performance but before I do I have some questions:

• could we cache the Gaussian Weights?

This is my most important question to you because it requires understanding of the Gaussain filter which I currently lack :)

In my implementation, yes, as the X-coordinates of the boundaries are changed for each regression run. In essence, the Gaussian filter rewards you for being close to the center of the regression analysis, and punishes it if it is on the edges (thus putting focus on the more reliable data for that datapoint). This absolute approach theoretically is more sound, but I'm far from a 100% certain that would be the practical case here as well.

There is a static variant I've played with that can be cached: don't use the absolute X-values, but the position (index) in the X-array. As the array length has a constant flankLength, results won't change from run to run. Theoretically, it loses some precission, but it works. That would allow the creation of a lookup table as your constructor could include window size, and create a lookup table for retrieval.

In practice it has quite a similar result, but I must do a long term tests to actually see the difference between these two approaches. Especially as I'm tweaking the entire filter chain, results can be extremely difficult to predict. Given that you are much more CPU constrained, I'd go for the static version.

• I would provide a function which can calculate the local goodness of fit for an array of data, because that would avoid repeated calls to series.average() (it can be cached and called once per range instead of once per point)

These micro optimisations exists all around your code but as you said its not much worse it might not worth the hustle.

Thanks for observing this. My average function leans on two stored variables (a sum and a count). Admitted, it is stored in another object, and a division has to be made, but I'm not gaining much CPU power here.

I'm experimenting with an additional filter which aims to reduce systematic noise. Technically relatively simple:

• I calculate the perfect X value for each datapoint that reaches the front of the flank based on the regression algorithm.
• I use that perfect currentDt to calculate a "correction factor": the relative error that is between the raw currentDt and the perfect currentDt for the same position.
• I store this correction factor per individual magnet (essentially a ringbuffer with the average of the correction factor per magnet)

My initial tests show I just measure, I always converge to a stable filter. The same pattern (correction values) I also find across multiple sessions (albeit shifted, as the 1st magnet is not always the same for each session), suggesting that it is machine dependent (as one would expect) and not session dependent. Even with raw recordings years apart, I find the same sinoid-like pattern, similar to our manual visualisations of the Concept2 systematic error. This suggests we could remove this type of error. Even potentially allowing to reduce the flankLength, which would reduce CPU-load and improve responsiveness of the results to true changes.

The current key challenge is that when I actually apply the filter (and simultanuously training it), it seems to run away after 7K ending in total chaos. As a person who typically rows 10K's, not really achieving my design goal :). Might be caused by stupid bugs, it might be due to compkex feedback mechanisms in the overall algorithm. I'll drop the code as soon as I find a solution to this issue.

[Abasz]

Thanks for observing this. My average function leans on two stored variables (a sum and a count). Admitted, it is stored in another object, and a division has to be made, but I'm not gaining much CPU power here.

Thanks for the above. I agree that micro optimisation is not for JS really. For me even a division is expensive, because of the emulation of the double precision on the MCU :) so even a stupid simple division adds to time because of the single precision of FPU.

Anyway I will do the benchmark this weekend and see what the costs are. I have modernised the code base recently for c++ 26 (as ESP IDF has now support for this) and now it is using gcc 14.2!!! so the compiler got rather smart compared to the 8.x that I used previously.

This is very useful. Once I see the cost I will try to see some force curves. I agree that if this can shave off one flanklength the benefit is bigger than the cost as that is a very big saving.

[Abasz]

I have run some preliminary benchmarks for the static and the dynamic version and the cost of the dynamic is 2.5 times more than the cost of the static across all tested flankLegnth :)

I still need to do more tests but the dynamic version must out perform the static massively to justify it self to be implemented for in depth testing (which based on your previous comments is highly unlikely). This is on double

With using single (float) precision dynamic is not too bad, cost is fractional in comparison so I could see potential likelihood for choosing dynamic. But this would yield increased code complexity (biggest issue is code readability with compile time branches) that probably not worth it as all current profiles are calibrated for double precision and tests from long ago showed that double produces significantly out preformed float with better more reliable metrics across machines (kayak and rowing) I tested.

[Abasz]

I have plotted a few force curves but I made an example. stat = static gaussian, old (no-filter) and the number is 12-10 flankLength (not this is a kayak ergometer rather than rower)

stat is the static Gaussian. Based on the plots its clear that dynamic does not give much benefit. Actually I have specific force data and the difference of the points is less than 1% so its clear that if any its the static version that needs to be used.

Any way I have one observation, namely that with the filter on (regardless if dynamic or static) the force size is kind of flattened. For the old algorithm (no-filter) we go up to almost 80, while with the filter on it does not reach 60. This was so surprising to me so I rerun the simulation twice because I thought I made a mistake.

Yes, the smoothing effect is absolutely visible (I have chosen this stroke because it has those waves on 10 impulse which comes from the magnet in accuracy) so the smoothing effect is present but it seems that at the significant cost of flattening the curve (the shape is crisp which is the aim).

What I am wondering, if this reduction in force values would cause a reduction of the wattage (and hence distance). My test data and its change based on the different setting and algorithm is inconclusive in this sense (some runs increased some stayed and some decreased).

Just for reference Below is the increase (delta) in max (worst case) execution time for the different version (dynamic vs. static) basis is the no-filter version for 3-21 flank length in micro sec.

[Abasz]

I like better when things are coupled to where they belong. I agree that calculating the whole thing at once is a huge overhead. But also having the need for slope as well as GoF calculated for each point in the recovery means additional computation, while in the async approach there is no such thing.

I mean the performance overhead if we calculate GoF on each new data point in contrast with calculating it at the end of the recovery (and indeed buffering).

But processing using best possible regression result is a lot more robust.

So this is the debate/tradeoff :)

. And a simple projection of a linear function literally is executing "y = a * x + b", so not really a performance killer.

The thing is with this is that we dont know how many data points we have. In ESPRM I have the drag factor period consider the number of impulse per revolution so the two settings cannot be set up in a way that theoretical worst case recovery points exceed 1000 but still. Actually I had to do this because of memory (I run out of heap with extreme recovery when I did tests).

It isn't a big issue, as long as we keep feeding the filter a fair amount of reliable datapoints. I actually "abuse" being time limited by focusing on the middle 50% of the recovery datapoints (essentially assuming a 1 to 2 ratio), which essentially prevents any noisy flanks to enter the equation.

So this is also a part I dont like design wise, this assumption. This 1 to 2 ratio may have the issue on a kayak where the recovery already quite limited (lets say 10-20 points maximum). Though there since, drive is longer its almost guaranteed to be able to process all points.

IDEA:
What if the error filter would have two buffers and it internally switches between the two (alternate) that would allow to process points while in the recovery as well (because we always fill the other). When recovery ends we make the switch an start processing the new (and empty the old if not all points have been processed which is not likely).

IDEA2:
this is ESP specific. The main loop there runs constantly and calls the update() methods of each service (this imitates asynchronous behaviour as it utilises free cpu cylces) so they can do work. most of these are on a timer (i.e. before they do work they check if certain amount of time has passed or a variable has changed). this also means that between new impulses there is potential free cpu cylces that I can fill in. I could use the free cpu cycles to pick up the calculation one by one and process the buffer because one pass is cheap and does not block for long and also when there is time to process the new impulse it also does not block that.

IDEA3: in js you can wrap the buffer calculation into a promise to achieve similar result and process the whole buffer at once.

[JaapvanEkris]

The thing is with this is that we dont know how many data points we have. In ESPRM I have the drag factor period consider the number of impulse per revolution so the two settings cannot be set up in a way that theoretical worst case recovery points exceed 1000 but still. Actually I had to do this because of memory (I run out of heap with extreme recovery when I did tests).

In ORM we do, simply measure the length of the array. In the worst case, you can start binning the head.

It isn't a big issue, as long as we keep feeding the filter a fair amount of reliable datapoints. I actually "abuse" being time limited by focusing on the middle 50% of the recovery datapoints (essentially assuming a 1 to 2 ratio), which essentially prevents any noisy flanks to enter the equation.

So this is also a part I dont like design wise, this assumption. This 1 to 2 ratio may have the issue on a kayak where the recovery already quite limited (lets say 10-20 points maximum). Though there since, drive is longer its almost guaranteed to be able to process all points.

It is an assumption, but not a truly relevant one. You want the filter to learn from the recovery, and 5 to 10 datapoints is already enough to prime and adjust the filter. It will be a bit more crude with fewer datapoints, but kayaks also have much higher SPM's, so it might configure the filter sooner than you think. Letting the algorithm do what it does without too much tricks around it, often is the best approach.

I used a buffer of a 1000 datapoints per magnet to prevent wild swings. However, I am now testing with a weighed version where I use a buffer of 16 per magnet, which works good as well. It converges much faster as I expect around 16 datapoints per magnet per recovery. With roughly 100 datapoints per recovery on a C2 RowErg (16 per magnet), it converges around the 8th stroke to its maximum GoF of 0.9996. In the second stroke it is already on a GoF of 0.9990.

IDEA: What if the error filter would have two buffers and it internally switches between the two (alternate) that would allow to process points while in the recovery as well (because we always fill the other). When recovery ends we make the switch an start processing the new (and empty the old if not all points have been processed which is not likely).

Part of the calculation is also the projection via the regression function. So two buffers also means two regression-functions being active. It is a messy process that requires quite some complexity to manage.

And culling the head and tail has the huge benefit of being fairly certain no messy flanks are included due to stroke detection errors. I rather have 20 rock solid datapoints, than 30 sloppy ones. Especially when a set of bad values can really mess with future data at considerable lengths. So making sure the filter doesn't get fed bad data and starts to misbehave is key here.

And as said, what the filter learns, it will learn. No need to push on with all kind of tricks. With literally hunderds of thousands of datapoints in a session, it will converge quickly. Trying to scrape the bottom of the barrel won't get you much.

IDEA2: this is ESP specific. The main loop there runs constantly and calls the update() methods of each service (this imitates asynchronous behaviour as it utilises free cpu cylces) so they can do work. most of these are on a timer (i.e. before they do work they check if certain amount of time has passed or a variable has changed). this also means that between new impulses there is potential free cpu cylces that I can fill in. I could use the free cpu cycles to pick up the calculation one by one and process the buffer because one pass is cheap and does not block for long and also when there is time to process the new impulse it also does not block that.

That would be an approach. On the Raspberry Pi we have no issues. The cyclic filter is so lightweight compared to the TS regressors, it hardly has an impact on the CPU (stable between 10% and 13%). I have some work to stabilize CPU load (array shifting of the Force curve cleaning can be optimized), but I like to keep code simple.

IDEA3: in js you can wrap the buffer calculation into a promise to achieve similar result and process the whole buffer at once.

I like the CPU load to be nicely distributed in the runtime. And in this way it flattens out: in the drive ORM does some extra work to calculate the filter, force curve, etc.., in the recovery it is calculating the dragfactor. And the extra complexity of a promise is not something I look forward to.

[Abasz]

So two buffers also means two regression-functions being active. It is a messy process that requires quite some complexity to manage.

Yes you are right we would need to keep the old regression function. But I think its sufficient to keep the formula of the function, we do not need the whole class and other underlying data (we can save the GoF by value, we need the slope and the strtpoint). But anyway I agree that this is more involved.

kayaks also have much higher SPM's,

good point indeed that is very true, 3x-4x generally. So that means actually over the same time we should have similar amount of points (still a bit less though but not substantially).

Trying to scrape the bottom of the barrel won't get you much.

its not really about scraping the bottom, rather that the calculations and the result are more deterministic. We can still use the interquartile range so to speak (so the middle 50% of the points). What I was trying to refer to is that processing the points will be dependent on the drive length (how many points are cumming in during the drive) that logically is not correlating to the points to be processed. What I am saying is that you tie a process to an other totally irrelevant process which in my view could potentially lead to nondeterministic behaviour. I am not saying that I can point to one right now, but that does not mean it cannot be.

I like the CPU load to be nicely distributed in the runtime. And in this way it flattens out: in the drive ORM does some extra work to calculate the filter, force curve, etc.., in the recovery it is calculating the dragfactor. And the extra complexity of a promise is not something I look forward to.

Actually I think you can use generator/yield with await that would result in similar behaviour than on the MCU event loop (JS eventl loop and the MCU in this sense relatively similar). Basically generators do some processing and then with the inclusion of a await Promise.resolve() the hand back temporarily the control to the event loop and they wait for the next round.

I agree you will need to handle certain things like removing the array or updating and now you do that in an async loop which is more difficult to reason about. This will not hurt the CPU more than would in the processing during the main loop. It would however be more efficient as it would not delay processing of any new incoming data point.

You can paste the below in a broser console and see how it works.

async function* asyncProcessor(arr) {
  for (const item of arr) {
    // simulate CPU yielding
    await Promise.resolve();

    yield item * 2; // or your processing logic
  }
}

const items1 = [...Array(1000)].map(_=>Math.ceil(Math.random()*40));
const items2 = [...Array(1000)].map(_=>Math.ceil(Math.random()*40));

(async () => {
  console.log("Start1");
  for await (const result of asyncProcessor(items1)) {
    console.log("item1", result);
  }
})();

(async () => {
  console.log("Start2");
  for await (const result of asyncProcessor(items2)) {
    console.log("item2", result);
  }
})(); 

What you will see is alternating of item1 and item2 values meaning that after one round of processing control is yielded to the other process. This means that it processes one value of the filter schedules the next round and then check if anything else needs to run and so on.

Anyway, this was just an idea for detaching the filter processing from the incoming points while not creating a potential blocking processing.

[JaapvanEkris]

Trying to scrape the bottom of the barrel won't get you much.

its not really about scraping the bottom, rather that the calculations and the result are more deterministic. We can still use the interquartile range so to speak (so the middle 50% of the points). What I was trying to refer to is that processing the points will be dependent on the drive length (how many points are comming in during the drive) that logically is not correlating to the points to be processed. What I am saying is that you tie a process to an other totally irrelevant process which in my view could potentially lead to nondeterministic behaviour. I am not saying that I can point to one right now, but that does not mean it cannot be.

Technically, it is a deterministic algorithm: given the identical input, it will produce the same output each and every time.

The key issue for you seems to be that you don't know in advance whether all datapoints in a recovery are handled. The key design choice is that it doesn't matter as, by design, the effect of a single datapoint (or more precisely, a full flywheel rotation of datapoints) is extremely limited. The algorithm looks for a repeating bias across multiple datapoints for the same magnet. So, a single datapoint should never have a significant effect. Ideally, you even want several independent dragfactor calculations (regression line attempts) before filtering really starts to do its thing (hence the use of a buffer of 1000 in the initial version), preventing regression errors to mess up the filter. Only if several datapoints support the same trend, the filter should react to it. In fact, more advanced versions, like the Kalman filter, use OLS to project the filter for future datapoints per cyclic error. However, this is too complex while the current solution converges pretty fast already, without any weird artifacts.

So, on the whole, one or two datapoints per magnet shouldn't produce noticeable deviations as the algorithm looks for the underlying structural element, and it doesn't matter when they are injected. Only real requirement is that you prevent starvation of the filter in the early stages. But requiring to process all data is far from a requirement. As said, not having a single regression run dominate the results is even a good thing (hence my current buffer setting of 16, which forces the algorithm to see several strokes before converging to its final state).

I like the CPU load to be nicely distributed in the runtime. And in this way it flattens out: in the drive ORM does some extra work to calculate the filter, force curve, etc.., in the recovery it is calculating the dragfactor. And the extra complexity of a promise is not something I look forward to.

Actually I think you can use generator/yield with await that would result in similar behaviour than on the MCU event loop (JS eventl loop and the MCU in this sense relatively similar). Basically generators do some processing and then with the inclusion of a await Promise.resolve() the hand back temporarily the control to the event loop and they wait for the next round.

I agree you will need to handle certain things like removing the array or updating and now you do that in an async loop which is more difficult to reason about. This will not hurt the CPU more than would in the processing during the main loop. It would however be more efficient as it would not delay processing of any new incoming data point.

The calculation is extremely light. It literally is just a hand full of retrievals, additions and multiplications. Why bother doing this kind of suboptimisation if it doesn't even register on the CPU-load.

[Abasz]

Ok, this is fair enough :)

I will play with the code over the weekend too.

[Abasz]

i have another question regarding hiw the totalSpinningTime is used:

openrowingmonitor/app/engine/Flywheel.js

Line 102 in 2b13b59

currentDt.recordRawDatapoint(totalNumberOfImpulses, totalTimeSpinning, currentDt.raw.atSeriesBegin())

why are you feeding the rror filter with series begin delta time?

what I am struggling with is the fact that when we record the raw data we use the totalNumberOfImpulses but that does not get inclemented when we are in the stop state while the flywhhel could still be rotating. so after restart I think the slots get misplaced. shouldn't we actually use the number of impulses that physically have been made?

EDIT:
Sorry i can see now that you reset the filter and restart when measuring starts after stop.

But I suppose we could use absolute raw values which could be maintained across stops hence no need for restart (I am considering the option for stopping the data generation for the filter lets say after 10000 pints for the entire session - depending on the calculation cost whuch i also expect to be minimal)

[JaapvanEkris]

Now the strange part here is that it seems it recovers but many strokes later it gets lots and logs on 594 stokes (instead of the 730)

It gives *** Stroke duration of 328.0639558899858 sec is considered unreliable, skipped update cycle statistics

I was not able to found out what is wrong. I know that its unlikely that there are two consecutive points skipped.

Could be that there is some messy data in the currentDT filter? I added an explicit check on ranges, with an arbitrary +/- 10% change. Perhaps reduce that to 5%?

[JaapvanEkris]

The GoF used is only calculated once per stroke, so it takse at least two to three strokes to detect a sudden drop and distinguish it from a one-off. As my example above shows, a missed stroke shows, finding out what is going on will takes several strokes.

On the regression. I understand that the algorithm would need a few strokes and that may delay the reset but actually we are running the regression on the GoF of the recovery slopes so when the first bad GoF comes in after a stable 0.999 the regression model slope will drop as well as the GoF of the "GoF regression" will drop too. Its up the strictness of the settings how big of a drop or how many "bad" strokes would be necessary to trigger recalculation. I mean technically we can even trigger a reset on a one off drop (to be extreme conservative). there is no harm in it as it will re calibrate it self within a few strokes and there is less of a risk that due to the total misalignment of the slots things potentially get messed up.

I do trigger a reset on detecting a too small currentDt. Essentially, being too fast for the minimum filter is easily user-adjustable. And a bounce error is handled much better that way. That was quite a surprise to me to be honest, as resetting the filter would trigger an unexpected jump in currentDt values.

[Abasz]

I do trigger a reset on detecting a too small currentDt. Essentially, being too fast for the minimum filter is easily user-adjustable.

I have seen the code change you pushed (above tests are run with that code, though I did not test the bounce but rather missing signal). I think in general this is ok but the issue is with this is that the bounce depends on speed generally. Which means that if we have bounce on slower rotation (e.g. during the recovery spin down but within the stop threshold) this will not prevent anything, the slots will get misaligned as the bounce can be above minimumTimeBetweenImpulses. So in my view, while this is a good place to reset, it does not treat the whole issue in a robust way.

That was quite a surprise to me to be honest, as resetting the filter would trigger an unexpected jump in currentDt values.

Now this is a valid point, I have not yet though about this, indeed we cannot reset blindly because if we do than I suppose its possible that the "time dilation" correction via the weight does not yet kicked in? Though I am not sure how fast the effect of that correction kicks in.

Could be that there is some messy data in the currentDT filter? I added an explicit check on ranges, with an arbitrary +/- 10% change. Perhaps reduce that to 5%?

No the loss of sync is clearly because of the missed points. The blue line is the (or supposed to be) the clean and orange is raw.

You can see that before the missed points blue is very straight and orange is a little bouncy after the missed points its the other way around and then the algorithm is not able to recover. When finally it detects a recovery by accident it can obtain some GoF for recovery and recalculate. After the first detected stroke things go back good almost instantly:

This is the recovery that was finally recognised after a few minutes (still misaligned to some extent):

Then we start processing the collected data points during the drive and it is apparent that after like the first or two drive points things are fixed

And by the time we get to the next recovery things are aligned again.

The issue is that if we dont detect a stroke (recovery) we dont have a way to update the filter so it cannot fix itself. I mean yes its possible to add reset if the drive is longer than x sec or has more points than x (in ESPRM I actually have a max drive point number length after which recovery is forced triggered, which originally was introduced to avoid the need for sending a massive force curve array via BLE, but now I use it as part of detecting erroneously long drive phase).

[Abasz]

ok. so for the bounce (downwards) I think this is a more robust checking (sorry its in cpp but the logic is the same, you can use Math.abs on the difference instead of the min max, I use that because its more efficient cpu wise than std::abs):

        auto deltaTimeDiffPair = std::minmax<volatile unsigned long>(currentDeltaTime, lastDeltaTime);
        auto deltaImpulseTimeDiff = deltaTimeDiffPair.second - deltaTimeDiffPair.first;

        lastDeltaTime = currentDeltaTime;
        // We disregard rotation signals that are non sensible (the absolute difference of the current and the previous deltas exceeds the current delta)
        if (deltaImpulseTimeDiff > currentDeltaTime)
        {
            cyclicFilter.reset();
        }

This basically checks the current dataPoint against the last and if the diff is bigger than the current it signals an error. This is better than checking against a static value of minimum time between impulses as it scales and my experience with bounce is that it scales too. I actually have this is an optional filter on the GPIO signal input. I used it for reed switches specifically before I moved to hall.

EDIT:

just as a test I plugged this into the flywheel.js:

    if (dataPoint > rowerSettings.maximumTimeBetweenImpulses && maintainMetrics) {
      // This shouldn't happen, but let's log it to clarify there is some issue going on here
      log.debug(`*** WARNING: currentDt of ${dataPoint} sec is above maximumTimeBetweenImpulses (${rowerSettings.maximumTimeBetweenImpulses} sec)`)
    }

    if (currentDt.raw.length() >= flankLength && Math.abs(dataPoint - currentDt.raw.atSeriesEnd()) > dataPoint) {
      log.info(`*** NOTICE: Detected large jump in currentDt from ${currentDt.raw.atSeriesEnd()} µsec to ${dataPoint*} µsec, resetting systematic error filter`)
       currentDt.reset()
    }

    if (dataPoint < rowerSettings.minimumTimeBetweenImpulses) {
      if (_deltaTime.length() >= flankLength && maintainMetrics) {

This correctly works for upwards misses as well it seems if there are too many points skipped:

*** NOTICE: Detected large jump in currentDt from 0.042651 µsec to 0.017887 µsec, resetting systematic error filter

I had to add a length check to avoid triggering at startup.

EDIT2:

potential issue and fine tuning opportunity to fix restart after pause for instance (because currentDt.raw will not be empty but there will be big jumps). This is not a big issue per se because you reset the filter on a pause any way.

[JaapvanEkris]

ok. so for the bounce (downwards) I think this is a more robust checking (sorry its in cpp but the logic is the same, you can use Math.abs on the difference instead of the min max, I use that because its more efficient cpu wise than std::abs):

I'm not too keen on resetting the filter on a single value. Especially if big jumps might occur naturally. A filter reset is a last resort, as a sudden filter change messes with stroke detection.

I think that when a machine is prone to bounce errors, the solution shouldn't be keep resetting the filter. The filter is designed to reduce systematic noise, but it can't function when there is too much bounce or too much random noise. The filter should then be set a lot less agressive (hence the default value of 0),

An approach would be to introduce a more dynamic offset: let the filter check if the previous or next average is closer to the calculated correction than the current one, and adjust the offset accordingly.

[Abasz]

I wonder whether you have considered using exponential moving average for the cyclicfilter instead of the ring buffer type averager? Reason for asking is that I did some tests and it handled the errors in my data seemingly slightly better or equivalent (though in the ESP port), while it is extremely light weight. It had smoother and faster response to these errors.

However, theoretically erroneous values would never completely "disappear" but converge to zero. Also since newer values are considered more the initial incoming error effect would potentially be higher I suppose.

[Abasz]

However, in this case, the key approach should be that older values should be as relevant as newer ones as we are looking for a systematic error across all measurements and the physics of the flywheel doesn't change.

This statement wtr. young vs. old points is only true if we are slot aligned (i.e. the algorithm "learned" the offsets and data is coming in in that order ) otherwise completely wrong. Once slot tracking is misaligned (e.g. bounce, double count or whatever) the past is garbage and the algorithm needs to relearn quickly to get back on track (for this statement I assume shifting the current correction or rotating the alignment).

That isn't true. First, we assume that the correction factor is stable, where in fact it might be dependent on flywheel speed, etc.. Secondly, stroke detection throws us a curve ball by detection errors, thus slightly skewing the ideal slope and thus offsetting a whole series of filter values. On average it will converge, but it is far from stable.

While you bring the above as an argument against weighing between new vs. old values in my view this proves the point that older and newer values should not be equal in weight. If the correction depends on speed or stroke conditions (which I suppose cannot be excluded), then weighting purely uniform in time can bias results toward data collected at speeds that dominate your window while in reality we are already speeding up /slowing down so adaptation should be done and younger older points are not equal. Any way in my view in this sense EMA (with correct decay) is not inherently worse: you can pick windows size so that it spans the time/operating range you care about. What I am trying to say if you row steadily for 30 min there is no issue, but if you do sprints of 500m and then 1000m on slower pace you need to adapt. Based on the statement that speed is important newer values should be more important as we need to track the speed changed for the next 1000m and what we learned in the past 500 as offset should be corrected quickly.

I dont challenge that median is better for filtering the noise, but the practical issue is actually index misalignment (skipped/duplicate/bounce pulses) that temporarily makes new samples correspond to the wrong slot:

This is a "naturally" occurring error in one of the data set I test with (so this has been recorded like this from the rower itself). This drops the GoF to extreme low. In my view and based on my testing this is a bigger issue than random noise occasionally that may potentially poison the average (as this latter is a window size issue).

A median-only approach avoids accepting bad points but does not correct the mapping and re-learning ; recovery then requires replacing half the window of points which could be slow.

I also strongly disagree that decaying filter isn't an approach that fits identifying a structural bias. It actually does identify structural bias just as well as a non-decaying method, as long as the decay is chosen appropriately — and unlike a fixed window, it gives you tunable adaptation speed when you need it. But even if we accept this statement, the sliding window median helps with outliers, but does not fundamentally make structural bias more identifiable; it merely protects the estimator when the data contains large spikes or otherwise outliers with a smaller window size. Actually I would argue that median is the one that does not fit structural bias because that is the one that reduces magnitude of systematic bias needed to adapt when changes are happening (speed, stroke type etc.).

Any way, in my view the mean with a sufficiently big window, while indeed more sensitive to outliers, is often more statistically "efficient" because it uses every data point in its calculation as well as it is unbiased towards a point being outlier or non-outlier it allows adaptation to changes in the systematic bias.

When I switch to a decaying filter, GoF drops significantly, as some random noise keeps messing with it.

This is probably a "windows size" (decay factor) issue. Stability will come from bigger window size (which is the same to some extent for the median, the sliding average etc.) for small outliers (at the cost of adaptiveness so this is the balance that needs to be found). But the good part in extraordinary decay (as it does not change the average it self) is that technically if its a false alarm we can continue without issue (with the small potential risk of the new data making a bigger impact and nudging off the filter which requires re-adaptation ). But these are settings, the smaller the extraordinary decay is, the less adaptive the system will be to changes (hence more resilient to random noise).

I developed a weighed median filter (essentially a more general BST with weights instead of a leaf-count). A median has similar properties like an average

yes, the median-based approach in my view is excellent as a robustness layer against outliers but by itself is suboptimal for accurate, fast, multiplicative calibration and for recovering from slot-index shifts as it does not correct the mapping as quickly. I mean yes, the median will help robustness against outliers and allow you to use a smaller window without being overwhelmed by spikes but I think median is not good for slot shift correction.

My recommendation is to use it as a guard (instead of the shifting/array rotation logic you are implementing), not as the sole estimator:

• For each incoming point, compare it to your current geometric/arithmetic EMA. If the median of the short window differs wildly from the EMA and this remains across multiple incoming data for the same slot, you can trigger fast adaptation by decaying the values in the EMA buffer (I dont like rotation or shifting the slots because its difficult to detect how to rotation so there is a high risk that it gets even more mess it up, so fast learning in my view is the best approach)..
• If a single sample is an outlier (median says it’s an outlier), reject or heavily down weight it before feeding the EMA.

Current ORM implementation test and issues I had

Any way, I am running ORM with the new median and for that session where the drop is recorded as is, I see that it is rather unstable with a window size of 31.
OldDanube-session-with-bounces.txt

  Old_Danube: {
    numOfImpulsesPerRevolution: 6,
    sprocketRadius: 1.55,
    minimumTimeBetweenImpulses: 0.007,
    smoothing: 1,
    flankLength: 10,
    minimumForceBeforeStroke: 10,
    minimumStrokeQuality: 0.4,
    minimumDriveTime: 0.160, // minimum time of the drive phase
    minimumRecoveryTime: 0.145, // minimum time of the recovery phase
    dragFactor: 108,
    autoAdjustDragFactor: true,
    dragFactorSmoothing: 1,
    minimumDragQuality: 0.6,
    flywheelInertia: 0.0894,
    maximumStrokeTimeBeforePause: 6.0
  },

Please note that for some reason these settings give me a window size of zero so I set it manually to 31 in the CyclicErrorFilter.js:

  let _numberOfFilterSamples = (minimumDragFactorSamples / numberOfMagnets) * 2

    _numberOfFilterSamples = 31

When I run this I get a random error during one of the strokes for no reason:

Then in the logs:
*** DRIVE phase started at time: 153.3342 sec
*** Calculated drag factor: 83.0660, no. samples: 683, Goodness of Fit: 0.9968
*** Calculated recovery slope: 0.000973, Goodness of Fit: 0.9968, not used as autoAdjustRecoverySlope isn't set to true
*** Stroke duration of 10.729483620684533 sec is considered unreliable, skipped update cycle statistics
stroke: 170, dist: 401.2m, No heartrate detected, pace: 3:08/500m, stroke dist: 2.0m, strokerate: 81.0 SPM, drive dur: 0.34s, rec. dur: 0.40s, stroke dur: 0.74s
*** WARNING: cyclic error filter detected a radical increase of 1.0168557556911768 at magnet 0, where about 1.0047289612092474 is expected, clipping
*** WARNING: cyclic error filter detected a radical increase of 1.0094916787564898 at magnet 1, where about 0.9964254466584569 is expected, clipping
*** WARNING: cyclic error filter detected a radical increase of 1.0167902733273106 at magnet 2, where about 1.0018419441928503 is expected, clipping
*** WARNING: cyclic error filter detected a positive shift at magnet 3, most likely due to an unhandled switch bounce
*** WARNING: cyclic error filter detected a positive shift at magnet 3, most likely due to an unhandled switch bounce
[...]
And than more of these (20 ish) start to show up.

The only thing here is that I change the damper (increased a little, hence you can see the bend in the recovery curve) but I have a very good GoF stroke detection is working, and then suddenly out of the blue it start applying some random correction while we are in the recovery and then everything goes haywire and it never actually recovers (this happens if I run it with EMA actually).

I disabled the clipping and bounce detection logic and than things work as it should so this suggests to me that there might be some bug on the clipping or rotation logic when stopping. I am not sure what data you are testing with but this could be the source of the unexpected behaviour you noticed and mentioned above.

This is how recovery GoF chart looks like:

The bigger straight lines are when I stopped paddling.

[JaapvanEkris]

However, in this case, the key approach should be that older values should be as relevant as newer ones as we are looking for a systematic error across all measurements and the physics of the flywheel doesn't change.

This statement wtr. young vs. old points is only true if we are slot aligned (i.e. the algorithm "learned" the offsets and data is coming in in that order ) otherwise completely wrong.

When you run a median buffer, in theory you could run it on 5 datapoints and be completely isolated from a single outlier. Only when it becomes systematic, it will start to include it.

We need more datapoints as you must exclude noise from stroke detection and from speed variations inside the recovery. And you need some room for gradual change, as sudden changes could offset stroke detection.

Once slot tracking is misaligned (e.g. bounce, double count or whatever) the past is garbage and the algorithm needs to relearn quickly to get back on track (for this statement I assume shifting the current correction or rotating the alignment).

There is such a thing as too quick a response. My testing shows that resetting the filter mid-session (i.e. not on a stop) will create total chaos in stroke detection. It is the first time in years I've seen that happen. In fact, the filter will not recover at all: it will get stuck into the random noise. GoF will not return to higher values (stuck around 0.9840, where before it was at 0.9997), even when dozens of strokes have passed and the stroke algorithm has stabilized, Gof stays low.

Please note: 0.9840 is a decent (but a bit on the lower side) GoF found on the RowErg. So essentially GoF returns to the state without filter.

That isn't true. First, we assume that the correction factor is stable, where in fact it might be dependent on flywheel speed, etc.. Secondly, stroke detection throws us a curve ball by detection errors, thus slightly skewing the ideal slope and thus offsetting a whole series of filter values. On average it will converge, but it is far from stable.

While you bring the above as an argument against weighing between new vs. old values in my view this proves the point that older and newer values should not be equal in weight. If the correction depends on speed or stroke conditions (which I suppose cannot be excluded), then weighting purely uniform in time can bias results toward data collected at speeds that dominate your window while in reality we are already speeding up /slowing down so adaptation should be done and younger older points are not equal.

You forget that flywheel speeds vary wildly within strokes already. And that recovery slope also varies across strokes. So there is significant random noise. To mitigate it, you need some algorithm that measures across several strokes to prevent that random noise to dominate the filter. A decaying filter can't do that, as it assumes that older strokes are less relevant (which they aren't).

A good approach would be to use OLS and have it map raw value versus perfect value, this would allow more complex relations (technically, the correction factor approximates the gradient). But it might be computationally too heavy and we can't prevent time delution.

Any way in my view in this sense EMA (with correct decay) is not inherently worse: you can pick windows size so that it spans the time/operating range you care about. What I am trying to say if you row steadily for 30 min there is no issue, but if you do sprints of 500m and then 1000m on slower pace you need to adapt. Based on the statement that speed is important newer values should be more important as we need to track the speed changed for the next 1000m and what we learned in the past 500 as offset should be corrected quickly.

This is not the point I made. The correction factors are approximations which work well across different paces. As the buffer gets refreshed often enough, it will adapt if necessary. If this is an issue, the agrssiveness should be toned down, allowing the algorithm a bit more wiggle room.

I dont challenge that median is better for filtering the noise, but the practical issue is actually index misalignment (skipped/duplicate/bounce pulses) that temporarily makes new samples correspond to the wrong slot:

Hence the detection of these issues.

This is a "naturally" occurring error in one of the data set I test with (so this has been recorded like this from the rower itself). This drops the GoF to extreme low.

This filter is not designed to fit each and every machine. You need dragfactor calculation to be robust, which might exclude this machine.

A median-only approach avoids accepting bad points but does not correct the mapping and re-learning ; recovery then requires replacing half the window of points which could be slow.

Current implementation is close to that: it refreshes the entire buffer about every two strokes. So as long as recovery detection keeps functioning decently, it will adjust itself. It will do in a controlled way, in order to prevent creating chaos in stroke detection.

I also strongly disagree that decaying filter isn't an approach that fits identifying a structural bias. It actually does identify structural bias just as well as a non-decaying method, as long as the decay is chosen appropriately — and unlike a fixed window, it gives you tunable adaptation speed when you need it.

No, because random noise in the youngest datapoint will actually be more relevant than the structual bias in the 10th datapoint before it. That is the whole issue with decaying approaches: the most recent element will by definition be more important than all its individual predecessors. Only when you compare the delta across several datapoints, you can determine structural error. A decaying filter focusses on the last error, which might contain too much random noise to be usefull.

That is in fact my experience on the RowErg: it contains quite a bit random noise, making decaying filters useless. Once you use a measurement across several values, it converges.

But even if we accept this statement, the sliding window median helps with outliers, but does not fundamentally make structural bias more identifiable; it merely protects the estimator when the data contains large spikes or otherwise outliers with a smaller window size. Actually I would argue that median is the one that does not fit structural bias because that is the one that reduces magnitude of systematic bias needed to adapt when changes are happening (speed, stroke type etc.).

No, because the median takes in account all datapoints equally (or weighed for GoF) and by averaging/median them, you can distinguish between the structural bias and the random noise. And you need some decent sampling for getting any decent estimates.

Any way, in my view the mean with a sufficiently big window, while indeed more sensitive to outliers, is often more statistically "efficient" because it uses every data point in its calculation as well as it is unbiased towards a point being outlier or non-outlier it allows adaptation to changes in the systematic bias.

A median isn't ignoring outliers. It will shift, causing the median to change in the right durection. But a median won't shift in a big way in the presence of an outlier. Only the behaviour becomes more structural, it will be included.

When I switch to a decaying filter, GoF drops significantly, as some random noise keeps messing with it.

This is probably a "windows size" (decay factor) issue. Stability will come from bigger window size (which is the same to some extent for the median, the sliding average etc.) for small outliers (at the cost of adaptiveness so this is the balance that needs to be found). But the good part in extraordinary decay (as it does not change the average it self) is that technically if its a false alarm we can continue without issue (with the small potential risk of the new data making a bigger impact and nudging off the filter which requires re-adaptation ). But these are settings, the smaller the extraordinary decay is, the less adaptive the system will be to changes (hence more resilient to random noise).

I've tried alphas of 1% and 0.5%, still not really working on a RowErg as random noise is more or less the same level as structural bias.

And the point you make goes for all algorithms here. Buffer size increase will make a metric less sensitive to sudden changes, which is a good thing when it is just random noise you want to ignore, but a bad thing when it is killing the signal you wanted to follow. Only difference is how much influence a single outlier can have and what buffer size you can safely maintain without being thrown off by a single outlier.

I developed a weighed median filter (essentially a more general BST with weights instead of a leaf-count). A median has similar properties like an average

yes, the median-based approach in my view is excellent as a robustness layer against outliers but by itself is suboptimal for accurate, fast, multiplicative calibration and for recovering from slot-index shifts as it does not correct the mapping as quickly. I mean yes, the median will help robustness against outliers and allow you to use a smaller window without being overwhelmed by spikes but I think median is not good for slot shift correction.

There is no single algorithm that will solve all your issues. A shift is a nasty issue, but overreacting to it and throw all other algorithms into total chaos is even a worse approach.

In all honesty, it is a supporting filter, helping to reduce structural errors for the machines that can support it. If it is off for a stroke, as long as the key metrics aren't thrown off too much (and ORM's basic metrics are really tough to throw off), I couldn't care less. A temporary dip in GoF is absolutely no big deal as long as stroke detection keeps doing its thing. I've seen GoF dip, but not to unacceptable levels (i.e. a dragfactor rejection based on GoF).

One thing that can throw off stroke detection is a filter resetting as it introduces sudden patterns across several datapoints.

My recommendation is to use it as a guard (instead of the shifting/array rotation logic you are implementing), not as the sole estimator:

Nope, as a median is much more robust. You don't want sudden changes to this filter, so resetting mid-session is no option, at all. Further reducing buffer size might be, but then I really need to baseline a working set of parameters.

  Old_Danube: {
    numOfImpulsesPerRevolution: 6,
    sprocketRadius: 1.55,
    minimumTimeBetweenImpulses: 0.007,
    smoothing: 1,
    flankLength: 10,
    minimumForceBeforeStroke: 10,
    minimumStrokeQuality: 0.4,
    minimumDriveTime: 0.160, // minimum time of the drive phase
    minimumRecoveryTime: 0.145, // minimum time of the recovery phase
    dragFactor: 108,
    autoAdjustDragFactor: true,
    dragFactorSmoothing: 1,
    minimumDragQuality: 0.6,
    flywheelInertia: 0.0894,
    maximumStrokeTimeBeforePause: 6.0
  },

Are you sure the minimumStrokeQuality is correct? I usually run with 0.28. This might be too tight.

Then in the logs: *** DRIVE phase started at time: 153.3342 sec *** Calculated drag factor: 83.0660, no. samples: 683, Goodness of Fit: 0.9968 *** Calculated recovery slope: 0.000973, Goodness of Fit: 0.9968, not used as autoAdjustRecoverySlope isn't set to true
...
The only thing here is that I change the damper (increased a little, hence you can see the bend in the recovery curve) but I have a very good GoF stroke detection is working, and then suddenly out of the blue it start applying some random correction while we are in the recovery and then everything goes haywire and it never actually recovers (this happens if I run it with EMA actually).

You realize that the autoAdjustRecoverySlope isn't set to true, so changing the DF will probably kill stroke detection as the recovery slopes won't match, and thus the filter isn't updated. And to be honest, that is a setting that should be true fir this filter to become active.

I disabled the clipping and bounce detection logic and than things work as it should so this suggests to me that there might be some bug on the clipping or rotation logic when stopping. I am not sure what data you are testing with but this could be the source of the unexpected behaviour you noticed and mentioned above.

A stop should reset the filter as you are almost guaranteed to lose sync. But timing that without filling up the filter again is challenging and still part of the design.

[Abasz]

I created a new branch with the data I test (same setting same everything, i.e. ceteris paribus).

This is the same with EMA (this is without any kind of additional reset or extraordinary decay apart from what you have in your branch), It realigns nicely. and it tracks things well (this is windows size 31):

This is the point that is at issue when tacking with median (this is with a window of 31)

If I test this with a very small window size, (like 10). Median works and produces almost identical values as EMA with the same with 10:
Just an example of one recovery:
EMA:

Median:

Even the realignment after error happens very close in time (note this is with a small window of 10 as bigger window messes with MEdian):
EMA:

Median:

Wtr. the above I must note that the missed pulse happens before a reset to the CyclicERrorFilter as part of the stopped rowing state. So when rowing restarts both algorithm starts from scratch (median is admittedly faster at a fresh start). But things a very different when the error happens during actual rowing. Also what is important here that with median correction happens suddenly (after around the 135k mark) not gradually.

My testing on the branch I have pushed shows median start breaking around a windows of 20 and below 8-9. Then there is a comeback (working again) with a window size of 40 again

Only equal weighting across many datapoints can detect bias.

I am not trying to convince you (and I am sure you have better understanding of the math) that EMA is better or the right approach, also that median does not work at all, but the results I am seeing for the median and your arguments contradict each other in my view.

For one, in my test I noticed that there is a window range below and above everything works. Based on my understanding of median this above is logical considering the characteristics of it: the window captures long/short-term nonstationary effects of different but actual changes to the bias (e.g. speed variation) and tries to apply it to something that has already changed. So if the shape of the distribution of your correction factors changes — even slightly — the median can jump or drift in unpredictable ways (I understand that this is not your experience), because the median depends only on order, not magnitude. In a system where correction drift is multiplicative and gradual, the median will first lag and then jump, because it only moves when the central ordering shifts past 50% of the window. This behaviour in my view is not inline

What I am saying is that purely median does not help distinguish better between structural bias and noise. The median removes magnitude information by being a positional measure that is only influenced by the middle value(s) of a dataset, not the values of the outliers or other data points. Which means in my view is that it tries to find the adjustment factor that is the middle ground across all adjustment factors. But the correction factors are not independent as they depend on the previous one (so if one overshoots the next will compensate into the other direction, i.e. If one correction overshoots, the next one undershoots and the median destroys this information).

So median essentially hides dynamic structure in the signal in my view which might not be a problem when it is totally stable but as you said it could be speed, stroke style, etc dependent. I think this is supported by the observation that certain bigger window sizes mess the filter up some do not (while EMA was resistant to this behaviour).

Second, I observed that the correction effect of the median is rather sudden instead of gradual. This was present after every reset (because of rowing stop) or correction after shift error (i.e. learning). While there could be other explanation for this, I think this is rather a symptom of the correction happening when the bad values go out of the ring buffer and new good values finally dominate. This in my view also does not support the non-volatile incremental learning behaviour statement (while the EMA exhibited this gradual learning very well for the same window size).

In a system where correction drift is multiplicative and gradual, the median will lag and then jump, because it only moves when the central ordering shifts past 50%. So the gradual change (which you specifically state that is required) is exactly what the median does not satisfy in case of adapting to new order.

Below are two re-learning, first is EMA second is the Median with a window size of 10. The EMA gradually reduces the outlying spikes, stroke by stroke (it is even visible to some extent that during the drive it reduces as it is processing the values), while in the median they just get reduced in "blocks" or increments. Which is logical since as the buffer will return the same/similar average from the previous batch it should/will receive the same correction factor over and over, and when finally there is sufficient number of this new correction factor the old will get pushed out and replaced by the new (but this is not gradual). The EMA will push in the new value and incrementally get closer to the true bias.

[JaapvanEkris]

A good approach would be to use OLS and have it map raw value versus perfect value, this would allow more complex relations (technically, the correction factor approximates the gradient). But it might be computationally too heavy and we can't prevent time delution.

In
https://github.com/JaapvanEkris/openrowingmonitor/tree/0.9.7d---WLS-for-CyclicErrorCorrection I implemented a Weighted Least Squares algorithm to do the corrections. It prevents time delution. Haven't tested this version yet, but it should allow for a bit more accurate corrections.

[Abasz]

Thanks. I had a look at the code, though I did not test directly. Logically and mathematically I agree that this tracking is generally probably the best, though I still think adaptation capability to sudden misalignment is a bigger issue than actual initial learning accuracy so in ESPRM I have chosen a different route and tests on different machines show very good result.

The main concept is that it uses the decaying exponential mean with an approximated windows size of 50 points but it has two different detection mechanisms that allows fast adaption in case of misalignment. The implementation is very very light weight adds 1-2% overhead to the stroke detection algorithm, while the heavy lifting (which is still very light weight) is done "async" during the free CPU cycles between impulses.

There are two fail safes:

1. I track the raw and the clean delta times when recording a data point in the error filter and I calculate OLS Goodness of Fit of these when stroke detection GoF drops (based on clean) below the minimum drag factor goodness set in the settings. I check the GoF of clean and raw and in case raw GoF is better with a threshold than clean, based on the difference I apply a variable amount of extraordinary decay to the values in the filter series partially in validating the data init (which would be similar to a reset, only that it allows adaptation rather starting from scratch avoiding the time dilation to get materialised or fixed because of the "relearning"). In such case I also update the error filter regardless of the GoF of the recovery to allow adaptation (since GoF is the weight of the new value if its crap its effect will be reduced).
2. I created a slot error tracker class that basically uses a fixed size ring buffer to store the signed deviation of the applied new correction for a given slot. Then I check on every new added datapoint if there is consistent shift in sufficiently "high" amount in the bias (if all elements in the ring buffer have the same sign i.e. trying to push the error correction filter to + or - as well as if these pushes are higher than a threshold) and then based on these I calculate a variable "boost factor" which assign higher weights to that particular data point. Basically what this does is track if there is a potential shift in bias and if yes it tells the algorithm that the new point is more important than the old ones as the alignment is probably off and the error filter is working hard to adapt to the new alignment.

The above allowed to avoid using reset() at all (I use raw impulses as even when rowing stopped ESPRM tracks the impulse count anyway).

I have been testing this with data from the 4 different machines (as well as on a kayak erg on which I exercise) and it is very good balance between reliability and fast adaptation without missed strokes even in case there is so many missed impulses that the algorithm is never able to get the hang of the pattern.

Actually I created a - planned to be simple but turned out to be rather complex - tests suite (with which I probably spent more time than I should have...) that can randomly drop points, or add bounce, change the magnet "alignment" by adding consistent offsets to points systematically (so I have tested symmetric, non-symmetric errors, I even tested random errors where of course the algorithm is not able to learn anything).

[Abasz]

One thing I noticed on ORM is that the calculation of the windows size has a bug for default settings if the minimumRecoveryTime is a small value (like for a kayak erg) as the initial seed and the window size is set to zero (or a very small number):

  const minimumDragFactorSamples = Math.floor(rowerSettings.minimumRecoveryTime / rowerSettings.maximumTimeBetweenImpulses) 

Where the default maximumTimeBetweenImpulses: 0.5 while the minimumRecoveryTime: 0.145 (but the same issue on Geneir_Air settings). It works only if the minimumRecoveryTime is greater than the maximumTimeBetweenImpulses. So I think either a clamp (with a minimum value for windows size and initial seed) should be introduced or somehow change the calculation logic.

I have been running a lot of simulation tests with the different data sets on both ORM and the ESP port and I have good results with clamping windows between 15 and 50 and using minimumRecoveryTime / minimumTimeBetweenImpulses for the windows size.

Also on the misalignment I have very good results in terms of robustness vs responsiveness in the version where I detect misalignment by checking relative deviation of the current and proposed new correction factor and whether this "big" deviation occurs consecutively (i.e. the algorithm is clearly trying to adapt something new) for a number of times (for now I use 4 which is basically 4 rotation of the flywheel) that triggers an extraordinary decay of the average making the new values have a more relative weight that helps quicker adaptation to the potentially new slot alignment. My finding based on my different test data for machines is that this provides good stability while when needed, ensures quick response and adaptation to misalignment.

Also my experience is that on a rowing machine (which symmetric magnet misalignment) which has a relatively long recovery, things are much simpler and adaptation is quick and the errors effect is less pronounced any way. In this case the filter is less sensitive to what settings (windows size, initial seed etc.) is being set. For reference below is the goodness of fits for the section that includes a missed impulse where blue is ORM default and orange is the exponential decay average (this is generic Air) with an extraordinary decay of 50% on a windows size of 31. Basically the question is whether its 2 strokes or like 3-4 strokes before w ego back to almost perfect alignment.

On the C2 test data you have in the repo there is basically no difference

The situation is totally different on kayak ergs (blue is ORM default):

The above results are consistent on the ESP implementation:

Generic:

OldDanube:

[JaapvanEkris]

One thing I noticed on ORM is that the calculation of the windows size has a bug for default settings if the minimumRecoveryTime is a small value (like for a kayak erg) as the initial seed and the window size is set to zero (or a very small number):

  const minimumDragFactorSamples = Math.floor(rowerSettings.minimumRecoveryTime / rowerSettings.maximumTimeBetweenImpulses) 

Where the default maximumTimeBetweenImpulses: 0.5 while the minimumRecoveryTime: 0.145 (but the same issue on Geneir_Air settings). It works only if the minimumRecoveryTime is greater than the maximumTimeBetweenImpulses. So I think either a clamp (with a minimum value for windows size and initial seed) should be introduced or somehow change the calculation logic.

Good point, we need a minimum size, if the filter is enabled.

I discovered that taking an aggressiveness of 0.90 retains much of the filters benefits, but reduces issues.

I have been running a lot of simulation tests with the different data sets on both ORM and the ESP port and I have good results with clamping windows between 15 and 50 and using minimumRecoveryTime / minimumTimeBetweenImpulses for the windows size.

A minimum number of datapoints indeed would be wise. The median filter might even work better here (as you won't need the buffer size to dampen the effect of outliers).

Also on the misalignment I have very good results in terms of robustness vs responsiveness in the version where I detect misalignment by checking relative deviation of the current and proposed new correction factor and whether this "big" deviation occurs consecutively (i.e. the algorithm is clearly trying to adapt something new) for a number of times (for now I use 4 which is basically 4 rotation of the flywheel) that triggers an extraordinary decay of the average making the new values have a more relative weight that helps quicker adaptation to the potentially new slot alignment. My finding based on my different test data for machines is that this provides good stability while when needed, ensures quick response and adaptation to misalignment.

I use two, and it seems reliable enough with my dataset (and in the ORM version I can look ahead, so I can actually shift on the first deviating datapoint).

[JaapvanEkris]

One thing I noticed on ORM is that the calculation of the windows size has a bug for default settings if the minimumRecoveryTime is a small value (like for a kayak erg) as the initial seed and the window size is set to zero (or a very small number):

  const minimumDragFactorSamples = Math.floor(rowerSettings.minimumRecoveryTime / rowerSettings.maximumTimeBetweenImpulses) 

Where the default maximumTimeBetweenImpulses: 0.5 while the minimumRecoveryTime: 0.145 (but the same issue on Geneir_Air settings). It works only if the minimumRecoveryTime is greater than the maximumTimeBetweenImpulses.

This is a bad config. Essentially you are saying that there are situations that the recovery can pass without seeing any magnet.

[Abasz]

This is a bad config. Essentially you are saying that there are situations that the recovery can pass without seeing any magnet.

Yes I know, but this is what you have in flywheel.js:

openrowingmonitor/app/engine/Flywheel.js

Line 35 in 1890702

const minimumDragFactorSamples = Math.floor(rowerSettings.minimumRecoveryTime / rowerSettings.maximumTimeBetweenImpulses)

[JaapvanEkris]

This is a bad config. Essentially you are saying that there are situations that the recovery can pass without seeing any magnet.

Yes I know, but this is what you have in flywheel.js:

openrowingmonitor/app/engine/Flywheel.js

Line 35 in 1890702

const minimumDragFactorSamples = Math.floor(rowerSettings.minimumRecoveryTime / rowerSettings.maximumTimeBetweenImpulses)

Yeah, is it such a weird assumption to expect this number of datapoints? Having a recoverytime that is bigger than your maximum time between impulses in a training (excluding startup and closedown) is far from normal. Let alone 3 times as long. It would mean that a a recovery regularly would pass without seeing any magnet. That isn't a realistic approach.

[Abasz]

Yeah, is is such a weird assumption to expect this kind of datapoints? Having a recoverytime that is bigger than your maximum time between impulses in a training (excluding startup and clisediwn) is far from normal. Let alone 3 times as long. It would mean that a a recovery reguraly would pass without seeing any magnet. That isn't a realistic approach.

This is the default settings:

openrowingmonitor/config/rowerProfiles.js

Line 29 in 1890702

maximumTimeBetweenImpulses: 0.5,

But anyway this formula is an issue in my view because on a kayak erg we have a short recovery 0.145. So if I set the maximumTimeBetweenInpulses to 0.025 I still only get 6 point in the window while 0.025 is too low, there is meaningful work there even in case of 6 points per rotation not to mention if we have 4.

This in fact less relevant on a rower where you have a long minimum recovery. The clamping between two values would help here I suppose.