Confidence score jump

[lebron-2016]

Dear author,

Based on your work, I still have a little doubt.

In your experiment, is there a jump of the confidence score in each frame of the MOT dataset? That is, is the polarization of confidence scores a common phenomenon?

Looking forward to your reply.

Thanks!!

[linh-gist]

Hi,

Thank you for being interested in our research and a good suggestion.
The adaptive confidence threshold should be changed according to the detection quality in each frame.
We have not experimentally shown these changes in our paper yet. But here is my quick check for the MOT20-01 sequence in the MOT20 dataset using the YOLOX detector, there is a drop in confidence score from 0.8 in frame #50 to 0.5 in frame #52.

50 0.8196244239807129
51 0.6545143127441406
52 0.5326142311096191
53 0.801461935043335

More details for sequence MOT20-01

{Frame} {Our adaptive confidence threshold}
0 0.7431707382202148
1 0.5948281288146973
2 0.6621150970458984
3 0.8344516754150391
4 0.8442084789276123
5 0.8337364196777344
6 0.7590265274047852
7 0.8484997749328613
8 0.8511543273925781
9 0.8365480899810791
10 0.8236527442932129
11 0.8156347274780273
12 0.829535722732544
13 0.8448958396911621
14 0.8458242416381836
15 0.8416213989257812
16 0.8487582206726074
17 0.7586166858673096
18 0.7085394859313965
19 0.7958784103393555
20 0.5935561656951904
21 0.6839025020599365
22 0.7110118865966797
23 0.7658166885375977
24 0.8015613555908203
25 0.676668643951416
26 0.6919205188751221
27 0.50299072265625
28 0.7732720375061035
29 0.4759712219238281
30 0.7846641540527344
31 0.7262916564941406
32 0.6777842044830322
33 0.8035283088684082
34 0.7901766300201416
35 0.7767529487609863
36 0.7963156700134277
37 0.7872295379638672
38 0.7360377311706543
39 0.8243980407714844
40 0.8387675285339355
41 0.8323571681976318
42 0.840116024017334
43 0.8244037628173828
44 0.828214168548584
45 0.8293471336364746
46 0.8272409439086914
47 0.5256056785583496
48 0.48253822326660156
49 0.8235137462615967
50 0.8196244239807129
51 0.6545143127441406
52 0.5326142311096191
53 0.801461935043335
54 0.8516407012939453
55 0.8257811069488525
56 0.823775053024292
57 0.8441457748413086
58 0.8266723155975342
59 0.43872928619384766
60 0.47887372970581055
61 0.38759613037109375
62 0.39276695251464844
63 0.7777299880981445
64 0.7288985252380371
65 0.767054557800293
66 0.6577541828155518
67 0.7626945972442627
68 0.7054800987243652
69 0.777191162109375

[lebron-2016]

Hi,

Thank you for being interested in our research and a good suggestion. The adaptive confidence threshold should be changed according to the detection quality in each frame. We have not experimentally shown these changes in our paper yet. But here is my quick check for the MOT20-01 sequence in the MOT20 dataset using the YOLOX detector, there is a drop in confidence score from 0.8 in frame #50 to 0.5 in frame #52.

50 0.8196244239807129
51 0.6545143127441406
52 0.5326142311096191
53 0.801461935043335

More details for sequence MOT20-01

{Frame} {Our adaptive confidence threshold}
0 0.7431707382202148
1 0.5948281288146973
2 0.6621150970458984
3 0.8344516754150391
4 0.8442084789276123
5 0.8337364196777344
6 0.7590265274047852
7 0.8484997749328613
8 0.8511543273925781
9 0.8365480899810791
10 0.8236527442932129
11 0.8156347274780273
12 0.829535722732544
13 0.8448958396911621
14 0.8458242416381836
15 0.8416213989257812
16 0.8487582206726074
17 0.7586166858673096
18 0.7085394859313965
19 0.7958784103393555
20 0.5935561656951904
21 0.6839025020599365
22 0.7110118865966797
23 0.7658166885375977
24 0.8015613555908203
25 0.676668643951416
26 0.6919205188751221
27 0.50299072265625
28 0.7732720375061035
29 0.4759712219238281
30 0.7846641540527344
31 0.7262916564941406
32 0.6777842044830322
33 0.8035283088684082
34 0.7901766300201416
35 0.7767529487609863
36 0.7963156700134277
37 0.7872295379638672
38 0.7360377311706543
39 0.8243980407714844
40 0.8387675285339355
41 0.8323571681976318
42 0.840116024017334
43 0.8244037628173828
44 0.828214168548584
45 0.8293471336364746
46 0.8272409439086914
47 0.5256056785583496
48 0.48253822326660156
49 0.8235137462615967
50 0.8196244239807129
51 0.6545143127441406
52 0.5326142311096191
53 0.801461935043335
54 0.8516407012939453
55 0.8257811069488525
56 0.823775053024292
57 0.8441457748413086
58 0.8266723155975342
59 0.43872928619384766
60 0.47887372970581055
61 0.38759613037109375
62 0.39276695251464844
63 0.7777299880981445
64 0.7288985252380371
65 0.767054557800293
66 0.6577541828155518
67 0.7626945972442627
68 0.7054800987243652
69 0.777191162109375

Thank you for your quick and detailed analysis！

Maybe I didn't express myself clearly. I mean, is there naturally such a clear cutoff for confidence scores in every frame? Namely, Is there a situation where all the detection confidence scores in a certain frame are so smooth that it is impossible to find a clear and convincing threshold to define high and low scores? This may be a problem with the ByteTrack algorithm itself. But I'd like to know if you think it's a strong prior information that there is always a cliff-like interval in the distribution of confidence scores in a frame.

Thanks!

[linh-gist]

Hi, the answer is "It depends on the detector we use to detect objects". For YOLOX, "there is always (high possibility) a cliff-like interval" with an analysis in our paper.

I hope this somehow clears your concerns.

[lebron-2016]

Hi, the answer is "It depends on the detector we use to detect objects". For YOLOX, "there is always (high possibility) a cliff-like interval" with an analysis in our paper.

I hope this somehow clears your concerns.

OK I understand it, thank you！！