[#2317] feat(server): Introduce server flush benchmark

[maobaolong]

What changes were proposed in this pull request?

Introduce server flush benchmark.

Why are the changes needed?

Fix: #2317

Does this PR introduce any user-facing change?

No.

How was this patch tested?

curl http://localhost:19976/bench/run \
  -H "Content-Type: application/json" \
  -d '{
        "blockSize": "1k",
        "blockNumPerEvent": 1000,
        "eventNum": 1000,
        "partitionNum": 1000
      }'


$ curl -s http://localhost:19976/bench/get | python -m json.tool
{
    "ShuffleServerBenchmarkAppId1": {
        "argument": {
            "blockNumPerEvent": 1000,
            "blockNumThresholdInMemory": 10000000,
            "blockSize": "1k",
            "configFile": null,
            "eventNum": 1000,
            "help": false,
            "partitionNum": 1000,
            "sleepMsExceedBlockNumThreshold": 10,
            "threadNumGenerateEvent": 10
        },
        "benchTrace": {
            "end": true,
            "totalWriteBlockSize": 1000000000.0,
            "totalWriteDataSize": 1024000000000.0,
            "totalWriteTime": 9490889.0
        },
        "duration": 397233,
        "endTime": "2024-12-16 15:24:41",
        "id": "ShuffleServerBenchmarkAppId1",
        "log": [
            "2024-12-16 15:18:04: [ShuffleServerBenchmarkAppId1] register closable : heartbeat threadPool: rss-heartbeat-ShuffleServerBenchmarkAppId1",
            "2024-12-16 15:18:04: [ShuffleServerBenchmarkAppId1] register closable : runBenchTask",
            "2024-12-16 15:18:04: [ShuffleServerBenchmarkAppId1] register closable : registerShuffle",
            "2024-12-16 15:18:04: [ShuffleServerBenchmarkAppId1] register closable : Bench-ThreadPool-ShuffleServerBenchmarkAppId1",
            "2024-12-16 15:18:04: [ShuffleServerBenchmarkAppId1] register closable : trace",
            "2024-12-16 15:18:04: [ShuffleServerBenchmarkAppId1] 2024-12-16T15:18:04.434 Start to generate 1000000 events\n",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closing trace",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closed trace",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closing Bench-ThreadPool-ShuffleServerBenchmarkAppId1",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closed Bench-ThreadPool-ShuffleServerBenchmarkAppId1",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closing registerShuffle",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closed registerShuffle",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closing runBenchTask",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closed runBenchTask",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closing heartbeat threadPool: rss-heartbeat-ShuffleServerBenchmarkAppId1",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closed heartbeat threadPool: rss-heartbeat-ShuffleServerBenchmarkAppId1",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] Closed all registered closable. Count is: 5",
            "2024-12-16 15:24:41: [ShuffleServerBenchmarkAppId1] 2024-12-16T15:24:41.667 ShuffleServerBenchmark arguments blockSize: 1.00KiB, block num: 1000, event num: 1000, partition num: 1000, config file: null, flushHandler=org.apache.uniffle.server.DefaultFlushEventHandler, flushThreadPoolSize=24\n\tShuffleServerBenchmark run Success. Cost: total=397233 ms, flushTotal=397233 ms, lock=0.00 ms, eventCostMs=9.49 ms\n"
        ],
        "startTime": "2024-12-16 15:18:04"
    }
}

$ curl -X POST http://localhost:19976/bench/remove?id=ShuffleServerBenchmarkAppId0

[github-actions]

Test Results

 2 966 files  ±0   2 966 suites  ±0   6h 27m 57s ⏱️ ±0s
 1 097 tests ±0   1 095 ✅ ±0   2 💤 ±0  0 ❌ ±0 
13 750 runs  ±0  13 720 ✅ ±0  30 💤 ±0  0 ❌ ±0 

Results for commit 568692c. ± Comparison against base commit 580b4f2.

♻️ This comment has been updated with latest results.

[roryqi]

Could you use JMH to do the work of benchmark?

[maobaolong]

@jerqi Sorry, I have no experience about JMH, I'm ok if other contributor can use JMH as an improvement refactor after this PR or instead of this PR.

[roryqi]

@jerqi Sorry, I have no experience about JMH, I'm ok if other contributor can use JMH as an improvement refactor after this PR or instead of this PR.

It's a little weird to use web page to trigger the benchmark. It maybe dangerous for production environment.

[maobaolong]

@jerqi Do you think it would be better if we set a benchmark enable config to control this? It is just a rest API, not an actual web page

[roryqi]

@jerqi Do you think it would be better if we set a benchmark enable config to control this? It is just a rest API, not an actual web page

It's also weird to use REST API.

[maobaolong]

@jerqi I see, do you have any suggestion?

[roryqi]

@jerqi I see, do you have any suggestion?

You can refer to Iceberg benchmark. https://github.com/apache/iceberg/tree/main/spark/v3.5/spark-extensions/src/jmh/java/org/apache/iceberg
Spark's benchmark is similar.

[zuston]

Nice improvement.

BTW, is it possible to introduce the feature of general end to end benchmark test, like using the client to create the fake data to send shuffle-server and then to read? WDYT? @maobaolong

[maobaolong]

Nice improvement.

BTW, is it possible to introduce the feature of general end to end benchmark test, like using the client to create the fake data to send shuffle-server and then to read? WDYT? @maobaolong

Is this what you motioned?

[zuston]

Nice improvement.
BTW, is it possible to introduce the feature of general end to end benchmark test, like using the client to create the fake data to send shuffle-server and then to read? WDYT? @maobaolong

Is this what you motioned?

I have seen the feature https://github.com/apache/incubator-uniffle/tree/master/tools/client-simulation-yarn, that is my thought of above mentioned. Thanks again for your effort @maobaolong .

For this PR, can you put the bench module into the root tool directory to avoid code pollution. If this, I think the api invoking is acceptable. Because it will only start the bench shuffle-server for flushing benchmark.

[roryqi]

You can use similar method to Alluxio. You use cli to start the benchmark tests.