List view

• MultiNet v0.1

  This milestone covers our target release for MultiNet v0.1

  Due by November 4, 2024

  •12/12 issues closed
  100% complete0 open12 closed

• Profiling SoTA models on Multinet Benchmark

  Profile SoTA Open-Source and Closed-Source LLMs, VLMs, and VLAs on Multinet Benchmark and report their performances.

  Due by October 31, 2024

  •53/55 issues closed
  96% complete2 open53 closed

• Multinet Benchmark

  Formulate a final benchmark to evaluate the performance of models on the Multinet dataset. This includes task-wise metrics, task-category-wise metrics, and an overall, unified metric.

  Due by August 6, 2024

  •5/6 issues closed
  83% complete1 open5 closed

• MultiNet V0

  Completing the Collection and Standardization of V0 of MultiNet

  Due by August 16, 2024

  •36/37 issues closed
  97% complete1 open36 closed