PaddleOCR-VL在5090服务化部署失败

[amlei]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

目标

• 使用服务化 / VLM加速推理的接口方式，调用PaddleOCR-VL处理文档，并能够将文档处理过程文件及结果（含MD 或 Json 格式输出保存）。
• 使用 Go 作为客户端，对部署的模型服务接口 Openai HTTP 协议发起交互请求

尝试过 官方教程 所举的三种部署方式，最终确认是服务化部署可满足该目标，下面是问题路径描述。

问题描述

• 对表格/图表识别，无法处理，重复\n等字符
• 内容输出是纯文本，非MD格式
• 无页眉、页脚数据识别

环境

• Ubuntu 24.04
• Nvidia RTX 5090：
  • NVIDIA-SMI 580.105.08
  • Driver Version: 580.105.08
  • CUDA Version: 13.0
• NVCC：
  • Cuda compilation tools, release 12.8, V12.8.93
  • Build cuda_12.8.r12.8/compiler.35583870_0

部署方式

尝试过三种50系显卡方式：

• PaddleOCR NVIDIA Blackwell
• VLLM 部署指南

其中，根据PaddleOCR官方指南尝试过：

• 方法一，使用Docker镜像
• 方法二，VLM加速推理
• 方法三，Docker Compose服务化部署（服务启动，接口均404。与 使用 Dockers Compose 运行 PaddleOCR-VL 服务无法正常访问 #17359 相同的问题）

测试

测试图片

图1-纯文本 - Base64大小:149.37 KiB

图2-含页眉、页脚、图表 - Base64大小:668.88 KiB

图3-英文文档测试 - Base64大小:380.57 KiB

测试方式

使用 curl Openai API 协议，调用测试：

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer EMPTY" \
  -d '{
    "model": "PaddlePaddle/PaddleOCR-VL",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {
              "url": "data:image/png;base64,'$(cat receipt.b64)'"
            }
          },
          {
            "type": "text",
            "text": "OCR:"
          }
        ]
      }
    ],
    "temperature": 0.0
  }'

提示词方面，尝试过OCR:、Table Recognition:、Formula Recognition:、Chart Recognition: 及自定义提示词五种。

结果

图1-纯文本 和 图2-含页眉、页脚、图表 均可识别，结果如下：

图1-纯文本结果：无误

图2-含页眉、页脚、图表 结果：无误

图3-英文文档测试 结果：识别关系混乱

补充描述

1. 接口返回都是非MD、Json格式，同时无处理过程数据，如demo中展示的关系识别高亮区域，及其它文件输出。
   
2. 使用 Go 作为客户端，对部署的模型服务接口 Openai HTTP 协议发起交互请求。看Paddle官方指出，use_chart_recognition需要设置为True，这需要如何正确传递？同时如何使用如下，设置识别导出路径？

def paddle_ocr(image_path: str, maxPixels: int, minPixels: int):
    # 每个进程独立初始化 PaddleOCR 实例，防止共享内存

    output = pipeline.predict(
        image_path,
        use_layout_detection=True,
        format_block_content=True,
        use_chart_recognition=False,
        merge_layout_blocks=True,
        maxPixels=maxPixels,
        minPixels=minPixels,
        sort_filter_box=True,
    )

似乎要采用服务化部署方式，才能通过客户端完成该需求？其中包含⌈版面解析⌋ ⌈layoutParsingResults⌋ 和 ⌈MarkDown结果⌋，但目前测试50系显卡的 Docker Compose 部署，服务成功启动，但端点都是 404 #17359

尝试 服务化部署-手动部署 服务启动成功，接口响应成功。

预期目标/结果符合 PaddleOCR-VL在线测试响应结果

即使成功看到 docker compose 启动服务化部署方式所没有的 /docs 及 /openapi.json，但调用接口出现：

INFO:     127.0.0.1:52998 - "POST /layout-parsing HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 410, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1135, in __call__
    await super().__call__(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 115, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 101, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 355, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 243, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/serving/basic_serving/_pipeline_apps/paddleocr_vl.py", line 54, in _infer
    result = await pipeline.infer(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/serving/basic_serving/_app.py", line 104, in infer
    return await self.call(_infer, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/serving/basic_serving/_app.py", line 111, in call
    return await fut
           ^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/serving/basic_serving/_app.py", line 126, in _worker
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/serving/basic_serving/_app.py", line 95, in _infer
    for item in it:
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/pipelines/_parallel.py", line 129, in predict
    yield from self._pipeline.predict(
  File "/mnt/data/model/PaddlePaddle/.venv/lib/python3.12/site-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.py", line 715, in predict
    raise RuntimeError(
RuntimeError: Exception from the 'vlm' worker: only 0-dimensional arrays can be converted to Python scalars

与 #17495 问题类似

🏃‍♂️ Environment (运行环境)

• OS: Ubuntu 24.04
• Nvidia RTX 5090：
  • NVIDIA-SMI 580.105.08
  • Driver Version: 580.105.08
  • CUDA Version: 13.0
• NVCC：
  • Cuda compilation tools, release 12.8, V12.8.93
  • Build cuda_12.8.r12.8/compiler.35583870_0
• Install: Docker Compose & Paddlex
• RAM: 64.00 GB
• CPU: I9 14900K
• Python 3.12

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

1. 手动服务化部署使用的：官方：多语言调用服务实例

2. Docker compose 部署出现与 使用 Dockers Compose 运行 PaddleOCR-VL 服务无法正常访问 #17359 相同的问题

[amlei]

使用 docker compose 方式 #17359 后，enterpoint 出现 404 是接口调用失败，我是用手动部署方式成后，拿到 /docs 和 /openai.json 文件如下：

{"openapi":"3.1.0","info":{"title":"FastAPI","version":"0.1.0"},"paths":{"/health":{"get":{"summary":" Check Health","operationId":"checkHealth","responses":{"200":{"description":"Successful Response","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AIStudioNoResultResponse"}}}}}}},"/layout-parsing":{"post":{"summary":" Infer","operationId":"infer","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/InferRequest"}}},"required":true},"responses":{"200":{"description":"Successful Response","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AIStudioResultResponse_InferResult_"}}}},"422":{"description":"Unprocessable Entity","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AIStudioNoResultResponse"}}}},"500":{"description":"Internal Server Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AIStudioNoResultResponse"}}}}}}}},"components":{"schemas":{"AIStudioNoResultResponse":{"properties":{"logId":{"type":"string","title":"Logid"},"errorCode":{"type":"integer","title":"Errorcode"},"errorMsg":{"type":"string","title":"Errormsg"}},"type":"object","required":["logId","errorCode","errorMsg"],"title":"AIStudioNoResultResponse"},"AIStudioResultResponse_InferResult_":{"properties":{"logId":{"type":"string","title":"Logid"},"result":{"$ref":"#/components/schemas/InferResult"},"errorCode":{"type":"integer","const":0,"title":"Errorcode","default":0},"errorMsg":{"type":"string","const":"Success","title":"Errormsg","default":"Success"}},"type":"object","required":["logId","result"],"title":"AIStudioResultResponse[InferResult]"},"ImageInfo":{"properties":{"width":{"type":"integer","title":"Width"},"height":{"type":"integer","title":"Height"},"type":{"type":"string","const":"image","title":"Type","default":"image"}},"type":"object","required":["width","height"],"title":"ImageInfo"},"InferRequest":{"properties":{"file":{"type":"string","title":"File"},"fileType":{"anyOf":[{"type":"integer","enum":[0,1]},{"type":"null"}],"title":"Filetype"},"useDocOrientationClassify":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Usedocorientationclassify"},"useDocUnwarping":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Usedocunwarping"},"useLayoutDetection":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Uselayoutdetection"},"useChartRecognition":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Usechartrecognition"},"layoutThreshold":{"anyOf":[{"type":"number"},{"additionalProperties":true,"type":"object"},{"type":"null"}],"title":"Layoutthreshold"},"layoutNms":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Layoutnms"},"layoutUnclipRatio":{"anyOf":[{"type":"number"},{"prefixItems":[{"type":"number"},{"type":"number"}],"type":"array","maxItems":2,"minItems":2},{"additionalProperties":true,"type":"object"},{"type":"null"}],"title":"Layoutunclipratio"},"layoutMergeBboxesMode":{"anyOf":[{"type":"string"},{"additionalProperties":true,"type":"object"},{"type":"null"}],"title":"Layoutmergebboxesmode"},"promptLabel":{"anyOf":[{"type":"string"},{"type":"null"}],"title":"Promptlabel"},"formatBlockContent":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Formatblockcontent"},"repetitionPenalty":{"anyOf":[{"type":"number"},{"type":"null"}],"title":"Repetitionpenalty"},"temperature":{"anyOf":[{"type":"number"},{"type":"null"}],"title":"Temperature"},"topP":{"anyOf":[{"type":"number"},{"type":"null"}],"title":"Topp"},"minPixels":{"anyOf":[{"type":"integer"},{"type":"null"}],"title":"Minpixels"},"maxPixels":{"anyOf":[{"type":"integer"},{"type":"null"}],"title":"Maxpixels"},"maxNewTokens":{"anyOf":[{"type":"integer"},{"type":"null"}],"title":"Maxnewtokens"},"mergeLayoutBlocks":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Mergelayoutblocks"},"markdownIgnoreLabels":{"anyOf":[{"items":{"type":"string"},"type":"array"},{"type":"null"}],"title":"Markdownignorelabels"},"prettifyMarkdown":{"type":"boolean","title":"Prettifymarkdown","default":true},"showFormulaNumber":{"type":"boolean","title":"Showformulanumber","default":false},"visualize":{"anyOf":[{"type":"boolean"},{"type":"null"}],"title":"Visualize"}},"type":"object","required":["file"],"title":"InferRequest"},"InferResult":{"properties":{"layoutParsingResults":{"items":{"$ref":"#/components/schemas/LayoutParsingResult"},"type":"array","title":"Layoutparsingresults"},"dataInfo":{"oneOf":[{"$ref":"#/components/schemas/ImageInfo"},{"$ref":"#/components/schemas/PDFInfo"}],"title":"Datainfo","discriminator":{"propertyName":"type","mapping":{"image":"#/components/schemas/ImageInfo","pdf":"#/components/schemas/PDFInfo"}}}},"type":"object","required":["layoutParsingResults","dataInfo"],"title":"InferResult"},"LayoutParsingResult":{"properties":{"prunedResult":{"additionalProperties":true,"type":"object","title":"Prunedresult"},"markdown":{"$ref":"#/components/schemas/MarkdownData"},"outputImages":{"anyOf":[{"additionalProperties":{"type":"string"},"type":"object"},{"type":"null"}],"title":"Outputimages"},"inputImage":{"anyOf":[{"type":"string"},{"type":"null"}],"title":"Inputimage"}},"type":"object","required":["prunedResult","markdown"],"title":"LayoutParsingResult"},"MarkdownData":{"properties":{"text":{"type":"string","title":"Text"},"images":{"anyOf":[{"additionalProperties":{"type":"string"},"type":"object"},{"type":"null"}],"title":"Images"}},"type":"object","required":["text"],"title":"MarkdownData"},"PDFInfo":{"properties":{"numPages":{"type":"integer","title":"Numpages"},"pages":{"items":{"$ref":"#/components/schemas/PDFPageInfo"},"type":"array","title":"Pages"},"type":{"type":"string","const":"pdf","title":"Type","default":"pdf"}},"type":"object","required":["numPages","pages"],"title":"PDFInfo"},"PDFPageInfo":{"properties":{"width":{"type":"integer","title":"Width"},"height":{"type":"integer","title":"Height"}},"type":"object","required":["width","height"],"title":"PDFPageInfo"}}}}

使用多语言调用服务示例成功获得处理与响应：

[Bobholamovic]

看起来是官方Docker镜像存在一些问题，我们将尽快排查，感谢指出问题！

[GitHubmalajava]

为什么我运行docker pose然后本地访问时显示错误，好像是日志里好像是依赖版本有问题，你们有遇到吗

[amlei]

为什么我运行docker pose然后本地访问时显示错误，好像是日志里好像是依赖版本有问题，你们有遇到吗

在目前的镜像版本来看，这不会影响调用 /layout-parsing 接口识别文档。你可以看我 #17507 (comment) 这个获取的 openai.json 进行接口调用，详细的调用方式可以见：PaddleOCR-VL官方 客户端调用方法

[scyyh11]

关于 404 错误的说明

API 路由本身是正常工作的。出现 404 错误的原因是调用了错误的端点。

错误的调用方式

POST /chat/completions
POST /v1/chat/completions

正确的调用方式

POST /layout-parsing

请求示例：

curl -X POST http://localhost:8000/layout-parsing \
  -H "Content-Type: application/json" \
  -d '{
    "file": "https://example.com/test.jpg"
  }'

---

关于 /openapi.json 返回 500 错误

这是一个 Pydantic v2 兼容性问题。

问题原因

paddlex/inference/serving/infra/models.py 第 76 行的类型定义存在问题：

# 当前代码（有问题）
DataInfo: TypeAlias = Annotated[Union[ImageInfo, PDFInfo], Discriminator("type")]

这种写法在 Pydantic v2 中，当 FastAPI 生成 OpenAPI schema 时会触发以下错误：

TypeError: Discriminator must be used with a Union type, not typing.Annotated[...]

影响范围

功能	状态
POST /layout-parsing	正常工作
POST /ocr	正常工作
GET /health	正常工作
GET /openapi.json	500 错误
GET /docs	无法加载 API 文档

我们将尽快修复这个问题。