Fixed summarization UDF to work with latest upstream changes

[KyleZheng1284]

Description

Fixed changes within summarization UDF

1. Pre Split (32 pages) was causing multiple summaries per file depending on the length. Example if total pages in PDF is 32-64 it will 2 summaries, if it has 64-96 pages it will have 3 summaries, ...

   • Added simple logic to skip non-first sections, so only the first and last chunk of the first section (1-32 pages) will be used for summarization.

2. Fixed custom pipeline to use nemotron_parse* from depreciated nemoretriever-parse*

3. Minor upstream fixes

   • Now using httpx instead of openai for API calls to build since we have removed that dep with https://github.com/NVIDIA/nv-ingest/pull/1101/changes
   • Changed max token limit to 4000, since the max token limit for the nvidia/nemotron-mini-4b-instruct is 4096

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.
• If adjusting docker-compose.yaml environment variables have you ensured those are mimicked in the Helm values.yaml file.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[jperez999]

Is this the only place this code will be used to call the summarizer? If not, should we consider making this into something that we can call in other places like a class or simply a function?