feat: debug "network volume model paths" issues #185
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Check if extra_model_paths.yaml exists at /comfyui/ - Print yaml content to logs - Check if /runpod-volume is mounted - List contents of /runpod-volume and /runpod-volume/models This helps debug why models on network volumes might not be detected.
- add logging to check if extra_model_paths.yaml exists and print content - add logging to check if /runpod-volume is mounted and list contents - replaces bash echo statements that were not captured by runpod serverless this diagnostic output will appear in runpod logs to help debug why models on network volumes are not being detected by comfyui.
- diagnostics now run on every request instead of module import time - use print() instead of logger for better stdout capture - this ensures diagnostic output appears in runpod serverless logs
…ooting - Add NETWORK_VOLUME_DEBUG environment variable (default: false) - Create comprehensive diagnostic function that shows: - Configuration file status - Network volume mount status - Directory structure validation - Model files found (with size and extension validation) - Expected structure guidance when issues found - Document network volume configuration in docs/configuration.md - Expected directory structure - Supported file extensions by model type - Step-by-step debugging instructions - Common issues and solutions Resolves user reports of models not being detected on network volumes. Root cause: user configuration issues (wrong directory structure or missing file extensions), not a bug in worker-comfyui.
- move NETWORK_VOLUME_DEBUG from debugging to logging configuration table - remove redundant TIP block about COMFY_LOG_LEVEL - fix customization.md tip block syntax to proper github admonition format - correct network volume paths in customization guide: - clarify /runpod-volume mount point for serverless endpoints - update example paths to show /runpod-volume/models/ structure - fix note block syntax to proper github admonition format - add links to network-volumes.md for detailed structure and debugging - mention s3-compatible api as upload option
- move network volume diagnostic functions from handler.py to src/network_volume.py - remove obsolete bash diagnostics from src/start.sh (replaced by python diagnostics) - update handler.py to import diagnostics from network_volume module - update dockerfile to include src/network_volume.py in image build - reduces handler.py size by ~150 lines for better maintainability
TimPietruskyRunPod
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
/runpod-volume/checkpoints/instead of/runpod-volume/models/checkpoints/)NETWORK_VOLUME_DEBUGenvironment variableIssues closed
fixes #91