Skip to content

Conversation

@hect0x7
Copy link
Owner

@hect0x7 hect0x7 commented Apr 8, 2025

Summary by CodeRabbit

  • Chores
    • Updated the application version to 2.5.35.
  • New Features
    • Enhanced album download reliability by adding an option to check for download exceptions.
    • Introduced a new alert for partially failed downloads.
  • Bug Fixes
    • Improved error messages to provide more context.
  • Refactor
    • Streamlined failure tracking by clearly separating image and photo download issues.
    • Refined entity displays for clearer, more descriptive output.
    • Updated tutorial documentation to enhance exception handling for multi-threaded downloads.

@coderabbitai
Copy link

coderabbitai bot commented Apr 8, 2025

Walkthrough

This pull request updates multiple modules. The version variable in the library’s initialization file is bumped from "2.5.34" to "2.5.35". The download_album and download_photo functions now accept a new check_exception parameter to control error checking. Enhanced error reporting is added in the API client, while the downloader class introduces a catch_exception decorator, separates failure tracking, renames a method, and adds new properties/methods for download failure handling. Additionally, entity string representations are improved via a unified alias method, and a new exception is defined for partial download failures.

Changes

File(s) Change Summary
src/jmcomic/__init__.py Updated the __version__ variable from "2.5.34" to "2.5.35".
src/jmcomic/api.py Added a new check_exception parameter to download_album and download_photo with updated docstrings and logic to conditionally call raise_if_has_exception().
src/jmcomic/jm_client_impl.py Enhanced the error message in check_special_text to include both the error reason and the triggering content for better debugging.
src/jmcomic/jm_downloader.py Introduced the catch_exception decorator (applied to download methods); removed download_failed_list in favor of separate lists for images and photos; renamed execute_by_condition to execute_on_condition; added properties/methods for failure checks.
src/jmcomic/jm_entity.py Added a new class method __alias__ in JmBaseEntity and updated the __str__/__repr__ methods in DetailEntity and JmImageDetail to incorporate alias information into their string representations.
src/jmcomic/jm_exception.py Introduced PartialDownloadFailedException for signaling incomplete downloads; removed placeholder pass statements in JsonResolveFailException and RequestRetryAllFailException.
assets/docs/sources/tutorial/0_common_usage.md Updated exception handling logic in documentation to specify exceptions related to fetching details and added handling for PartialDownloadFailedException during multi-threaded downloads.
tests/test_jmcomic/test_jm_api.py Added a new test method test_partial_exception to check for PartialDownloadFailedException when conditions are met in the Test_Api class.

Sequence Diagram(s)

sequenceDiagram
    participant C as Client
    participant A as API.download_album
    participant D as Downloader
    C->>A: Call download_album(check_exception=True)
    A->>D: Process download (e.g., download_by_image_detail)
    D->>D: @catch_exception intercepts call
    alt Exception Occurs
        D-->>D: Log failure in image/photo list
    else Successful Download
        D-->>D: Return normally
    end
    A->>D: Call raise_if_has_exception()
    alt Failures exist
        D-->>C: Raise PartialDownloadFailedException
    else
        D-->>C: Complete download successfully
    end
Loading

Possibly related PRs

  • Dev #279 – Updates the __version__ variable in __init__.py, matching the core version bump here.
  • Dev #326 – Also involves modifications to the __version__ variable in __init__.py, providing a direct code-level connection.
  • 修复下载图片失败的异常处理 #284 – Related changes to the __version__ variable and updates to error handling in the jm_downloader.py file, focusing on improving exception management during download processes.

Poem

I hop through lines of freshly forged code,
In digital meadows my updates now glow,
With versions bumped and errors now caught,
My little paws fix what bugs have wrought,
Hoppy insights in every refactored row—
A bunny cheers as the new commits flow! 🐇

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (5)
src/jmcomic/api.py (1)

74-75: Consider adding exception handling logic to download_photo for consistency.

The check_exception parameter and exception handling has been added to download_album but not to download_photo. For API consistency, consider adding similar functionality to the download_photo function.

 def download_photo(jm_photo_id,
                    option=None,
                    downloader=None,
-                   callback=None):
+                   callback=None,
+                   check_exception=True):
     """
     下载一个章节(photo),参数同 download_album
+    
+    :param check_exception: 是否检查异常, 如果为True,会检查downloader是否有下载异常,并上抛PartialDownloadFailedException
     """
     if not isinstance(jm_photo_id, (str, int)):
         return download_batch(download_photo, jm_photo_id, option)
 
     with new_downloader(option, downloader) as dler:
         photo = dler.download_photo(jm_photo_id)
 
         if callback is not None:
             callback(photo, dler)
+
+        if check_exception:
+            dler.raise_if_has_exception()
 
         return photo, dler
src/jmcomic/jm_downloader.py (3)

4-24: Consider adding safeguards and a docstring to clarify usage.
The decorator relies on args[0] to retrieve the entity, which might lead to unexpected behavior if the entity is passed via keyword arguments. Also, consider ensuring a docstring and type hints to guide contributors on how to correctly pass the detail object as the first positional argument.

🧰 Tools
🪛 Ruff (0.8.2)

10-10: JmBaseEntity may be undefined, or defined from star imports

(F405)


12-12: JmImageDetail may be undefined, or defined from star imports

(F405)


13-13: jm_log may be undefined, or defined from star imports

(F405)


17-17: JmPhotoDetail may be undefined, or defined from star imports

(F405)


18-18: jm_log may be undefined, or defined from star imports

(F405)


196-196: Double-check the negated property usage.
if not self.has_no_download_failed_exception: is a double negative and can obscure meaning. Consider renaming for clarity.


209-212: Rename for clearer semantics.
The property name has_no_download_failed_exception can be confusing. A positive phrasing—like has_download_failures—aids readability.

src/jmcomic/jm_entity.py (1)

38-44: Fallback checks recommended for __alias__.
If 'm' or 'Detail' is missing in cls.__name__, slicing can cause errors. Consider verifying the class name beforehand or adding a fallback alias.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e4584d0 and e99af0c.

📒 Files selected for processing (6)
  • src/jmcomic/__init__.py (1 hunks)
  • src/jmcomic/api.py (3 hunks)
  • src/jmcomic/jm_client_impl.py (1 hunks)
  • src/jmcomic/jm_downloader.py (8 hunks)
  • src/jmcomic/jm_entity.py (3 hunks)
  • src/jmcomic/jm_exception.py (2 hunks)
🧰 Additional context used
🧬 Code Definitions (3)
src/jmcomic/jm_client_impl.py (1)
src/jmcomic/jm_client_interface.py (1)
  • content (25-26)
src/jmcomic/api.py (1)
src/jmcomic/jm_downloader.py (1)
  • raise_if_has_exception (270-285)
src/jmcomic/jm_downloader.py (3)
src/jmcomic/jm_entity.py (13)
  • JmBaseEntity (16-43)
  • is_image (23-24)
  • is_image (257-258)
  • JmImageDetail (168-263)
  • download_url (208-217)
  • is_photo (27-28)
  • is_photo (424-425)
  • JmPhotoDetail (266-425)
  • id (74-75)
  • id (407-408)
  • id (486-487)
  • JmAlbumDetail (428-540)
  • album_id (337-338)
src/jmcomic/jm_option.py (1)
  • build_jm_client (380-386)
src/jmcomic/jm_client_interface.py (3)
  • get_album_detail (142-143)
  • get_photo_detail (145-150)
  • check_photo (152-172)
🪛 Ruff (0.8.2)
src/jmcomic/jm_downloader.py

10-10: JmBaseEntity may be undefined, or defined from star imports

(F405)


12-12: JmImageDetail may be undefined, or defined from star imports

(F405)


13-13: jm_log may be undefined, or defined from star imports

(F405)


17-17: JmPhotoDetail may be undefined, or defined from star imports

(F405)


18-18: jm_log may be undefined, or defined from star imports

(F405)


78-78: Dict may be undefined, or defined from star imports

(F405)


78-78: JmAlbumDetail may be undefined, or defined from star imports

(F405)


78-78: Dict may be undefined, or defined from star imports

(F405)


78-78: JmPhotoDetail may be undefined, or defined from star imports

(F405)


78-78: List may be undefined, or defined from star imports

(F405)


78-78: Tuple may be undefined, or defined from star imports

(F405)


78-78: JmImageDetail may be undefined, or defined from star imports

(F405)


80-80: List may be undefined, or defined from star imports

(F405)


80-80: Tuple may be undefined, or defined from star imports

(F405)


80-80: JmImageDetail may be undefined, or defined from star imports

(F405)


81-81: List may be undefined, or defined from star imports

(F405)


81-81: Tuple may be undefined, or defined from star imports

(F405)


81-81: JmPhotoDetail may be undefined, or defined from star imports

(F405)


88-88: JmAlbumDetail may be undefined, or defined from star imports

(F405)


105-105: JmPhotoDetail may be undefined, or defined from star imports

(F405)


119-119: JmImageDetail may be undefined, or defined from star imports

(F405)


281-281: ExceptionTool may be undefined, or defined from star imports

(F405)


284-284: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


311-311: JmImageDetail may be undefined, or defined from star imports

(F405)


326-326: JmImageDetail may be undefined, or defined from star imports

(F405)

⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: test (3.13, ubuntu-latest)
🔇 Additional comments (16)
src/jmcomic/__init__.py (1)

5-5: Version bump for feature release.

The version has been updated from 2.5.34 to 2.5.35, correctly reflecting the new error handling enhancements added in this PR.

src/jmcomic/jm_client_impl.py (1)

569-569: Enhanced error message with content context.

This change improves the error reporting by including the content that triggered the error along with the reason, making debugging and troubleshooting easier.

src/jmcomic/api.py (1)

51-51: Good enhancement to error handling with configurable exception checking.

Adding the check_exception parameter with a default value of True maintains backward compatibility while giving users control over exception handling behavior. The docstring clearly explains the parameter's purpose and the potential exception that could be raised.

This feature enables more flexible error handling strategies in client code, allowing them to either catch partial download failures or continue despite them.

Also applies to: 62-63, 74-75

src/jmcomic/jm_exception.py (2)

48-48: Removed redundant pass statements.

The removal of empty pass statements is a good practice as they're unnecessary when the class already inherits all behavior from its parent class.

Also applies to: 59-60


62-63: Added new exception for partial download failures.

This new exception class enhances the error handling by providing a specific exception type for partial download failures, which makes error handling more granular and precise. The descriptive message helps in understanding the nature of the failure.

src/jmcomic/jm_downloader.py (9)

76-76: Looks good.
Initializing the client through self.client = option.build_jm_client() is straightforward and consistent with the available API.


80-81: Clear separation of failure tracking.
Using dedicated lists for failed images vs. failed photos improves readability and tracking.

🧰 Tools
🪛 Ruff (0.8.2)

80-80: List may be undefined, or defined from star imports

(F405)


80-80: Tuple may be undefined, or defined from star imports

(F405)


80-80: JmImageDetail may be undefined, or defined from star imports

(F405)


81-81: List may be undefined, or defined from star imports

(F405)


81-81: Tuple may be undefined, or defined from star imports

(F405)


81-81: JmPhotoDetail may be undefined, or defined from star imports

(F405)


84-85: Clean approach for fetching and applying album downloads.
The straightforward call to download_by_album_detail is consistent with the rest of the class design.


88-88: Method call flow looks appropriate.
The logic to dispatch the album’s photos with execute_on_condition is consistent with concurrent execution strategies in this codebase.

Also applies to: 92-96

🧰 Tools
🪛 Ruff (0.8.2)

88-88: JmAlbumDetail may be undefined, or defined from star imports

(F405)


100-101: Concise flow for single photo download.
Fetching the photo detail first and then delegating to download_by_photo_detail embraces the new decorator approach for exception handling.


104-106: Decorator usage is coherent.
Decorating download_by_photo_detail ensures photo-level failures are uniformly handled. The check_photo(photo) call prior to the download loop is helpful for data enrichment.

Also applies to: 111-115

🧰 Tools
🪛 Ruff (0.8.2)

105-105: JmPhotoDetail may be undefined, or defined from star imports

(F405)


270-285: Robust summary of partial downloads.
Raising PartialDownloadFailedException with a clear message and stored state is a practical way to surface the failures.

🧰 Tools
🪛 Ruff (0.8.2)

281-281: ExceptionTool may be undefined, or defined from star imports

(F405)


284-284: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


311-311: No-op download method is consistent with design.
By overriding download_by_image_detail to do nothing, this class effectively tests scenarios without actual file writes.

🧰 Tools
🪛 Ruff (0.8.2)

311-311: JmImageDetail may be undefined, or defined from star imports

(F405)


325-326: JustDownloadSpecificCountImage decorator usage is coherent.
Applying @catch_exception and then forwarding to super().download_by_image_detail(image) aligns well with partial or restricted download testing.

Also applies to: 331-331

🧰 Tools
🪛 Ruff (0.8.2)

326-326: JmImageDetail may be undefined, or defined from star imports

(F405)

src/jmcomic/jm_entity.py (2)

134-137: Enhanced string representation for DetailEntity.
Including the alias, ID, and title in the __str__ method greatly improves readability and debugging.


260-263: Consistent string representation for JmImageDetail.
Aligning JmImageDetail with the new alias approach ensures uniform logging and debugging across entity types.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
src/jmcomic/api.py (1)

79-84: Documentation improvement needed for download_photo.

While the implementation correctly adds the check_exception parameter, the docstring is too brief and doesn't explain this new parameter.

-    """
-    下载一个章节(photo),参数同 download_album
-    """
+    """
+    下载一个章节(photo)
+
+    :param jm_photo_id: 章节的禁漫车号
+    :param option: 下载选项
+    :param downloader: 下载器类
+    :param callback: 返回值回调函数,可以拿到 photo 和 downloader
+    :param check_exception: 是否检查异常, 如果为True,会检查downloader是否有下载异常,并上抛PartialDownloadFailedException
+    :return: 对于的章节实体类,下载器
+    """
src/jmcomic/jm_downloader.py (2)

4-25: Well-designed exception handling decorator.

The decorator elegantly centralizes exception handling logic, categorizing failures as either image or photo failures, and provides appropriate logging. This reduces code duplication and improves maintainability.

Consider using functools.wraps to preserve the function's metadata:

+from functools import wraps
+
 def catch_exception(func):
+    @wraps(func)
     def wrapper(self, *args, **kwargs):
         self: JmDownloader
         try:
             return func(self, *args, **kwargs)
         except Exception as e:
             detail: JmBaseEntity = args[0]
             if detail.is_image():
                 detail: JmImageDetail
                 jm_log('image.failed', f'图片下载失败: [{detail.download_url}], 异常: [{e}]')
                 self.download_failed_image.append((detail, e))
 
             elif detail.is_photo():
                 detail: JmPhotoDetail
                 jm_log('photo.failed', f'章节下载失败: [{detail.id}], 异常: [{e}]')
                 self.download_failed_photo.append((detail, e))
 
             raise e
-
-    wrapper.__name__ = func.__name__
     return wrapper
🧰 Tools
🪛 Ruff (0.8.2)

10-10: JmBaseEntity may be undefined, or defined from star imports

(F405)


12-12: JmImageDetail may be undefined, or defined from star imports

(F405)


13-13: jm_log may be undefined, or defined from star imports

(F405)


17-17: JmPhotoDetail may be undefined, or defined from star imports

(F405)


18-18: jm_log may be undefined, or defined from star imports

(F405)


270-286: Well-implemented exception raising logic.

The raise_if_has_exception method intelligently checks for failures and raises a detailed PartialDownloadFailedException with information about the specific failures.

For large numbers of failures, consider limiting the output to avoid excessively verbose error messages:

     def raise_if_has_exception(self):
         if not self.has_download_failures:
             return
         msg_ls = ['部分下载失败', '', '']
 
         if len(self.download_failed_photo) != 0:
-            msg_ls[1] = f'共{len(self.download_failed_photo)}个章节下载失败: {self.download_failed_photo}'
+            # Limit the number of failures shown to avoid excessively verbose messages
+            failed_photos_summary = str(self.download_failed_photo[:5])
+            if len(self.download_failed_photo) > 5:
+                failed_photos_summary = failed_photos_summary[:-1] + ", ... (and more)]"
+            msg_ls[1] = f'共{len(self.download_failed_photo)}个章节下载失败: {failed_photos_summary}'
 
         if len(self.download_failed_image) != 0:
-            msg_ls[2] = f'共{len(self.download_failed_image)}个图片下载失败: {self.download_failed_image}'
+            # Limit the number of failures shown to avoid excessively verbose messages
+            failed_images_summary = str(self.download_failed_image[:5])
+            if len(self.download_failed_image) > 5:
+                failed_images_summary = failed_images_summary[:-1] + ", ... (and more)]"
+            msg_ls[2] = f'共{len(self.download_failed_image)}个图片下载失败: {failed_images_summary}'
 
         ExceptionTool.raises(
             '\n'.join(msg_ls),
             {'downloader': self},
             PartialDownloadFailedException,
         )
🧰 Tools
🪛 Ruff (0.8.2)

281-281: ExceptionTool may be undefined, or defined from star imports

(F405)


284-284: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e99af0c and 58cf648.

📒 Files selected for processing (5)
  • assets/docs/sources/tutorial/0_common_usage.md (1 hunks)
  • src/jmcomic/api.py (4 hunks)
  • src/jmcomic/jm_downloader.py (8 hunks)
  • src/jmcomic/jm_entity.py (2 hunks)
  • src/jmcomic/jm_exception.py (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/jmcomic/jm_exception.py
  • src/jmcomic/jm_entity.py
🧰 Additional context used
🧠 Learnings (1)
src/jmcomic/jm_downloader.py (1)
Learnt from: hect0x7
PR: hect0x7/JMComic-Crawler-Python#412
File: src/jmcomic/jm_downloader.py:118-119
Timestamp: 2025-04-08T16:35:25.735Z
Learning: Python list.append operations are thread-safe due to the Global Interpreter Lock (GIL) in CPython, making explicit locks unnecessary for simple list modifications.
🧬 Code Definitions (1)
src/jmcomic/jm_downloader.py (6)
src/jmcomic/jm_entity.py (11)
  • JmBaseEntity (16-36)
  • is_image (23-24)
  • is_image (257-258)
  • is_photo (27-28)
  • is_photo (424-425)
  • JmPhotoDetail (266-425)
  • id (67-68)
  • id (407-408)
  • id (486-487)
  • JmAlbumDetail (428-540)
  • album_id (337-338)
src/jmcomic/jm_config.py (1)
  • jm_log (373-375)
src/jmcomic/jm_option.py (1)
  • build_jm_client (380-386)
src/jmcomic/jm_client_interface.py (2)
  • get_album_detail (142-143)
  • get_photo_detail (145-150)
src/jmcomic/jm_client_impl.py (6)
  • get_album_detail (271-272)
  • get_album_detail (663-666)
  • get_album_detail (1105-1109)
  • get_photo_detail (274-288)
  • get_photo_detail (668-679)
  • get_photo_detail (1129-1176)
src/jmcomic/jm_exception.py (2)
  • ExceptionTool (69-191)
  • PartialDownloadFailedException (62-67)
🪛 Ruff (0.8.2)
src/jmcomic/jm_downloader.py

10-10: JmBaseEntity may be undefined, or defined from star imports

(F405)


12-12: JmImageDetail may be undefined, or defined from star imports

(F405)


13-13: jm_log may be undefined, or defined from star imports

(F405)


17-17: JmPhotoDetail may be undefined, or defined from star imports

(F405)


18-18: jm_log may be undefined, or defined from star imports

(F405)


78-78: Dict may be undefined, or defined from star imports

(F405)


78-78: JmAlbumDetail may be undefined, or defined from star imports

(F405)


78-78: Dict may be undefined, or defined from star imports

(F405)


78-78: JmPhotoDetail may be undefined, or defined from star imports

(F405)


78-78: List may be undefined, or defined from star imports

(F405)


78-78: Tuple may be undefined, or defined from star imports

(F405)


78-78: JmImageDetail may be undefined, or defined from star imports

(F405)


80-80: List may be undefined, or defined from star imports

(F405)


80-80: Tuple may be undefined, or defined from star imports

(F405)


80-80: JmImageDetail may be undefined, or defined from star imports

(F405)


81-81: List may be undefined, or defined from star imports

(F405)


81-81: Tuple may be undefined, or defined from star imports

(F405)


81-81: JmPhotoDetail may be undefined, or defined from star imports

(F405)


88-88: JmAlbumDetail may be undefined, or defined from star imports

(F405)


105-105: JmPhotoDetail may be undefined, or defined from star imports

(F405)


119-119: JmImageDetail may be undefined, or defined from star imports

(F405)


281-281: ExceptionTool may be undefined, or defined from star imports

(F405)


284-284: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


311-311: JmImageDetail may be undefined, or defined from star imports

(F405)


326-326: JmImageDetail may be undefined, or defined from star imports

(F405)

🔇 Additional comments (13)
assets/docs/sources/tutorial/0_common_usage.md (2)

85-91: Improved error handling specificity.

The updated comment now correctly specifies that the exceptions being caught relate specifically to fetching album or chapter details, which provides better context for users.


105-117: Excellent documentation for the new partial download exception handling.

The new example provides clear guidance on how to handle partial download failures in a multi-threaded context. This is a valuable addition for users implementing error handling in their applications.

src/jmcomic/api.py (4)

47-52: Clear addition of error checking parameter.

The new check_exception parameter with a default value of True maintains backward compatibility while providing users with the option to control exception checking behavior.

🧰 Tools
🪛 Ruff (0.8.2)

52-52: Union may be undefined, or defined from star imports

(F405)


52-52: Set may be undefined, or defined from star imports

(F405)


62-63: Good docstring update.

The parameter documentation clearly explains the purpose and behavior of the new check_exception parameter.


74-76: Appropriate implementation of exception checking.

The conditional call to dler.raise_if_has_exception() correctly implements the behavior described in the docstring.


96-98: Consistent implementation of exception checking.

The implementation for download_photo correctly mirrors the implementation in download_album, ensuring consistent behavior.

src/jmcomic/jm_downloader.py (7)

80-81: Improved failure tracking with separate lists.

Separating failure tracking into download_failed_image and download_failed_photo lists is a good design improvement that allows for more specific failure reporting and analysis.

🧰 Tools
🪛 Ruff (0.8.2)

80-80: List may be undefined, or defined from star imports

(F405)


80-80: Tuple may be undefined, or defined from star imports

(F405)


80-80: JmImageDetail may be undefined, or defined from star imports

(F405)


81-81: List may be undefined, or defined from star imports

(F405)


81-81: Tuple may be undefined, or defined from star imports

(F405)


81-81: JmPhotoDetail may be undefined, or defined from star imports

(F405)


84-86: Simplified client initialization.

Direct usage of self.client and removing the indirect call improves code clarity and reduces unnecessary method calls.


104-107: Clean application of the decorator.

The @catch_exception decorator is properly applied to the download_by_photo_detail method, centralizing error handling logic.

🧰 Tools
🪛 Ruff (0.8.2)

105-105: JmPhotoDetail may be undefined, or defined from star imports

(F405)


118-142: Improved encapsulation for image download.

The @catch_exception decorator is properly applied to the download_by_image_detail method, and the direct usage of client methods with cleaner parameter passing improves code clarity.

🧰 Tools
🪛 Ruff (0.8.2)

119-119: JmImageDetail may be undefined, or defined from star imports

(F405)


123-123: file_exists may be undefined, or defined from star imports

(F405)


146-147: Better method naming.

Renaming execute_by_condition to execute_on_condition improves readability and better reflects the method's purpose.

🧰 Tools
🪛 Ruff (0.8.2)

147-147: DetailEntity may be undefined, or defined from star imports

(F405)


210-212: Useful property for failure detection.

The has_download_failures property provides a clean and concise way to check for download failures, improving code readability wherever this check is needed.


325-326: Consistent decorator application.

The @catch_exception decorator is consistently applied to the subclass implementation of download_by_image_detail, ensuring that error handling behavior is preserved in subclasses.

🧰 Tools
🪛 Ruff (0.8.2)

326-326: JmImageDetail may be undefined, or defined from star imports

(F405)

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
src/jmcomic/jm_downloader.py (2)

4-26: Good implementation of the exception-handling decorator.

The catch_exception decorator is well-structured with proper use of @wraps to preserve function metadata. It cleanly separates error handling for different entity types and provides informative logging.

A minor suggestion would be to consider adding type annotations instead of type comments for better IDE support and static analysis:

-def catch_exception(func):
+def catch_exception(func: Callable) -> Callable:
-        detail: JmBaseEntity = args[0]
+        detail = args[0]  # type: JmBaseEntity
🧰 Tools
🪛 Ruff (0.8.2)

13-13: JmBaseEntity may be undefined, or defined from star imports

(F405)


15-15: JmImageDetail may be undefined, or defined from star imports

(F405)


16-16: jm_log may be undefined, or defined from star imports

(F405)


20-20: JmPhotoDetail may be undefined, or defined from star imports

(F405)


21-21: jm_log may be undefined, or defined from star imports

(F405)


272-288: Well-implemented error reporting mechanism.

The raise_if_has_exception method provides a clean way to check for failures and raise a descriptive exception with all the relevant information. This helps with error diagnosis and reporting.

Consider enhancing the error message by including more specific details about each failure:

- msg_ls[1] = f'共{len(self.download_failed_photo)}个章节下载失败: {self.download_failed_photo}'
+ msg_ls[1] = f'共{len(self.download_failed_photo)}个章节下载失败: ' + \
+           ', '.join([f"{p[0].id}: {str(p[1])}" for p in self.download_failed_photo[:5]])

This would provide more readable error messages, especially when there are many failures.

🧰 Tools
🪛 Ruff (0.8.2)

283-283: ExceptionTool may be undefined, or defined from star imports

(F405)


286-286: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 58cf648 and 9bc9a00.

📒 Files selected for processing (1)
  • src/jmcomic/jm_downloader.py (8 hunks)
🧰 Additional context used
🧠 Learnings (1)
src/jmcomic/jm_downloader.py (1)
Learnt from: hect0x7
PR: hect0x7/JMComic-Crawler-Python#412
File: src/jmcomic/jm_downloader.py:118-119
Timestamp: 2025-04-08T16:35:25.735Z
Learning: Python list.append operations are thread-safe due to the Global Interpreter Lock (GIL) in CPython, making explicit locks unnecessary for simple list modifications.
🧬 Code Definitions (1)
src/jmcomic/jm_downloader.py (4)
src/jmcomic/jm_entity.py (13)
  • JmBaseEntity (16-36)
  • is_image (23-24)
  • is_image (257-258)
  • JmImageDetail (168-263)
  • download_url (208-217)
  • is_photo (27-28)
  • is_photo (424-425)
  • JmPhotoDetail (266-425)
  • id (67-68)
  • id (407-408)
  • id (486-487)
  • JmAlbumDetail (428-540)
  • album_id (337-338)
src/jmcomic/jm_option.py (1)
  • build_jm_client (380-386)
src/jmcomic/jm_client_impl.py (6)
  • get_album_detail (271-272)
  • get_album_detail (663-666)
  • get_album_detail (1105-1109)
  • get_photo_detail (274-288)
  • get_photo_detail (668-679)
  • get_photo_detail (1129-1176)
src/jmcomic/jm_client_interface.py (2)
  • get_album_detail (142-143)
  • get_photo_detail (145-150)
🪛 Ruff (0.8.2)
src/jmcomic/jm_downloader.py

13-13: JmBaseEntity may be undefined, or defined from star imports

(F405)


15-15: JmImageDetail may be undefined, or defined from star imports

(F405)


16-16: jm_log may be undefined, or defined from star imports

(F405)


20-20: JmPhotoDetail may be undefined, or defined from star imports

(F405)


21-21: jm_log may be undefined, or defined from star imports

(F405)


80-80: Dict may be undefined, or defined from star imports

(F405)


80-80: JmAlbumDetail may be undefined, or defined from star imports

(F405)


80-80: Dict may be undefined, or defined from star imports

(F405)


80-80: JmPhotoDetail may be undefined, or defined from star imports

(F405)


80-80: List may be undefined, or defined from star imports

(F405)


80-80: Tuple may be undefined, or defined from star imports

(F405)


80-80: JmImageDetail may be undefined, or defined from star imports

(F405)


82-82: List may be undefined, or defined from star imports

(F405)


82-82: Tuple may be undefined, or defined from star imports

(F405)


82-82: JmImageDetail may be undefined, or defined from star imports

(F405)


83-83: List may be undefined, or defined from star imports

(F405)


83-83: Tuple may be undefined, or defined from star imports

(F405)


83-83: JmPhotoDetail may be undefined, or defined from star imports

(F405)


90-90: JmAlbumDetail may be undefined, or defined from star imports

(F405)


107-107: JmPhotoDetail may be undefined, or defined from star imports

(F405)


121-121: JmImageDetail may be undefined, or defined from star imports

(F405)


283-283: ExceptionTool may be undefined, or defined from star imports

(F405)


286-286: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


313-313: JmImageDetail may be undefined, or defined from star imports

(F405)


328-328: JmImageDetail may be undefined, or defined from star imports

(F405)

🔇 Additional comments (7)
src/jmcomic/jm_downloader.py (7)

76-84: Improved class initialization and failure tracking.

Splitting the failure tracking into separate lists (download_failed_image and download_failed_photo) is a good design decision, making it easier to handle different types of failures separately.

Direct initialization of self.client is cleaner than using an intermediate method call.

🧰 Tools
🪛 Ruff (0.8.2)

76-76: JmOption may be undefined, or defined from star imports

(F405)


80-80: Dict may be undefined, or defined from star imports

(F405)


80-80: JmAlbumDetail may be undefined, or defined from star imports

(F405)


80-80: Dict may be undefined, or defined from star imports

(F405)


80-80: JmPhotoDetail may be undefined, or defined from star imports

(F405)


80-80: List may be undefined, or defined from star imports

(F405)


80-80: Tuple may be undefined, or defined from star imports

(F405)


80-80: JmImageDetail may be undefined, or defined from star imports

(F405)


82-82: List may be undefined, or defined from star imports

(F405)


82-82: Tuple may be undefined, or defined from star imports

(F405)


82-82: JmImageDetail may be undefined, or defined from star imports

(F405)


83-83: List may be undefined, or defined from star imports

(F405)


83-83: Tuple may be undefined, or defined from star imports

(F405)


83-83: JmPhotoDetail may be undefined, or defined from star imports

(F405)


85-88: Simplified album download method.

The refactoring to directly use self.client instead of an intermediate method simplifies the code and improves readability.


94-95: Method name change improves clarity.

Renaming execute_by_condition to execute_on_condition is a good choice as it better represents the function's purpose - executing a function on a collection of objects when certain conditions are met.

Also applies to: 113-114, 148-149


106-109: Good application of the decorator pattern.

Using the catch_exception decorator for error handling is cleaner than embedding try-except blocks in each method. This follows the DRY principle and centralizes error handling logic.

🧰 Tools
🪛 Ruff (0.8.2)

107-107: JmPhotoDetail may be undefined, or defined from star imports

(F405)


120-144: Clean refactoring of the download method.

The image download logic is now properly decorated with catch_exception, simplifying the method by removing direct exception handling code while maintaining the same functionality.

🧰 Tools
🪛 Ruff (0.8.2)

121-121: JmImageDetail may be undefined, or defined from star imports

(F405)


125-125: file_exists may be undefined, or defined from star imports

(F405)


211-214: Useful helper property for error state detection.

The has_download_failures property provides a clean way to check for download failures without having to repeatedly write the condition manually throughout the code.


327-334: Consistent error handling in derived class.

The catch_exception decorator is properly applied to the overridden method in the JustDownloadSpecificCountImage class, ensuring consistent error handling throughout the inheritance hierarchy.

🧰 Tools
🪛 Ruff (0.8.2)

328-328: JmImageDetail may be undefined, or defined from star imports

(F405)

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🔭 Outside diff range comments (1)
tests/test_jmcomic/test_jm_api.py (1)

1-1: 🛠️ Refactor suggestion

Consider replacing star imports with explicit imports

The line from test_jmcomic import * makes it difficult to track which symbols are being used in the file. Static analysis is flagging several undefined symbols like JmDownloader, JmImageDetail, catch_exception, and PartialDownloadFailedException.

Replace star imports with explicit imports to improve code maintainability and readability.

-from test_jmcomic import *
+from test_jmcomic import JmTestConfigurable, jmcomic
+from jmcomic.jm_downloader import JmDownloader, catch_exception
+from jmcomic.entity import JmImageDetail, JmPhotoDetail
+from jmcomic.exception import PartialDownloadFailedException
+# Import other required symbols explicitly
🧰 Tools
🪛 Ruff (0.8.2)

1-1: from test_jmcomic import * used; unable to detect undefined names

(F403)

🧹 Nitpick comments (2)
tests/test_jmcomic/test_jm_api.py (2)

80-99: Improve test clarity and structure

The test successfully verifies the new exception handling behavior, but could benefit from a few improvements:

  1. Consider adding a docstring explaining the purpose of this test and the expected behavior.
  2. The TestDownloader class could be moved outside the test method for better readability.
+    """
+    Tests that partial download failures are properly caught and raised as PartialDownloadFailedException
+    when check_exception is set to True.
+    """
     def test_partial_exception(self):
🧰 Tools
🪛 Ruff (0.8.2)

81-81: JmDownloader may be undefined, or defined from star imports

(F405)


82-82: catch_exception may be undefined, or defined from star imports

(F405)


83-83: JmImageDetail may be undefined, or defined from star imports

(F405)


86-86: catch_exception may be undefined, or defined from star imports

(F405)


87-87: JmPhotoDetail may be undefined, or defined from star imports

(F405)


93-93: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


94-94: download_album may be undefined, or defined from star imports

(F405)


97-97: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


98-98: download_photo may be undefined, or defined from star imports

(F405)

🪛 GitHub Check: test (3.11, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.13, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.10, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.9, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


83-85: Consider using a custom exception subclass for testing

Instead of using a generic Exception, consider creating a custom exception class specifically for testing. This would make it easier to identify test-related exceptions in logs.

-            def download_by_image_detail(self, image: JmImageDetail):
-                raise Exception('test_partial_exception')
+            def download_by_image_detail(self, image: JmImageDetail):
+                class TestException(Exception): pass
+                raise TestException('test_partial_exception')
🧰 Tools
🪛 Ruff (0.8.2)

83-83: JmImageDetail may be undefined, or defined from star imports

(F405)

🪛 GitHub Check: test (3.11, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception

🪛 GitHub Check: test (3.13, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception

🪛 GitHub Check: test (3.10, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception

🪛 GitHub Check: test (3.9, ubuntu-latest)

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9bc9a00 and 99dde9a.

📒 Files selected for processing (1)
  • tests/test_jmcomic/test_jm_api.py (1 hunks)
🧰 Additional context used
🪛 Ruff (0.8.2)
tests/test_jmcomic/test_jm_api.py

81-81: JmDownloader may be undefined, or defined from star imports

(F405)


82-82: catch_exception may be undefined, or defined from star imports

(F405)


83-83: JmImageDetail may be undefined, or defined from star imports

(F405)


86-86: catch_exception may be undefined, or defined from star imports

(F405)


87-87: JmPhotoDetail may be undefined, or defined from star imports

(F405)


93-93: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


94-94: download_album may be undefined, or defined from star imports

(F405)


97-97: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


98-98: download_photo may be undefined, or defined from star imports

(F405)

🪛 GitHub Check: test (3.11, ubuntu-latest)
tests/test_jmcomic/test_jm_api.py

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.13, ubuntu-latest)
tests/test_jmcomic/test_jm_api.py

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.10, ubuntu-latest)
tests/test_jmcomic/test_jm_api.py

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.9, ubuntu-latest)
tests/test_jmcomic/test_jm_api.py

[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 84-84:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🔇 Additional comments (2)
tests/test_jmcomic/test_jm_api.py (2)

87-91: Good conditional exception handling

The conditional exception based on photo.index value is a good approach to test partial failures, where some downloads succeed and others fail.

🧰 Tools
🪛 Ruff (0.8.2)

87-87: JmPhotoDetail may be undefined, or defined from star imports

(F405)

🪛 GitHub Check: test (3.11, ubuntu-latest)

[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.13, ubuntu-latest)

[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.10, ubuntu-latest)

[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception

🪛 GitHub Check: test (3.9, ubuntu-latest)

[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


[failure] 89-89:
test_partial_exception


92-95:

Details

✅ Verification successful

Verify test failure is expected behavior

This test intentionally throws exceptions to verify error handling. The GitHub Check failures for this test are likely expected as part of testing the exception handling mechanism.

The test uses hardcoded album ID 182150. Make sure this ID is a real test album ID that can be reliably used in tests.


🏁 Script executed:

#!/bin/bash
# Check if these test IDs are consistently used in other tests
grep -r "182150\|182151" --include="*.py" .

Length of output: 301


Action: Confirm intentional exception test behavior and validate album ID usage

The test is intentionally triggering an exception to verify that error handling via PartialDownloadFailedException works as expected. The hardcoded album ID (182150) is consistently used within these tests (as seen alongside photo ID 182151), which indicates that it's a deliberate choice for testing purposes. No changes are required.

🧰 Tools
🪛 Ruff (0.8.2)

93-93: PartialDownloadFailedException may be undefined, or defined from star imports

(F405)


94-94: download_album may be undefined, or defined from star imports

(F405)

@hect0x7 hect0x7 merged commit ebf0ea6 into master Apr 8, 2025
4 checks passed
@hect0x7 hect0x7 deleted the dev branch April 8, 2025 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants