Skip to content

script/download_files.py: improve logging #25

@benoistlaurent

Description

@benoistlaurent

Logging level

Logging level is set to logging.DEBUG which makes the script very verbose (at least on stderr), as the user gets debug messages from urllib3.

Better error highlighting

Errors are, IMHO, quite difficult to spot, for two reasons:

  • they are lost in the infinity of debug/info messages.
  • error messages are not displayed using logging.error which makes the message somewhat harder to spot in the log file (you have to use grep -i!).

Example:

2023-08-17 10:59:59 INFO Exception message: 429 Client Error: TOO MANY REQUESTS for url: https://zenodo.org/api/files/dd1e0822-c0f7-43fe-866c-4bc688750c20/PCPS_KCL_500.gro

A summary message at the end of the download process would be nice, e.g. "Successfully downloaded XX files. YY errors. Total size: ZZ Go"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions