Training Dataset

[rakage]

Hi, i want to train with my dataset but i got an error like this. Can you help me? Thanks

[mantasu]

The downloaded .zip datasets should be put inside data folder which should be at the same directory from where the pre-processing script is called.

For example, assuming all your .zip files are inside /root/sunglass_detector/training/data, you could simply run:

cd /root/sunglass_detector/training
python preprocess.py

[rakage]

i have downloaded the datasets, but got error when doing
python run.py fit --task classification:eyeglasses --size medium --batch-size 64 --trainer.max_epochs 300 --checkpoint.dirpath ckpt --root /home/rakage/training_glasses/data --seed 1

Error message:
Seed set to 1 Traceback (most recent call last): File "/home/rakage/training_glasses/run.py", line 204, in <module> cli_main() File "/home/rakage/training_glasses/run.py", line 200, in cli_main cli = RunCLI(create_wrapper_callback, seed_everything_default=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/pytorch_lightning/cli.py", line 385, in __init__ self.instantiate_classes() File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/pytorch_lightning/cli.py", line 535, in instantiate_classes self.config_init = self.parser.instantiate_classes(self.config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_deprecated.py", line 141, in patched_instantiate_classes cfg = self._unpatched_instantiate_classes(cfg, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_core.py", line 1184, in instantiate_classes cfg[subcommand] = subparser.instantiate_classes(cfg[subcommand], instantiate_groups=instantiate_groups) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_deprecated.py", line 141, in patched_instantiate_classes cfg = self._unpatched_instantiate_classes(cfg, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_core.py", line 1178, in instantiate_classes component.instantiate_class(component, cfg) File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_signatures.py", line 580, in group_instantiate_class parent[key] = instantiator_fn(group.group_class, **value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_common.py", line 128, in default_class_instantiator return class_type(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/jsonargparse/_util.py", line 397, in __new__ return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/training_glasses/run.py", line 196, in create_wrapper_callback return wrapper_cls(model, *data_cls.create_loaders(**kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/training_glasses/glasses_detector/_data/base_categorized_dataset.py", line 102, in create_loaders train_loader = cls.create_loader(split_type="train", **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/training_glasses/glasses_detector/_data/base_categorized_dataset.py", line 97, in create_loader return DataLoader(**default_loader_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 350, in __init__ sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rakage/miniconda3/envs/training_glasses/lib/python3.12/site-packages/torch/utils/data/sampler.py", line 143, in __init__ raise ValueError(f"num_samples should be a positive integer value, but got num_samples={self.num_samples}") ValueError: num_samples should be a positive integer value, but got num_samples=0

[mantasu]

This could happen if there are no images in no_eyeglasses or eyeglasses subfolder in at least one dataset/split.

I think something has failed during data preprocessing. Could you delete tmp directory under data and then rerun preprocess.py with flags --force --tasks classification? eyeglasses should contain more than 4 datasets if all are used. Alternatively, please ensure no subfolder is empty for both positive and negative sample directories per dataset folder.

Let me know if it helps!