diff --git a/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/README.md b/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/README.md
index aef8a18..896418f 100644
--- a/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/README.md
+++ b/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/README.md
@@ -1,7 +1,7 @@
# Self-Supervised Learning for Strong Gravitational Lensing :sparkles: :milky_way:
*Special thanks to (mentors) Anna Parul, Yurii Halychanskyi, and Sergei Gleyzer.*
-
+
Click on this button below to read the detailed blog (Part 1):
[](https://yaashwardhan.medium.com/self-supervised-learning-for-strong-gravitational-lensing-part1-5a049e976b51)
@@ -9,32 +9,208 @@ Click on this button below to read the detailed blog (Part 1):
[](https://skillicons.dev)
+---
+
+## Project Overview
+
+This module implements **self-supervised learning (SSL)** techniques — specifically **Contrastive Learning** (Rotation & Gaussian Noise pretext tasks) and **Bootstrap Your Own Latent (BYOL)** — for classifying strong gravitational lensing images from [DeepLense](https://github.com/ML4SCI/DeepLense) datasets.
+
+**Key goals:**
+- Pre-train a ResNet50 encoder on unlabelled lensing images using SSL pretext tasks.
+- Fine-tune the pre-trained encoder on labelled data for downstream classification (axion / CDM / no sub-structure).
+- Benchmark SSL performance against a fully supervised ResNet50 baseline.
+
+The pipeline covers:
+1. Data loading and augmentation
+2. Self-supervised pre-training (BYOL / Contrastive Learning)
+3. Fine-tuning on labelled data
+4. Evaluation (AUC, ROC curves, confusion matrix)
+
+---
+
+## Installation
+
+### Prerequisites
+- Python ≥ 3.8
+- CUDA-compatible GPU (recommended for training)
+
+### 1. Clone the repository
+
+```bash
+git clone https://github.com/ML4SCI/DeepLense.git
+cd DeepLense
+```
+
+### 2. (Optional) Create a virtual environment
+
+```bash
+python -m venv venv
+# On Linux/macOS
+source venv/bin/activate
+# On Windows
+venv\Scripts\activate
+```
+
+### 3. Install dependencies
+
+```bash
+pip install tensorflow>=2.6 keras numpy pandas scikit-learn pillow imageio tqdm matplotlib
+```
+
+> **Note:** TensorFlow ≥ 2.6 is required. If you have a GPU, install the GPU-enabled build:
+> ```bash
+> pip install tensorflow-gpu>=2.6
+> ```
+
+---
+
+## Repository Structure
+
+```
+Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/
+│
+├── byol_learning/ # BYOL self-supervised pipeline
+│ ├── 1.imports.py # All imports
+│ ├── 2.parameters.py # Hyperparameters (image size, epochs, lr, batch size)
+│ ├── 3.augmentation.py # Data augmentation strategies
+│ ├── 4.data_generator.py # Custom Keras data generator
+│ ├── 5.encoder.py # Encoder/projection head definition
+│ ├── 6.resnet50_baseline.py # Supervised ResNet50 baseline
+│ ├── 7.byol_pretraining.py # BYOL pre-training loop
+│ ├── 8.finetuning.py # Fine-tuning on labelled data
+│ ├── 9.eval.py # Evaluation: AUC, ROC, confusion matrix
+│ └── notebooks/ # Jupyter notebooks for exploration
+│
+├── contrastive_learning/ # Contrastive Learning pipeline
+│ ├── 1.imports.py
+│ ├── 2.parameters.py
+│ ├── 3.data_generator.py
+│ ├── 4.augmentations.py # Rotation & Gaussian Noise augmentations
+│ ├── 5.encoder.py
+│ ├── 5.vit_encoder.py # Vision Transformer encoder (experimental)
+│ ├── 6.resnet50_baseline.py
+│ ├── 7.rotation_pretraining.py # Contrastive pre-training (Rotation pretext)
+│ ├── 8.rotation_finetuning.py
+│ ├── 9.gaussian_pretraining.py # Contrastive pre-training (Gaussian Noise pretext)
+│ ├── 10.gaussian_finetuning.py
+│ ├── 11.eval.py
+│ └── notebooks/
+│
+├── real_data/ # Scripts tested on real observational data
+├── regression_notebooks/ # Regression experiments with SSL representations
+│
+├── header.png
+├── byol.png
+├── contrastive.png
+├── augmentations.png
+└── README.md
+```
+
+---
+
+## Usage
+
+Each numbered Python script is designed to be run **in order** within its respective folder. Below are the steps for each pipeline.
+
+### BYOL Pipeline
+
+```bash
+cd byol_learning
+
+# Step 1 – (Optional) Review/adjust imports
+python 1.imports.py
+
+# Step 2 – (Optional) Adjust hyperparameters
+# Edit 2.parameters.py: width, num_epochs, batch_size, lr, input_shape
+
+# Step 3 – Pre-train the encoder using BYOL
+python 7.byol_pretraining.py
+
+# Step 4 – Fine-tune on labelled data
+python 8.finetuning.py
+
+# Step 5 – Evaluate the model
+python 9.eval.py
+```
+
+### Contrastive Learning Pipeline (Rotation Pretext)
+
+```bash
+cd contrastive_learning
+
+# Pre-train with rotation pretext task
+python 7.rotation_pretraining.py
+
+# Fine-tune
+python 8.rotation_finetuning.py
+
+# Evaluate
+python 11.eval.py
+```
+
+### Contrastive Learning Pipeline (Gaussian Noise Pretext)
+
+```bash
+cd contrastive_learning
+
+# Pre-train with Gaussian noise pretext task
+python 9.gaussian_pretraining.py
+
+# Fine-tune
+python 10.gaussian_finetuning.py
+
+# Evaluate
+python 11.eval.py
+```
+
+### Key Hyperparameters (`2.parameters.py`)
+
+| Parameter | Default | Description |
+|:-------------:|:-------:|:----------------------------------:|
+| `width` | 128 | Width of the projection head |
+| `num_epochs` | 30 | Number of training epochs |
+| `batch_size` | 128 | Batch size |
+| `lr` | 1e-4 | Learning rate (Adam optimizer) |
+| `input_shape` | (64,64,3) | Input image shape |
+
+---
+
## Contrastive Learning
-Contrastive learning is a type of self-supervised learning method, that tries to learn similar and dissimilar representations of data by contrasting positive and negative examples.
+Contrastive learning is a type of self-supervised learning method that learns similar and dissimilar representations by contrasting positive and negative examples.
## Bootstrap Your Own Latent (BYOL) Learning
-BYOL trains two networks, the target network and the online network, both in parallel. There are no positive or negative pairs here like there are in contrastive learning. Two different augmented views of the ‘same’ image are brought, and representations are learned using the online network, while the target network is a moving average of the online network, giving it a slower parameter update.
+BYOL trains two networks — an **online network** and a **target network** — in parallel. Unlike contrastive learning, there are no negative pairs. Two augmented views of the same image are passed through both networks; the online network is trained to predict the target network's representation, while the target network is updated as a moving average of the online network.
+---
+
## Classification Results
-The values in the Model columns represent AUC for axion, cdm and no_sub respectively.
-e.g (0.97, 0.96, 1.0) represents 0.97 AUC for axion, 0.96 AUC for cdm and 1.0 AUC for no_sub.
+
+The values in the Model columns represent AUC for axion, cdm and no_sub respectively.
+e.g. `(0.97, 0.96, 1.0)` → 0.97 AUC for axion, 0.96 AUC for cdm, 1.0 AUC for no_sub.
All results were calculated on a **separate test set**.
+| NN Architecture | Model I | Model II | Model III |
+| :-----------------------------------: | :------------: | :---------: | :----------: |
+| Baseline ResNet50 | 0.97, 0.96, 1.0 | 0.98, 0.92, 0.98 | 0.96,0.95,0.99 |
+| Contrastive Learning Rotation Pretext | 0.92, 0.91, 1.0 |**0.99, 0.99, 1.0** | **1.0, 0.99, 1.0** |
+|Contrastive Learning Gaussian Noise Pretext| 0.96,0.95,1.0 | **0.99, 0.99, 1.0** | **1.0, 0.99, 1.0** |
+| Bootstrap Your Own Latent | 0.95, 0.93, 1.0 | **1.0, 0.98, 1.0** | 0.98, 0.92, 0.98 |
-| NN Architecture | Model I | Model II | Model III |
-| :-----------------------------------: | :------------: | :---------: | :----------: |
-| Baseline ResNet50 | 0.97, 0.96, 1.0 | 0.98, 0.92, 0.98 | 0.96,0.95,0.99 |
-| Contrastive Learning Rotation Pretext | 0.92, 0.91, 1.0 |**0.99, 0.99, 1.0** | **1.0, 0.99, 1.0** |
-|Contrastive Learning Gaussian Noise Pretext| 0.96,0.95,1.0 | **0.99, 0.99, 1.0** | **1.0, 0.99, 1.0** |
-| Bootstrap Your Own Latent | 0.95, 0.93, 1.0 | **1.0, 0.98, 1.0** | 0.98, 0.92, 0.98 |
+---
## Conclusion and Future Goals
-So far, we can see that the results for self-supervised learning seem superior to their ResNet50 Baseline.
-Future goals consist of testing models for regression, implementing vision transformers (twin networks) and testing some more self-supervised learning methods.
+
+Self-supervised learning results are consistently **superior** to the supervised ResNet50 baseline across all three models.
+
+Future goals:
+- Test SSL representations for regression tasks
+- Implement Vision Transformer (ViT) twin networks for SSL
+- Explore additional SSL methods (SimSiam, MoCo, etc.)
+- Evaluate on real observational lensing data
diff --git a/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/requirements.txt b/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/requirements.txt
new file mode 100644
index 0000000..a5de5ce
--- /dev/null
+++ b/Deeplens_Self_Supervised_Learning_Yashwardhan_Deshmukh/requirements.txt
@@ -0,0 +1,9 @@
+tensorflow>=2.10.0
+keras>=2.10.0
+numpy>=1.21.0
+pandas>=1.3.0
+matplotlib>=3.5.0
+pillow>=9.0.0
+scikit-learn>=1.0.0
+tqdm>=4.62.0
+imageio>=2.19.0
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/README.md b/Transformers_Classification_DeepLense_Kartik_Sachdev/README.md
index cf5a39a..d0fab56 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/README.md
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/README.md
@@ -88,6 +88,7 @@ python3 main.py \
| train_config | Transformer config: [CvT, CCT, TwinsSVT, LeViT, CaiT, CrossViT, PiT, Swin, T2TViT, CrossFormer] |
| cuda | Use cuda |
| no-cuda | Not use cuda |
+| seed | Random seed for reproducible experiments (default: 42) |
### __Self-Supervised Learning__
@@ -136,6 +137,7 @@ python3 main_ray.py \
| cuda | Use cuda |
| no-cuda | Not use cuda |
| num_samples | Number of samples for [ASHA scheduler](https://docs.ray.io/en/latest/tune/api_docs/schedulers.html) |
+| seed | Random seed for reproducible experiments (default: 42) |
@@ -155,6 +157,26 @@ sbatch < jobscript.sh
```
+# __Reproducibility__
+
+All training scripts support a `--seed` argument to ensure reproducible experiments. This sets seeds for Python's `random`, NumPy, PyTorch CPU, and CUDA:
+
+```bash
+# Supervised training with a custom seed
+python3 main.py --dataset_name Model_II --train_config CvT --seed 123
+
+# SSL pre-training with a custom seed
+python3 pretrain.py --seed 123
+
+# Finetuning with a custom seed
+python3 finetune.py --seed 123
+python3 finetune_byol.py --seed 123
+```
+
+The default seed is `42` if `--seed` is not specified.
+
+
+
# __Results__
## __Self-Supervised Learning__
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune.py
index cddae46..c022583 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune.py
@@ -1,3 +1,4 @@
+import os
import torch
import torch.nn as nn
import torch.optim as optim
@@ -6,13 +7,27 @@
from models.cnn_zoo import CustomResNet
from utils.losses.contrastive_loss import ContrastiveLossEuclidean
from utils.train import train_simplistic
-from utils.util import load_model_add_head
+from utils.util import load_model_add_head, seed_everything
from torchsummary import summary
+from argparse import ArgumentParser
+
+parser = ArgumentParser()
+parser.add_argument(
+ "--seed", type=int, default=42, help="random seed for reproducible experiments"
+)
+args = parser.parse_args()
+
+# Set reproducible seed
+seed_everything(seed=args.seed)
+
# Set device
device = "cuda" # torch.device("cuda" if torch.cuda.is_available() else "cpu")
learning_method = "contrastive_embedding"
-saved_model_path = "/home/kartik/git/deepLense_transformer_ssl/output/pretrained_contrastive_embedding.pth"
+
+# Define base directory relative to script location
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+saved_model_path = os.path.join(BASE_DIR, "output", "pretrained_contrastive_embedding.pth")
# Set hyperparameters
batch_size = 128
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune_byol.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune_byol.py
index c90729b..6afac41 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune_byol.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/finetune_byol.py
@@ -1,3 +1,4 @@
+import os
import torch
import torch.nn as nn
import torch.optim as optim
@@ -12,16 +13,37 @@
load_model_add_head,
get_second_last_layer,
get_last_layer_features,
+ seed_everything,
)
from torchsummary import summary
from models.byol import BYOLSingleChannel, FinetuneModelByol
import torchvision
from models.utils.finetune_model import FinetuneModel
+from argparse import ArgumentParser
+
+parser = ArgumentParser()
+parser.add_argument(
+ "--seed", type=int, default=42, help="random seed for reproducible experiments"
+)
+args = parser.parse_args()
+
+# Set reproducible seed
+seed_everything(seed=args.seed)
+
# Set device
device = "cuda" # torch.device("cuda" if torch.cuda.is_available() else "cpu")
learning_method = "contrastive_embedding"
-saved_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-23-13-30-24/checkpoint/Resnet_finetune_Model_II_2023-07-23-13-30-24.pt"
+
+# Define base directory relative to script location
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+saved_model_path = os.path.join(
+ BASE_DIR,
+ "logger",
+ "2023-07-23-13-30-24",
+ "checkpoint",
+ "Resnet_finetune_Model_II_2023-07-23-13-30-24.pt"
+)
# Set hyperparameters
batch_size = 512
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/inference.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/inference.py
index 5bda766..9fd9dcf 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/inference.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/inference.py
@@ -1,5 +1,5 @@
from __future__ import print_function
-
+import os
from turtle import down
from utils.dataset import DefaultDatasetSetupSSL
@@ -19,8 +19,19 @@ def main():
labels_map = {0: "axion", 1: "cdm", 2: "no_sub"}
image_size = 224
channels = 1
- log_dir = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-23-13-30-24"
- finetune_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-23-13-30-24/checkpoint/Resnet_finetune_Model_II.pt"
+
+ # Define base directory relative to script location
+ BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+
+ # Use relative paths instead of hardcoded absolute paths
+ log_dir = os.path.join(BASE_DIR, "logger", "2023-07-23-13-30-24")
+ finetune_model_path = os.path.join(
+ BASE_DIR,
+ "logger",
+ "2023-07-23-13-30-24",
+ "checkpoint",
+ "Resnet_finetune_Model_II.pt"
+ )
batch_size = 512
num_workers = 8
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/main.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/main.py
index 8293b99..4ea5e0c 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/main.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/main.py
@@ -94,6 +94,9 @@
parser.add_argument(
"--no-cuda", dest="cuda", action="store_false", help="not to use cuda"
)
+parser.add_argument(
+ "--seed", type=int, default=42, help="random seed for reproducible experiments"
+)
parser.set_defaults(cuda=True)
args = parser.parse_args()
@@ -155,7 +158,7 @@ def main():
train_len = len(trainset) - valid_len
trainset, testset = random_split(trainset, [train_len, valid_len])
- seed_everything(seed=42)
+ seed_everything(seed=args.seed)
device = get_device(use_cuda=use_cuda, cuda_idx=0)
# logging
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/pretrain.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/pretrain.py
index cd88f35..685dcce 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/pretrain.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/pretrain.py
@@ -6,6 +6,17 @@
from models.cnn_zoo import CustomResNet
from utils.losses.contrastive_loss import ContrastiveLossEuclidean, ContrastiveLossEmbedding
from utils.train import train_contrastive_with_labels, train_contrastive
+from utils.util import seed_everything
+from argparse import ArgumentParser
+
+parser = ArgumentParser()
+parser.add_argument(
+ "--seed", type=int, default=42, help="random seed for reproducible experiments"
+)
+args = parser.parse_args()
+
+# Set reproducible seed
+seed_everything(seed=args.seed)
# Set device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/finetune_byol.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/finetune_byol.py
index 5f54c71..1a7977e 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/finetune_byol.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/finetune_byol.py
@@ -1,3 +1,4 @@
+import os
import torch
import torch.nn as nn
import torch.optim as optim
@@ -106,8 +107,31 @@ def get_transfomer_finetuned(saved_model_path, device, num_classes) -> nn.Module
def main():
# Set device
device = "cuda" # torch.device("cuda" if torch.cuda.is_available() else "cpu")
- pretrained_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-25-06-27-13/checkpoint/CrossFormer_pretrained_Model_II_2023-07-25-06-27-13.pt"
- finetuned_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-25-06-27-13/checkpoint/CrossFormer_finetuned_Model_II_2023-07-25-06-27-13.pt"
+
+ # Define base directory relative to script location
+ BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+
+ # Use relative paths instead of hardcoded absolute paths
+ pretrained_model_path = os.path.join(
+ BASE_DIR,
+ "..",
+ "..",
+ "..",
+ "logger",
+ "2023-07-25-06-27-13",
+ "checkpoint",
+ "CrossFormer_pretrained_Model_II_2023-07-25-06-27-13.pt"
+ )
+ finetuned_model_path = os.path.join(
+ BASE_DIR,
+ "..",
+ "..",
+ "..",
+ "logger",
+ "2023-07-25-06-27-13",
+ "checkpoint",
+ "CrossFormer_finetuned_Model_II_2023-07-25-06-27-13.pt"
+ )
# Set hyperparameters
batch_size = 512
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/inference_byol.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/inference_byol.py
index 893a4f1..1f85ed2 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/inference_byol.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/inference_byol.py
@@ -1,4 +1,5 @@
from __future__ import print_function
+import os
from turtle import down
@@ -107,8 +108,22 @@ def main():
labels_map = {0: "axion", 1: "cdm", 2: "no_sub"}
image_size = 224
channels = 1
- log_dir = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-25-06-27-13"
- finetune_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-25-06-27-13/checkpoint/CrossFormer_finetuned_Model_II_2023-07-25-06-27-13.pt"
+
+ # Define base directory relative to script location
+ BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+
+ # Use relative paths instead of hardcoded absolute paths
+ log_dir = os.path.join(BASE_DIR, "..", "..", "..", "logger", "2023-07-25-06-27-13")
+ finetune_model_path = os.path.join(
+ BASE_DIR,
+ "..",
+ "..",
+ "..",
+ "logger",
+ "2023-07-25-06-27-13",
+ "checkpoint",
+ "CrossFormer_finetuned_Model_II_2023-07-25-06-27-13.pt"
+ )
batch_size = 512
num_workers = 8
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/load_pretrained_byol.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/load_pretrained_byol.py
index 570d123..77f96ce 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/load_pretrained_byol.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/self_supervised/examples/byol/load_pretrained_byol.py
@@ -1,3 +1,4 @@
+import os
import torch
import torch.nn as nn
import torch.optim as optim
@@ -21,8 +22,29 @@
# Set device
device = "cuda" # torch.device("cuda" if torch.cuda.is_available() else "cpu")
learning_method = "contrastive_embedding"
-pretrained_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-23-13-30-24/checkpoint/Resnet_finetune_Model_II_2023-07-23-13-30-24.pt"
-finetuned_model_path = "/home/kartik/git/DeepLense/Transformers_Classification_DeepLense_Kartik_Sachdev/logger/2023-07-23-13-30-24/checkpoint/Resnet_finetune_Model_II.pt"
+
+# Define base directory relative to script location
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+pretrained_model_path = os.path.join(
+ BASE_DIR,
+ "..",
+ "..",
+ "..",
+ "logger",
+ "2023-07-23-13-30-24",
+ "checkpoint",
+ "Resnet_finetune_Model_II_2023-07-23-13-30-24.pt"
+)
+finetuned_model_path = os.path.join(
+ BASE_DIR,
+ "..",
+ "..",
+ "..",
+ "logger",
+ "2023-07-23-13-30-24",
+ "checkpoint",
+ "Resnet_finetune_Model_II.pt"
+)
# Set hyperparameters
batch_size = 512
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/simsiam.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/simsiam.py
index 096bf59..80b7e06 100755
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/simsiam.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/simsiam.py
@@ -75,7 +75,7 @@ def main(args):
num_classes = len(classes)
make_directories([dataset_dir])
- seed_everything(seed=42)
+ seed_everything(seed=args.seed)
device = get_device(use_cuda=use_cuda, cuda_idx=0)
# logging
@@ -332,6 +332,9 @@ def main(args):
parser.add_argument(
"--no-cuda", dest="cuda", action="store_false", help="not to use cuda"
)
+ parser.add_argument(
+ "--seed", type=int, default=42, help="random seed for reproducible experiments"
+ )
parser.set_defaults(cuda=True)
args = parser.parse_args()
diff --git a/Transformers_Classification_DeepLense_Kartik_Sachdev/utils/train.py b/Transformers_Classification_DeepLense_Kartik_Sachdev/utils/train.py
index 79d1982..334b72a 100644
--- a/Transformers_Classification_DeepLense_Kartik_Sachdev/utils/train.py
+++ b/Transformers_Classification_DeepLense_Kartik_Sachdev/utils/train.py
@@ -159,7 +159,8 @@ def train_contrastive_with_labels(
for epoch in range(epochs):
epoch_loss = 0.0
- for batch_idx, (img1, img2, label) in enumerate(train_loader):
+ pbar = tqdm(train_loader, desc=f"Epoch [{epoch}/{epochs}]")
+ for batch_idx, (img1, img2, label) in enumerate(pbar):
img1, img2, label = img1.to(device), img2.to(device), label.to(device)
optimizer.zero_grad()
@@ -170,10 +171,7 @@ def train_contrastive_with_labels(
loss.backward()
optimizer.step()
- if batch_idx % 10 == 0:
- print(
- f"Epoch [{epoch}/{epochs}], Batch [{batch_idx}/{len(train_loader)}], Loss: {loss.item()}"
- )
+ pbar.set_postfix({"Loss": loss.item()})
epoch_loss = epoch_loss / len(train_loader)
@@ -198,7 +196,8 @@ def train_contrastive_pair(
# Training loop
for epoch in range(epochs):
epoch_loss = 0.0
- for batch_idx, (img1, img2, label) in enumerate(train_loader):
+ pbar = tqdm(train_loader, desc=f"Epoch [{epoch}/{epochs}]")
+ for batch_idx, (img1, img2, label) in enumerate(pbar):
if img1.shape[0] != batch_size:
continue
@@ -217,10 +216,7 @@ def train_contrastive_pair(
loss.backward()
optimizer.step()
- if batch_idx % 10 == 0:
- print(
- f"Epoch [{epoch}/{epochs}], Batch [{batch_idx}/{len(train_loader)}], Loss: {loss.item()}"
- )
+ pbar.set_postfix({"Loss": loss.item()})
epoch_loss = epoch_loss / len(train_loader)
@@ -244,7 +240,8 @@ def train_contrastive(
# Training loop
for epoch in range(epochs):
epoch_loss = 0.0
- for batch_idx, (img1, _, _) in enumerate(train_loader):
+ pbar = tqdm(train_loader, desc=f"Epoch [{epoch}/{epochs}]")
+ for batch_idx, (img1, _, _) in enumerate(pbar):
img1 = img1.to(device)
optimizer.zero_grad()
@@ -257,10 +254,7 @@ def train_contrastive(
loss.backward()
optimizer.step()
- if batch_idx % 10 == 0:
- print(
- f"Epoch [{epoch}/{epochs}], Batch [{batch_idx}/{len(train_loader)}], Loss: {loss.item()}"
- )
+ pbar.set_postfix({"Loss": loss.item()})
epoch_loss = epoch_loss / len(train_loader)
@@ -288,7 +282,8 @@ def train_simplistic(
for epoch in range(epochs):
epoch_loss = 0.0
- for batch_idx, (img1, _, label) in enumerate(train_loader):
+ pbar = tqdm(train_loader, desc=f"Epoch [{epoch}/{epochs}]")
+ for batch_idx, (img1, _, label) in enumerate(pbar):
img1 = img1.to(device)
label = label.to(device)
optimizer.zero_grad()
@@ -299,10 +294,7 @@ def train_simplistic(
optimizer.step()
epoch_loss += loss
- if batch_idx % 10 == 0:
- print(
- f"Epoch [{epoch}/{epochs}], Batch [{batch_idx}/{len(train_loader)}], Loss: {loss.item()}"
- )
+ pbar.set_postfix({"Loss": loss.item()})
epoch_loss = epoch_loss / len(train_loader)
@@ -354,7 +346,8 @@ def train_byol(
for epoch in range(epochs):
total_loss = 0
momentum_val = cosine_schedule(epoch, epochs, 0.996, 1)
- for batch_idx, (img1, img2, label) in enumerate(train_loader):
+ pbar = tqdm(train_loader, desc=f"Epoch [{epoch}/{epochs}]")
+ for batch_idx, (img1, img2, label) in enumerate(pbar):
x0, x1 = img1, img2
x0 = x0.to(device)
x1 = x1.to(device)
@@ -367,10 +360,7 @@ def train_byol(
optimizer.step()
optimizer.zero_grad()
- if batch_idx % 10 == 0:
- print(
- f"Epoch [{epoch}/{epochs}], Batch [{batch_idx}/{len(train_loader)}], Loss: {loss.item()}"
- )
+ pbar.set_postfix({"Loss": loss.item()})
if total_loss < best_loss:
best_loss = total_loss