how to analys the results, whatever values are coming in the end in the dictonary, how to evaluate them , what are they : are they probability value, are they kaapa score

[Shrishti120]

[Adesoji1]

@Shrishti120 please give more guideline, or codebase issues

[Shrishti120]

Ok, So I was trying to access the code which is in process.py which is import os,sys sys.path.insert(0,"..") from glob import glob import matplotlib.pyplot as plt import numpy as np import argparse import skimage, skimage.io import pprint import torch import torch.nn.functional as F import torchvision, torchvision.transforms import torchxrayvision as xrv parser = argparse.ArgumentParser() parser.add_argument('-f', type=str, default="", help='') parser.add_argument('img_path', type=str) parser.add_argument('-weights', type=str,default="densenet121-res224-all") parser.add_argument('-feats', default=False, help='', action='store_true') parser.add_argument('-cuda', default=False, help='', action='store_true') parser.add_argument('-resize', default=False, help='', action='store_true') cfg = parser.parse_args() img = skimage.io.imread(cfg.img_path) img = xrv.datasets.normalize(img, 255) # Check that images are 2D arrays if len(img.shape) > 2: img = img[:, :, 0] if len(img.shape) < 2: print("error, dimension lower than 2 for image") # Add color channel img = img[None, :, :] # the models will resize the input to the correct size so this is optional. if cfg.resize: transform = torchvision.transforms.Compose([xrv.datasets.XRayCenterCrop(), xrv.datasets.XRayResizer(224)]) else: transform = torchvision.transforms.Compose([xrv.datasets.XRayCenterCrop()]) img = transform(img) model = xrv.models.get_model(cfg.weights) output = {} with torch.no_grad(): img = torch.from_numpy(img).unsqueeze(0) if cfg.cuda: img = img.cuda() model = model.cuda() if cfg.feats: feats = model.features(img) feats = F.relu(feats, inplace=True) feats = F.adaptive_avg_pool2d(feats, (1, 1)) output["feats"] = list(feats.cpu().detach().numpy().reshape(-1)) preds = model(img).cpu() output["preds"] = dict(zip(xrv.datasets.default_pathologies,preds[0 ].detach().numpy())) if cfg.feats: print(output) else: pprint.pprint(output) and I got output as 'preds': {'Atelectasis': 0.52231383, 'Cardiomegaly': 0.14971305, 'Consolidation': 0.5107231, 'Edema': 0.018210562, 'Effusion': 0.55934864, 'Emphysema': 0.5045767, 'Enlarged Cardiomediastinum': 0.5371645, 'Fibrosis': 0.43839028, 'Fracture': 0.15676177, 'Hernia': 0.03712785, 'Infiltration': 0.34870642, 'Lung Lesion': 0.0014005175, 'Lung Opacity': 0.6327572, 'Mass': 0.7738221, 'Nodule': 0.5024437, 'Pleural_Thickening': 0.25367537, 'Pneumonia': 0.50162745, 'Pneumothorax': 0.5065439} Now I am not able to understand how to evaluate this result and what these values are, are these kappa score or probability values The image I took says that it has Pneumonia but the output gives significantly less percent of Pneumonia. I will be very grateful if you could help. Than you. Regard Shrishti Pandey

…

On Fri, 21 Jul, 2023, 4:45 pm Adesoji Alu, ***@***.***> wrote: @Shrishti120 <https://github.com/Shrishti120> please give more guideline, or codebase issues — Reply to this email directly, view it on GitHub <#138 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APAWHGAANYW2O2O2SBBQK7LXRJQFLANCNFSM6AAAAAA2SWKQFA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ieee8023]

are these kappa score or probability values

They are probability values

The image I took says that it has Pneumonia but the output gives

The model may be wrong or the image is labelled wrong. This is the challenge with AI models. You can process the image with this tool and use the explain function to generate a heatmap to explain the predictions: https://mlmed.org/tools/xray

You may also want to read "Section 2.5 Model Calibration" in the paper https://arxiv.org/abs/2111.00595 which discusses the calibration of the model and our attempt to make the 0.5 prediction meaningful. Also, this paper discusses the generalization issues of these types of models: https://arxiv.org/abs/2002.02497

[Shrishti120]

I am getting poor results I have a labelled dataset but that also model predicted really bad results, I tested it on 30 images out of 30 images I am able to get correct results only on 9-10 Is it possible that I can train it if I have the dataset?