SkinSense is a comprehensive web application that detects and classifies skin lesions using deep learning models trained on the HAM10000 dataset. The platform features an integrated AI chatbot powered by the Gemini API to provide interactive dermatology-related guidance, explainable results, and personalized health recommendations.
- Deep Learning-Based Skin Lesion Classification: Accurately detects and classifies 7 types of skin lesions using EfficientNet architecture trained on HAM10000 dataset
- Interactive AI Chatbot: Gemini API-powered chatbot for real-time dermatology queries and result explanation
- Real-Time Image Processing: FastAPI backend with optimized preprocessing pipeline for instant predictions
- Multi-Format Upload Support: Drag-and-drop, file upload, and webcam capture functionality
- Responsive UI: TypeScript + Next.js frontend with TailwindCSS for seamless cross-device experience
- Explainable Results: Detailed confidence scores and probability distributions for each lesion classification
- User-Friendly Interface: Intuitive design enabling non-technical users to interact with medical-grade AI
| Metric | Value |
|---|---|
| Accuracy | ~91% |
| Framework | EfficientNet + DenseNet(Ensemble model) |
| Dataset | HAM10000 (10,000+ dermatoscopic images) |
| Classes | 7 skin lesion types |
| Preprocessing | Image normalization, augmentation, and enhancement |
- Melanoma (MEL)
- Melanocytic Nevi (NV)
- Basal Cell Carcinoma (BCC)
- Actinic Keratoses (AK)
- Benign Keratosis (BKL)
- Dermatofibroma (DF)
- Vascular Lesions (VASC)
- Framework: Next.js with TypeScript
- Styling: TailwindCSS
- State Management: React Hooks
- UI Components: Custom React components with responsive design
- Framework: FastAPI (Python)
- Model Inference: PyTorch with EfficientNet
- API Integration: Gemini API for AI chatbot functionality
- Image Processing: OpenCV, PIL for preprocessing and enhancement
- Deployment: Hugging Face Spaces
- Dataset: HAM10000 (10,000 dermatoscopic images)
- Preprocessing: Normalization, augmentation, histogram equalization
- Train-Test Split: Optimized for balanced class distribution
- Feature Engineering: Transfer learning from ImageNet pre-trained weights
- Frontend: Hosted on Vercel
- Backend: Deployed on Hugging Face Spaces
- Live URL: https://skinsense-five.vercel.app/
| Component | Technology |
|---|---|
| Frontend | Next.js, TypeScript, React, TailwindCSS |
| Backend | FastAPI, Python, PyTorch, EfficientNet |
| ML/AI | EfficientNet, Transfer Learning, Gemini API |
| Database | N/A (Stateless API) |
| Deployment | Vercel (Frontend), Hugging Face Spaces (Backend) |
| Image Processing | OpenCV, PIL, NumPy |
| Dataset | HAM10000 |
- Transfer learning from ImageNet pre-trained weights
- EfficientNet architecture for optimal accuracy-efficiency tradeoff
- Data augmentation techniques (rotation, flipping, color jittering)
- Class balancing via resampling to handle dataset imbalance
- Loss Function: Weighted Cross-Entropy Loss
- Optimizer: Adam
- Learning Rate: 0.001 with learning rate scheduling
- Batch Size: 32
- Epochs: 50+ with early stopping
- Stratified train-test split (80-20)
- Confusion matrix analysis for per-class performance
- Per-class metrics: Precision, Recall, F1-Score
The integrated Gemini API chatbot provides:
- Disease Information: Detailed explanations of diagnosed skin conditions
- Symptom Guidance: Interactive Q&A about symptoms and causes
- Treatment Recommendations: Evidence-based treatment options
- Follow-up Questions: Contextual understanding of user queries
- Result Explanation: Simplified explanation of model predictions
- Upload Image: Use drag-and-drop or file picker to upload skin lesion image
- Get Prediction: Model processes image and returns classification with confidence scores
- Ask AI Chatbot: Interact with Gemini API-powered chatbot for detailed information
- Review Results: View detailed diagnosis, probabilities, and recommendations
- Download Report: Export results as PDF (optional feature)
- Client-side image processing where possible
- HTTPS-only communication
- No permanent storage of user images
- GDPR-compliant data handling
SkinSense is an AI-assisted tool for informational purposes only and should not be considered a substitute for professional medical diagnosis. Always consult with a qualified dermatologist for proper diagnosis and treatment. This application is designed to support healthcare professionals and provide preliminary assessment guidance.
This project is licensed under the MIT License - see the LICENSE file for details.
Souresh Mondal
- GitHub: @soureshmondal
- Live Demo: skinsense-five.vercel.app