Convolutional Neural Networks (CNNs) have become indispensable tools in skin cancer classification, aiding clinical experts to achieve earlier and more accurate diagnoses, improving treatment outcomes, and driving advancements in medical research. Despite their pivotal role, the most popular CNN architecture families exhibit a critical issue related to the distribution and quantity of available data, potentially compromising their performance and generalization abilities. This challenge is commonly overlooked in most skin lesion classification papers, which mainly rely on weighted classification techniques. Directly using appropriately dataset balancing or Transfer Learning (TL) methods, as suggested in recent studies, has the potential to deliver more satisfactory results, providing a more effective approach to addressing this issue. In the effort to tackle this problem, we provide a comprehensive quantitative evaluation aimed at identifying the most critical new emerging computational aspects and the related effective techniques. Specifically, we propose twelve Computational Models (CMs) based on four prominent CNN models with increasing structural complexity. We assess their effectiveness in both pretrained and unpretrained versions, incorporating TL strategies as well. Our experiments focus on the ISIC 2018 image dataset, a benchmark widely recognized for its extensive application in skin cancer research yet challenged by significant class imbalance issues. To mitigate this, we also randomly extracted a balanced image subset from ISIC 2018 for evaluation purposes. The experimental results, regarding four different scenarios, provide valuable insights into the design and utilization of CNNs for skin lesion classification, laying the groundwork for further investigations.
