Dr. Geetha Malika Nallamotu

R and Python are popular software used for statistics and ML. In Python, you can find libraries like - scikit-learn for ML tasks TensorFlow, PyTorch for Deep learning R has its own ML packages like caret, randomForest, xgboost, mlr..etc which offer similar functionalities to scikit-learn. These libraries enable tasks such as prediction, grouping, and pattern recognition in data analysis. They are widely used in both research and real-world projects due to their flexibility and ability to handle advanced techniques.

Integrating machine learning with statistics blends two data perspectives. Machine learning predicts using complex algorithms, while statistics understands patterns with simpler methods. Balancing data abundance is crucial; too much can confuse models, leading to overfitting. Cleaning data before analysis is vital, but it can alter its meaning. Finding a balance between complex algorithms and simplicity is key, while being cautious of bias and model interpretability.

In healthcare, predictive analysis playing a major role in conditions like cancer treatment, gene therapy, and rare diseases. For instance, it can help predict the effectiveness of different cancer treatments based on patient characteristics and tumor genetics. In gene therapy, predictive models can forecast the likelihood of treatment success or identify potential adverse reactions based on genetic profiles. Similarly, for rare diseases, predictive analysis can aid in early detection, prognosis assessment, and personalized treatment planning by analyzing patient data and genetic markers. However, in rare diseases where the disease mechanisms are poorly understood or data is scarce, predictive analysis may be less effective.

According to me - Understand the task type: classification, regression, etc. - Search for open-source papers and GitHub repositories for reference. - Set measurable success criteria: increased accuracy, improved predictability, reduced errors, minimized false positives. - Collaborate as needed and conduct thorough research before presenting results. Note I m not someone with CS background but this is from my experience as data scientist there might be better ways but this is what I would do

My response will be helpful for those who wanted to become a data analyst/data scientist with a healthcare background. First to assess the skills we should know what are the ‘basic’ essential skills to start with. Statistical Analysis: Descriptive - Measures of central tendency & dispersion, Graphs Inferential - Hypothesis testing, CI, distributions, regression , ANOVA. Tools : Excel , SQL, Python , R ( follow the order of priority as a beginner) Types of Data : -categorical , continuous - nominal , ordinal , ratio , interval If you know all of this good luck you are already one step closer.

Handling missing data based on proportion: - <1%: Safe to remove or use basic imputation (mean/median). - 1-5%: Remove if randomness, otherwise, use simple imputation. - 5-20%: Be cautious about removal; try advanced imputation - 20-50%: Avoid removing; use ML-based imputation methods - >50%: Explore alternatives, consult domain experts to avoid bias