Prediction of breast cancer based upon several features computed for each subject is a binary classification problem. Several discriminant methods exist for this problem, some of the commonly used methods are: Decision Trees, Random Forest, Neural Network, Support Vector Machine (SVM), and Logistic Regression (LR). Except for Logistic Regression, the other listed methods are predictive in nature; LR yields an explanatory model that can also be used for prediction, and for this reason it is commonly used in many disciplines including clinical research. In this article, we demonstrate the method of Bayesian LR to predict breast cancer using the Wisconsin Diagnosis Breast Cancer (WDBC) data set available at the UCI Machine Learning Repository.

Chang M, Dalpatadu RJ, Phanord D, Singh AK (2018) Breast Cancer Prediction Using Bayesian Logistic Regression. Ann Community Med Pract 4(3): 1039.

Cancer is a group of diseases characterized by the uncontrolled growth and spread of abnormal cells [1]. Globally, breast cancer is the most frequently diagnosed cancer and the leading cause of cancer death among females, accounting for 23% of the total cancer cases and 14% of the cancer deaths [2]. In US as well, breast cancer is the most frequent type of cancer (Figure 1). Bozorgi, Taghva, and Singh [3] used logistic regression for the prediction of breast cancer survivability using the SEER (Surveillance, Epidemiology, and End Results) database NCI (2016) of 338,596 breast cancer patients. Salama, Abdelhalim and Zeid [4], compared different classifiers (decision tree, Multi-Layer Perception, Naive Bayes, Sequential Minimal Optimization, and K-Nearest neighbor) on three different data sets of breast cancer and found a hybrid of the four methods to be the best classifier. Delen, Walker and Kadam [5], used artificial neural networks (ANN), decision trees (DT) and logistic regression (LR) to predict breast cancer survivability using a dataset of over 200,000 cases, using 10-fold cross-validation for performance comparison. The overall accuracies of the three methods turned out to be 93.6% (ANN), 91.2% (DT), and 89.2% (LR). Peretti and Amenta [6] used logistic regression to predict breast cancer tumor on a data set with 569 cases and obtained overall accuracy of 85%. Barco et al. [7], used LR on a data set of 1254 breast cancer patients to predict high tumour burden (HTB), as defined by the presence of three or more involved nodes with macro metastasis. Three predictors (tumour size, lymphovascular invasion and histological grade) were found to be statistically significant. LR and ANN are commonly used in many medical data classification tasks. Dreiseitl, and Ohno-Machado [8] summarize the differences and similarities of these models and compare them with a few other machine learning algorithms. Van Domelen et al. [9], estimated the LR model from a Bayesian approach in situations when the predictors are random variables with measurement errors. In a study to determine the main causes of complications after radical cystectomy (urinary bladder removal) [10], multivariate logistic regression was used to show that the main causes of complications were anemia before surgery, weight loss, intraoperative blood loss, intra-abdominal infection.

In the present article, we use the Wisconsin Diagnostic Breast Cancer Data Set of 569 observations on 32 variables [11] to predict breast cancer using the method of Bayesian LR. We provide a description of the Bayesian LR in the next section.

The Logistic Regression (LR) model is a special type of regression model fitted to a binary

(0-1) response variable Y, which relates the probability that Y equals 1 to a set of predictor variables:

                              (1)

Where X1, …, XP are K predictors, which can be continuous or discrete. The above model can be expressed in terms of log-odds as follows [12]

                  (2)

In the frequentist approach, given the random sample

Yj are n independent realizations of a Bernoulli experiment with probability of success P (Yj=1) given by (1); the model coefficients βj are unknown constants to be estimated from data. The likelihood function of the sample is

The LR model parameters are determined by the method of maximum likelihood estimation (MLE), which finds the β-coefficients that maximize the logarithm of the likelihood function

In the Bayesian approach, the model coefficients (β1 , β2 , …,K) are realizations of a K-variate random vector generated from the joint prior distribution; any prior knowledge about the β-coefficients can be incorporated in this joint prior distribution. All inferences drawn using the Bayesian approaches are conditional on data, and large sample theory of estimates is not needed. The conditional sample likelihood given by expression (3) is combined with the joint prior distribution of the parameters via the Bayes theorem [13] to obtain the joint posterior distribution of the model parameters, as shown below.

where g*( β | Y) is the joint posterior distribution,

and g(β) the joint posterior distribution of the parameters β .

If very little prior knowledge exists about the model parameters, we can use a vague prior. The marginal posterior distributions are numerically computed from the joint posterior distribution, and the means of these distributions are the parameter estimates. We can also obtain 95% confidence intervals of the parameters from these marginal posterior distributions. In Bayesian framework, these confidence intervals are called credible sets. In computing a credible set, it is desirable to obtain a credible set with shortest interval. The 95% highest posterior density (HPD) credible set contains only those points with largest posterior probability distribution [14]. A comparison of Bayesian and Frequentist approaches for estimation of predictive models is provided in [15-18].

Performance measures for prediction of a binary response

A large number of performance measures for multi-level classifiers exist in machine learning literature [19]. Commonly used performance measures of classifiers are accuracy, precision, recall and the geometric mean F1 of precision and recall [20,21]. To compute these measures, we first need to calculate the 2x2 confusion matrix shown in Table (1).

Here Ci,j = number of times true response of j get predicted as i (i, j = 0, 1).

The performance measures accuracy, precision, recall and F1 are calculated for each category 0 and 1 from the following formulas:

Bayesian prediction of breast cancer

The data set used here is the Wisconsin Diagnostic Breast Cancer (WDBC) Data Set, which is well-known in Machine Learning literature [9]. This data set has 569 observations on 32 variables including the binary response variable “Diagnosis” which takes values M (malignant) and B (benign). There are 10 features computed for each cell nucleus:

a) Radius (average distance from center to points on the perimeter)

b) Texture (standard deviation of gray-scale values)

c) Perimeter

d) Area

e) Smoothness (local variation in radius lengths)

f) Compactness (perimeter^2 / area - 1.0)

g) Concavity (severity of concave portions of the contour)

h) Concave points (number of concave portions of the contour)

i) Symmetry

j) Fractal dimension (“coastline approximation” - 1)

The mean, standard error, and “worst” or largest (mean of the three largest values) of these features were computed for each image, resulting in a total of 30 features for each of the 569 patients. Detailed descriptions of how these features are computed can be found in [22,23]. Since 20 of the 30 predictors were computed from data, high multicollinearity is expected in this data set. This can be seen in Figure 2, which is a plot of the correlations among the predictors in the WDBC data set.

There are three common approaches for fitting a LR model when high multicollinearity exists in the data. Aguilera, Escabias, Valderrama [24] used Principal Components Analysis (PCA) to obtain independent predictors (Principal Components) and then used LR; simulated data was used in this study. Asar [25] proposed shrinkage type estimators for fitting LR models, and used Monte Carlo simulation experiments to show that the shrinkage estimators perform better than the standard MLE estimator. Another simpler and more common approach is to drop predictors with high variance inflation factor (VIF) values and obtain a model in which largest VIF is 5 [26]. This is the approach taken in this article.

Table 1: Confusion matrix for a binary classifier.

Table 2: Bayesian LR model with all 30 predictors in the model fitted to the training set.

Table 3: Final Bayesian LR model fitted to the training set.

Table 4: Confusion Matrix for the Training set.

Table 5: Confusion Matrix for the Test set.

Table 6: Precision, recall and F1 measures for both training and test data sets.

All of the analyses presented here are performed using the statistical software environment R [27]. The WDBC data set of 569 cases was first split into a 75% training set of 427 observations and 25% test set of 142 observations. The LR Model for the training set, with all 30 predictors in the model had VIF falling in the range 78 to 123630, with none of the predictor’s significant (Table 1); this is due to extremely high multicollinearities among the 30 predictors. After eliminating predictors with VIF > 5 one by one, the final LR model was obtained (Table 2) with Texture, Area, Concavity, and Symmetry in the model. A comparison of Tables 2 and 3 shows how multicollinearities affect the estimation of LR model coefficients:

I. In the LR model with all predictors, all P-values are 1 i.e., none of the predictors are significant,

II. The estimated coefficients of the final predictors in the LR model with all predictors are all negative, when these coefficients should all be positive,

III. The standard errors (SE) of the final predictors in the LR model with all predictors are orders of magnitude higher than the corresponding estimates, and

IV. The final LR model, which has Texture, Area, Concavity, and Symmetry as the significant predictors, does not suffer from any of the above three issues; each coefficient is positive as it should be, and each predictor is highly significant.

The Figure 3 shows the posterior distributions and the 95% HPD credible sets for the coefficients of the predictors in the final LR model; the 95% HPD credible sets are:

βTexture: (0.16, 0.37), βArea: (0.008, 0.016), βConcavity: (16.65, 36.30), βArea: (3.22, 40.28).

Observe that all four 95% HPD credible sets fall to the right of 0.

Elimination of predictors with large VIF values leads to the final Bayesian LR model, given in Table 2.

The final LR model was next used to predict response “Diagnosis” for both the training and test data sets. The confusion matrices and overall accuracies for the training and test sets are shown in Tables 4 and 5.

The values of precision, recall and F1 measures for both training and test data are all quite high, as shown in Table 6.

The fitted Bayesian LR model has a total of four significant predictors: texture, area, concavity, and symmetry, with each predictor coefficient positive, as to be expected; the 95% HPD credible sets for these coefficients are shown in Figure 3; in each case, the entire 95% credible set falls to the right of 0, showing statistical significance of these predictors. Note that the Bayesian credible sets have a simple explanation – for example, we can say with 95% confidence that the random parameter βTexture falls inside the interval (0.16, 0.37) with the most likely value of 0.28.

We have used the Bayesian method for estimating the LR model for prediction of breast cancer; the Bayesian method comes with a much higher computational cost but has certain advantages over the classical method. The classical or frequentist approach to fitting an LR model is more common but has two major disadvantages: (i) it does not allow the user to formally incorporate any prior knowledge into parameter estimation [28], and (ii) it yields confidence intervals that are harder to interpret [29], with confidence going with the method or formula of computing the confidence interval, and not with the calculated confidence interval itself. Bayesian LR allows for formally using expert opinion and prior knowledge in the estimation of parameters, and typically yields better results than the classical method (Gordóvil-Merino et al., and Ogunsakin and Siaka).

1. American Cancer Society. Cancer Facts and Figures. 2018.

2. Jemal A, Bray F, Center MM, Ferlay J, Ward E, Forman D. Global Cancer Statistics. Ca Cancer J Clin. 2011; 61: 69-90.

3. Bozorgi M, Taghva K, Singh AK. Cancer Survivability with Logistic Regression. Computing Conference, 2017-18- July 20. UK. London: IEEE. 2018.

4. Salama GI, Abdelhalim MB, Zeid MA. Breast Cancer Diagnosis on Three Different Datasets Using Multi-Classifiers. Int J Comp Infor Technol. 2012; 1: 2277-2764.

5. Delen D, Walker G, Kadam A. Predicting breast cancer survivability: a comparison of three data mining methods. Artif Intell Med. 2005; 34: 113-127.

6. Peretti A, Amenta F. Breast Cancer Prediction by Logistic Regression with CUDA Parallel Programming Support. Breast Can Curr Res: 2016; 1: 111.

7. Barco I, Garcia Font M, Garcia Fernandez A, Giménez N, Fraile M, Lain JM, et al. A logistic regression model predicting high axillary tumour burden in early breast cancer patients. Clin Transl Oncol. 2007; 19: 1393-1399.

8. Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform. 2002; 35: 352-359.

9. Van Domelen DR, Mitchell EM, Perkins NJ, Schisterman EF, Manatunga AK. Logistic regression with a continuous exposure measured inpools and subject to errors. Stat Med. 2018; 1-15.

10. Atduev V, Gasrataliev V, Ledyaev D, Belsky V, Lyubarskaya Y, Mamedov H. Predictors of 30-Day Complications after Radical Cystectomy. Exp Tech Urol Nephrol. 2018; 1.

11. Dua D, Karra Taniskidou E. UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science.

12. Kleinbaum DG, Klein M. Logistic Regression - A Self-Learning Text. 3rd Edn. New York: Springer. 2007.

13. Kruschke J. Doing Bayesian Data Analysis. 2nd Edn. Netherlands: Elsevier. 2014.

14. Rohan D, Gewali L, Singh AK. Computing the Bayesian highest posterior density credible sets for the lognormal mean. Environmetrics. 2002; 13: 465-472.

15. Newcombe PJ, Reck BH, Sun J, Platek GT, Verzilli C, Kader AK, et al. A comparison of Bayesian and frequentist approaches to incorporating external information for the prediction of prostate cancer risk. Genet Epidemiol. 2012; 36: 71-83.

16. Ambrose PG, Hammel JP, Bhavnani SM, Rubino CM, Ellis-Grosse EJ, Drusano GL. Frequentist and Bayesian Pharmacometric-Based Approaches To Facilitate Critically Needed New Antibiotic Development: Overcoming Lies, Damn Lies, and Statistics. Antimicrob Agents Chemother. 2012; 3: 1466-1470.

17. Austin PC, Naylor CD, Tu JV. A comparison of a Bayesian vs. a frequentist method for profiling hospital performance. J Eval Clin Pract. 2001; 7: 35-45.

18. Grzenda W. The advantages of Bayesian methods over Classical methods in the context of credible intervals. Infor Sys Manag. 2015; 4: 53-63.

19. Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Infor Process Manag. 2009; 45: 427- 437.

20. James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning. New York: Springer. 2013.

21. Guillet F, Hamilton HJ. Quality measures in data mining. New York: Springer. 2007.

22. Street WN, Wolberg WH, Mangasarian OL. Nuclear Feature Extraction for Breast Tumor Diagnosis. Int Sym on Electron Imaging: Sci Technol. 1905: 1993; 861-870.

23. Wolberg WH, Street WN, Mangasarian OL. Machine learning techniques to diagnose breast cancer from: image-processed nuclear features of fine needle aspirates. Cancer Lett. 1994; 77: 163-171.

24. Ana M. Aguilera, Manuel Escabias, Mariano J. Valderrama. Using principal components for estimating logistic regression with high-dimensional multicollinear data. Comput. Stat Data Anal. 2006; 50: 1905-1924.

25. Yasin Asar. Some new methods to solve multicollinearity in logistic regression. Commun Stat Simul Comput. 2014; 46: 2576-2586.

26. Montgomery DC, Peck EA, Vining GG. Introduction to Linear regression Analysis. 3rd Edn. New York: Wiley. 2012.

27. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2017.

28. Amalia Gordóvil-Merino, Joan Guàrdia-Olmos, Maribel Peró-Cebollero, Emilia I. de la Fuente-Solanas. Classical and Bayesian estimation in the logistic regression model applied to diagnosis of child attention deficit hyperactivity disorder. Psychol Rep. 2010; 106: 519-533.

29. Grzenda W. The advantages of Bayesian methods over Classical methods in the context of credible intervals. Infor Sys Manag. 2015; 4: 53-63.