[MXML-11-04] Extreme Gradient Boosting (XGBoost) [4/9] - Classification: Training & Predict process

*** Dubbing: [ English ] [ 한국어 ] Until the last video, we looked at the regression, but from this video, we will look at the classification. Let's look at the training and the prediction process through a simple example. In the next video, we will derive the formulas used in this video and analyze the classification algorithm in detail. Let's take a look at the Exact Greedy Algorithm for classification. We'll look at the training process first and then the prediction process later. For regression, we use the initial prediction to calculate the residuals and learn the residuals to estimate the next prediction. And we keep repeating this process. However, for classification, as we saw in GBM, we use the initial prediction to compute the initial logodds, F0 and the residuals r. We then use these residuals to estimate the new logodds and then convert the logodds values into probabilities. And we keep repeating this process. In the case of regression, y-hat is directly estimated from the residuals, but in the case of classification, logodds is estimated first from the residuals, and then y-hat is estimated from the logodds. The reason for this process is that in classification, the y-hat is bounded to 0 and 1. For regression, y is unbounded, from the negative infinity to the positive infinity. And the residuals are also unbounded. However, in classification, y is bounded to 0 and 1, while the residuals are unbounded, from the negative infinity to the positive infinity. We cannot directly estimate the bounded y value from the unbounded residuals. Therefore, we first estimate the unbounded logodds F, and then use the logistic function to convert the logodds to the y value. For this reason, the classification is a bit more complex then the regression. #ExtremeGradientBoosting #XGBoost #XGBoostClassification #ExactGreedyAlgorithmForSplitFinding