[MXML-11-06] Extreme Gradient Boosting (XGBoost) [6/9] - Implement the classification from scratch

*** Dubbing: [ English ] [ 한국어 ] This is part 6 of a series on XGBoost. In this video, we will implement the XGBoost Classification model in code from scratch. In the last video, we analyzed the Exact Greedy Algorithm for Split Finding presented in the section 2.1 of the paper, and applied this to XGBoost Classification. In this video, we'll implement this algorithm in code from scratch. This code may be a bit difficult for machine learning beginners. Beginners are recommended to skip this video and move on to the next video. Since GBM Classification uses Decision Trees constructed from the CART algorithm, we were able to implement GBM from scratch using the DecisionTreeRegressor. In the CART algorithm, the optimal split point was found using the gain obtained from the Gini index or entropy. However, rather than using the typical CART algorithm, XGBoost classification uses regularization and pruning, uses scores and gains to split nodes, and computes output values to predict the next target class. The DecisionTreeRegressor cannot be used for XGBoost classification. Therefore, if you want to implement XGBoost from scratch, you first need to implement a base learner That's why this code may seem a bit difficult. To understand this code, you must first understand the Exact Greedy Algorithm for Split Finding from the previous video, as well as the code for creating a binary tree using recursion covered in the Decision Trees video, [MXML-2-07]. This code is an upgraded version of the code covered in the video. Here we will only discuss the changes in XGBoost. #ExtremeGradientBoosting #XGBoost #GreedyAlgorithmforSplitFinding #XGBoostFromScratch