Ensemble learning combines the predictions of several models so the group is more accurate and stable than any single member. Questions contrast bagging, which trains models in parallel on bootstrap samples, with boosting, which trains them sequentially so each corrects earlier errors. Random forest items cover feature subsampling and out-of-bag error, while AdaBoost questions ask how misclassified samples receive larger weights. Gradient boosting appears often, including learning rate, subsample and tree depth parameters, and differences among XGBoost, LightGBM and CatBoost. Stacking with a meta-learner, hard versus soft voting, and the theory linking boosting to PAC learning also feature.
Below are 30 practice questions from a pool of 210 Ensemble Learning MCQs, one of 17 topics in AI / Machine Learning & Data Analytics. Each shows the correct answer with an explanation; when you are ready, take a timed quiz to test recall under exam conditions.
Each question below shows the correct answer with a full explanation. Use these to build conceptual understanding before attempting a timed quiz.
Q1. What is ensemble learning?
Ensemble learning combines predictions from multiple models to produce more accurate and robust results than any single model.
Q2. Random Forest is an ensemble of:
Random Forest builds multiple decision trees and combines their predictions through majority voting (classification) or averaging (regression).
Q3. What is bagging in ensemble learning?
Bagging (Bootstrap Aggregating) trains multiple models on different random subsets of the training data drawn with replacement.
Q4. What is boosting?
Boosting trains models sequentially, with each new model focusing on correcting the errors made by previous models.
Q5. The final prediction in a classification ensemble is typically made by:
In classification ensembles, the final prediction is typically determined by majority voting among all the individual models.
Q6. Which of these is a popular boosting algorithm?
XGBoost (Extreme Gradient Boosting) is one of the most popular and effective boosting algorithms.
Q7. How does Random Forest handle overfitting?
Random Forest reduces overfitting by averaging predictions from many trees, each trained on different data subsets and feature subsets.
Q8. What is a base learner in ensemble methods?
A base learner is an individual model (often a weak learner) that is combined with others in an ensemble.
Q9. AdaBoost stands for:
AdaBoost (Adaptive Boosting) adjusts sample weights to focus subsequent classifiers on previously misclassified examples.
Q10. In Random Forest, what is feature randomness?
Feature randomness means each tree in the forest considers only a random subset of features when making splits, increasing diversity among trees.
Q11. What is the difference between bagging and boosting?
Bagging trains independent models in parallel on bootstrap samples, while boosting trains sequentially with each model correcting previous errors.
Q12. In AdaBoost, how are misclassified samples handled?
AdaBoost increases the weights of misclassified samples so subsequent learners focus more on the difficult examples.
Q13. What is stacking in ensemble learning?
Stacking trains a meta-learner on the predictions of base models, learning the optimal way to combine their outputs.
Q14. What is the out-of-bag (OOB) error in Random Forest?
OOB error uses samples not selected in each tree's bootstrap sample as a natural validation set, providing an unbiased error estimate without a separate test set.
Q15. Gradient Boosting minimizes the loss function by:
Gradient Boosting fits each new tree to the negative gradient (pseudo-residuals) of the loss function, performing gradient descent in function space.
Q16. What is the max_features parameter in Random Forest?
max_features controls how many features each tree considers when looking for the best split, affecting model diversity and performance.
Q17. Why does ensemble learning generally outperform single models?
Ensembles reduce variance (bagging), bias (boosting), or both (stacking) by combining diverse models that make different types of errors.
Q18. What is a weak learner?
A weak learner is a model that performs only slightly better than random chance, yet can be combined in ensembles to create strong learners.
Q19. In XGBoost, what is the purpose of the learning rate?
The learning rate (eta) shrinks the contribution of each tree, requiring more trees but providing better generalization and preventing overfitting.
Q20. What is voting in ensemble methods?
Voting combines predictions from multiple models. Hard voting uses majority class; soft voting averages predicted probabilities.
Q21. What is the bias-variance decomposition of ensemble methods?
Bagging reduces variance by averaging independent models. Boosting reduces bias by sequentially fitting residuals. Both can improve overall performance.
Q22. How does XGBoost handle regularization differently from traditional Gradient Boosting?
XGBoost explicitly adds L1 and L2 regularization terms to its objective function, controlling model complexity and reducing overfitting more effectively.
Q23. What is the difference between XGBoost and LightGBM's tree growing strategy?
XGBoost grows trees level-wise (breadth-first), while LightGBM grows leaf-wise (choosing the leaf with max loss reduction), making LightGBM faster on large datasets.
Q24. In CatBoost, how are categorical features handled?
CatBoost uses ordered target statistics with random permutations to encode categorical features, preventing target leakage that traditional target encoding can cause.
Q25. What is the effect of increasing the number of trees in a Random Forest?
Unlike boosting, adding more trees to a Random Forest improves performance up to a point and then plateaus without overfitting, due to the law of large numbers.
Q26. What is the role of the subsample parameter in Gradient Boosting?
Subsample controls the fraction of training data used for each tree, introducing randomness (stochastic gradient boosting) that reduces overfitting and can improve generalization.
Q27. How does the isolation mechanism work in Isolation Forest for ensemble anomaly detection?
Isolation Forest isolates anomalies faster (fewer splits) because outliers are few and have attribute values that differ significantly from normal instances.
Q28. What is Bayesian Model Averaging?
Bayesian Model Averaging weights each model's predictions by its posterior probability given the data, providing a principled approach to ensemble combination.
Q29. What is negative correlation learning in ensemble methods?
Negative correlation learning adds a penalty term during training that encourages base learners to make diverse, uncorrelated errors, improving ensemble performance.
Q30. How does DART (Dropouts meet Multiple Additive Regression Trees) improve boosting?
DART applies dropout to boosting by randomly dropping previously built trees during each iteration, preventing individual trees from dominating and reducing overfitting.
Practice MCQs with explanations
Practice MCQs with explanations
Practice MCQs with explanations
Practice MCQs with explanations
Practice MCQs with explanations
Practice MCQs with explanations
Take a timed quiz drawn from 210+ questions on this topic. No signup required — your progress saves in your browser.
Start Ensemble Learning Quiz