![]() ![]() The problem is not the issue between one-vs-one or one-vs-all solutions, but that all implementations that I know, will use the same hyperparameters for all (OVO or OVA) classifiers. I have never seriously worked on multiclass problems, but my feeling is that SVM are not so good on them. Once we settle in the best features (according to LR) we used a RF (selecting for the best hyperparameters) to get the final classifier. And I used logistic regression to select among the many alternatives (because there is no hyperparameter search in LR). I was doing some "intense" feature engineering on a image classification problem (such as - combine or not two different descriptions of the image, and the dimensionality of the descriptions). So my personal experience is that although SVM may get you some extra bit of accuracy, it is almost always a better choice to use a RF.Īlso for larger problems, it may be impossible to use a batch SVM solver (I have never used a online SVM solver such as LASVM or others).įinally I only used logistic regression in one situation. I am no longer that emphatic in recommending RF (on speed grounds). ![]() The paper is available at It turns out that after the full analysis RF and SVM are almost equivalent in terms of expected error rate and SVM is fastest (to my surprise!!). Hopefully, in some weeks I can re-edit this answer and point to a technical report with the results. I have not yet finished analyzing the results or writing the paper so I cannot even point to a technical report with the results. I did not try standard logistic regression in these problems, but I tried an elastic net (L1 and L2 regularized LR) but it did not perform well (mean rank 8.3)~ My take is that SVM passed RF because the original dataset contained many multiclass problems - which I will discuss in the speculation part - should be a problem for SVM.īut RF is way way faster, and it was the 2nd best algorithm (mean rank 5.6) followed by gbm (5.8), nnets (7.2), and so on). My results is that the SVM was the "best algorithm" (mean rank 4.9). ![]() I also think I was more careful on the selection of hyperparameters then they were. I ran some 15 different algorithms on 100+ binary datasets (from Delgado's set of datasets). I believe that papers such as Delgado et all very important for the machine learning community, so I tried to replicate their results under some different conditions. The limits on generalizing that result are that the datasets are almost all tall and skinny (n>p), not very sparse - which I speculate should be more of a problem for RF, and not very big (both n and p).įinally, still on the published evidence, I recommend two sites that compare different implementations of random forests:īenchmarking Random Forest Implementationsīenchmarking Random Forest Classification If you consider that their selection of datasets is a "good sample" of real world problems, than SVM are definitively an algorithm that should be tried on new problems but one should try random forest first! They find that although RBF SVM are not the "best" algorithm (it is random forests if I remember correctly), it is among the top 3 (or 5). The only paper I know that help answer the question is Delgado et al 2014 - Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? - JMLR which runs hundreds of different algorithms and implementations on 121 datasets fro UCI. I will try to answer this question with a combination of published evidence, personal experience, and speculation. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |