Cross-Validation

How good are supervised data mining methods on your classification data set? Here's a workflow that scores various classification techniques on a data set from medicine. The central widget here is the one for testing and scoring, which is given the data and a set of learners, does cross-validation and scores predictive accuracy, and outputs the scores for further examination.

File

Test & Score

Logistic Regression

Random Forest Classification

SVM

Confusion Matrix

Data Table

Data Table (1)

Choose class-labeled data set. Say, "iris.tab" from documentation data sets.
It's always a good idea to check out the data first.
Select a cell in confusion matrix to obtain related data instances. Here we examine them in the spreadheet.
Use for additional analysis of cross-validation results.
Cross-validation takes place here. Double click to see the performance scores.
Several learners can be scored in cross-validation at the same time.
gAN9cQAoWBAAAABjb250ZXh0X3NldHRpbmdzcQFdcQJYAwAAAHVybHEDWAAAAABxBFgGAAAAc291 cmNlcQVLAFgLAAAAcmVjZW50X3VybHNxBl1xB1gLAAAAc2hlZXRfbmFtZXNxCH1xCVgTAAAAc2F2 ZWRXaWRnZXRHZW9tZXRyeXEKQy4B2dDLAAEAAAAAAhMAAAE8AAADqgAAAjUAAAITAAABUgAAA6oA AAI1AAAAAAAAcQtYDAAAAHJlY2VudF9wYXRoc3EMXXENY09yYW5nZS53aWRnZXRzLnV0aWxzLmZp bGVkaWFsb2dzClJlY2VudFBhdGgKcQ4pgXEPfXEQKFgHAAAAcmVscGF0aHERWAgAAABpcmlzLnRh YnESWAYAAABwcmVmaXhxE1gPAAAAc2FtcGxlLWRhdGFzZXRzcRRYBwAAAGFic3BhdGhxFVgwAAAA L1VzZXJzL2FuemUvZGV2L29yYW5nZTMvT3JhbmdlL2RhdGFzZXRzL2lyaXMudGFicRZYBQAAAHRp dGxlcRdoBHViYXUu
gAN9cQAoWAcAAABrX2ZvbGRzcQFLClgQAAAAY29udGV4dF9zZXR0aW5nc3ECXXEDY09yYW5nZS53 aWRnZXRzLnNldHRpbmdzCkNvbnRleHQKcQQpgXEFfXEGKFgEAAAAdGltZXEHR0HVwfPRXzJRWAYA AAB2YWx1ZXNxCH1xCShoAUsKWBIAAABzaHVmZmxlX3N0cmF0aWZpZWRxCohYCAAAAG5fcmVwZWF0 cQtLClgKAAAAcmVzYW1wbGluZ3EMSwBYCAAAAHNhbXBsZV9wcQ1LS1gPAAAAY2xhc3Nfc2VsZWN0 aW9ucQ5YFgAAAChBdmVyYWdlIG92ZXIgY2xhc3NlcylxD1gNAAAAY3Zfc3RyYXRpZmllZHEQiFgT AAAAc2F2ZWRXaWRnZXRHZW9tZXRyeXERY3NpcApfdW5waWNrbGVfdHlwZQpxElgMAAAAUHlRdDQu UXRDb3JlcRNYCgAAAFFCeXRlQXJyYXlxFEMuAdnQywABAAAAAAHLAAAA9QAABFYAAAJqAAABywAA AQsAAARWAAACagAAAAAAAHEVhXEWh3EXUnEYdVgHAAAAY2xhc3Nlc3EZXXEaKFgLAAAASXJpcy1z ZXRvc2FxG1gPAAAASXJpcy12ZXJzaWNvbG9ycRxYDgAAAElyaXMtdmlyZ2luaWNhcR1ldWJhaAqI aAtLCmgMSwBoDUtLaBCIaBFoEmgTWAoAAABRQnl0ZUFycmF5cR5DLgHZ0MsAAQAAAAABywAAAPUA AARWAAACagAAAcsAAAELAAAEVgAAAmoAAAAAAABxH4VxIIdxIVJxInUu
{'C_index': 61, 'penalty_type': 1, 'learner_name': 'Logistic Regression', 'savedWidgetGeometry': None}
{'use_max_features': False, 'max_features': 5, 'use_max_depth': False, 'max_depth': 3, 'use_random_state': False, 'savedWidgetGeometry': None, 'index_output': 0, 'use_max_leaf_nodes': True, 'random_state': 0, 'max_leaf_nodes': 5, 'learner_name': 'Random Forest Learner', 'n_estimators': 10}
{'degree': 3, 'tol': 0.001, 'probability': False, 'limit_iter': True, 'kernel_type': 0, 'savedWidgetGeometry': None, 'max_iter': 100, 'gamma': 0.0, 'svmtype': 0, 'C': 1.0, 'coef0': 0.0, 'nu': 0.5, 'learner_name': 'SVM Learner'}
gAN9cQAoWBIAAABhcHBlbmRfcHJlZGljdGlvbnNxAYhYFAAAAGFwcGVuZF9wcm9iYWJpbGl0aWVz cQKJWBEAAABzZWxlY3RlZF9xdWFudGl0eXEDSwBYCgAAAGF1dG9jb21taXRxBIhYEAAAAHNlbGVj dGVkX2xlYXJuZXJxBV1xBksAYVgTAAAAc2F2ZWRXaWRnZXRHZW9tZXRyeXEHY3NpcApfdW5waWNr bGVfdHlwZQpxCFgMAAAAUHlRdDQuUXRDb3JlcQlYCgAAAFFCeXRlQXJyYXlxCkMuAdnQywABAAAA AAOBAAABvgAABm4AAAO9AAADgQAAAdQAAAZuAAADvQAAAAAAAHELhXEMh3ENUnEOdS4=
gAN9cQAoWBAAAABjb250ZXh0X3NldHRpbmdzcQFdcQJjT3JhbmdlLndpZGdldHMuc2V0dGluZ3MK Q29udGV4dApxAymBcQR9cQUoWA4AAABvcmRlcmVkX2RvbWFpbnEGXXEHKFgMAAAAc2VwYWwgbGVu Z3RocQhLAoZxCVgLAAAAc2VwYWwgd2lkdGhxCksChnELWAwAAABwZXRhbCBsZW5ndGhxDEsChnEN WAsAAABwZXRhbCB3aWR0aHEOSwKGcQ9YBAAAAGlyaXNxEEsBhnERZVgGAAAAdmFsdWVzcRJ9cRMo WA0AAABzZWxlY3RlZF9yb3dzcRRdcRVYEwAAAHNhdmVkV2lkZ2V0R2VvbWV0cnlxFk5K/v///4Zx F1gNAAAAc2VsZWN0ZWRfY29sc3EYXXEZWA4AAABjb2xvcl9ieV9jbGFzc3EaiEr+////hnEbWBUA AABzaG93X2F0dHJpYnV0ZV9sYWJlbHNxHIhK/v///4ZxHVgOAAAAZGlzdF9jb2xvcl9SR0JxHihL 3EvcS9xL/3RxH0r+////hnEgWBIAAABzaG93X2Rpc3RyaWJ1dGlvbnNxIYlK/v///4ZxIlgLAAAA YXV0b19jb21taXRxI4hK/v///4ZxJFgLAAAAc2VsZWN0X3Jvd3NxJYhK/v///4ZxJnVYBQAAAG1l dGFzcSd9cShYCgAAAGF0dHJpYnV0ZXNxKX1xKihoEF1xKyhYCwAAAElyaXMtc2V0b3NhcSxYDwAA AElyaXMtdmVyc2ljb2xvcnEtWA4AAABJcmlzLXZpcmdpbmljYXEuZWgOSwJoCEsCaAxLAmgKSwJ1 WAQAAAB0aW1lcS9HQdXB89Fd5591YmFoFk5oGohoHIhoHmgfaCWIaCOIaCGJdS4=
gAN9cQAoWBAAAABjb250ZXh0X3NldHRpbmdzcQFdcQJjT3JhbmdlLndpZGdldHMuc2V0dGluZ3MK Q29udGV4dApxAymBcQR9cQUoWA4AAABvcmRlcmVkX2RvbWFpbnEGXXEHKFgMAAAAc2VwYWwgbGVu Z3RocQhLAoZxCVgLAAAAc2VwYWwgd2lkdGhxCksChnELWAwAAABwZXRhbCBsZW5ndGhxDEsChnEN WAsAAABwZXRhbCB3aWR0aHEOSwKGcQ9YBAAAAGlyaXNxEEsBhnERZVgGAAAAdmFsdWVzcRJ9cRMo WA4AAABjb2xvcl9zZXR0aW5nc3EUTkr+////hnEVWAsAAABzZWxlY3Rfcm93c3EWiEr+////hnEX WA0AAABzZWxlY3RlZF9yb3dzcRhdcRlYDQAAAHNlbGVjdGVkX2NvbHNxGl1xG1gOAAAAY29sb3Jf YnlfY2xhc3NxHIhK/v///4ZxHVgVAAAAc2hvd19hdHRyaWJ1dGVfbGFiZWxzcR6ISv7///+GcR9Y FQAAAHNlbGVjdGVkX3NjaGVtYV9pbmRleHEgSwBK/v///4ZxIVgOAAAAZGlzdF9jb2xvcl9SR0Jx IihL3EvcS9xL/3RxI0r+////hnEkWBIAAABzaG93X2Rpc3RyaWJ1dGlvbnNxJYlK/v///4ZxJlgL AAAAYXV0b19jb21taXRxJ4hK/v///4ZxKFgTAAAAc2F2ZWRXaWRnZXRHZW9tZXRyeXEpTkr+//// hnEqdVgFAAAAbWV0YXNxK31xLFgKAAAAYXR0cmlidXRlc3EtfXEuKGgQXXEvKFgLAAAASXJpcy1z ZXRvc2FxMFgPAAAASXJpcy12ZXJzaWNvbG9ycTFYDgAAAElyaXMtdmlyZ2luaWNhcTJlaA5LAmgI SwJoDEsCaApLAnVYBAAAAHRpbWVxM0dB1ZLULuJgA3ViYVgTAAAAc2F2ZWRXaWRnZXRHZW9tZXRy eXE0TlgOAAAAY29sb3JfYnlfY2xhc3NxNYhYFQAAAHNob3dfYXR0cmlidXRlX2xhYmVsc3E2iFgO AAAAZGlzdF9jb2xvcl9SR0JxN2gjWAsAAABzZWxlY3Rfcm93c3E4iFgLAAAAYXV0b19jb21taXRx OYhYEgAAAHNob3dfZGlzdHJpYnV0aW9uc3E6iXUu