Multilevel models (model ensembles)

Concept of model ensemble

Flame can build models using as object descriptors (X variables) the results of other (1 or n) models (Y variables). This feature can be used to develop multilevel models, when the top models are built with the results of the lower models or to integrate the results of multiple models for the same endpoint.

Features

This feature is not limited to two-levels. In theory it is possible to build a model A using the prediction of models B, C and D, where model C uses the results of model E and F.

The key idea is that the top model uses as "input" the output of 1/n models defined as an "ensemble". This output contains both the prediction and the confidence of the prediction (when available). This allow to integrate the results at the top model in a smart way which gives more importance (weight) to the more reliable results.

The system will run the lower models in multiple threads, whenever the number of models is higher than a certain cutoff value (MAX_MODELS_SINGLE_CPU = 2)

How building a model ensemble

For building a model ensemble, we must define the parameter input_type as model_ensemble and define a list of existing models by providing a list of names in parameter ensemble_names. By default the models are built with version 0, but an especific version can also be entered using parameter ensemble_versions

input_type : model_ensemble

ensemble_names : ['nDIPL2', 'nDIPL3']

ensemble_versions : [1,1]

Ensemble models must be built, just as regular models. This will not build the lower level models, only submit the compounds of the training series to the lower level models for prediction. The results will be then used to validate the model. This building operation does not check if the training series is coincident with the training series of the lower level models and therefore the results obtained can represent either an internal or an external validation

Combo models

The input variables (prediction produced by the lower level models) will be considered regular X variables by the top models and can be used to build any kind of model. However, they can also be combined using simple methods like:

1. mean (for quantitative variables)
2. median (for quantitative variables)
3. majority voting (for qualitative variables)

This is obtained defining in the modeling parameters the values mean, median and majority respectively

When the lower levels provide also confidence information, this will be used by these methods to produce weigthed means, weigthed medians, for quantitative variables. In the case of majority voting, the number of models producing the assignment to class 0 (negative) or class 1 (negative) will be taken into account for generating an aggregated assignment.

This feature is under development and testing and the values obtained must not be used for production