Test Run with Example Predictor

[BradKML]

Experimenting with dataset predicting assessment Z-score as last item to personality binary responses for the other columns

data_example.csv

Observation: This bug repeats for ElasticNetCV and LassoCV in Gram matrix passed in via 'precompute' parameter did not pass validation when a single element was checked - please check that it was computed properly. For element (283,284) we computed -22.94281768798828 but the user-supplied value was -22.94300651550293. astype(np.float32) is unknown to be useful or not prior to the edit.

!pip install lazypredict

from pandas import read_csv
from lazypredict.Supervised import LazyRegressor
from sklearn import datasets
from sklearn.model_selection import train_test_split
import numpy as np
df = read_csv('data_example.csv').dropna()
X, y = df.iloc[: , :-1].astype(np.float64), df.iloc[: , -1].astype(np.float64)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=13)
reg = LazyRegressor(verbose=0,ignore_warnings=False, custom_metric=None)
models,predictions = reg.fit(X_train, X_test, y_train, y_test)

Observation:

1. There is a weird mix of CV-based estimators and non-CV estimators, with some having no CV optimization.
2. GammaRegressor, BayesianRidge, TweedieRegressor all do not collapse redundant columns to Zero.
3. ElasticNetCV has ... live variables, LassoLarsIC has 292 live variables, LassoLarsCV has 282 live variables, but more variables means more accurate in terms of RMSE or R^2. A second run with random_state=15 shows 340, 295 and 289 variables, which is slightly different.
4. RFE and SFS can be used to reduce Ridge-like redundancy of variables.
5. This whole testing procedure does not do multiple runs to confirm if one model is better than the other in general if the random_state is adjusted. This also can cause different configuration of noise reduction. https://www.scikit-yb.org/en/latest/api/model_selection/rfecv.html

x = {i: (reg.provide_models(X_train, X_test, y_train, y_test)[i][1].coef_,
 reg.provide_models(X_train, X_test, y_train, y_test)[i][1].intercept_)
for i in ['GammaRegressor', 'BayesianRidge', 'TweedieRegressor',
          'LassoLarsIC', 'LassoLarsCV']}

print(sum([1 for i in x['ElasticNetCV'][0] if i != 0]),
      sum([1 for i in x['LassoLarsIC'][0] if i != 0]),
      sum([1 for i in x['LassoLarsCV'][0] if i != 0]))

[BradKML]

For testing and tuning purposes, tests with

from pandas import read_csv
from sklearn.linear_model import *
from yellowbrick.regressor import ManualAlphaSelection
from sklearn.model_selection import RepeatedKFold
from numpy import logspace

df = read_csv('https://files.catbox.moe/4nztka.csv')
df = df.drop(columns=df.columns[0], axis=1)
X, y = df.drop(columns=['AFQT']), df[['AFQT']]

alphas, cv = logspace(-5.0,  0.0, num=200, base=10.0), RepeatedKFold(n_splits=5, n_repeats=20)
# https://numpy.org/doc/stable/reference/generated/numpy.logspace.html
# https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RepeatedKFold.html

def image_maker(model, alphas=alphas, cv=cv):
    # https://www.scikit-yb.org/en/latest/api/regressor/alphas.html
    visualizer = ManualAlphaSelection(
        model,
        alphas=alphas, cv=cv,
        scoring="neg_mean_absolute_error"
        # https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics
        # https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics
    )
    visualizer.fit(X, y)
    visualizer.show(outpath=str(model.__name__)+".png")
    pass

# LassoLars has no alpha
for model in [
    TweedieRegressor(tol=0.001, max_iter=1000), 
    Lasso(tol=0.001, max_iter=1000), 
    ElasticNet(tol=0.001, max_iter=1000),
    GammaRegressor(tol=0.001, max_iter=1000),
    BayesianRidge(tol=0.001, n_iter=1000)
    LassoLars(eps=0.001, max_iter=1000)
    ]:
    image_maker(model, alphas=alphas, cv=cv)