gbnet.gblinear

Classes

GBLinear

A linear gradient boosting module that uses gradient boosting for updates.

Functions

ridge_regression(X, y, lambd)

Solves ridge regression using Cholesky decomposition.

Module Contents

class gbnet.gblinear.GBLinear(input_dim, output_dim, bias=True, lr=0.5, min_hess=0.0, lambd=0.01)

Bases: gbnet.base.BaseGBModule

A linear gradient boosting module that uses gradient boosting for updates.

This module implements a linear layer that can be trained using gradient boosting. It maintains state between iterations and updates parameters using computed gradients and hessians.

Parameters:

• input_dim (int) – Input feature dimension

• output_dim (int) – Output prediction dimension

• bias (bool, optional) – Whether to include a bias term. Defaults to True.

• lr (float, optional) – Learning rate for parameter updates. Defaults to 0.5.

• min_hess (float, optional) – Minimum hessian threshold. Defaults to 0.0.

• lambd (float, optional) – L2 regularization parameter. Defaults to 0.01.

Variables:

• linear (nn.Linear) – The underlying linear layer

• FX (torch.Tensor) – Current predictions tensor

• input (numpy.ndarray) – Input data cache

• g (torch.Tensor) – Gradient cache

• h (torch.Tensor) – Hessian cache

input_dim

output_dim

min_hess = 0.0

bias = True

lr = 0.5

lambd = 0.01

linear

FX = None

input = None

g = None

h = None

_input_checking_setting(x)

Parameters:

x (Union[torch.Tensor, numpy.ndarray, pandas.DataFrame])

forward(x)

Parameters:

x (Union[torch.Tensor, numpy.ndarray, pandas.DataFrame])

gb_calc()

Calculate gradients and stores in the object

gb_step()

Uses stored gradients to update weights

gbnet.gblinear.ridge_regression(X, y, lambd)

Solves ridge regression using Cholesky decomposition.

Fastest method tested.

Parameters:

• X (np.ndarray) – Design matrix of shape (n_samples, n_features)

• y (np.ndarray) – Target values of shape (n_samples,)

• lambd (float) – Ridge regularization parameter

Returns:

Fitted coefficients of shape (n_features,)

Return type:

np.ndarray

The function solves the ridge regression problem: min_beta ||X beta - y||^2 + lambd ||beta||^2 using the normal equations and Cholesky decomposition for numerical stability.