mfpml.models

mfpml.models.kernels

class KernelCore

Bases: object

Base class for kernel

property _get_bounds: ndarray

Get the parameters’ bounds

Returns:

design bounds of the parameters

Return type:

np.ndarray

property _get_bounds_list: list

Get the parameters’ bounds with list

Returns:

design bounds of the parameters

Return type:

list

property _get_high_bound: list

Get the high bound of the parameters

Returns:

high bound of kernel’s parameters

Return type:

list

property _get_low_bound: list

Get the low bound of the parameters

Returns:

low bound of kernel’s parameters

Return type:

list

property _get_num_para: int

Return number of parameters of the kernel

Returns:

number of parameters

Return type:

int

property _get_param: ndarray

Get the parameters of the corelation

Returns:

parameters

Return type:

np.ndarray

property hyperparameters: list

Returns a list of all hyper_parameters specifications

class RBF(theta, parameters=['theta'], bounds=[-2, 3])

Bases: KernelCore

calculate the rbf kernel

Parameters:

• theta (np.ndarray, shape=((1,num_dims))) – initial guess of the parameters

• parameters (list, optional) – list the parameters to fit, by default [‘theta’]

• bounds (list, optional) – log bounds of the parameters, by default [-4, 3]

__init__(theta, parameters=['theta'], bounds=[-2, 3])

calculate the rbf kernel

Parameters:

• theta (np.ndarray, shape=((1,num_dims))) – initial guess of the parameters

• parameters (list, optional) – list the parameters to fit, by default [‘theta’]

• bounds (list, optional) – log bounds of the parameters, by default [-4, 3]

get_kernel_matrix(X, Y)

Return the kernel matrix k(x, y).

Parameters:

• X (np.ndarray) – array of the first samples shape=((n1, num_dims))

• Y (np.ndarray) – array of the second samples shape=((n2, num_dims))

Returns:

kernel matrix with shape ((n1, n2))

Return type:

np.ndarray

set_params(params)

Set the parameters of the kernel.

Return type:

None

kronDelta(X, Y)

Computes Kronecker delta for rows in x and y.

Parameters:

• x (np.ndarray, shape=((n, nfeatures)))

• y (np.ndarray, shape=((n, nfeatures)))

Returns:

Kronecker delta between row pairs of X and Xstar.

Return type:

np.ndarray

mfpml.models.gaussian_process

class GaussianProcessRegression(design_space, kernel=None, regr=<mfpml.models.basis_functions.Ordinary object>, optimizer=None, noise_prior=0.0, optimizer_restart=5)

Bases: object

Gaussian Process Regressor, for noisy data or noise-free data

Initialize the Gaussian Process Regressor

Parameters:

• design_space (np.ndarray) – design space of the problem with shape=(num_dim, 2), it is used to normalize the input samples

• kernel (Any, optional) – kernel function for spatial correlation of samples, by default None

• regr (Any, optional) – mean function, by default Ordinary()

• optimizer (Any, optional) – optimizer for the log likelihood function , by default None

• noise_prior (float, optional) – noise prior of the , by default 0.0 (noise-free data), if the noise is not None, the type II maximum likelihood should be used to optimize the for both the hyper-parameters and noise value

• optimizer_restart (int, optional) – restart the optimizer if needed, by default 5

__init__(design_space, kernel=None, regr=<mfpml.models.basis_functions.Ordinary object>, optimizer=None, noise_prior=0.0, optimizer_restart=5)

Initialize the Gaussian Process Regressor

Parameters:

• design_space (np.ndarray) – design space of the problem with shape=(num_dim, 2), it is used to normalize the input samples

• kernel (Any, optional) – kernel function for spatial correlation of samples, by default None

• regr (Any, optional) – mean function, by default Ordinary()

• optimizer (Any, optional) – optimizer for the log likelihood function , by default None

• noise_prior (float, optional) – noise prior of the , by default 0.0 (noise-free data), if the noise is not None, the type II maximum likelihood should be used to optimize the for both the hyper-parameters and noise value

• optimizer_restart (int, optional) – restart the optimizer if needed, by default 5

_hyper_paras_optimization()

Return type:

None

_logLikelihood(params)

Compute the concentrated ln-likelihood

Parameters:

params (np.ndarray) – parameters of the kernel

Returns:

log likelihood

Return type:

np.ndarray

property _num_samples: int

Return the number of samples

Returns:

num_samples – num samples

Return type:

int

_update_kernel_matrix()

update the kernel matrix with optimized parameters

Return type:

None

change_optimizer(optimizer)

Change the optimizer of the model

Parameters:

optimizer (Any) – optimizer for optimizing the log likelihood function

Return type:

None

static normalize_input(X, bounds)

Normalize samples to range [0, 1]

Parameters:

• X (np.ndarray) – samples to scale

• bounds (np.ndarray) – bounds with shape=((num_dim, 2))

Returns:

normalized samples

Return type:

np.ndarray

static normalize_output(Y)

Normalize output to range [0, 1]

Parameters:

sample_y (np.ndarray) – output to scale

Returns:

normalized output

Return type:

np.ndarray

predict(X, return_std=False)

Predict responses through the Kriging model

Parameters:

• X (np.ndarray) – new sample need to predict

• return_std (bool, optional) – whether return the standard deviation , by default False

Returns:

return the prediction with shape (#Xinput, 1)

Return type:

np.ndarray

train(X, Y)

training procedure of gpr

Parameters:

• X (np.ndarray) – sample array of sample

• Y (np.ndarray) – responses of the sample

Return type:

None

property y_mean: float

Return the mean of the response

Returns:

mean of the response

Return type:

float

property y_std: float

Return the standard deviation of the response

Returns:

standard deviation of the response

Return type:

float

mfpml.models.mf_gaussian_process

class _mfGaussianProcess(design_space)

Bases: object

_eval_corr(X, Xprime, fidelity=0)

Evaluate the correlation values based on current multi- fidelity model

Parameters:

• X (np.ndarray) – x

• Xprime (np.ndarray) – x’

• fidelity (str, optional) – str indicating fidelity level, by default ‘hf’

Returns:

correlation matrix

Return type:

np.ndarray

property _get_lfGP: Any

Get the low-fidelity model

Returns:

low-fidelity model instance

Return type:

Any

property _get_sample_hf: ndarray

Return samples of high-fidelity

Returns:

high-fidelity samples

Return type:

np.ndarray

property _get_sample_lf: ndarray

Return samples of high-fidelity

Returns:

high-fidelity samples

Return type:

np.ndarray

property _num_xh: int

Return the number of high-fidelity samples

Returns:

#high-fidelity samples

Return type:

int

property _num_xl: int

Return the number of low-fidelity samples

Returns:

#low-fidelity samples

Return type:

int

_train_hf(X, Y)

Train the high-fidelity model

Parameters:

• X (np.ndarray) – array of high-fidelity samples

• Y (np.ndarray) – array of high-fidelity responses

Return type:

None

_train_lf(X, Y)

Train the low-fidelity model

Parameters:

• X (np.ndarray) – low-fidelity samples

• Y (np.ndarray) – low-fidelity responses

Return type:

None

normalize_hf_output(outputs)

Normalize output to normal distribution

Parameters:

outputs (np.ndarray) – output to scale

Returns:

normalized output

Return type:

np.ndarray

normalize_input(inputs)

Normalize samples to range [0, 1]

Parameters:

inputs (np.ndarray) – samples to scale

Returns:

normalized samples

Return type:

np.ndarray

predict(X, return_std=False)

Predict the response of the model

Parameters:

• X (np.ndarray) – array of samples to be predicted

• return_std (bool, optional) – whether to return the standard deviation, by default False

Returns:

prediction of the model

Return type:

np.ndarray

predict_lf(X, return_std=False)

Predict low-fidelity responses

Parameters:

• X (np.ndarray) – array of low-fidelity to be predicted

• return_std (bool, optional) – whether to return the standard deviation, by default False

Returns:

prediction of low-fidelity

Return type:

np.ndarray

train(samples, responses)

training for multi-fidelity Gaussian process regression model for two-fidelity model, where the first fidelity is high-fidelity and second fidelity is low-fidelity.

Parameters:

• samples (List) – list with two elements, where each element is a np.ndarray of samples. The first element is high-fidelity samples and the second element is low-fidelity samples.

• responses (List) – list with two elements, where each element is a np.ndarray of responses. The first element is high-fidelity responses and the second element is low-fidelity responses.

Return type:

None

mfpml.models.hierarchical_kriging

class HierarchicalKriging(design_space, optimizer=None, optimizer_restart=5, kernel_bound=[-2.0, 3.0], noise_prior=None)

Bases: _mfGaussianProcess

_logLikelihood(params)

Compute the concentrated ln-likelihood

Parameters:

params (np.ndarray) – parameters of the kernel

Returns:

log likelihood

Return type:

np.ndarray

_optHyp()

Optimize the hyperparameters

_train_hf(X, Y)

Train the high-fidelity model

Parameters:

• X (np.ndarray) – array of high-fidelity samples

• Y (np.ndarray) – array of high-fidelity responses

Return type:

None

_update_parameters()

Update parameters of the model

Return type:

None

predict(X, return_std=False)

Predict high-fidelity responses

Parameters:

• X (np.ndarray) – array of high-fidelity to be predicted

• return_std (bool, optional) – whether to return std values, by default False

Returns:

prediction of high-fidelity

Return type:

np.ndarray

mfpml.models.co_kriging

class CoKriging(design_space, optimizer=None, optimizer_restart=0, kernel_bound=[-2.0, 3.0], rho_bound=[0.01, 100.0], noise_prior=None)

Bases: object

co-kriging model for handling multi-fidelity data

Parameters:

• design_space (np.ndarray) – array of shape=((num_dim,2)) where each row describes the bound for the variable on specfic dimension

• optimizer (Any, optional) – instance of the optimizer used to optimize the hyperparameters with the use style optimizer.run_optimizer(objective function, number of dimension, design space of variables), if not assigned, the ‘L-BFGS-B’ method in scipy is used

• kernel_bound (list, optional) – log bound of the kernel for difference Gaussian process model, by default [-4, 3]

• rho_bound (list, optional) – bound for scale factor, by default [1e-2, 1e2]

__init__(design_space, optimizer=None, optimizer_restart=0, kernel_bound=[-2.0, 3.0], rho_bound=[0.01, 100.0], noise_prior=None)

co-kriging model for handling multi-fidelity data

Parameters:

• design_space (np.ndarray) – array of shape=((num_dim,2)) where each row describes the bound for the variable on specfic dimension

• optimizer (Any, optional) – instance of the optimizer used to optimize the hyperparameters with the use style optimizer.run_optimizer(objective function, number of dimension, design space of variables), if not assigned, the ‘L-BFGS-B’ method in scipy is used

• kernel_bound (list, optional) – log bound of the kernel for difference Gaussian process model, by default [-4, 3]

• rho_bound (list, optional) – bound for scale factor, by default [1e-2, 1e2]

property _get_lf_model: Any

Get the low-fidelity model

Returns:

low-fidelity model instance

Return type:

Any

property _get_sample_hf: ndarray

Return samples of high-fidelity

Returns:

high-fidelity samples

Return type:

np.ndarray

property _get_sample_lf: ndarray

Return samples of high-fidelity

Returns:

high-fidelity samples

Return type:

np.ndarray

_logLikelihood(params)

Compute the concentrated ln-likelihood

Parameters:

params (np.ndarray) – parameters of the kernel

Returns:

log likelihood

Return type:

np.ndarray

property _num_xh: int

Return the number of high-fidelity samples

Returns:

#high-fidelity samples

Return type:

int

property _num_xl: int

Return the number of low-fidelity samples

Returns:

#low-fidelity samples

Return type:

int

_optHyp()

Optimize the hyperparameters

Return type:

None

_update_optimizer_hf(optimizer)

Change the optimizer for high-fidelity hyper parameters

Parameters:

optimizer (any) – instance of optimizer

Return type:

None

_update_optimizer_lf(optimizer)

Change the optimizer for low-fidelity hyper parameters

Parameters:

optimizer (Any) – instance of optimizer

Return type:

None

_update_parameters()

Update parameters of the model

Return type:

None

normalize_hf_output(outputs)

Normalize output to normal distribution

Parameters:

outputs (np.ndarray) – output to scale

Returns:

normalized output

Return type:

np.ndarray

normalize_input(inputs)

Normalize samples to range [0, 1]

Parameters:

inputs (np.ndarray) – samples to scale

Returns:

normalized samples

Return type:

np.ndarray

predict(X, return_std=False)

Predict high-fidelity responses

Parameters:

• X (np.ndarray) – array of high-fidelity to be predicted

• return_std (bool, optional) – whether to return std values, by default False

Returns:

prediction of high-fidelity

Return type:

np.ndarray

predict_lf(X, return_std=False)

Predict low-fidelity responses

Parameters:

• X (np.ndarray) – array of low-fidelity to be predicted

• return_std (bool, optional) – whether to return std values, by default False

Returns:

prediction of low-fidelity

Return type:

np.ndarray

train(samples, responses)

Train the hierarchical Kriging model

Parameters:

• samples (dict) – dict with two keys, ‘hf’ contains np.ndarray of high-fidelity sample points and ‘lf’ contains low-fidelity

• responses (dict) – dict with two keys, ‘hf’ contains high-fidelity responses and ‘lf’ contains low-fidelity ones

Return type:

None

class _GP(design_space, kernel=None, regr=<mfpml.models.basis_functions.Ordinary object>, optimizer=None, optimizer_restart=0, noise_prior=0.0)

Bases: object

single fidelity Gaussian process regression model, there is no normalization for the input and output. This is because we have to adapt the model to the Co-Kriging model.

_hyper_paras_optimization()

Return type:

None

_logLikelihood(params)

Compute the concentrated ln-likelihood

Parameters:

params (np.ndarray) – parameters of the kernel

Returns:

nll – negative log likelihood

Return type:

np.ndarray

property _num_samples: int

Return the number of samples

Returns:

num_samples – num samples

Return type:

int

_update_kernel_matrix()

Update the kernel matrix with optimized parameters.

Return type:

None

predict(x_predict, return_std=False)

Predict responses through the Kriging model

Parameters:

• x_predict (np.ndarray) – new sample need to predict

• return_std (bool, optional) – whether return the standard deviation , by default False

Returns:

return the prediction with shape (#Xinput, 1)

Return type:

np.ndarray

train(sample_x, sample_y)

training procedure of gpr models

Parameters:

• sample_x (np.ndarray) – sample array of sample

• sample_y (np.ndarray) – responses of the sample

Return type:

None

update_model(update_x, update_y)

update the model with new samples

Parameters:

• update_x (np.ndarray) – update sample array

• update_y (np.ndarray) – update responses

Return type:

None

update_optimizer(optimizer)

Change the optimizer for optimizing hyper parameters

Parameters:

optimizer (any) – instance of optimizer

Return type:

None

mfpml.models.scale_kriging

class ScaledKriging(design_space, lfGP=None, disc_model=None, optimizer=None, optimizer_restart=0, kernel_bound=[-4.0, 3.0], rho_optimize=False, rho_method='error', rho_bound=[0.01, 10.0], rho_optimizer=None, noise_prior=None)

Bases: _mfGaussianProcess

Multi-fidelity Kriging model with scaled function

Parameters:

• design_space (np.ndarray) – array of shape=((num_dim,2)) where each row describes the bound for the variable on specfic dimension

• lfGP (any) – instance of low-fidelity model, the model should have the method: train(x: np.ndarray, y: np.ndarray), predict(x: np.ndarray, return_std: bool)

• disc_model (any, optional) – instance of discrepancy model, the model should have the method: train(x: np.ndarray, y: np.ndarray), predict(x: np.ndarray, return_std: bool). Default Kriging

• optimizer (any, optional) – instance of the optimizer used to optimize the hyperparameters with the use style optimizer.run_optimizer(objective function, number of dimension, design space of variables), if not assigned, the ‘L-BFGS-B’ method in scipy is used

• kernel_bound (list, optional) – log bound of the kernel for discrepancy Kriging model, by default [-3, 2]

• rho_optimize (bool, optional) – whether to optimize the scale factor, if not the scale factor is 1, by default False

• rho_method (str, optional) – method to choose rho, can choose from [‘error’, ‘bumpiness’]

• rho_bound (list, optional) – bound for the factor rho if optimizing rho, by default [1e-1, 1e1]

• rho_optimizer (any, optional) – optimizer for the parameter rho, by default ‘L-BFGS-B’

__init__(design_space, lfGP=None, disc_model=None, optimizer=None, optimizer_restart=0, kernel_bound=[-4.0, 3.0], rho_optimize=False, rho_method='error', rho_bound=[0.01, 10.0], rho_optimizer=None, noise_prior=None)

Multi-fidelity Kriging model with scaled function

Parameters:

• design_space (np.ndarray) – array of shape=((num_dim,2)) where each row describes the bound for the variable on specfic dimension

• lfGP (any) – instance of low-fidelity model, the model should have the method: train(x: np.ndarray, y: np.ndarray), predict(x: np.ndarray, return_std: bool)

• disc_model (any, optional) – instance of discrepancy model, the model should have the method: train(x: np.ndarray, y: np.ndarray), predict(x: np.ndarray, return_std: bool). Default Kriging

• optimizer (any, optional) – instance of the optimizer used to optimize the hyperparameters with the use style optimizer.run_optimizer(objective function, number of dimension, design space of variables), if not assigned, the ‘L-BFGS-B’ method in scipy is used

• kernel_bound (list, optional) – log bound of the kernel for discrepancy Kriging model, by default [-3, 2]

• rho_optimize (bool, optional) – whether to optimize the scale factor, if not the scale factor is 1, by default False

• rho_method (str, optional) – method to choose rho, can choose from [‘error’, ‘bumpiness’]

• rho_bound (list, optional) – bound for the factor rho if optimizing rho, by default [1e-1, 1e1]

• rho_optimizer (any, optional) – optimizer for the parameter rho, by default ‘L-BFGS-B’

_eval_bumpiness(rho)

Evaluate the bumpiness

Parameters:

rho (np.ndarray) – array of rho

Returns:

measure of bumpiness

Return type:

np.ndarray

_eval_error(rho)

Evaluate the summation of squared error for high-fidelity samples

Parameters:

rho (np.ndarray) – array of rho

Returns:

sum of error

Return type:

np.ndarray

_getDisc()

Compute the discrepancy between low-fidelity prediction at high-fidelity samples and high-fidelity responses

Returns:

discrepancy

Return type:

np.ndarray

_rho_optimize()

Optimize the rho value

Return type:

None

_train_hf(sample_xh, sample_yh)

Train the discrepancy model in mf models

Return type:

None

Parameters:

sample_xhnp.ndarray

array of high-fidelity samples

sample_yhnp.ndarray

array of high-fidelity responses

predict(X, return_std=False)

Predict high-fidelity responses

Parameters:

• X (np.ndarray) – array of high-fidelity to be predicted

• return_std (bool, optional) – whether to return std values, by default False

Returns:

prediction of high-fidelity

Return type:

np.ndarray