hepstats.hypotests.calculators.basecalculator module#

class hepstats.hypotests.calculators.basecalculator.BaseCalculator(input, minimizer, **kwargs)[source]#

Bases: HypotestsObject

Base class for calculator.

Parameters:

  • input – loss or fit result

  • minimizer – minimizer to use to find the minimum of the loss function

Example with zfit:
>>> import zfit
>>> from zfit.core.loss import UnbinnedNLL
>>> from zfit.minimize import Minuit
>>>
>>> obs = zfit.Space('x', limits=(0.1, 2.0))
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = UnbinnedNLL(model=model, data=data)
>>>
>>> calc = BaseCalculator(input=loss, minimizer=Minuit())
obs_nll(pois)[source]#

Compute observed negative log-likelihood values for given parameters of interest.

Parameters:

pois (POIarray) – parameters of interest.

Return type:

ndarray

Returns:

Observed nll values.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> nll = calc.obs_nll(poi)
qobs(poinull, onesided=True, onesideddiscovery=True, qtilde=False)[source]#

Computes observed values of the \(\Delta\) log-likelihood test statistic.

Parameters:

  • poinull (POI) – parameters of interest for the null hypothesis.

  • qtilde (bool) – if True use the \(\tilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Returns:

Observed values of q.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> q = calc.qobs(poi)
pvalue(poinull, poialt=None, qtilde=False, onesided=True, onesideddiscovery=False)[source]#

Computes pvalues for the null and alternative hypothesis.

Parameters:

  • poinull (POI | POIarray) – parameters of interest for the null hypothesis.

  • poialt (POI | None) – parameters of interest for the alternative hypothesis.

  • qtilde (bool) – if True use the \(\widetilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Return type:

tuple[ndarray, ndarray]

Returns:

Tuple of arrays for pnull, palt

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> pvalues = calc.pavalue(poinull, poialt)
expected_pvalue(poinull, poialt, nsigma, CLs=False, qtilde=False, onesided=True, onesideddiscovery=False)[source]#

Computes the expected pvalues and error bands for different values of \(\sigma\) (0=expected/median)

Parameters:

  • poinull (POI | POIarray) – parameters of interest for the null hypothesis.

  • poialt (POI | POIarray) – parameters of interest for the alternative hypothesis.

  • nsigma (list[int]) – list of values of \(\sigma\) to compute the expected pvalue.

  • CLs (bool) – if True computes pvalues as \(p_{cls}=p_{null}/p_{alt}=p_{clsb}/p_{clb}\) else as \(p_{clsb} = p_{null}\).

  • qtilde (bool) – if True use the \(\widetilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

list[array]

Returns:

Array of expected pvalues for each \(\sigma\) value

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> nll = calc.expected_pvalue(poinull, poialt)
static check_pois(pois)[source]#

Checks if the parameter of interest is a hepstats.parameters.POIarray instance.

Parameters:

pois (POI | POIarray) – the parameter of interest to check.

Raises:

TypeError – if pois is not an instance of hepstats.parameters.POIarray.

static check_pois_compatibility(poi1, poi2)[source]#

Checks compatibility between two lists of hepstats.parameters.POIarray() instances.

Parameters:

  • poi1 (POI | POIarray) – the first parameter of interest.

  • poi2 (POI | POIarray) – the second parameter of interest.

Raises:

ValueError – if the two parameters of interests don’t have the same parameters, check by their names.

q(nll1, nll2, poi1, poi2, onesided=True, onesideddiscovery=False)[source]#

Compute values of the test statistic q defined as the difference between negative log-likelihood values \(q = nll1 - nll2\).

Parameters:

  • nll1 (array) – array of nll values #1, evaluated with poi1.

  • nll2 (array) – array of nll values #2, evaluated with poi2.

  • poi1 (POIarray) – POI’s #1.

  • poi2 (POIarray) – POI’s #2.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

ndarray

Returns:

Array of \(q\) values.

property bestfit#

Returns the best fit values of the model parameters.

property constraints#

Returns the constraints on the loss / likehood function.

property data#

Returns the data.

get_parameter(name)#

Returns the parameter in loss function with given input name.

Parameters:

name (str) – name of the parameter to return

property loss#

Returns the loss / likelihood function.

lossbuilder(model, data, weights=None, oldloss=None)#

Method to build a new loss function.

Parameters:

  • model (*) – The model or models to evaluate the data on

  • data (*) – Data to use

  • weights (*) – the data weights

  • oldloss (*) – Previous loss that has data, models, type

Example with zfit:
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = calc.lossbuilder(model, data)
Returns:

Loss function

property minimizer#

Returns the minimizer.

property model#

Returns the model.

property parameters#

Returns the list of free parameters in loss / likelihood function.

set_params_to_bestfit()#

Set the values of the parameters in the models to the best fit values

class hepstats.hypotests.calculators.basecalculator.BaseToysCalculator(input, minimizer, **kwargs)[source]#

Bases: BaseCalculator

Basis for toys calculator class.

Parameters:

  • input – loss or fit result.

  • minimizer – minimizer to use to find the minimum of the loss function.

  • sampler – function used to create sampler with models, number of events and floating parameters in the sample.

  • sample – function used to get samples from the sampler.

property bestfit#

Returns the best fit values of the model parameters.

static check_pois(pois)#

Checks if the parameter of interest is a hepstats.parameters.POIarray instance.

Parameters:

pois (POI | POIarray) – the parameter of interest to check.

Raises:

TypeError – if pois is not an instance of hepstats.parameters.POIarray.

static check_pois_compatibility(poi1, poi2)#

Checks compatibility between two lists of hepstats.parameters.POIarray() instances.

Parameters:

  • poi1 (POI | POIarray) – the first parameter of interest.

  • poi2 (POI | POIarray) – the second parameter of interest.

Raises:

ValueError – if the two parameters of interests don’t have the same parameters, check by their names.

property constraints#

Returns the constraints on the loss / likehood function.

property data#

Returns the data.

expected_pvalue(poinull, poialt, nsigma, CLs=False, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes the expected pvalues and error bands for different values of \(\sigma\) (0=expected/median)

Parameters:

  • poinull (POI | POIarray) – parameters of interest for the null hypothesis.

  • poialt (POI | POIarray) – parameters of interest for the alternative hypothesis.

  • nsigma (list[int]) – list of values of \(\sigma\) to compute the expected pvalue.

  • CLs (bool) – if True computes pvalues as \(p_{cls}=p_{null}/p_{alt}=p_{clsb}/p_{clb}\) else as \(p_{clsb} = p_{null}\).

  • qtilde (bool) – if True use the \(\widetilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

list[array]

Returns:

Array of expected pvalues for each \(\sigma\) value

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> nll = calc.expected_pvalue(poinull, poialt)
get_parameter(name)#

Returns the parameter in loss function with given input name.

Parameters:

name (str) – name of the parameter to return

property loss#

Returns the loss / likelihood function.

lossbuilder(model, data, weights=None, oldloss=None)#

Method to build a new loss function.

Parameters:

  • model (*) – The model or models to evaluate the data on

  • data (*) – Data to use

  • weights (*) – the data weights

  • oldloss (*) – Previous loss that has data, models, type

Example with zfit:
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = calc.lossbuilder(model, data)
Returns:

Loss function

property minimizer#

Returns the minimizer.

property model#

Returns the model.

obs_nll(pois)#

Compute observed negative log-likelihood values for given parameters of interest.

Parameters:

pois (POIarray) – parameters of interest.

Return type:

ndarray

Returns:

Observed nll values.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> nll = calc.obs_nll(poi)
property parameters#

Returns the list of free parameters in loss / likelihood function.

pvalue(poinull, poialt=None, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes pvalues for the null and alternative hypothesis.

Parameters:

  • poinull (POI | POIarray) – parameters of interest for the null hypothesis.

  • poialt (POI | None) – parameters of interest for the alternative hypothesis.

  • qtilde (bool) – if True use the \(\widetilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Return type:

tuple[ndarray, ndarray]

Returns:

Tuple of arrays for pnull, palt

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> pvalues = calc.pavalue(poinull, poialt)
q(nll1, nll2, poi1, poi2, onesided=True, onesideddiscovery=False)#

Compute values of the test statistic q defined as the difference between negative log-likelihood values \(q = nll1 - nll2\).

Parameters:

  • nll1 (array) – array of nll values #1, evaluated with poi1.

  • nll2 (array) – array of nll values #2, evaluated with poi2.

  • poi1 (POIarray) – POI’s #1.

  • poi2 (POIarray) – POI’s #2.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

ndarray

Returns:

Array of \(q\) values.

qobs(poinull, onesided=True, onesideddiscovery=True, qtilde=False)#

Computes observed values of the \(\Delta\) log-likelihood test statistic.

Parameters:

  • poinull (POI) – parameters of interest for the null hypothesis.

  • qtilde (bool) – if True use the \(\tilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Returns:

Observed values of q.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> q = calc.qobs(poi)
set_params_to_bestfit()#

Set the values of the parameters in the models to the best fit values

class hepstats.hypotests.calculators.basecalculator.ToysCalculator(input, minimizer, ntoysnull=100, ntoysalt=100, sampler=<function base_sampler>, sample=<function base_sample>)[source]#

Bases: BaseToysCalculator, ToysManager

Class for calculators using toys.

Toys calculator class.

Parameters:

  • input – loss or fit result.

  • minimizer – minimizer to use to find the minimum of the loss function.

  • ntoysnull (int) – minimum number of toys to generate for the null hypothesis.

  • ntoysalt (int) – minimum number of toys to generate for the alternative hypothesis.

  • sampler (Callable) – function used to create sampler with models, number of events and floating** parameters. in the sample Default is hepstats.utils.fit.sampling.base_sampler().

  • sample (Callable) – function used to get samples from the sampler. Default is hepstats.utils.fit..sampling.base_sample().

classmethod from_yaml(filename, input, minimizer, sampler=<function base_sampler>, sample=<function base_sample>, **kwargs)[source]#

ToysCalculator constructor with the toys loaded from a yaml file.

Parameters:

  • filename (str) – the yaml file name.

  • input – loss or fit result.

  • minimizer – minimizer to use to find the minimum of the loss function.

  • sampler (Callable) – function used to create sampler with models, number of events and floating parameters in the sample Default is hepstats.utils.fit.sampling.base_sampler().

  • sample (Callable) – function used to get samples from the sampler. Default is hepstats.fitutils.sampling.base_sample().

property ntoysnull: int#

Returns the number of toys generated for the null hypothesis.

property ntoysalt: int#

Returns the number of toys generated for the alternative hypothesis.

add_toyresult(toy)#

Add ToyResult to the manager.

Parameters:

toy (ToyResult) – the toy result to add

property bestfit#

Returns the best fit values of the model parameters.

static check_pois(pois)#

Checks if the parameter of interest is a hepstats.parameters.POIarray instance.

Parameters:

pois (POI | POIarray) – the parameter of interest to check.

Raises:

TypeError – if pois is not an instance of hepstats.parameters.POIarray.

static check_pois_compatibility(poi1, poi2)#

Checks compatibility between two lists of hepstats.parameters.POIarray() instances.

Parameters:

  • poi1 (POI | POIarray) – the first parameter of interest.

  • poi2 (POI | POIarray) – the second parameter of interest.

Raises:

ValueError – if the two parameters of interests don’t have the same parameters, check by their names.

property constraints#

Returns the constraints on the loss / likehood function.

property data#

Returns the data.

expected_pvalue(poinull, poialt, nsigma, CLs=False, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes the expected pvalues and error bands for different values of \(\sigma\) (0=expected/median)

Parameters:

  • poinull (POI | POIarray) – parameters of interest for the null hypothesis.

  • poialt (POI | POIarray) – parameters of interest for the alternative hypothesis.

  • nsigma (list[int]) – list of values of \(\sigma\) to compute the expected pvalue.

  • CLs (bool) – if True computes pvalues as \(p_{cls}=p_{null}/p_{alt}=p_{clsb}/p_{clb}\) else as \(p_{clsb} = p_{null}\).

  • qtilde (bool) – if True use the \(\widetilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

list[array]

Returns:

Array of expected pvalues for each \(\sigma\) value

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> nll = calc.expected_pvalue(poinull, poialt)
generate_and_fit_toys(ntoys, poigen, poieval)#

Generate and fit toys for at a given POI (poigen). The toys are then fitted, and the likelihood is profiled at the values of poigen and poieval.

Parameters:

  • ntoys (int) – number of toys to generate

  • poigen (POI) – POI used to generate the toys

  • poieval (POIarray) – POI values to evaluate the loss function

get_parameter(name)#

Returns the parameter in loss function with given input name.

Parameters:

name (str) – name of the parameter to return

get_toyresult(poigen, poieval)#

Getter function.

Parameters:

  • poigen (POI) – POI used to generate the toys

  • poieval (POIarray) – POI values to evaluate the loss function

Return type:

ToyResult

get_toys_null(poigen, poieval=None, qtilde=False)[source]#

Return the generated toys for the null hypothesis.

Parameters:

  • poigen (POI | POIarray) – POI used to generate the toys.

  • poieval (POI | POIarray | None) – POI values to evaluate the loss function.

  • qtilde (bool) – if True use the :math:` ilde{q}` test statistics else (default) use the \(q\) test statistic.

Return type:

dict[POI, ToyResult]

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POIarray(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> for p in poinull:
...     calc.get_toys_alt(p, poieval=poialt)
keys()#

Returns keys of the ToysManager instance defined as key = (toy.poigen, toy.poieval) for a given ToyResult instance toy.

property loss#

Returns the loss / likelihood function.

lossbuilder(model, data, weights=None, oldloss=None)#

Method to build a new loss function.

Parameters:

  • model (*) – The model or models to evaluate the data on

  • data (*) – Data to use

  • weights (*) – the data weights

  • oldloss (*) – Previous loss that has data, models, type

Example with zfit:
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = calc.lossbuilder(model, data)
Returns:

Loss function

property minimizer#

Returns the minimizer.

property model#

Returns the model.

ntoys(poigen, poieval)#

Return the number of toys generated from given value of a POI, and scanned/evaluated for given values_equal of the same POI.

Parameters:

  • poigen (POI) – POI used to generate the toys

  • poieval (POIarray) – POI values to evaluate the loss function

Return type:

int

obs_nll(pois)#

Compute observed negative log-likelihood values for given parameters of interest.

Parameters:

pois (POIarray) – parameters of interest.

Return type:

ndarray

Returns:

Observed nll values.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> nll = calc.obs_nll(poi)
property parameters#

Returns the list of free parameters in loss / likelihood function.

pvalue(poinull, poialt=None, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes pvalues for the null and alternative hypothesis.

Parameters:

  • poinull (POI | POIarray) – parameters of interest for the null hypothesis.

  • poialt (POI | None) – parameters of interest for the alternative hypothesis.

  • qtilde (bool) – if True use the \(\widetilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Return type:

tuple[ndarray, ndarray]

Returns:

Tuple of arrays for pnull, palt

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> pvalues = calc.pavalue(poinull, poialt)
q(nll1, nll2, poi1, poi2, onesided=True, onesideddiscovery=False)#

Compute values of the test statistic q defined as the difference between negative log-likelihood values \(q = nll1 - nll2\).

Parameters:

  • nll1 (array) – array of nll values #1, evaluated with poi1.

  • nll2 (array) – array of nll values #2, evaluated with poi2.

  • poi1 (POIarray) – POI’s #1.

  • poi2 (POIarray) – POI’s #2.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

ndarray

Returns:

Array of \(q\) values.

qobs(poinull, onesided=True, onesideddiscovery=True, qtilde=False)#

Computes observed values of the \(\Delta\) log-likelihood test statistic.

Parameters:

  • poinull (POI) – parameters of interest for the null hypothesis.

  • qtilde (bool) – if True use the \(\tilde{q}\) test statistics else (default) use the \(q\) test statistic.

  • onesided (bool) – if True (default) computes onesided pvalues.

  • onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Returns:

Observed values of q.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> q = calc.qobs(poi)
sample(sampler, ntoys, poi, constraints=None)#

Generator function of samples from the sampler for a given value of a parameter of interest. Returns a dictionnary of samples constraints in any.

Parameters:

  • sampler (list) – generator of samples

  • ntoys (int) – number of samples to generate

  • poi (POI) – in the sampler

  • constraints (list, optional) – list of constraints to sample

Example with zfit:
>>> mean = zfit.Parameter("mean")
>>> sampler = calc.sampler()
>>> sample = calc.sample(sampler, 1000, POI(mean, 1.2))
Returns:

dictionnary of sampled values of the constraints at each iteration

sampler()#

Create sampler with models.

>>> sampler = calc.sampler()
set_params_to_bestfit()#

Set the values of the parameters in the models to the best fit values

to_yaml(filename)#

Save the toys into a yaml file under the key toys.

Parameters:

filename (str) – the yaml file name.

toyresults_to_dict()#

Returns a list of all the toy results converted into dictionaries.

Return type:

list[dict]

toys_loss(parameter_name)#

Construct a loss function constructed with a sampler for a given floating parameter

Parameters:

parameter_name (str) – name floating parameter in the sampler

Returns:

Loss function

Example with zfit:
>>> loss = calc.toys_loss(zfit.Parameter("mean"))
toysresults_from_yaml(filename)#

Extract toy results from a yaml file.

Parameters:

filename (str) – the yaml file name.

Return type:

list[ToyResult]

values()#

Returns values of ToysManager instance that are ToyResult instances.

get_toys_alt(poigen, poieval=None, qtilde=False)[source]#

Return the generated toys for the alternative hypothesis.

Parameters:

  • poigen (POI | POIarray) – POI used to generate the toys.

  • poieval (POI | POIarray | None) – POI values to evaluate the loss function.

  • qtilde (bool) – if True use the :math:` ilde{q}` test statistics else (default) use the \(q\) test statistic.

Return type:

dict[POI, ToyResult]

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POIarray(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> calc.get_toys_alt(poialt, poieval=poinull)
On this page
Edit on GitHub
Show Source