^jw ddlZddlZddlmZmZddlmZddl m Z m Z m Z m Z mZmZmZddlmZmZmZmZddlmZddlmZmZmZgdZd Zd Zd Zd Z e d geedde dddddddZ!e ddgddgdeedde dddddddZ"e d dgdeedd ed!dd"d*dd#d$Z#e d d%gdeedded&dd"d+dd#d'Z$dd(d)Z%y),N)statsspecial)array_api_extra)xp_capabilitiesarray_namespace xp_promoteis_numpy _share_masks_count_nonmasked is_marray) _SimpleNormalSignificanceResult _get_pvalue _rankdata)_axis_nan_policy_factory)TheilslopesResult_n_samples_optional_xSiegelslopesResult) chatterjeexi spearmanrho theilslopes siegelslopesc|jd}|j|d}|j||\}}|j||d}t j |dd}t j | dd}|j |j|j|dd}|rdd|z|dzdz z z } n'd|j ||z |zdz} d||z| z z } | ||fS)Naxismax)methodr) shapeargsortbroadcast_arraystake_along_axisrrankdatasumabsdiff) xy y_continuousxpnjrlnum statisticdens Q/opt/ringagent/.cad-venv/lib/python3.12/site-packages/scipy/stats/_correlation.py _xi_statisticr6s  A 12 A  q! $DAq 1ab)A qR0A r%b1A &&+,2& 6CC16A:.. "&&!a%12&..C# % a?c|jd}|rF|jtjdtj|z |jS|j d|dz|j}|j |d}|j|d}d|dzz |jd|zd|zz dz|dzzdz}d|dzz |j|||z |zzdzdz} d|d zz |jd|zd|zz dz|zdz} d|d zz |j|||z zdz} |d| zz | dzz| dzz } |j| tj|z S) Nrg?dtyperrr!r ) r"asarraymathsqrtr:arangesortcumulative_sumr') r0r1r,r-r.iuvanbncndntau2s r5_xi_stdrK-s  Azz$))E*TYYq\9zII !QU!'' *A A !"%A QTBFFAaC!A#IMQT1F; ;B QTBFFAQ MA-BF7 7B QTBFFAaC!A#IMQ.RF8 8B QTBFFAQKrF2 2B 2IA Q &D 774=499Q< ''r7c|dvr tdt|tjs"|j }d}|dk7r t|||fS)N>FTz`y_continuous` must be boolean.z@`method` must be 'asymptotic' or a `PermutationMethod` instance. asymptotic) ValueError isinstancerPermutationMethodlower)r,rmessages r5_chatterjeexi_ivrSGsV=(:;; fe55 6T \ !W% %  r7c2|j|jfSN)r3pvalue)res_s r5_unpackrYWs ==#** $$r7) dask.arrayzno take_along_axis) skip_backendsTr!r)paired n_samplesresult_to_tuple n_outputs too_smallFrM)rr,rc& t| t|\}t|d \}d}|dk(r?t| \}}}t || } t } t || z | | } ndt|tjrJtjd |f fd|dd |jd d i} | j| j} }jd k(r|d n|} jd k(r| d n| } t|| S)aCompute the xi correlation and perform a test of independence. The xi correlation coefficient is a measure of association between two variables; the value tends to be close to zero when the variables are independent and close to 1 when there is a strong association. Unlike other correlation coefficients, the xi correlation is effective even when the association is not monotonic. Parameters ---------- x, y : array-like The samples: corresponding observations of the independent and dependent variable. The (N-d) arrays must be broadcastable. axis : int, default: 0 Axis along which to perform the test. y_continuous : bool, default: False Whether `y` is assumed to be drawn from a continuous distribution. If `y` is drawn from a continuous distribution, results are valid whether this is assumed or not, but enabling this assumption will result in faster computation and typically produce similar results. method : 'asymptotic' or `PermutationMethod` instance, optional Selects the method used to calculate the *p*-value. Default is 'asymptotic'. The following options are available. * ``'asymptotic'``: compares the standardized test statistic against the normal distribution. * `PermutationMethod` instance. In this case, the p-value is computed using `permutation_test` with the provided configuration options and other appropriate settings. Returns ------- res : SignificanceResult An object containing attributes: statistic : float The xi correlation statistic. pvalue : float The associated *p*-value: the probability of a statistic at least as high as the observed value under the null hypothesis of independence. See Also -------- scipy.stats.pearsonr, scipy.stats.spearmanr, scipy.stats.kendalltau Notes ----- There is currently no special handling of ties in `x`; they are broken arbitrarily by the implementation. [1]_ recommends: "if there are ties among the Xi's, then choose an increasing rearrangement as above by breaking ties uniformly at random." This is easily accomplished by adding a small amount of random noise to `x`; see examples. [1]_ notes that the statistic is not symmetric in `x` and `y` *by design*: "...we may want to understand if :math:`Y` is a function :math:`X`, and not just if one of the variables is a function of the other." See [1]_ Remark 1. References ---------- .. [1] Chatterjee, Sourav. "A new coefficient of correlation." Journal of the American Statistical Association 116.536 (2021): 2009-2022. :doi:`10.1080/01621459.2020.1758115`. Examples -------- Generate perfectly correlated data, and observe that the xi correlation is nearly 1.0. >>> import numpy as np >>> from scipy import stats >>> rng = np.random.default_rng(348932549825235) >>> x = rng.uniform(0, 10, size=100) >>> y = np.sin(x) >>> res = stats.chatterjeexi(x, y) >>> res.statistic np.float64(0.9012901290129013) The probability of observing such a high value of the statistic under the null hypothesis of independence is very low. >>> res.pvalue np.float64(2.2206974648177804e-46) As noise is introduced, the correlation coefficient decreases. >>> noise = rng.normal(scale=[[0.1], [0.5], [1]], size=(3, 100)) >>> res = stats.chatterjeexi(x, y + noise, axis=-1) >>> res.statistic array([0.79507951, 0.41824182, 0.16651665]) Because the distribution of `y` is continuous, it is valid to pass ``y_continuous=True``. The statistic is identical, and the p-value (not shown) is only slightly different. >>> stats.chatterjeexi(x, y + noise, y_continuous=True, axis=-1).statistic array([0.79507951, 0.41824182, 0.16651665]) Consider a case in which there are ties in `x`. >>> x = rng.integers(10, size=1000) >>> y = rng.integers(10, size=1000) [1]_ recommends breaking the ties uniformly at random. >>> d = rng.uniform(1e-5, size=x.size) >>> res = stats.chatterjeexi(x + d, y) >>> res.statistic -0.029919991638798438 Since this gives a randomized estimate of the statistic, [1]_ also suggests considering the average over all possibilities of breaking ties. This is computationally infeasible when there are many ties, but a randomized estimate of *this* quantity can be obtained by considering many random possibilities of breaking ties. >>> d = rng.uniform(1e-5, size=(9999, x.size)) >>> res = stats.chatterjeexi(x + d, y, axis=1) >>> np.mean(res.statistic) 0.001186895213756626 Tforce_floatingr-greaterrMr-) alternativer-c(t|dS)Nrer)r6)r+rr*r-r,s r5zchatterjeexi..smAq,2&Nq&Qr7pairings)datar3rfpermutation_typerrr)rrSrr6rKr rrOrrPpermutation_test_asdictr3rVndimr)r*r+rr,rrfxir0r1stdnormrVrWr-s` ` @r5rr[s#z A B,L&AL& a4B 7DAq K  A|;AqaLR0R#XtL FE33 4$$Q#j EKNNDT   ]]CJJF77a<BRB!;;!+VBZF b& ))r7 jax.numpy)rZz+not supported by rankdata (take_along_axis)z:Only the default `method` is compatible with MArray input.)cpu_only exceptionsmarrayr[ extra_notez two-sided)rfrrc t||}t|||\}}tj||}tj||}tj|||||}t |j |jS)aCalculate a Spearman rho correlation coefficient with associated p-value. The Spearman rank-order correlation coefficient is a nonparametric measure of the monotonicity of the relationship between two datasets. Like other correlation coefficients, it varies between -1 and +1 with 0 implying no correlation. Coefficients of -1 or +1 are associated with an exact monotonic relationship. Positive correlations indicate that as `x` increases, so does `y`; negative correlations indicate that as `x` increases, `y` decreases. The p-value is the probability of an uncorrelated system producing datasets with a Spearman correlation at least as extreme as the one computed from the observed dataset. Parameters ---------- x, y : array-like The samples: corresponding observations of the independent and dependent variable. The (N-d) arrays must be broadcastable. alternative : {'two-sided', 'less', 'greater'}, optional Defines the alternative hypothesis. Default is 'two-sided'. The following options are available: * 'two-sided': the correlation is nonzero * 'less': the correlation is negative (less than zero) * 'greater': the correlation is positive (greater than zero) method : ResamplingMethod, optional Defines the method used to compute the p-value. If `method` is an instance of `PermutationMethod`/`MonteCarloMethod`, the p-value is computed using `scipy.stats.permutation_test`/`scipy.stats.monte_carlo_test` with the provided configuration options and other appropriate settings. Otherwise, the p-value is computed using an asymptotic approximation of the null distribution. axis : int or None, optional If axis=0 (default), then each column represents a variable, with observations in the rows. If axis=1, the relationship is transposed: each row represents a variable, while the columns contain observations. If axis=None, then both arrays will be raveled. Like other `scipy.stats` functions, `axis` is interpreted after the arrays are broadcasted. Returns ------- res : SignificanceResult An object containing attributes: statistic : floating point array or NumPy scalar Spearman correlation coefficient pvalue : floating point array NumPy scalar The p-value - the probabilitiy of realizing such an extreme statistic value under the null hypothesis that two samples have no ordinal correlation. See `alternative` above for alternative hypotheses. Warns ----- `~scipy.stats.ConstantInputWarning` Raised if an input is a constant array. The correlation coefficient is not defined in this case, so ``np.nan`` is returned. Notes ----- `spearmanrho` was created to make improvements to SciPy's implementation of the Spearman correlation test without making backward-incompatible changes to `spearmanr`. Advantages of `spearmanrho` over `spearmanr` include: - `spearmanrho` follows standard array broadcasting rules. - `spearmanrho` is compatible with some non-NumPy arrays. - `spearmanrho` can compute exact p-values, even in the presence of ties, when an appropriate instance of `PermutationMethod` is provided via the `method` argument. References ---------- .. [1] Zwillinger, D. and Kokoska, S. (2000). CRC Standard Probability and Statistics Tables and Formulae. Chapman & Hall: New York. 2000. Section 14.7 .. [2] Kendall, M. G. and Stuart, A. (1973). The Advanced Theory of Statistics, Volume 2: Inference and Relationship. Griffin. 1973. Section 31.18 Examples -------- Univariate samples, approximate p-value. >>> import numpy as np >>> from scipy import stats >>> x = [1, 2, 3, 4, 5] >>> y = [5, 6, 7, 8, 7] >>> res = stats.spearmanrho(x, y) >>> res.statistic np.float64(0.8207826816681233) >>> res.pvalue np.float64(0.08858700531354405) Univariate samples, exact p-value. >>> res = stats.spearmanrho(x, y, method=stats.PermutationMethod()) >>> res.statistic np.float64(0.8207826816681233) >>> res.pvalue np.float64(0.13333333333333333) Batch of univariate samples, one vectorized call. >>> rng = np.random.default_rng(98145152315484) >>> x2 = rng.standard_normal((2, 100)) >>> y2 = rng.standard_normal((2, 100)) >>> res = stats.spearmanrho(x2, y2, axis=-1) >>> res.statistic array([ 0.16585659, -0.12151215]) >>> res.pvalue array([0.0991155 , 0.22846869]) Bivariate samples using standard broadcasting rules. >>> res = stats.spearmanrho(x2[np.newaxis, :], x2[:, np.newaxis], axis=-1) >>> res.statistic array([[ 1. , -0.14670267], [-0.14670267, 1. ]]) >>> res.pvalue array([[0. , 0.14526128], [0.14526128, 0. ]]) rer)rrfr)rr rr&pearsonrrr3rV) r*r+rfrrr-rxryrWs r5rrsnJ A B 1 $DAq  %B  %B ..RKd SC cmmSZZ 88r7)rsznon-concrete boolean indexing)r[rvr;ct|SrUtupler*rXs r5rhrh uQxr7) default_axisr_r]r^r\r`rc"t||||dS)at Computes the Theil-Sen estimator for a set of points (x, y). `theilslopes` implements a method for robust linear regression. It computes the slope as the median of all slopes between paired values. Parameters ---------- y : array_like Dependent variable. x : array_like or None, optional Independent variable. If None, use ``arange(len(y))`` instead. alpha : float, optional Confidence degree between 0 and 1. Default is 95% confidence. Note that `alpha` is symmetric around 0.5, i.e. both 0.1 and 0.9 are interpreted as "find the 90% confidence interval". method : {'joint', 'separate'}, optional Method to be used for computing estimate for intercept. Following methods are supported, * 'joint': Uses ``np.median(y - slope * x)`` as intercept. * 'separate': Uses ``np.median(y) - slope * np.median(x)`` as intercept. The default is 'separate'. .. versionadded:: 1.8.0 axis : int or tuple of ints, default: None If an int or tuple of ints, the axis or axes of the input along which to compute the statistic. The statistic of each axis-slice (e.g. row) of the input will appear in a corresponding element of the output. If ``None``, the input will be raveled before computing the statistic. Returns ------- result : ``TheilslopesResult`` instance The return value is an object with the following attributes: slope : float Theil slope. intercept : float Intercept of the Theil line. low_slope : float Lower bound of the confidence interval on `slope`. high_slope : float Upper bound of the confidence interval on `slope`. See Also -------- siegelslopes : a similar technique using repeated medians Notes ----- The implementation of `theilslopes` follows [1]_. The intercept is not defined in [1]_, and here it is defined as ``median(y) - slope*median(x)``, which is given in [3]_. Other definitions of the intercept exist in the literature such as ``median(y - slope*x)`` in [4]_. The approach to compute the intercept can be determined by the parameter ``method``. A confidence interval for the intercept is not given as this question is not addressed in [1]_. For compatibility with older versions of SciPy, the return value acts like a ``namedtuple`` of length 4, with fields ``slope``, ``intercept``, ``low_slope``, and ``high_slope``, so one can continue to write:: slope, intercept, low_slope, high_slope = theilslopes(y, x) References ---------- .. [1] P.K. Sen, "Estimates of the regression coefficient based on Kendall's tau", J. Am. Stat. Assoc., Vol. 63, pp. 1379-1389, 1968. .. [2] H. Theil, "A rank-invariant method of linear and polynomial regression analysis I, II and III", Nederl. Akad. Wetensch., Proc. 53:, pp. 386-392, pp. 521-525, pp. 1397-1412, 1950. .. [3] W.L. Conover, "Practical nonparametric statistics", 2nd ed., John Wiley and Sons, New York, pp. 493. .. [4] https://en.wikipedia.org/wiki/Theil%E2%80%93Sen_estimator Examples -------- >>> import numpy as np >>> from scipy import stats >>> import matplotlib.pyplot as plt >>> x = np.linspace(-5, 5, num=150) >>> y = x + np.random.normal(size=x.size) >>> y[11:15] += 10 # add outliers >>> y[-5:] -= 7 Compute the slope, intercept and 90% confidence interval. For comparison, also compute the least-squares fit with `linregress`: >>> res = stats.theilslopes(y, x, 0.90, method='separate') >>> lsq_res = stats.linregress(x, y) Plot the results. The Theil-Sen regression line is shown in red, with the dashed red lines illustrating the confidence interval of the slope (note that the dashed red lines are not the confidence interval of the regression as the confidence interval of the intercept is not included). The green line shows the least-squares fit for comparison. >>> fig = plt.figure() >>> ax = fig.add_subplot(111) >>> ax.plot(x, y, 'b.') >>> ax.plot(x, res[1] + res[0] * x, 'r-') >>> ax.plot(x, res[1] + res[2] * x, 'r--') >>> ax.plot(x, res[1] + res[3] * x, 'r--') >>> ax.plot(x, lsq_res[1] + lsq_res[0] * x, 'g-') >>> plt.show() r)r*alpharpfun_robust_slopes)r+r*rrrs r5rrsp !qf= QQr7)rszquantile needs lazy nan_policyct|SrUr}rs r5rhrhrr7c t|||dS)a Computes the Siegel estimator for a set of points (x, y). `siegelslopes` implements a method for robust linear regression using repeated medians (see [1]_) to fit a line to the points (x, y). The method is robust to outliers with an asymptotic breakdown point of 50%. Parameters ---------- y : array_like Dependent variable. x : array_like or None, optional Independent variable. If None, use ``arange(len(y))`` instead. method : {'hierarchical', 'separate'} If 'hierarchical', estimate the intercept using the estimated slope ``slope`` (default option). If 'separate', estimate the intercept independent of the estimated slope. See Notes for details. axis : int or tuple of ints, default: None If an int or tuple of ints, the axis or axes of the input along which to compute the statistic. The statistic of each axis-slice (e.g. row) of the input will appear in a corresponding element of the output. If ``None``, the input will be raveled before computing the statistic. Returns ------- result : ``SiegelslopesResult`` instance The return value is an object with the following attributes: slope : float Estimate of the slope of the regression line. intercept : float Estimate of the intercept of the regression line. See Also -------- theilslopes : a similar technique without repeated medians Notes ----- With ``n = len(y)``, compute ``m_j`` as the median of the slopes of the lines from the point ``(x[j], y[j])`` to all other ``n-1`` points. ``slope`` is then the median of all slopes ``m_j``. Two ways are given to estimate the intercept in [1]_ which can be chosen via the parameter ``method``. The hierarchical approach uses the estimated slope ``slope`` and computes ``intercept`` as the median of ``y - slope*x``. The other approach estimates the intercept separately as follows: for each point ``(x[j], y[j])``, compute the intercepts of all the ``n-1`` lines through the remaining points and take the median ``i_j``. ``intercept`` is the median of the ``i_j``. The implementation computes `n` times the median of a vector of size `n` which can be slow for large vectors. There are more efficient algorithms (see [2]_) which are not implemented here. For compatibility with older versions of SciPy, the return value acts like a ``namedtuple`` of length 2, with fields ``slope`` and ``intercept``, so one can continue to write:: slope, intercept = siegelslopes(y, x) References ---------- .. [1] A. Siegel, "Robust Regression Using Repeated Medians", Biometrika, Vol. 69, pp. 242-244, 1982. .. [2] A. Stein and M. Werman, "Finding the repeated median regression line", Proceedings of the Third Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 409-413, 1992. Examples -------- >>> import numpy as np >>> from scipy import stats >>> import matplotlib.pyplot as plt >>> x = np.linspace(-5, 5, num=150) >>> y = x + np.random.normal(size=x.size) >>> y[11:15] += 10 # add outliers >>> y[-5:] -= 7 Compute the slope and intercept. For comparison, also compute the least-squares fit with `linregress`: >>> res = stats.siegelslopes(y, x) >>> lsq_res = stats.linregress(x, y) Plot the results. The Siegel regression line is shown in red. The green line shows the least-squares fit for comparison. >>> fig = plt.figure() >>> ax = fig.add_subplot(111) >>> ax.plot(x, y, 'b.') >>> ax.plot(x, res[1] + res[0] * x, 'r-') >>> ax.plot(x, lsq_res[1] + lsq_res[0] * x, 'g-') >>> plt.show() r)r*rrr)r+r*rrs r5rrsX !qn EEr7)rc P !|dk(rdnd}||dhvrtd|d|dt||!t||d! \}}|*!j|jd |j n|}!j ||\}}t||! \}}|jd d ks|jd d kr td|ddd!jf|d!jddfz }|ddd!jf|d!jddfz }|dk(rщ!j!j!j|jddd!j}t!r|d|f|d|f}}nn!j||j}||||}}!j|g|jddd }!j|g|jddd }t!!r|j"dk(n|dk(} t%j&|| j)!j*}||z } t!!r!fd} nd} d} |dk(r| | | d d n | | d } |dvr$| || dt,jf|zz d }n\|dk(r| |d | | |d zz }n>|ddd!jf| |ddd!jfzz }| | |d d }|dk(rt/| d|dS|dkDrd|z }t1t3j4|dz }t7|dd \}}}t7|dd \}}}!j9!j;| d d!}!j=||j }t?|dd "}d#||dz zd |zd$zz!jA||dz zd |zd$zzd d!z !jA||dz zd |zd$zzd d!z z}!jC!jE|!jG|}!jI!j!jK|||zz dz !jL!j|!jLdz }!jE!j!jK|||zzdz !jLdz !j=d!jL }!jOt%jP|d%t%jP|d%fd }!jS| d } !jU| |d }!j|dk|j}t%j&||j)!j*}| jVdk(r| dn| }|jVdk(r|dn|}|d&}|d'} |jVdk(r|dn|}| jVdk(r| dn| } tY|||| (S))Nrjoint hierarchicalseparatezmethod must be either 'z' or 'separate'. 'z ' is invalid.Trbrr9rer!z(`x` and `y` must have length at least 2..r)krcj|j|j|jj |jz}t j |d|S)N)mask?r)r=rjr_xpisnanrquantile)r*r x_nans_maskedr-s r5 nanmedianz!_robust_slopes..nanmediansFJJqvvAFFQUU[[=P4PJQM>>-4@ @r7c4tj|d|dS)Nromit)r nan_policyrrr*rs r5rz!_robust_slopes..nanmedians>>!StG Gr7c2tj|d|S)Nrrrrs r5medianz_robust_slopes..mediansq#D AAr7rr>rrrl)slope interceptrg?g@min)r return_ties)rkeepdims)rrgqq?r<)ro).r).r)rr low_slope high_slope)-rNrrr@r"r:r$r newaxisastypetriuonesboolr broadcast_toreshaper rjxpxatsetnannprfloatrndtrir count_nonzeroisfiniter=r r'r?maximum zeros_likeminimumroundint64concat atleast_ndrAr%ror)"r+r*rrr other_methoddeltaxdeltayrCdeltax0slopesrrmedslopemedinter interceptszrXnxrepsnyrepsntnysigsqsigmaRuRlRdeltai_nanrrrrr-s" @r5rros#"m37L lJ//2<.A#HM34 4 A B a4B 7DAq12 !''"+QWW -A  q! $DAq 1 $DAqwwr{Q!''"+/CDDsArzz! "QsBJJ'9%: :F sArzz! "QsBJJ'9%: :F } IIbggbggfll23&78Ag> H B<#CF^VCF^FF6<<0A#AYq FFZZ(;!''#2,(;(;>"1-s~~bq/IJQS TA WWV"W %F   vqr  2E OOEAIu{{ 3E FF5M% $ $RVV ,E$MMQ.HRLHE (  2 If IvJ!*1!4 " )I#-??a#7BZJ 5I'0Z IIr7)Ngffffff?r)Nr)&numpyrr>scipyrrscipy._externalrrscipy._lib._array_apirrrr r r r scipy.stats._stats_pyr rrrscipy.stats._axis_nan_policyr scipy.stats._stats_mstats_commonrrr__all__r6rKrSrYrrrrrrlr7r5rs 2XXX..ABB I8(4  % DEF,TQ*1Q!M u\W*MGW*t$K=PQK,TQ*1Q!M(3DqD9M  D9N D N P+$!$9*?$%'qR$qR 'qRh D O Q,41$9*?$%'eF4eF 'eFP#'lIr7