Detection of Correlated Random Vectors
0
Sign in to get full access
Overview
- Presents a method for detecting correlated random vectors
- Focuses on determining if two sets of random variables are correlated or independent
- Proposes a detection test based on the generalized likelihood ratio
Plain English Explanation
The paper addresses the problem of detecting correlated random vectors. Imagine you have two sets of measurements or observations, and you want to know if they are related or independent. This could be useful in many fields, like analyzing sensor data or studying statistical relationships.
The paper proposes a statistical test to determine if the two sets of random variables are correlated. It develops a generalized likelihood ratio test that compares the likelihood of the data under the assumption of correlation versus independence. This allows us to quantify the evidence for or against correlation and make a decision.
Technical Explanation
The paper starts by defining the problem and setting up the mathematical model. It considers two sets of random vectors, x and y, and wants to determine if they are correlated or independent.
The key idea is to construct a generalized likelihood ratio test that compares the likelihood of the data under the null hypothesis of independence versus the alternative hypothesis of correlation. This test statistic is shown to asymptotically follow a chi-squared distribution, allowing us to set a threshold for detection.
The paper analyzes the statistical properties of this test, including its false alarm rate and detection probability. It also provides guidelines for choosing the appropriate threshold based on the desired level of confidence.
Critical Analysis
The paper provides a principled statistical approach to the problem of detecting correlated random vectors. However, it assumes the random variables follow a Gaussian distribution, which may not always be the case in practice. Extending the method to handle non-Gaussian data could be an area for future research.
Additionally, the paper does not address the impact of data corruption or outliers on the performance of the detection test. Developing robust versions of the method could be a valuable direction to explore.
Conclusion
This paper presents a generalized likelihood ratio test for detecting correlated random vectors. The proposed method provides a statistically rigorous approach to determining if two sets of measurements or observations are related. This could have applications in various fields, such as sensor data analysis and statistical inference.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Detection of Correlated Random Vectors
Dor Elimelech, Wasim Huleihel
In this paper, we investigate the problem of deciding whether two standard normal random vectors $mathsf{X}inmathbb{R}^{n}$ and $mathsf{Y}inmathbb{R}^{n}$ are correlated or not. This is formulated as a hypothesis testing problem, where under the null hypothesis, these vectors are statistically independent, while under the alternative, $mathsf{X}$ and a randomly and uniformly permuted version of $mathsf{Y}$, are correlated with correlation $rho$. We analyze the thresholds at which optimal testing is information-theoretically impossible and possible, as a function of $n$ and $rho$. To derive our information-theoretic lower bounds, we develop a novel technique for evaluating the second moment of the likelihood ratio using an orthogonal polynomials expansion, which among other things, reveals a surprising connection to integer partition functions. We also study a multi-dimensional generalization of the above setting, where rather than two vectors we observe two databases/matrices, and furthermore allow for partial correlations between these two.
Read more7/26/2024
🧠
0
Information-Theoretic Thresholds for the Alignments of Partially Correlated Graphs
Dong Huang, Xianwen Song, Pengkun Yang
This paper studies the problem of recovering the hidden vertex correspondence between two correlated random graphs. We propose the partially correlated ErdH{o}s-R'enyi graphs model, wherein a pair of induced subgraphs with a certain number are correlated. We investigate the information-theoretic thresholds for recovering the latent correlated subgraphs and the hidden vertex correspondence. We prove that there exists an optimal rate for partial recovery for the number of correlated nodes, above which one can correctly match a fraction of vertices and below which correctly matching any positive fraction is impossible, and we also derive an optimal rate for exact recovery. In the proof of possibility results, we propose correlated functional digraphs, which partition the edges of the intersection graph into two types of components, and bound the error probability by lower-order cumulant generating functions. The proof of impossibility results build upon the generalized Fano's inequality and the recovery thresholds settled in correlated ErdH{o}s-R'enyi graphs model.
Read more6/11/2024
0
Variance-based sensitivity analysis in the presence of correlated input variables
Thomas Most
In this paper we propose an extension of the classical Sobol' estimator for the estimation of variance based sensitivity indices. The approach assumes a linear correlation model between the input variables which is used to decompose the contribution of an input variable into a correlated and an uncorrelated part. This method provides sampling matrices following the original joint probability distribution which are used directly to compute the model output without any assumptions or approximations of the model response function.
Read more8/12/2024
0
Robust Kernel Hypothesis Testing under Data Corruption
Antonin Schrab, Ilmun Kim
We propose two general methods for constructing robust permutation tests under data corruption. The proposed tests effectively control the non-asymptotic type I error under data corruption, and we prove their consistency in power under minimal conditions. This contributes to the practical deployment of hypothesis tests for real-world applications with potential adversarial attacks. One of our methods inherently ensures differential privacy, further broadening its applicability to private data analysis. For the two-sample and independence settings, we show that our kernel robust tests are minimax optimal, in the sense that they are guaranteed to be non-asymptotically powerful against alternatives uniformly separated from the null in the kernel MMD and HSIC metrics at some optimal rate (tight with matching lower bound). Finally, we provide publicly available implementations and empirically illustrate the practicality of our proposed tests.
Read more5/31/2024