2018年,傅等.
[18]提出了一种基于步长自适应的IVA盲分离算法。该算法利用特征矩阵联合近似对角化算法初始化分离矩阵,自适应优化步长参数。即避免局部收敛,还可以显著提高算法的收敛速度,进一步提高分离性能。根据迭代步长与估计成本函数之间的关系变化。2012年,王等.
[19]提出了一种基于最大块速度-步长下降的可变步长IVA梯度算法。此外,根据迭代步长与待得到的分离矩阵变化之间的关系,提出一种基于估计函数的变步长IVA梯度算法。2010年,金·
[12]提出了一种具有非完全闭合约束的修正梯度和归一化IVA方法。梯度归一化提高了收敛速度,计算复杂度较低的非全息约束梯度表现出更好的性能,同时与其他方法相比具有更简单的结构。2018年,科尔多夫斯基等人.
[20]基于IVA算法的独立向量提取(IVE),提出了一种在复杂非高斯场景中采用自适应步长方法的IVE算法,以加快收敛速度。
优化基于负熵的目标函数时,最简单的方法是使用 GD。虽然基于GD的方法具有良好的分离效果,但使用起来相对简单。该方法的整体收敛速度很慢,并且取决于学习速率序列的良好选择,即每次迭代的步长。尽管上一节总结了步长因子的各种优化,但GD方法依赖于合适的步长进行分离。
其中 (⋅)H 表示
(⋅) 的共轭转置。为了能够直接应用牛顿方法推导出复变量的快速算法,在复数符号中引入了二次泰勒多项式。使用这种形式的泰勒级数展开使推导更简单,并且对于直接将牛顿方法应用于复值变量的客观函数很有用。2000年,闫等.
[26]提供了独立的等效物。
最近,在 2021 年,科尔多夫斯基等人。
[27]提出了一种基于FastICA和FastIVA静态混合算法的扩展快速动态独立矢量分析(FastDIVA)算法,用于从时变混合信号中盲目提取或分离一个或多个信号源。在允许所需源移动的逐源分离混合物模型中,混合物要么串联,要么并联。该算法继承了FastIVA的优点,在运动源分离方面表现出良好的性能,表现出优越的收敛速度和分离超高斯和亚高斯信号的能力。
2021年,阿莫尔等人.
[28]使用 FastDIVA 对具有恒定分离载体 CSV 的混合物模型进行盲源提取。此外,它在嘈杂环境中的运动扬声器、运动大脑活动的提取和运动源三种环境中显示出新的潜力和良好的分离性能。2021年,科尔多夫斯基等人.
[29]提出了一种新的动态IVA算法。它基于一个混合模型,其中与兴趣源(SOI)相关的混合参数是时变的,分离参数是时不变的。采用牛顿-拉夫森方法在准似然法的基础上对目标函数进行优化,然后在不施加正交约束的情况下进行迭代更新,然后进行正交性。该算法是对快速定点算法的优化,在性能上优于梯度算法和辅助函数法。
4. 辅助功能
The update method based on the auxiliary function technology is also a method that does not include tuning parameters such as step size, which is an iterative algorithm with a convergence guarantee. This is a stable and fast update rule derived from the majorize-minimization principle
[30][31]. Find its minimum by exploiting the convexity of the function. When the objective function
f(θ)
is difficult to optimize, and the optimization algorithm used cannot directly find the optimal solution to the objective function, an easy-to-optimize objective function
g(θ) can be found instead. Then, the substitution function is solved, and the optimal solution of
g(θ) is close to the optimal solution of
f(θ). In each iteration, a new surrogate function for the next iteration is reconstructed from the solution. Then, the new substitute function is optimized and solved to obtain the objective function of the next iteration. After several iterations, the optimal solution that is closer and closer to the original objective function that can be obtained. It was first proposed in the literature
[32] to accelerate the convergence speed of the ICA algorithm. This rule consists of two optional updates:
-
The update of the weighted covariance matrix (that is, the auxiliary function variable).
-
The update of the separation matrix ensures that the objective function decreases monotonically at each update and finally achieves convergence.
Equation (12) is the auxiliary function variable update:
Among them, Vn denotes a covariance matrix of the observed signals, U(⋅) denotes a continuous and differentiable function of a real variable · satisfying, and U′(⋅) usually takes the constant 1. ∥⋅∥2 denotes the 2-norm of ·. Equation (13) is the update of the unmixing matrix:
In 2011, Nobutaka Ono
[33] used the auxiliary function technique in the objective function of the IVA algorithm and similarly derived an efficient update rule suitable for the IVA algorithm, called AuxIVA. In 2012, Nobutaka Ono
[34] proposed an AuxIVA algorithm based on a generalized Gaussian source model or a Gaussian source model with time-varying variance. In 2012 and 2013, Nobutaka Ono
[35][36] proposed a faster algorithm that can update two separation vectors simultaneously by solving the generalized eigenvalue problem for the AuxIVA algorithm with two sources and two microphones. Compared with the one-by-one update method, this method has faster convergence speed and better performance. This pairwise update method is also applicable to the pairwise separation of vectors in the case of three or more sources
[37]. In 2014, Taniguchi et al.
[38] used the AuxIVA algorithm based on the auxiliary function method for online real-time blind speech separation. In experimental comparisons with commonly used real-time IVA algorithms, the proposed online algorithm achieves a higher signal-to-noise ratio without environment-sensitive tuning parameters such as step factor.
In 2021, Brendel et al.
[39] further optimized the IVA algorithm based on auxiliary functions under the same computational cost. The convergence speed of the AuxIVA algorithm is enhanced by three methods:
-
Turn the differential term into a tuning parameter via the differential term in the NG approximation algorithm.
-
Approximate the differential term as a matrix using the quasi-Newton method.
-
Use the square iteration method to speed it up.
5. EM Method
In signal processing, a common problem is estimating the parameters of a probability distribution function. The situation is more complicated in many parameter estimation problems because the data needed to estimate the parameters are not directly accessible, or some data are missing. EM-based optimization algorithms are well-suited for solving this class of problems because the EM algorithm produces maximum likelihood (ML) estimates of the parameters when there is a many-to-one mapping from the underlying distribution to the distribution of the control observations, while taking additive noise into account. The EM algorithm overcomes the problem of unanalyzable solutions and has been widely used in statistics, signal processing, and machine learning
[40].
The EM algorithm is an iterative optimization method
[41] that is used to estimate some unknown parameters given measurement data. The solution is divided into two steps.
E-step: First assign an initial distribution to each hidden variable empirically, that is, assume distribution parameters. Then, according to the parameters of the distribution, the expectation of the hidden variables in each data tuple can be obtained, that is, the classification operation is performed. The posteriors of the source signal can be obtained by
where ∝ denotes it is proportional to the previous term, and q denotes posterior probability.
M-step: Calculate the maximum likelihood value of the distribution parameter (vector) based on the classification result, and then in turn recalculate the expectation of the hidden variable for each data tuple based on this maximum likelihood value. The update rules for mixing matrices A are
where
<⋅>q denotes expectation over
q.
Through the repetition of the above two steps, when the expectation of the hidden variable and the maximum likelihood value of the parameter tends to be stable, the entire iteration is completed.
In 2004 and 2008, Varadhan et al.
[42][43] used the square iteration method in the EM algorithm to accelerate its convergence speed. In 2008, Lee et al.
[44] deduced the expectation-maximization algorithm, and the algorithm was used in the updated iteration of the IVA algorithm. The EM algorithm could estimate the parameters of the separation matrix and the unknown source at the same time, showing a good separation performance. In 2010, Hao et al.
[45] proposed a unified probabilistic framework for the IVA algorithm with the Gaussian mixture model as the source prior model; this flexible prior source enables the IVA algorithm to separate different types of signals, deduce different EM algorithms, and test three models: noiseless IVA, online IVA, and noise IVA. The EM algorithm can effectively estimate the unmixing matrix without sensor noise. In online IVA, an online EM algorithm is derived to track the motion of the source under nonstationary conditions. Noise IVA includes sensor noise and denoising combined with separation. An EM algorithm suitable for this model is proposed which can effectively estimate the model parameters and separate the source signal at the same time.
In 2019, Gu et al.
[46] proposed a Gaussian mixture model IVA algorithm with time-varying parameters to accommodate temporal power fluctuations embedded in nonstationary speech signals, thus avoiding the pretraining process of the original Gaussian mixture model IVA (GMM-IVA) algorithm and using the corresponding improved EM algorithm to estimate the separation matrix and signal model. The experimental results confirm the effectiveness of the method in random initialization and the advantages in separation accuracy and convergence speed. In 2019, Rafique et al.
[47] proposed a new IVA algorithm based on Student’s t-mixture model as a source before adapting to the statistical properties of different speech sources. At the same time, an efficient EM algorithm is derived which estimates the location parameters of the source prior matrix and the decomposition matrix together, thereby improving the separation performance of the IVA algorithm. In 2020, Tang et al.
[48] proposed a complex generalized Gaussian mixture distribution with weighted variance to capture the non-Gaussian and nonstationary properties of speech signals to flexibly characterize real speech signals. At the same time, the optimization rules based on the EM method are used to estimate and update the mixing parameters.
6. BCD Method
Coordinate descent (CD) is a nongradient optimization algorithm. The algorithm does not need to calculate the gradient of the objective function and performs a linear search along a single dimension at a time. When a minimum value of the current dimension is obtained, different dimension directions are used repeatedly, and the optimal solution is finally converged. However, this algorithm is only suitable for smooth functions. When nonsmooth functions are used, they may fall into a nonstagnant point and fail to converge. In 2015, Wright
[49] proposed block coordinate descent (BCD), a generalization of the coordinate descent algorithm. It decomposes the original problem into multiple subproblems by simultaneously optimizing a subset of variables. The order of updates during the descent can be deterministic or random. This algorithm is mainly used to solve the nonconvex function, of which the objective function’s global optimal value is difficult to obtain.
Among them, the BCD algorithm has developed two methods with closed update formula for the BSS IVA algorithm’s
[50] IP and ISS methods.
6.1. Iterative Projection
The IVA algorithm based on iterative projection was first introduced in the AuxIVA
[33] algorithm.
This update rule is derived by solving a quadratic system of equations obtained by differentiating the cost function concerning the separation vector. In 2004, Dégerine et al.
[51] also proposed a similar scheme in the context of semiblind Gaussian source components. In 2016, Kitamura et al.
[52] used the IP algorithm in a BSS algorithm combining IVA and NMF, which provided good convergence speed and separation effect. In 2018, Yatabe et al.
[53] proposed an alternative to the AuxIVA-IP algorithm based on proximal splitting. In 2021, Nakashima et al.
[54] optimized it based on IP and extended each row vector of the separation matrix to update one by one to two rows of the separation matrix per update, resulting in a faster IP-2.
In 2020, Ikeshita et al.
[55] deduced IP-1 and IP-2 and used these two update rules to accelerate the OverIVA algorithm, forming the OverIVA-IP and OverIVA-IP2 update rules. In 2021, Scheibler
[56] proposed an iterative projection with adjustment (IPA) and a Newton conjugate gradient (NCG) to solve the hybrid exact-approximate diagonalization (HEAD) problem. IPA adopts a multiplicative update form, that is, the current separation matrix is multiplied by the rank 2 perturbation of the identity matrix. This method performs joint updates to the unmixing filters and additional rank-one updates to the remainder of the unmixing matrix. Simply put, the IPA optimization rule is a combination of IP and ISS methods. Updating one row and one column of the matrix in each update, performing IP- and ISS-style updates jointly, outperforms the IP and ISS methods.
6.2. Iterative Source Steering
ISS
[57] is an alternative to IP. Although IP has the advantages of good performance and fast convergence speed, in the iterative update process, it needs to recalculate a covariance matrix and invert for each source and each iteration. This greatly increases the overall complexity of the algorithm. The complexity of the algorithm is three times the number of microphones used. In addition to that, inverting a matrix is an inherently dangerous operation that can lead to unstable convergence when iterating. On this basis, the proposed ISS algorithm can effectively reduce the computational cost and complexity brought by the IP algorithm. ISS can also minimize the same cost function as the AuxIVA algorithm.
This update rule, which does not require matrix inversion, is used in a new method for joint deredundancy and BSS
[58]. This is a method based on an ILRMA framework, which combines the advantages of no inversion and low complexity of the ISS algorithm to achieve efficient BSS. In 2021, Du et al.
[59] proposed a computationally efficient optimization algorithm for BSS of overdetermined mixtures, an improved ISS algorithm for OverIVA algorithm, namely OverIVA-ISS. The algorithm combines the technology in OverIVA-IP with the technology in AuxIVA-ISS, which is more computationally efficient than the OverIVA-IP algorithm and can guarantee convergence. Additionally, the computational complexity is reduced from
O(M2) to
O(MN).
The overall performance of the ISS algorithm is better than the IP algorithm but inferior to the IP-2 algorithm. Therefore, an ISS-2 algorithm is proposed. In 2022, Ikeshita et al.
[60] extended the ISS algorithm to ISS-2.
At the same time, the advantage of the smaller time complexity of the ISS algorithm is maintained, and the separation performance is comparable to IP-2.
7. EVD Method
The EVD method is to find the most similar matrix to the original matrix. The optimization update rule based on EVD can be expressed as:
and
where
λM and
uM denote the smallest eigenvalue and eigenvector, respectively.