In 2011, Nobutaka Ono
[42][33] used the auxiliary function technique in the objective function of the IVA algorithm and similarly derived an efficient update rule suitable for the IVA algorithm, called AuxIVA. In 2012, Nobutaka Ono
[43][34] proposed an AuxIVA algorithm based on a generalized Gaussian source model or a Gaussian source model with time-varying variance. In 2012 and 2013, Nobutaka Ono
[44,45][35][36] proposed a faster algorithm that can update two separation vectors simultaneously by solving the generalized eigenvalue problem for the AuxIVA algorithm with two sources and two microphones. Compared with the one-by-one update method, this method has faster convergence speed and better performance. This pairwise update method is also applicable to the pairwise separation of vectors in the case of three or more sources
[46][37]. In 2014, Taniguchi et al.
[47][38] used the AuxIVA algorithm based on the auxiliary function method for online real-time blind speech separation. In experimental comparisons with commonly used real-time IVA algorithms, the proposed online algorithm achieves a higher signal-to-noise ratio without environment-sensitive tuning parameters such as step factor.
In 2021, Brendel et al.
[48][39] further optimized the IVA algorithm based on auxiliary functions under the same computational cost. The convergence speed of the AuxIVA algorithm is enhanced by three methods:
-
Turn the differential term into a tuning parameter via the differential term in the NG approximation algorithm.
-
Approximate the differential term as a matrix using the quasi-Newton method.
-
Use the square iteration method to speed it up.
5. EM Method
In signal processing, a common problem is estimating the parameters of a probability distribution function. The situation is more complicated in many parameter estimation problems because the data needed to estimate the parameters are not directly accessible, or some data are missing. EM-based optimization algorithms are well-suited for solving this class of problems because the EM algorithm produces maximum likelihood (ML) estimates of the parameters when there is a many-to-one mapping from the underlying distribution to the distribution of the control observations, while taking additive noise into account. The EM algorithm overcomes the problem of unanalyzable solutions and has been widely used in statistics, signal processing, and machine learning
[50][40].
The EM algorithm is an iterative optimization method
[51][41] that is used to estimate some unknown parameters given measurement data. The solution is divided into two steps.
E-step: First assign an initial distribution to each hidden variable empirically, that is, assume distribution parameters. Then, according to the parameters of the distribution, the expectation of the hidden variables in each data tuple can be obtained, that is, the classification operation is performed. The posteriors of the source signal can be obtained by
where ∝ denotes it is proportional to the previous term, and
q denotes posterior probability.
M-step: Calculate the maximum likelihood value of the distribution parameter (vector) based on the classification result, and then in turn recalculate the expectation of the hidden variable for each data tuple based on this maximum likelihood value. The update rules for mixing matrices
A are
where
<⋅>q denotes expectation over
q.
Through the repetition of the above two steps, when the expectation of the hidden variable and the maximum likelihood value of the parameter tends to be stable, the entire iteration is completed.
In 2004 and 2008, Varadhan et al.
[52,53][42][43] used the square iteration method in the EM algorithm to accelerate its convergence speed. In 2008, Lee et al.
[54][44] deduced the expectation-maximization algorithm, and the algorithm was used in the updated iteration of the IVA algorithm. The EM algorithm could estimate the parameters of the separation matrix and the unknown source at the same time, showing a good separation performance. In 2010, Hao et al.
[55][45] proposed a unified probabilistic framework for the IVA algorithm with the Gaussian mixture model as the source prior model; this flexible prior source enables the IVA algorithm to separate different types of signals, deduce different EM algorithms, and test three models: noiseless IVA, online IVA, and noise IVA. The EM algorithm can effectively estimate the unmixing matrix without sensor noise. In online IVA, an online EM algorithm is derived to track the motion of the source under nonstationary conditions. Noise IVA includes sensor noise and denoising combined with separation. An EM algorithm suitable for this model is proposed which can effectively estimate the model parameters and separate the source signal at the same time.
In 2019, Gu et al.
[56][46] proposed a Gaussian mixture model IVA algorithm with time-varying parameters to accommodate temporal power fluctuations embedded in nonstationary speech signals, thus avoiding the pretraining process of the original Gaussian mixture model IVA (GMM-IVA) algorithm and using the corresponding improved EM algorithm to estimate the separation matrix and signal model. The experimental results confirm the effectiveness of the method in random initialization and the advantages in separation accuracy and convergence speed. In 2019, Rafique et al.
[57][47] proposed a new IVA algorithm based on Student’s t-mixture model as a source before adapting to the statistical properties of different speech sources. At the same time, an efficient EM algorithm is derived which estimates the location parameters of the source prior matrix and the decomposition matrix together, thereby improving the separation performance of the IVA algorithm. In 2020, Tang et al.
[58][48] proposed a complex generalized Gaussian mixture distribution with weighted variance to capture the non-Gaussian and nonstationary properties of speech signals to flexibly characterize real speech signals. At the same time, the optimization rules based on the EM method are used to estimate and update the mixing parameters.
6. BCD Method
Coordinate descent (CD) is a nongradient optimization algorithm. The algorithm does not need to calculate the gradient of the objective function and performs a linear search along a single dimension at a time. When a minimum value of the current dimension is obtained, different dimension directions are used repeatedly, and the optimal solution is finally converged. However, this algorithm is only suitable for smooth functions. When nonsmooth functions are used, they may fall into a nonstagnant point and fail to converge. In 2015, Wright
[59][49] proposed block coordinate descent (BCD), a generalization of the coordinate descent algorithm. It decomposes the original problem into multiple subproblems by simultaneously optimizing a subset of variables. The order of updates during the descent can be deterministic or random. This algorithm is mainly used to solve the nonconvex function, of which the objective function’s global optimal value is difficult to obtain.
其中,Among them, the BCD
算法针对 algorithm has developed two methods with closed update formula for the BSS IVA
算法的[60] algorithm’s [50] IP
和 and ISS
方法开发了两种具有封闭更新公式的方法。 methods.
6.1. Iterative 迭代投影Projection
基于迭代投影的The IVA
算法最早是在 algorithm based on iterative projection was first introduced in the AuxIVA
[42]算法中引入的。[33] algorithm.
该更新规则是通过求解通过微分分离向量的成本函数而获得的二次方程组得出的。This update rule is derived by solving a quadratic system of equations obtained by differentiating the cost function concerning the separation vector. In 2004
年,, Dégerin
e et al. [51] also propose
等人[61]也在半盲高斯源分量的背景下提出了类似的方案。d a similar scheme in the context of semiblind Gaussian source components. In 2016
年,, Kitamura
等人[62]在结合 et al. [52] used the IP algorithm in a BSS algorithm combining IVA
和 and NMF
的BSS算法中使用了IP算法,提供了良好的收敛速度和分离效果。2018年,, which provided good convergence speed and separation effect. In 2018, Yatabe
等人[63]提出了一种基于近端分裂的 et al. [53] proposed an alternative to the AuxIVA-IP
算法的替代方案。 algorithm based on proximal splitting. In 2021
年,, Nakashima
等人et al. [54] optimized it [64]based 基于on IP
对其进行了优化,并将分离矩阵的每一行向量扩展为每次更新一行到两行分离矩阵,从而获得更快的 IP-2。and extended each row vector of the separation matrix to update one by one to two rows of the separation matrix per update, resulting in a faster IP-2.
In 2020
年,池下等[65]推导出, Ikeshita et al. [55] deduced IP-1
和 and IP-2
,并利用这两个更新规则加速 and used these two update rules to accelerate the OverIVA
算法,形成了 algorithm, forming the OverIVA-IP
和 and OverIVA-IP2
更新规则。2021 年, update rules. In 2021, Scheibler
[66][56] proposed an iterative projection with 提出了带调整的迭代投影adjustment (IPA) 和牛顿共轭梯度 (NCG) 来解决混合精确近似对角化 (HEAD) 问题。IPA采用乘法更新形式,即将当前分离矩阵乘以单位矩阵的秩2扰动。此方法对解混过滤器执行联合更新,并对解混矩阵的其余部分执行其他排名一更新。简单地说,IPA优化规则是IP和ISS方法的组合。在每次更新中更新矩阵的一行和一列,同时执行 IP 和 ISS 样式的更新,优于 IP 和 ISS 方法。(IPA) and a Newton conjugate gradient (NCG) to solve the hybrid exact-approximate diagonalization (HEAD) problem. IPA adopts a multiplicative update form, that is, the current separation matrix is multiplied by the rank 2 perturbation of the identity matrix. This method performs joint updates to the unmixing filters and additional rank-one updates to the remainder of the unmixing matrix. Simply put, the IPA optimization rule is a combination of IP and ISS methods. Updating one row and one column of the matrix in each update, performing IP- and ISS-style updates jointly, outperforms the IP and ISS methods.
6.2. Iterative 迭代源控制Source Steering
ISS
[67][57] is an alternative 是to IP
的替代品。虽然IP具有性能好、收敛速度快等优点,但在迭代更新过程中,需要重新计算协方差矩阵,并针对每个源和每次迭代进行反转。这大大增加了算法的整体复杂性。该算法的复杂性是所用麦克风数量的三倍。除此之外,反转矩阵本质上是一种危险的操作,可能导致迭代时收敛不稳定。在此基础上,所提出的ISS算法可以有效降低IP算法带来的计算成本和复杂度。ISS还可以最小化与. Although IP has the advantages of good performance and fast convergence speed, in the iterative update process, it needs to recalculate a covariance matrix and invert for each source and each iteration. This greatly increases the overall complexity of the algorithm. The complexity of the algorithm is three times the number of microphones used. In addition to that, inverting a matrix is an inherently dangerous operation that can lead to unstable convergence when iterating. On this basis, the proposed ISS algorithm can effectively reduce the computational cost and complexity brought by the IP algorithm. ISS can also minimize the same cost function as the AuxIVA
算法相同的成本函数。 algorithm.
此更新规则不需要矩阵反演,用于联合冗余和This update rule, which does not require matrix inversion, is used in a new method for joint deredundancy and BSS
的新方法[68]。该方法基于 [58]. This is a method based on an ILRMA
框架,结合了 framework, which combines the advantages of no inversion and low complexity of the ISS
算法无反演、复杂度低等优点,实现了高效的BSS。2021年,D algorithm to achieve efficient BSS. In 2021, Du et al. [59] proposed a compu
等人[69]提出了一种计算高效的超定混合物tationally efficient optimization algorithm for BSS
优化算法,一种用于 of overdetermined mixtures, an improved ISS algorithm for OverIVA
算法的改进ISS算法,即 algorithm, namely OverIVA-ISS
。该算法将. The algorithm combines the technology in OverIVA-IP
中的技术与 with the technology in AuxIVA-ISS
中的技术相结合,比, which is more computationally efficient than the OverIVA-IP
算法计算效率更高,可以保证收敛性。此外,计算复杂度从 algorithm and can guarantee convergence. Additionally, the computational complexity is reduced from O(M 2)降低到(M2) to O((MN)。).
The overall performance of the ISS
算法的整体性能优于IP算法,但不如IP-2算法。因此,提出了一种ISS-2算法。2022 年,池下等人 algorithm is better than the IP algorithm but inferior to the IP-2 algorithm. Therefore, an ISS-2 algorithm is proposed. In 2022, Ikeshita et al. [60] [70]extended 将the ISS
算法扩展到algorithm to ISS-2
。.
同时,保持了At the same time, the advantage of the smaller time complexity of the ISS
算法时间复杂度较小的优势,分离性能可与IP-2相媲美。 algorithm is maintained, and the separation performance is comparable to IP-2.
7. EVD 埃博拉病毒病方法Method
埃博拉病毒病方法是找到与原始基质最相似的矩阵。基于埃博拉病毒病的优化更新规则可以表示为:The EVD method is to find the most similar matrix to the original matrix. The optimization update rule based on EVD can be expressed as:
和and
其中where λM Mand 和 uM 分别表示最小特征值和特征向量。denote the smallest eigenvalue and eigenvector, respectively.
[11]中提出了基于The IVA algorithm based on the EVD
更新规则的IVA算法,用于快速独立载体提取( update rule was proposed in [61] for a fast independent vector extraction (FIVE
)算法。通过实验与) algorithm. By comparing with the OverIVA
和 and AuxIVA
算法的实验比较,所提算法只需几次迭代即可获得最优解,在收敛性能上远优于其他算法。2021年, algorithms experimentally, the proposed algorithm can obtain the optimal solution with only a few iterations and is far superior to other algorithms in terms of convergence performance. In 2021, Brendel
等人[71]将特征值分解的更新规则扩展到具有 et al. [62] extended the update rule of eigenvalue decomposition to an IVA source extraction algorithm with SOI
机制的 mechanism. The proposed update rule achieves fast convergence at lower computational cost and outperforms the I
VA源提取算法。该更新规则以较低的计算成本实现了快速收敛,在性能上优于IP更新规则。
P update rule in performance.