You're using an outdated browser. Please upgrade to a modern browser for the best experience.
Optimization of Independent Vector Analysis Algorithm
Edit
This entry is adapted from the peer-reviewed paper 10.3390/s23010493

With the advent of the era of big data information, artificial intelligence (AI) methods have become extremely promising and attractive. It has become extremely important to extract useful signals by decomposing various mixed signals through blind source separation (BSS). BSS has been proven to have prominent applications in multichannel audio processing. For multichannel speech signals, independent component analysis (ICA) requires a certain statistical independence of source signals and other conditions to allow blind separation. independent vector analysis (IVA) is an extension of ICA for the simultaneous separation of multiple parallel mixed signals. IVA solves the problem of arrangement ambiguity caused by independent component analysis by exploiting the dependencies between source signal components and plays a crucial role in dealing with the problem of convolutional blind signal separation.

blind source separation (BSS) independent vector analysis (IVA) optimization update rule

1. Introduction

With the advent of the era of big data information, people’s access to information has become more and more abundant. However, researchers usually only obtain the mixed information collected from the receiver, and the whole mixed information needs to be separated or extracted from the latent signals. The subsequent problem is how to effectively obtain useful signals from the received signals, which leads to the technology related to blind source separation (BSS) [1].
The theory of BSS can be traced back to the cocktail party problem, which has attracted much attention for decades. The cocktail party problem is when you are at a cocktail party and there are all kinds of people chatting around, but you can only concentrate on one of the discussions, or focus on the conversation of one of the people. BSS theory refers to observing the mixed signals of different sources and using these mixed signals to restore the original signal, and the prior information of the source signal and its mixed signal is minimal. A large number of applications of BSS in communication, speech, and medical signal processing has received extensive attention in recent years [2]. It is of great significance to realize blind estimation, blind equalization, and adaptive signal processing through blind characteristics.
Independent component analysis [3][4][5] (ICA) is one of the most important methods first proposed to deal with BSS. This is a classic BSS technology based on statistical independence of source signals and is the mainstream technology of BSS. ICA requires that source signals be statistically independent of each other. It is an unsupervised, data-driven signal processing technique based on non-Gaussian maximization to separate time-invariant mixture signals in the time domain.
However, consider that in a real scenario, the signal is often mixed with reverberation in the form of convolution. However, ICA cannot separate the common form of convolution mixing. Moreover, the convolution mixed signal is processed in the time domain with high computational complexity and a huge amount of computation, and the convergence speed is slow, which greatly reduces the separation performance. Taking advantage of the properties of convolution mixing: the convolution in the time domain is equal to the product in the frequency domain, a frequency domain ICA [6][7] (FD-ICA) algorithm is proposed. The entire convolutional mixed signal is converted from the time domain to the frequency domain for separation by the short-time Fourier transform (STFT). Compared with the time-domain convolution operation, the frequency-domain product operation has the advantages of convenient calculation, small computational complexity, and fast convergence speed.
To solve the above-mentioned problems of ICA, the independent vector analysis (IVA) [8][9] algorithm is proposed. It generalizes ICA to multiple datasets by exploiting statistical dependencies across datasets, addressing some of the uncertainty in the output of signal separation.

2. Optimizing IVA Algorithm—Optimizing Update Rules

GD [10] is one of the most primitive optimization algorithms. Gradient descent is a method that minimizes I by updating the model parameter in the opposite direction of the gradient of the objective function I. The learning rate η determines the size of the step size chosen to reach the local minimum, in other words, the descending hill along the slope of the surface produced by the objective function until a valley is reached. This is a separation method obtained by minimizing (1), a simple GD method is extrapolated as follows:
ΔW(k)=−∂I∂W(k)
(1)
Its main variants are batch gradient (BG), stochastic gradient (SG), and natural gradient (NG). Among them, the NG algorithm [11][12] is an effective and one of the most commonly used algorithms to solve the problem of BSS. The main idea is to take the NG direction of the objective function I as the iterative direction so that the algorithm can quickly converge, so as to realize the separation of source signals. Additionally, it is proved that the best descent direction is not the “negative” regular gradient direction but the "negative" Riemann gradient. It was first proposed in [13][14], and its main idea is to multiply the scaling matrix Q(k) to modify the gradient in the original GD method to obtain faster convergence speed. As Equation (2):
ΔW(k)=−∂I∂W(k)Q(k)
(2)
The update for the separation matrix is:
W(k)←W(k)+ηΔW(k)
(3)
 when solving the objective function. The choice of step size will directly affect the convergence speed and accuracy. In order to speed up the convergence speed of the algorithm, many scholars have also optimized and improved the classical NG algorithm. In 2011, Liang et al. [15] proposed a control mechanism that considers the step size to obtain fast and stable convergence. In 2011, Zhang et al. [16] proposed an NG blind separation algorithm that directly estimates the score function through function approximation, which uses a linear combination of a set of orthogonal polynomials to approximate the score function, and its performance is measured by the mean squared error. An improved momentum term method was proposed in [17] which can speed up the algorithm’s convergence.
In 2018, Fu et al. [18] proposed a blind separation algorithm for IVA based on step-size adaptation. The algorithm initializes the separation matrix using the feature matrix joint approximate diagonalization algorithm and adaptively optimizes the step-size parameter. That is, to avoid local convergence, it can also significantly improve the convergence speed of the algorithm and further improve the separation performance. According to the relationship between the iteration step size and the estimated cost function change. In 2012, Wang et al. [19] proposed a variable-step-size IVA gradient algorithm based on the most block speed step-size descent. Additionally, according to the relationship between the iterative step size and the change in the separation matrix to be obtained, a variable-step-size IVA gradient algorithm based on the estimation function is proposed. In 2010, Kim [12] proposed a modified gradient and normalized IVA method with nonfully closed constraints. Gradient normalization improves the convergence speed, and nonholographically constrained gradients with lower computational complexity show better performance, while possessing simpler structures compared with other methods. In 2018, Koldovský et al. [20], based on the independent vector extraction (IVE) of the IVA algorithm, proposed an IVE algorithm with an adaptive step-size method in complex non-Gaussian scenarios to speed up convergence.

3. Fast Fixed Point Method

The fast fixed point method was derived by introducing Newton’s method. The iterative update rule based on fast fixed point [21] was first proposed to optimize the objective function of ICA. It provides a very simple algorithm, one that does not depend on any defined parameters and that quickly converges to the most accurate update rule the data allow.
When optimizing a negative entropy-based objective function, the easiest way is to use GD. Although the GD-based method has a good separation effect, it is relatively simple to use. The overall convergence speed of this method is slow and depends on a good choice of the learning rate sequence, i.e., the step size per iteration. Although various optimizations for the step-size factor were summarized in the previous section, GD methods rely on a suitable step size for separation.
Therefore, in practical applications, it is very important to make the entire convergence process faster and more reliable. Therefore, a fast fixed point iterative algorithm [22] is proposed to achieve this. In fixed point algorithms, the entire computation is performed in batch or block mode, i.e., a large number of data points are used in one step of the algorithm. The fast fixed point algorithm has very attractive convergence properties, and in experiments, it converges much faster than the commonly used GD method. At the same time, in environments where fast real-time adaptation is not required, this method is a good alternative to adaptive learning rules. In 1997, Hyvarinen [23] described a more heuristic derivation of it.
In 2000, Bingham et al. [24] proposed a FastICA algorithm capable of separating complex-valued linear mixed-source signals. The method shows good performance in the ICA algorithm. The same [25] generalized fast fixed point method to the IVA algorithm, which was developed based on the idea of FastICA and used to optimize the traditional IVA algorithm. Under this method, the update is expressed as:

wn(k)←E[G′(∑k|yn(k)|2)+|yn(k)|2G″(∑k|yn(k)|2)]wn(k)−E[(yn(k))*G′(∑k|yn(k)|2)x(k)]
(4)
where E denotes the expectation, G(⋅) denotes a nonlinear function, and 
G(∑k|yn(k)|2)=−logg∧sn(yn)
(5)
After the updated matrix W is obtained through the update rule, decorrelation needs to be performed to ensure orthogonality as follows:
W[k]←W[k]W[k]H−1/2W[k]
(6)
where (⋅)H denotes the conjugate transpose of (⋅). To be able to directly apply Newton’s method to derive a fast algorithm for complex variables, a quadratic Taylor polynomial is introduced into the complex notation. Using this form of Taylor series expansion makes the derivation simpler and is useful for directly applying Newton’s method to objective functions of complex-valued variables. In 2000, Yan et al. [26] provided an independent equivalent.
Recently, in 2021, Koldovský et al. [27] proposed an extended fast dynamic independent vector analysis (FastDIVA) algorithm based on the FastICA and FastIVA static hybrid algorithms, used to blindly extract or separate one or more signal sources from a time-varying mixed signal. In a source-by-source separation mixture model that allows the desired source to move, the mixture is either in series or in parallel. The algorithm inherits the advantages of FastIVA, exhibits good performance in motion source separation, and exhibits superior convergence speed and ability to separate super-Gaussian and sub-Gaussian signals.
In 2021, Amor et al. [28] used FastDIVA for blind source extraction for mixture models with constant separation vector CSV. Additionally, it shows new potential and good separation performance in three environments: motion loudspeaker in a noisy environment, extraction of motion brain activity, and motion source. In 2021, Koldovský et al. [29] proposed a new dynamic IVA algorithm. It is based on a mixed model in which the source-of-interest (SOI)-related mixing parameters are time-varying, and the separation parameters are time-invariant. The Newton–Raphson method is used to optimize the objective function based on the quasi-likelihood method, then the iterative update is performed without imposing orthogonality constraints, and then orthogonality is performed. This algorithm is an optimization of the fast fixed point algorithm, which is better than the gradient algorithm and the auxiliary function method in performance.

4. Auxiliary Function

The update method based on the auxiliary function technology is also a method that does not include tuning parameters such as step size, which is an iterative algorithm with a convergence guarantee. This is a stable and fast update rule derived from the majorize-minimization principle [30][31]. Find its minimum by exploiting the convexity of the function. When the objective function f(θ)
is difficult to optimize, and the optimization algorithm used cannot directly find the optimal solution to the objective function, an easy-to-optimize objective function g(θ) can be found instead. Then, the substitution function is solved, and the optimal solution of g(θ) is close to the optimal solution of f(θ). In each iteration, a new surrogate function for the next iteration is reconstructed from the solution. Then, the new substitute function is optimized and solved to obtain the objective function of the next iteration. After several iterations, the optimal solution that is closer and closer to the original objective function that can be obtained. It was first proposed in the literature [32] to accelerate the convergence speed of the ICA algorithm. This rule consists of two optional updates:
  • The update of the weighted covariance matrix (that is, the auxiliary function variable).
  • The update of the separation matrix ensures that the objective function decreases monotonically at each update and finally achieves convergence.

Equation (7) is the auxiliary function variable update:

Vn=En[U′(‖yn‖2)‖yn‖2xn(xn)H]
(7)

Among them, Vn denotes a covariance matrix of the observed signals, U(⋅) denotes a continuous and differentiable function of a real variable · satisfying, and U′(⋅) usually takes the constant 1. ∥⋅∥2 denotes the 2-norm of ·. Equation (8) is the update of the unmixing matrix:

wn(k)=[WVn]−1enenT(Wn−HVn−1Wn−1)en
(8)

In 2011, Nobutaka Ono [33] used the auxiliary function technique in the objective function of the IVA algorithm and similarly derived an efficient update rule suitable for the IVA algorithm, called AuxIVA. In 2012, Nobutaka Ono [34] proposed an AuxIVA algorithm based on a generalized Gaussian source model or a Gaussian source model with time-varying variance. In 2012 and 2013, Nobutaka Ono [35][36] proposed a faster algorithm that can update two separation vectors simultaneously by solving the generalized eigenvalue problem for the AuxIVA algorithm with two sources and two microphones. Compared with the one-by-one update method, this method has faster convergence speed and better performance. This pairwise update method is also applicable to the pairwise separation of vectors in the case of three or more sources [37]. In 2014, Taniguchi et al. [38] used the AuxIVA algorithm based on the auxiliary function method for online real-time blind speech separation. In experimental comparisons with commonly used real-time IVA algorithms, the proposed online algorithm achieves a higher signal-to-noise ratio without environment-sensitive tuning parameters such as step factor.
In 2021, Brendel et al. [39] further optimized the IVA algorithm based on auxiliary functions under the same computational cost. The convergence speed of the AuxIVA algorithm is enhanced by three methods:
  • Turn the differential term into a tuning parameter via the differential term in the NG approximation algorithm.
  • Approximate the differential term as a matrix using the quasi-Newton method.
  • Use the square iteration method to speed it up.

5. EM Method

In signal processing, a common problem is estimating the parameters of a probability distribution function. The situation is more complicated in many parameter estimation problems because the data needed to estimate the parameters are not directly accessible, or some data are missing. EM-based optimization algorithms are well-suited for solving this class of problems because the EM algorithm produces maximum likelihood (ML) estimates of the parameters when there is a many-to-one mapping from the underlying distribution to the distribution of the control observations, while taking additive noise into account. The EM algorithm overcomes the problem of unanalyzable solutions and has been widely used in statistics, signal processing, and machine learning [40].
The EM algorithm is an iterative optimization method [41] that is used to estimate some unknown parameters given measurement data. The solution is divided into two steps.
E-step: First assign an initial distribution to each hidden variable empirically, that is, assume distribution parameters. Then, according to the parameters of the distribution, the expectation of the hidden variables in each data tuple can be obtained, that is, the classification operation is performed. The posteriors of the source signal can be obtained by
logq(x1(k),…,xN(k)|s1(k),…,sN(k))∝logg(y1(k),…,yN(k)|x1(k),…,xN(k))+(logg(x1(k)|s1(k))+…+logg(xN(k)|sN(k)))+const.
(9)
where ∝ denotes it is proportional to the previous term, and q denotes posterior probability.
M-step: Calculate the maximum likelihood value of the distribution parameter (vector) based on the classification result, and then in turn recalculate the expectation of the hidden variable for each data tuple based on this maximum likelihood value. The update rules for mixing matrices A are 
A(k)=(∑k<y(k)(x(k))T>q)(∑k<x(k)(x(k))T>q)−1
(10)
where <⋅>q denotes expectation over q.
Through the repetition of the above two steps, when the expectation of the hidden variable and the maximum likelihood value of the parameter tends to be stable, the entire iteration is completed.
In 2004 and 2008, Varadhan et al. [42][43] used the square iteration method in the EM algorithm to accelerate its convergence speed. In 2008, Lee et al. [44] deduced the expectation-maximization algorithm, and the algorithm was used in the updated iteration of the IVA algorithm. The EM algorithm could estimate the parameters of the separation matrix and the unknown source at the same time, showing a good separation performance. In 2010, Hao et al. [45] proposed a unified probabilistic framework for the IVA algorithm with the Gaussian mixture model as the source prior model; this flexible prior source enables the IVA algorithm to separate different types of signals, deduce different EM algorithms, and test three models: noiseless IVA, online IVA, and noise IVA. The EM algorithm can effectively estimate the unmixing matrix without sensor noise. In online IVA, an online EM algorithm is derived to track the motion of the source under nonstationary conditions. Noise IVA includes sensor noise and denoising combined with separation. An EM algorithm suitable for this model is proposed which can effectively estimate the model parameters and separate the source signal at the same time.
In 2019, Gu et al. [46] proposed a Gaussian mixture model IVA algorithm with time-varying parameters to accommodate temporal power fluctuations embedded in nonstationary speech signals, thus avoiding the pretraining process of the original Gaussian mixture model IVA (GMM-IVA) algorithm and using the corresponding improved EM algorithm to estimate the separation matrix and signal model. The experimental results confirm the effectiveness of the method in random initialization and the advantages in separation accuracy and convergence speed. In 2019, Rafique et al. [47] proposed a new IVA algorithm based on Student’s t-mixture model as a source before adapting to the statistical properties of different speech sources. At the same time, an efficient EM algorithm is derived which estimates the location parameters of the source prior matrix and the decomposition matrix together, thereby improving the separation performance of the IVA algorithm. In 2020, Tang et al. [48] proposed a complex generalized Gaussian mixture distribution with weighted variance to capture the non-Gaussian and nonstationary properties of speech signals to flexibly characterize real speech signals. At the same time, the optimization rules based on the EM method are used to estimate and update the mixing parameters.

6. BCD Method

Coordinate descent (CD) is a nongradient optimization algorithm. The algorithm does not need to calculate the gradient of the objective function and performs a linear search along a single dimension at a time. When a minimum value of the current dimension is obtained, different dimension directions are used repeatedly, and the optimal solution is finally converged. However, this algorithm is only suitable for smooth functions. When nonsmooth functions are used, they may fall into a nonstagnant point and fail to converge. In 2015, Wright [49] proposed block coordinate descent (BCD), a generalization of the coordinate descent algorithm. It decomposes the original problem into multiple subproblems by simultaneously optimizing a subset of variables. The order of updates during the descent can be deterministic or random. This algorithm is mainly used to solve the nonconvex function, of which the objective function’s global optimal value is difficult to obtain.
Among them, the BCD algorithm has developed two methods with closed update formula for the BSS IVA algorithm’s [50] IP and ISS methods.

6.1. Iterative Projection

The IVA algorithm based on iterative projection was first introduced in the AuxIVA [33] algorithm.
This update rule is derived by solving a quadratic system of equations obtained by differentiating the cost function concerning the separation vector. In 2004, Dégerine et al. [51] also proposed a similar scheme in the context of semiblind Gaussian source components. In 2016, Kitamura et al. [52] used the IP algorithm in a BSS algorithm combining IVA and NMF, which provided good convergence speed and separation effect. In 2018, Yatabe et al. [53] proposed an alternative to the AuxIVA-IP algorithm based on proximal splitting. In 2021, Nakashima et al. [54] optimized it based on IP and extended each row vector of the separation matrix to update one by one to two rows of the separation matrix per update, resulting in a faster IP-2.
In 2020, Ikeshita et al. [55] deduced IP-1 and IP-2 and used these two update rules to accelerate the OverIVA algorithm, forming the OverIVA-IP and OverIVA-IP2 update rules. In 2021, Scheibler [56] proposed an iterative projection with adjustment (IPA) and a Newton conjugate gradient (NCG) to solve the hybrid exact-approximate diagonalization (HEAD) problem. IPA adopts a multiplicative update form, that is, the current separation matrix is multiplied by the rank 2 perturbation of the identity matrix. This method performs joint updates to the unmixing filters and additional rank-one updates to the remainder of the unmixing matrix. Simply put, the IPA optimization rule is a combination of IP and ISS methods. Updating one row and one column of the matrix in each update, performing IP- and ISS-style updates jointly, outperforms the IP and ISS methods.

6.2. Iterative Source Steering

ISS [57] is an alternative to IP. Although IP has the advantages of good performance and fast convergence speed, in the iterative update process, it needs to recalculate a covariance matrix and invert for each source and each iteration. This greatly increases the overall complexity of the algorithm. The complexity of the algorithm is three times the number of microphones used. In addition to that, inverting a matrix is an inherently dangerous operation that can lead to unstable convergence when iterating. On this basis, the proposed ISS algorithm can effectively reduce the computational cost and complexity brought by the IP algorithm. ISS can also minimize the same cost function as the AuxIVA algorithm. 
W(k)←W(k)−vn(k)(wn(k))H
(11)
This update rule, which does not require matrix inversion, is used in a new method for joint deredundancy and BSS [58]. This is a method based on an ILRMA framework, which combines the advantages of no inversion and low complexity of the ISS algorithm to achieve efficient BSS. In 2021, Du et al. [59] proposed a computationally efficient optimization algorithm for BSS of overdetermined mixtures, an improved ISS algorithm for OverIVA algorithm, namely OverIVA-ISS. The algorithm combines the technology in OverIVA-IP with the technology in AuxIVA-ISS, which is more computationally efficient than the OverIVA-IP algorithm and can guarantee convergence. Additionally, the computational complexity is reduced from O(M2) to O(MN).
The overall performance of the ISS algorithm is better than the IP algorithm but inferior to the IP-2 algorithm. Therefore, an ISS-2 algorithm is proposed. In 2022, Ikeshita et al. [60] extended the ISS algorithm to ISS-2.
At the same time, the advantage of the smaller time complexity of the ISS algorithm is maintained, and the separation performance is comparable to IP-2.

7. EVD Method

The EVD method is to find the most similar matrix to the original matrix. The optimization update rule based on EVD can be expressed as:
w[k]←w[k]‖w[k]‖2
(12)
and
w(k)=1λM(k)uM(k)
(13)
where λM and uM denote the smallest eigenvalue and eigenvector, respectively.
The IVA algorithm based on the EVD update rule was proposed in [61] for a fast independent vector extraction (FIVE) algorithm. By comparing with the OverIVA and AuxIVA algorithms experimentally, the proposed algorithm can obtain the optimal solution with only a few iterations and is far superior to other algorithms in terms of convergence performance. In 2021, Brendel et al. [62] extended the update rule of eigenvalue decomposition to an IVA source extraction algorithm with SOI mechanism. The proposed update rule achieves fast convergence at lower computational cost and outperforms the IP update rule in performance.

References

  1. Cao, X.-R.; Liu, R. General approach to blind source separation. IEEE Trans. Signal Process. 1996, 44, 562–571.
  2. Luo, Z.; Li, C.; Zhu, L. A comprehensive survey on blind source separation for wireless adaptive processing: Principles, perspectives, challenges and new research directions. IEEE Access 2018, 6, 66685–66708.
  3. Hyvärinen, A.; Oja, E. Independent component analysis: Algorithms and applications. Neural Netw. 2000, 13, 411–430.
  4. Stone, J.V. Independent component analysis: An introduction. Trends Cogn. Sci. 2002, 6, 59–64.
  5. Torkkola, K. Blind separation of convolved sources based on information maximization. In Proceedings of the 1996 IEEE Signal Processing Society Workshop Neural Networks for Signal Processing VI, Kyoto, Japan, 4–6 September 1996.
  6. Sawada, H.; Araki, S.; Makino, S. Measuring dependence of bin-wise separated signals for permutation alignment in frequency-domain BSS. In Proceedings of the 2007 IEEE International Symposium on Circuits and Systems, New Orleans, LA, USA, 27–30 May 2007.
  7. Nesta, F.; Svaizer, P.; Omologo, M. Convolutive BSS of short mixtures by ICA recursively regularized across frequencies. IEEE Trans. Audio Speech Lang. Process. 2010, 19, 624–639.
  8. Kim, T.; Eltoft, T.; Lee, T. Independent vector analysis: An extension of ICA to multivariate components. In International Conference on Independent Component Analysis and Signal Separation; Springer: Berlin/Heidelberg, Gremany, 2006.
  9. Kim, T.; Lee, I.; Lee, T. Independent vector analysis: Definition and algorithms. In Proceedings of the 2006 Fortieth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA, 29 October–1 November 2006.
  10. Ruder, S. An overview of gradient descent optimization algorithms. arXiv 2016, arXiv:1609.04747.
  11. Kim, T. Real-time independent vector analysis for convolutive blind source separation. IEEE Trans. Circuits Syst. I Regul. Pap. 2010, 57, 1431–1438.
  12. Kim, T.; Attias, H.T.; Lee, S.Y.; Lee, T.W. Blind source separation exploiting higher-order frequency dependencies. IEEE Trans. Audio Speech Lang. Process. 2006, 15, 70–79.
  13. Amari, S.; Cichocki, A.; Yang, H. A new learning algorithm for blind signal separation. Adv. Neural Inf. Process. Syst. 1995, 8, 757–763.
  14. Cardoso, J.-F.; Laheld, B. Equivariant adaptive source separation. IEEE Trans. Signal Process. 1996, 44, 3017–3030.
  15. Liang, Y.; Naqvi, S.M.; Jonathon, A. Chambers. Adaptive step size independent vector analysis for blind source separation. In Proceedings of the 2011 17th International Conference on Digital Signal Processing (DSP), Corfu, Greece, 6–8 July 2011.
  16. Zhang, Y.; Lou, S.; Zhang, W.; Chang, H. Blind source separation algorithm of natural gradient based on estimation of score function. Shuju Caiji Chuli/Journal Data Acquis. Process. 2011, 26, 167–171.
  17. Das, N.; Tripathy, B.; Patnaik, S. Adaptive fast algorithm based on natural gradient for instantaneous blind source separation. Int. J. Inf. Commun. Technol. 2016, 8, 307–314.
  18. Fu, W.; Cong, Z. Convolutional Blind Separation Algorithm Based on Independent Vector Analysis Based on Adaptive Step Size. J. Electron. Inf. 2018, 40, 2158–2164.
  19. Wang, C. Research on Blind Separation of Speech Mixed Signal. Ph.D. Dissertation, University of Electronic Science and Technology of China, Chengdu, China, 2012.
  20. Koldovský, Z.; Tichavský, P. Gradient algorithms for complex non-gaussian independent component/vector extraction, question of convergence. IEEE Trans. Signal Process. 2018, 67, 1050–1064.
  21. Hyvärinen, A.; Oja, E. A fast fixed-point algorithm for independent component analysis. Neural Comput. 1997, 9, 1483–1492.
  22. Hyvarinen, A. Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 1999, 10, 626–634.
  23. Hyvarinen, A. A family of fixed-point algorithms for independent component analysis. In Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, Munich, Germany, 21–24 April 1997; Volume 5.
  24. Bingham, E.; Hyvärinen, A. A fast fixed-point algorithm for independent component analysis of complex valued signals. Int. J. Neural Syst. 2000, 10, 1–8.
  25. Lee, I.; Kim, T.; Lee, T. Fast fixed-point independent vector analysis algorithms for convolutive blind source separation. Signal Process. 2007, 87, 1859–1871.
  26. Yan, G.; Fan, H. A Newton-like algorithm for complex variables with applications in blind equalization. IEEE Trans. Signal Process. 2000, 48, 553–556.
  27. Koldovský, Z.; Kautský, V.; Tichavský, P.; Čmejla, J.; Málek, J. Dynamic independent component/vector analysis: Time-variant linear mixtures separable by time-invariant beamformers. IEEE Trans. Signal Process. 2021, 69, 2158–2173.
  28. Amor, N.; Čmejla, J.; Kautský, V.; Koldovský, Z.; Kounovský, T. Blind extraction of moving sources via independent component and vector analysis: Examples. In Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–11 June 2021.
  29. Koldovský, Z.; Kautský, V.; Kounovský, T.; Čmejla, J. Algorithm for independent vector extraction based on semi-time-variant mixing model. In Proceedings of the 2021 29th European Signal Processing Conference (EUSIPCO), Dublin, Ireland, 23–27 August 2021.
  30. Brendel, A.; Haubner, T.; Kellermann, W. Spatially guided independent vector analysis. In Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020.
  31. Lange, K. MM Optimization Algorithms. Available online: https://epubs.siam.org/doi/book/10.1137/1.9781611974409 (accessed on 15 November 2022).
  32. Ono, N.; Miyabe, S. Auxiliary-function-based independent component analysis for super-Gaussian sources. In International Conference on Latent Variable Analysis and Signal Separation; Springer: Berlin/Heidelberg, Germany, 2010.
  33. Ono, N. Stable and fast update rules for independent vector analysis based on auxiliary function technique. In Proceedings of the 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), Paltz, NY, USA, 16–19 October 2011.
  34. Ono, N. Auxiliary-function-based independent vector analysis with power of vector-norm type weighting functions. In Proceedings of the 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hollywood, CA, USA, 3–6 December 2012.
  35. Ono, N. Fast stereo independent vector analysis and its implementation on mobile phone. In Proceedings of the IWAENC 2012 International Workshop on Acoustic Signal Enhancement, Aachen, Germany, 4–6 September 2012.
  36. Ono, N. Blind source separation on iPhone in real environment. In Proceedings of the 21st European Signal Processing Conference (EUSIPCO 2013), Marrakech, Morocco, 9–13 September 2013.
  37. Ono, N. Fast algorithm for independent component low-rank matrix analysis with three or more sources. In Proceedings of the 2018 Spring Meeting Acoustical Society of Japan, Oita, Japan, 15 March 2018; pp. 437–438.
  38. Taniguchi, T.; Ono, N.; Kawamura, A.; Sagayama, S. An auxiliary-function approach to online independent vector analysis for real-time blind source separation. In Proceedings of the 2014 4th Joint Workshop on Hands-Free Speech Communication and Microphone Arrays (HSCMA), Villers-les-Nancy, France, 12–14 May 2014.
  39. Brendel, A.; Kellermann, W. Accelerating auxiliary function-based independent vector analysis. In Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–11 June 2021.
  40. Bishop, C.M.; Nasser, M.N. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2006; Volume 4.
  41. Moon, T.K. The expectation-maximization algorithm. IEEE Signal Process. Mag. 1996, 13, 47–60.
  42. Varadhan, R.; Roland, C. Squared Extrapolation Methods (SQUAREM): A New Class of Simple and Efficient Numerical Schemes for Accelerating the Convergence of the EM Algorithm; Working Paper 63; Johns Hopkins University, Dept. of Biostatistics Working Papers: Baltimore, MD, USA, 2004.
  43. Varadhan, R.; Roland, C. Simple and globally convergent methods for accelerating the convergence of any EM algorithm. Scand. J. Stat. 2008, 35, 335–353.
  44. Lee, I.; Hao, J.; Lee, T. Adaptive independent vector analysis for the separation of convoluted mixtures using EM algorithm. In Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA, 31 March–4 April 2008.
  45. Hao, J.; Lee, I.; Lee, T.W.; Sejnowski, T.J. Independent vector analysis for source separation using a mixture of Gaussians prior. Neural Comput. 2010, 22, 1646–1673.
  46. Gu, Z.; Lu, J.; Chen, K. Speech Separation Using Independent Vector Analysis with an Amplitude Variable Gaussian Mixture Model. In Proceedings of the Interspeech 2019, Graz, Austria, 15–19 September 2019; pp. 1358–1362.
  47. Rafique, W.; Chambers, J.; Sunny, A.I. An expectation–maximization-based IVA algorithm for speech source separation using student’st mixture model based source priors. Acoustics 2019, 1, 117–136.
  48. Tang, X.; Chen, R.; Wang, X.; Zhou, Y.; Su, D. Independent Vector Analysis for Blind Speech Separation Using Complex Generalized Gaussian Mixture Model with Weighted Variance. In Proceedings of the 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Auckland, New Zealand, 7–10 December 2020.
  49. Wright, S.J. Coordinate descent algorithms. Math. Program. 2015, 151, 3–34.
  50. Wright, S.; Nocedal, J. Numerical optimization. Science 1999, 35, 67–68.
  51. Dégerine, S.; Zaïdi, A. Separation of an instantaneous mixture of Gaussian autoregressive sources by the exact maximum likelihood approach. IEEE Trans. Signal Process. 2004, 52, 1499–1512.
  52. Kitamura, D.; Ono, N.; Sawada, H.; Kameoka, H.; Saruwatari, H. Determined blind source separation unifying independent vector analysis and nonnegative matrix factorization. IEEE/ACM Trans. Audio Speech Lang. Process. 2016, 24, 1626–1641.
  53. Yatabe, K.; Kitamura, D. Determined blind source separation via proximal splitting algorithm. In Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 15–20 April 2018.
  54. Nakashima, T.; Scheibler, R.; Wakabayashi, Y.; Ono, N. Faster independent low-rank matrix analysis with pairwise updates of demixing vectors. In Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Dublin, Ireland, 18–21 January 2021.
  55. Ikeshita, R.; Nakatani, T.; Araki, S. Overdetermined independent vector analysis. In Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020.
  56. Scheibler, R. Independent vector analysis via log-quadratically penalized quadratic minimization. IEEE Trans. Signal Process. 2021, 69, 2509–2524.
  57. Scheibler, R.; Ono, N. Fast and stable blind source separation with rank-1 updates. In Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020.
  58. Nakashima, T.; Scheibler, R.; Togami, M.; Ono, N. Joint dereverberation and separation with iterative source steering. In Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–11 June 2021.
  59. Du, Y.; Scheibler, R.; Togami, M.; Yoshii, K.; Kawahara, T. Computationally-efficient overdetermined blind source separation based on iterative source steering. IEEE Signal Process. Lett. 2021, 29, 927–931.
  60. Ikeshita, R.; Nakatani, T. ISS2: An Extension of Iterative Source Steering Algorithm for Majorization-Minimization-Based Independent Vector Analysis. arXiv 2022, arXiv:2202.00875.
  61. Scheibler, R.; Ono, N. Fast independent vector extraction by iterative SINR maximization. In Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020.
  62. Brendel, A.; Kellermann, W. Informed source extraction based on independent vector analysis using eigenvalue decomposition. In Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Dublin, Ireland, 18–21 January 2021.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Automation & Control Systems
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Ruiming Guo
,
Zhongqiang Luo
,
Mingchun Li
View Times: 982
Revisions: 6 times (View History)
Update Date: 02 Feb 2023
Table of Contents
Academic Video Service
Feedback