Guoqiang Zhang received his Bachelor degree from the University of Science and Technology of China (USTC) in 2003, and a master of philosophy (M.Phil.) from the Unversity of Hong Kong in 2006. He studied at the Royal Institute of Technology – KTH and obtained his PhD degree in 2010. He then worked as a post-doctoral researcher at Delft University of Technology full time until the end of 2014 and part time until the end of 2016. From 2015 to 2016, he worked as a senior researcher at Ercisson AB. He is now working as a senior lecturer at the University of Technology Sydney. His research interests include large scale optimisation, multimedia processing and machine learning.
- Camera-Radar Fusion Using Deep Learning (cooperation with Qamcom): Information fusion of multiple sensors brings robustness and high performance by combining the merits of different sensors. This project focuses on camera-radar fusion. In the project, the deep learning framework will be used to simutenously process video and radar signals for reliable object detection, classification, and tracking under various weather conditions.
Can supervise: YES
Distributed and Parallel Computing
- Algorithm Design and Theoretical Analysis
- Distributed Processing over Wireless Sensor Networks
- Supervised Learning of Neural Networks by Optimisation over Graphic Models
- Iterative Channel Decoding over Graphic Models
- Simultaneous Localisation and Mapping (SLAM) by Optimisation over Graphic Models
- Nonnegative Matrix Factorization by Optimisation over Graphic Models
Speech and Image Processing
- Speech packet loss concealment
- Adaptive jitter buffer control
- Multi-view camera geometry
- Camera Calibration
- Multiple description coding
- Multi-terminal source coding
PhD scholarships are available on distributed signal processing
- Distributed Signal Processing
- Information Theory and Source Coding
LU, S, Oberst, S, Zhang, G & Luo, Z 2019, 'Bifurcation analysis of dynamic pricing processes with nonlinear external reference effects', Communications in Nonlinear Science and Numerical Simulation, vol. 79, pp. 104929-104929.
Dynamic pricing has been widely implemented to hedge against
volatile demand. One challenging problem is the study of optimal price
choices under the influence of this volatility. Stochastic demand is a
prevalent assumption when it comes to model the volatility on pricing
decisions. However, the demand volatility might also be produced by
deterministic chaos, which has been rarely studied in this field of
research to-date. Disregarding deterministic dynamics may not only cause
revenue losses in practice but might also mislead regulators about the
underlying mechanisms used by market participants.
To improve pricing decisions and price regulations, we propose a
deterministic dynamic pricing process, of which the optimisation
objective is the revenue. Consumer expectations and discrete price
choices are considered to mimic a real pricing decision. Contradicted
expectations are quantitatively modelled on the consumer purchasing
decisions. Due to asymmetry in the perceptions of gains or losses, the
model becomes non-smooth. Period adding bifurcations, codimension-2
points and coexisting solutions can be observed.
Results highlight that an optimal pricing strategy should agree with the
dynamics of consumer expectations - which we show for the first time.
Based on that an optimal irregular pricing strategy is introduced: a
decision maker can make the first return iteration of each optimal price
choice non-periodic to follow non-periodic expectations when confronting
finite price choices. These results may justify implementing irregular
pricing strategies in the case of practical pricing decisions. Here, the
existence of coexisting solutions will assist to identify potential
market manipulations within a monopoly market. This not only contributes
to a fresh look on a volatile market but also spotlights the important
role of initial conditions to pricing decision and price regulations.
Zhang, G, Tao, J, Qiu, X & Burnett, I 2019, 'Decentralized Two-Channel Active Noise Control for Single Frequency by Shaping Matrix Eigenvalues', IEEE/ACM Transactions on Audio, Speech and Language Processing, vol. 27, no. 1, pp. 44-52.View/Download from: UTS OPUS or Publisher's site
© 2014 IEEE. In an active noise control (ANC) system, computational complexity is one major concern when designing practical control algorithms. For an ANC system with multiple secondary sources and error microphones, one approach to reducing computational complexity is to apply a decentralized control scheme rather than centralized approaches. A decentralized scheme attempts to control a number of small-size ANC subsystems independently. In this paper, we consider the decentralized control of a two-channel ANC system tackling a noise disturbance in the frequency domain, where each channel consists of one secondary source and one error microphone. We propose a decentralized control method that is able to achieve the same noise reduction performance as the centralized controller with guaranteed convergence. The key step in designing the control method is to properly shape the eigenvalues of a matrix that models the two-channel secondary paths for each frequency index.
Zhang, Y, Lu, W, Ou, W, Zhang, G, Zhang, X, Cheng, J & Zhang, W 2019, 'Chinese medical question answer selection via hybrid models based on CNN and GRU', Multimedia Tools and Applications.View/Download from: Publisher's site
© 2019, Springer Science+Business Media, LLC, part of Springer Nature. Question answer selection in the Chinese medical field is very challenging since it requires effective text representations to capture the complex semantic relationships between Chinese questions and answers. Recent approaches on deep learning, e.g., CNN and RNN, have shown their potential in improving the selection quality. However, these existing methods can only capture a part or one-side of semantic relationships while ignoring the other rich and sophisticated ones, leading to limited performance improvement. In this paper, a series of neural network models are proposed to address Chinese medical question answer selection issue. In order to model the complex relationships between questions and answers, we develop both single and hybrid models with CNN and GRU to combine the merits of different neural network architectures. This is different from existing works that can onpy capture partial relationships by utilizing a single network structure. Extensive experimental results on cMedQA dataset demonstrate that the proposed hybrid models, especially BiGRU-CNN, significantly outperform the state-of-the-art methods. The source codes of our models are available in the GitHub (https://github.com/zhangyuteng/MedicalQA-CNN-BiGRU).
Guo, J, Tan, ZH, Cho, SH & Zhang, G 2018, 'Wireless Personal Communications: Machine Learning for Big Data Processing in Mobile Internet', Wireless Personal Communications, vol. 102, no. 3, pp. 2093-2098.View/Download from: UTS OPUS or Publisher's site
O'Connor, M, Zhang, G, Kleijn, WB & Abhayapala, TD 2018, 'Function Splitting and Quadratic Approximation of the Primal-Dual Method of Multipliers for Distributed Optimization over Graphs', IEEE Transactions on Signal and Information Processing over Networks, vol. 4, no. 4, pp. 656-666.View/Download from: UTS OPUS or Publisher's site
Zhang, G & Heusdens, R 2018, 'Distributed Optimization Using the Primal-Dual Method of Multipliers', IEEE Transactions on Signal and Information Processing over Networks, vol. 4, no. 1, pp. 173-187.View/Download from: UTS OPUS or Publisher's site
© 2015 IEEE. In this paper, we propose the primal-dual method of multipliers (PDMM) for distributed optimization over a graph. In particular, we optimize a sum of convex functions defined over a graph, where every edge in the graph carries a linear equality constraint. In designing the new algorithm, an augmented primal-dual Lagrangian function is constructed which smoothly captures the graph topology. It is shown that a saddle point of the constructed function provides an optimal solution of the original problem. Further under both the synchronous and asynchronous updating schemes, PDMM has the convergence rate of O(1/K) (where K denotes the iteration index) for general closed, proper, and convex functions. Other properties of PDMM such as convergence speeds versus different parameter-settings and resilience to transmission failure are also investigated through the experiments of distributed averaging.
Klejsa, J, Zhang, G, Li, M & Kleijn, WB 2013, 'Multiple Description Distribution Preserving Quantization', IEEE Transactions on Signal Processing, vol. 61, no. 24, pp. 6410-6422.View/Download from: UTS OPUS or Publisher's site
Zhang, G & Heusdens, R 2012, 'Linear Coordinate-Descent Message Passing for Quadratic Optimization', Neural Computation, vol. 24, no. 12, pp. 3340-3370.View/Download from: UTS OPUS or Publisher's site
In this letter, we propose a new message-passing algorithm for quadratic optimization. The design of the new algorithm is based on linear coordinate descent between neighboring nodes. The updating messages are in a form of linear functions as compared to the min-sum algorithm of which the messages are in a form of quadratic functions. As a result, the linear coordinate-descent (LiCD) algorithm transmits only one parameter per message as opposed to the min-sum algorithm, which transmits two parameters per message. We show that when the quadratic matrix is walk-summable, the LiCD algorithm converges. By taking the LiCD algorithm as a subroutine, we also fix the convergence issue for a general quadratic matrix. The LiCD algorithm works in either a synchronous or asynchronous message-passing manner. Experimental results show that for a general graph with multiple cycles, the LiCD algorithm has comparable convergence speed to the min-sum algorithm, thereby reducing the number of parameters to be transmitted and the computational complexity.
Zhang, G, Klejsa, J & Kleijn, WB 2012, 'Optimal Index Assignment for Multiple Description Scalar Quantization With Translated Lattice Codebooks', IEEE TRANSACTIONS ON SIGNAL PROCESSING, vol. 60, no. 8, pp. 4444-4451.View/Download from: UTS OPUS or Publisher's site
Wong, K-YK, Zhang, G & Chen, Z 2011, 'A Stratified Approach for Camera Calibration Using Spheres', IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 20, no. 2, pp. 305-316.View/Download from: UTS OPUS or Publisher's site
Zhang, G, Ostergaard, J, Klejsa, J & Kleijn, WB 2011, 'High-Rate Analysis of Symmetric L-Channel Multiple Description Coding', IEEE TRANSACTIONS ON COMMUNICATIONS, vol. 59, no. 7, pp. 1846-1856.View/Download from: UTS OPUS or Publisher's site
Wong, KYK, Zhang, G, Liang, C & Zhang, H 2008, '1D camera geometry and its application to the self-calibration of circular motion sequences', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 12, pp. 2243-2248.View/Download from: Publisher's site
This paper proposes a novel method for robustly recovering the camera geometry of an uncalibrated image sequence taken under circular motion. Under circular motion, all the camera centers lie on a circle and the mapping from the plane containing this circle to the horizon line observed in the image can be modelled as a 1D projection. A 2×2 homography is introduced in this paper to relate the projections of the camera centers in two 1D views. It is shown that the two imaged circular points of the motion plane and the rotation angle between the two views can be derived directly from such a homography. This way of recovering the imaged circular points and rotation angles is intrinsically a multiple view approach, as all the sequence geometry embedded in the epipoles is exploited in the estimation of the homography for each view pair. This results in a more robust method compared to those computing the rotation angles using adjacent views only. The proposed method has been applied to self-calibrate turntable sequences using either point features or silhouettes, and highly accurate results have been achieved. © 2008 IEEE.
Zhang, H, Wong, KYK & Zhang, G 2007, 'Camera calibration from images of spheres', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 3, pp. 499-502.View/Download from: Publisher's site
This paper introduces a novel approach for solving the problem of camera calibration from spheres. By exploiting the relationship between the dual images of spheres and the dual image of the absolute conic (IAC), it is shown that the common pole and polar with regard to the conic images of two spheres are also the pole and polar with regard to the IAC. This provides two constraints for estimating the IAC and, hence, allows a camera to be calibrated from an image of at least three spheres. Experimental results show the feasibility of the proposed approach. © 2007 IEEE.
Zhang, G, Li, H & Wenger, F, 'Object Detection and 3D Estimation via an FMCW Radar Using a Fully Convolutional Network'.
This paper considers object detection and 3D estimation using an FMCW radar.
The state-of-the-art deep learning framework is employed instead of using
traditional signal processing. In preparing the radar training data, the ground
truth of an object orientation in 3D space is provided by conducting image
analysis, of which the images are obtained through a coupled camera to the
radar device. To ensure successful training of a fully convolutional network
(FCN), we propose a normalization method, which is found to be essential to be
applied to the radar signal before feeding into the neural network. The system
after proper training is able to first detect the presence of an object in an
environment. If it does, the system then further produces an estimation of its
3D position. Experimental results show that the proposed system can be
successfully trained and employed for detecting a car and further estimating
its 3D position in a noisy environment.
Zhang, G, Niwa, K & Kleijn, WB, 'Rapidly Adapting Moment Estimation'.
Adaptive gradient methods such as Adam have been shown to be very effective
for training deep neural networks (DNNs) by tracking the second moment of
gradients to compute the individual learning rates. Differently from existing
methods, we make use of the most recent first moment of gradients to compute
the individual learning rates per iteration. The motivation behind it is that
the dynamic variation of the first moment of gradients may provide useful
information to obtain the learning rates. We refer to the new method as the
rapidly adapting moment estimation (RAME). The theoretical convergence of
deterministic RAME is studied by using an analysis similar to the one used in
 for Adam. Experimental results for training a number of DNNs show promising
performance of RAME w.r.t. the convergence speed and generalization performance
compared to the stochastic heavy-ball (SHB) method, Adam, and RMSprop.
LU, S, Oberst, S, Zhang, G & Luo, Z 2019, 'Novel order patterns recurrence plot-based quantification measures to unveil deterministic dynamics from stochastic processes' in Valenzuela, O, Rojas, F, Pomares, H & Rojas, I (eds), Theory and Applications of Time Series Analysis, Springer.View/Download from: UTS OPUS
LU, S, Oberst, S, Zhang, G & Luo, Z 2019, 'Novel order patterns recurrence plot-based quantification measures to unveil deterministic dynamics from stochastic processes' in Valenzuela, O, Rojas, F, Pomares, H & Rojas, I (eds), Theory and Applications of Time Series Analysis, Springer International Publishing, Cham, Switzerland.View/Download from: Publisher's site
LU, S, Oberst, S, Zhang, G & Luo, Z 2019, 'Period adding bifurcations in dynamic pricing processes', IEEE Xplore, IEEE CIFEr 2019: 2019 IEEE Conference on Computational Intelligence for Financial Engineering and Economics, IEEE, Shenzhen.View/Download from: UTS OPUS or Publisher's site
Price information enables consumers to anticipate
a price and to make purchasing decisions based on their price
expectations, which are critical for agents with pricing decisions
or price regulations. A company with pricing decisions can
aim to optimise the short-term or the long-term revenue, each
of which leads to different pricing strategies thereby different
price expectations. The choices between the two optimisation
objectives consider the maximal revenue and the robustness of
a chosen pricing strategy against market volatility. However
the robustness is rarely identified in a volatile market. Here,
we investigate the robustness of optimal pricing strategies with
the short-term or long-term optimisation objectives through the
analysis of nonlinear dynamics of price expectations. Bifurcation
diagrams and period diagrams are introduced to compare their
change in dynamics. Our results highlight that period adding
bifurcations occur during the dynamic pricing processes studied.
These bifurcations would challenge the robustness of an optimal
pricing strategy. The consideration of the long-term revenue
allows a company to charge a higher price, which in turn
increases the revenue. However, the consideration of the shortterm
revenue can avoid period adding bifurcations, contributing
to a robust pricing strategy. This allows a company to harvest
a good revenue through a robust pricing strategy in a volatile
market and to satisfy regulations of a control in price volatility.
Niwa, K, Zhang, G & Kleijn, WB 2019, 'Fast Edge-Consensus Computing Based On Bregman Monotone Operator Splitting', ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, IEEE, Brighton, UK.
Xie, J, Ma, Z, Zhang, G, Xue, J, Tan, Z & Guo, J 2019, 'Soft Dropout and Its Variational Bayes Approximation', IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, Pittsburgh, PA, USA.
Zhang, G, Tao, J & Qiu, X 2019, 'Empirical Study of Decentralized Multi-Channel Active Noise Control Based on the Genetic Algorithm', PROCEEDINGS of the 23rd International Congress on Acoustics, the 23rd International Congress on Acoustics, Aachen, Germany.View/Download from: UTS OPUS
LU, S, Oberst, S, Zhang, G & Luo, Z 2018, 'Comparing complex dynamics using machine learning-reconstructed attracting sets', Colloquium on Irregular Engineering Oscillations and Signal Processing, TUHH, Hamburg, Germany.
LU, S, Oberst, S, Zhang, G & Luo, Z 2018, 'Order patterns recurrence plots and new quantifications to unveil nonlinear dynamics from stochastic systems', International Conference on Time Series and Forecasting 2018, Granada, Spain.View/Download from: UTS OPUS
Niwa, K, Zhang, G & Bastiaan Kleijn, W 2018, 'Edge Consensus Computing for Heterogeneous Data Sets', 2018 IEEE Statistical Signal Processing Workshop, SSP 2018, Statistical Signal Processing Workshop, IEEE, Freiburg, Germany, pp. 663-667.View/Download from: UTS OPUS or Publisher's site
© 2018 IEEE. Edge consensus computing is a framework to optimize a cost function when distributed nodes have distinct data sets available to them. The primal-dual method of multipliers (PDMM) is an optimization algorithm that forms a consensus among nodes by exchanging latent variables rather than the data sets. PDMM often has a high rate of convergence. However, when the nodes see statistically heterogeneous data sets then the performance of PDMM degrades. To overcome this problem, we propose quadratic PDMM. In this method, the original cost functions are replaced by their quadratic majorization based on the L2 norm to ensure homogeneous convexity among nodes. We describe a method to set its parameters optimally for fast convergence. Our experiments confirm that the proposed quadratic PDMM provides good performance even when the data sets are heterogeneous.
Xie, J, Ma, Z, Zhang, G, Xue, JH, Chien, JT, Lin, Z & Guo, J 2018, 'Balson: Bayesian least squares optimization with nonnegative L1-Norm constraint', IEEE International Workshop on Machine Learning for Signal Processing, MLSP, International Workshop on Machine Learning for Signal Processing, IEEE, Aalborg, Denmark.View/Download from: UTS OPUS or Publisher's site
© 2018 IEEE. A Bayesian approach termed the BAyesian Least Squares Optimization with Nonnegative L 1 -norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parameters is equivalent to finding the mode of the posterior distribution. In order to explicitly characterize the nonnegative L 1 -norm constraint of the parameters, we further approximate the true posterior distribution by a Dirichlet distribution. We estimate the moments of the approximated Dirichlet posterior distribution by sampling methods. Four sampling methods have been introduced and implemented. With the estimated posterior distributions, the original parameters can be effectively reconstructed in polynomial fitting problems, and the BALSON framework is found to perform better than conventional methods.
Zhang, G & Kleijn, WB 2018, 'Training Deep Neural Networks via Optimization Over Graphs', Proceedings of international on acoustics, Speech, and Signal Processing, International Conference on Acoustics, Speech, and Signal Processing, Calgary, Canada.
Zhang, G, Orconnor, M & Li, L 2018, 'On Convergence Analysis of Gradient Based Primal-Dual Method of Multipliers', 2018 IEEE Statistical Signal Processing Workshop, SSP 2018, Statistical Signal Processing Workshop, IEEE, Freiburg, Germany, pp. 353-357.View/Download from: UTS OPUS or Publisher's site
© 2018 IEEE. Recently, the primal-dual method of multipliers (PDMM) has been proposed and successfully applied to solve a number of decomposable convex optimizations distributedly and iteratively. In this work, we study the gradient based PDMM (GPDMM), where the objective functions are approximated using the gradient information per iteration. It is shown that for a certain class of decomposable convex optimizations, synchronous GPDMM has a sublinear convergence rate of O(1/K) (where K denotes the iteration index). Experiments on a problem of distributed ridge regularized logistic regression demonstrate the efficiency of synchronous GPDMM.
Zhang, G & Heusdens, R 2016, 'On Simplifying the Pirmal-Dual Method of Multipliers', Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Shanghai, China, pp. 4826-4830.View/Download from: UTS OPUS or Publisher's site
Recently, the primal-dual method of multipliers (PDMM) has been proposed to solve a convex optimization problem defined over a general graph. In this paper, we consider simplifying PDMM for a subclass of the convex optimization problems. This subclass includes the consensus problem as a special form. By using algebra, we show that the update expressions of PDMM can be simplified significantly. We then evaluate PDMM for training a support vector machine (SVM). The experimental results indicate that PDMM converges considerably faster than the alternating direction method of multipliers (ADMM).
Zhang, G & Heusdens, R 2013, 'Proximal Alternating-Direction Message-Passing for MAP LP Relaxation', Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Vancouver, BC, Canada, pp. 3402-3406.View/Download from: UTS OPUS or Publisher's site
Linear programming (LP) relaxation for MAP inference over (factor) graphic models is one of the fundamental problems in machine learning. In this paper, we propose a new message-passing algorithm for the MAP LP-relaxation by using the proximal alternating-direction method of multipliers (PADMM). At each iteration, the new algorithm performs two layers of optimization, that is node-oriented optimization and factor-oriented optimization. On the other hand, the recently proposed augmented primal LP (APLP) algorithm, based on the ADMM, has to perform three layers of optimization. Our algorithm simplifies the APLP algorithm by removing one layer of optimization, thus reducing the computational complexities and further accelerating the convergence rate. We refer to our new algorithm as the proximal alternating-direction (PAD) algorithm. Experimental results confirm that the PAD algorithm indeed converges faster than the APLP method.
Zhang, G & Heusdens, R 2015, 'Bi-Alternating Direction Method of Multipliers over Graphs', 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Brisbane, Australia, pp. 3571-3575.View/Download from: UTS OPUS or Publisher's site
In this paper, we extend the bi-alternating direction method of multipliers (BiADMM) designed on a graph of two nodes to a graph of multiple nodes. In particular, we optimize a sum of convex functions defined over a general graph, where every edge carries a linear equality constraint. In designing the new algorithm, an augmented primal-dual Lagrangian function is carefully constructed which naturally captures the associated graph topology. We show that under both the synchronous and asynchronous updating schemes, the extended BiADMM has the convergence rate of O(1/K) (where K denotes the iteration index) for general closed, proper and convex functions. As an example, we apply the new algorithm for distributed averaging. Experimental results show that the new algorithm remarkably outperforms the state-of-the-art methods.
Zhang, G & Heusdens, R 2014, 'Convergence of min-sum-min message-passing for quadratic optimization', Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Joint European Conference on Machine Learning and Knowledge Discovery in Databases ECML PKDD 2014, pp. 353-368.View/Download from: Publisher's site
We propose a new message-passing algorithm for the quadratic optimization problem. As opposed to the min-sum algorithm, the new algorithm involves two minimizations and one summation at each iteration. The new min-sum-min algorithm exploits feedback from last iteration in generating new messages, resembling the Jacobi- relaxation algorithm. We show that if the feedback signal is large enough, the min-sum-min algorithm is guaranteed to converge to the optimal solution. Experimental results show that the min-sum-min algorithm outperforms two reference methods w.r.t. the convergence speed. © 2014 Springer-Verlag.
Zhang, G, Heusdens, R & Kleijn, WB 2014, 'On the Convergence Rate of the Bi-Alternating Direction Method of Multipliers', ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Florence, Italy, pp. 3869-3873.View/Download from: UTS OPUS or Publisher's site
Zhang, G & Heusdens, R 2013, 'Bi-Alternating Direction Method of Multipliers', 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Vancouver, BC, Canada, pp. 3317-3321.View/Download from: UTS OPUS or Publisher's site
The alternating-direction method of multipliers (ADMM) has been widely applied in the field of distributed optimization and statistic learning. ADMM iteratively approaches the saddle point of an augmented Lagrangian function by performing three updates per-iteration. In this paper, we propose a bi-alternating direction method of multipliers (BiADMM) that iteratively minimizes an augmented bi-conjugate function. As a result, the convergence of BiADMM is naturally established. Unlike ADMM that always involves three updates per iteration, BiADMM opens up an avenue to perform either two or three updates per iteration, depending on the functional construction. As an application, we consider applying BiADMM for the lasso problem. Experimental results demonstrate the effectiveness of our new method.
Zhang, G & Heusdens, R 2013, 'Simplified Alternating-Direction Message Passing for Dual MAP LP-Relaxation', 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Vancouver, BC, Canada, pp. 3397-3401.View/Download from: UTS OPUS or Publisher's site
The approximate MAP inference over (factor) graphic models is of great importance in many applications. Due to its simplicity, linear-programming (LP) relaxation has become one of the most popular approaches to approximate MAP. In this paper, we propose a new message passing algorithm for the MAP LP-relaxation problem by using the alternating-direction method of multipliers (ADMM). At each iteration, the new algorithm performs two layers of optimization sequentially, that is node-oriented optimization and factor-oriented optimization. On the other hand, the recently proposed augmented dual LP (ADLP) algorithm, also based on the ADMM, has to perform three layers of optimization. We refer to our new algorithm as the simplified ADLP (SiADLP) algorithm. The design of the SiADLP algorithm stems from a new formulation for the dual LP problem. Experimental results show that the SiADLP algorithm outperforms the ADLP method.
Zhang, G & Heusdens, R 2012, 'Convergence of Generalized Linear Coordinate-Descent Message-Passing for Quadratic Optimization', 2012 IEEE International Symposium on Information Theory Proceedings, IEEE International Symposium on Information Theory, iEEE, Cambridge, MA, USA, pp. 1997-2001.View/Download from: UTS OPUS or Publisher's site
Zhang, G & Heusdens, R 2012, 'Generalized Linear Coordinate-Descent Message Passing for Convex Optimization', 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Kyoto, Japan, pp. 2009-2012.View/Download from: UTS OPUS or Publisher's site
In this paper we propose a generalized linear coordinate-descent (GLiCD) algorithm for a class of unconstrained convex optimization problems. The considered objective function can be decomposed into edge-functions and node-functions of a graphical model. The messages of the GLiCD algorithm are in a form of linear functions, as compared to the min-sum algorithm of which the form of messages depends on the objective function. Thus, the implementation of the GLiCD algorithm is much simpler than that of the min-sum algorithm. A theorem is stated according to which the algorithm converges to the optimal solution if the objective function satisfies a diagonal-dominant condition. As an application, the GLiCD algorithm is exploited in solving the averaging problem in sensor networks, where the performance is compared to that of the min-sum algorithm.
Zhang, G & Heusdens, R 2012, 'Linear Coordinate-Descent Message Passing for Quadratic Optimization', Proceedings fo the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Kyoto, Japan, pp. 2005-2008.View/Download from: UTS OPUS or Publisher's site
In this paper we propose a new message-passing algorithm for quadratic optimization. The design of the new algorithm is based on linear coordinate-descent between neighboring nodes. The updating messages are in a form of linear functions as compared to the min-sum algorithm of which the messages are in a form of quadratic functions. Therefore, the linear coordinate-descent (LiCD) algorithm has simpler updating rules than the min-sum algorithm. It is shown that when the quadratic matrix is walk-summable, the LiCD algorithm converges. As an application, the LiCD algorithm is utilized in solving general linear systems. The performance of the LiCD algorithm is found empirically to be comparable to that of the min-sum algorithm, but at lower complexity in terms of computation and storage.
Zhang, G 2011, 'On the Rate Region of the Vector Gaussian One-Helper Distributed Source Coding Problem', Data Compression Conference (DCC), 2011, Data Compression Conference, IEEE, Snowbird, UT, USA, pp. 263-272.View/Download from: UTS OPUS or Publisher's site
In this paper we consider the rate region of the vector Gaussian one-helper distributed source coding problem. In particular, we derive optimality conditions under which a weighted sum rate is minimum by using a contradiction-based argument. When the sources are specified to be scalar, the optimality conditions can always be constructed for any weighted sum rate. In the derivation of the optimality conditions, we introduce a new concept of "source enhancement", which can be viewed as a dual to the well known "channel enhancement" technique. In particular, source enhancement refers to the operation of increasing the covariance matrix of a Gaussian source in a partial ordering sense. This new technique makes the derivation of the optimality conditions straightforward.
Li, H, Zhang, G & Kleijn, WB 2010, 'Adaptive playout scheduling for voip using the k-Erlang distribution', European Signal Processing Conference, pp. 1494-1498.
We propose a new adaptive playout scheme for VoIP. The k- Erlang distribution is introduced to model the packet interarrival time distribution. A cost function is proposed for the next played out packet in the buffer based on modelling packet-arrival times with the k-Erlang distribution. The cost function essentially balances the average buffering delay and the packet-loss rate. The optimal playout length of the packet is determined by minimizing the cost function and realized by either inserting or dropping pitch cycles from the packet. Our real-world data experiments show that our scheme outperforms two reference methods for both low-jitter and highjitter cases. © EURASIP, 2010.
Zhang, G, Kleijn, WB & Østergaard, J 2010, 'Bounding the Rate Region of Vector Gaussian Multiple Descriptions with Individual and Central Receivers', Data Compression Conference (DCC), pp. 13-19.View/Download from: Publisher's site
Zhang, G, Kleijn, WB & Østergaard, J 2010, 'Bounding the rate region of vector Gaussian multiple descriptions with individual and central receivers', Data Compression Conference Proceedings, pp. 13-19.View/Download from: Publisher's site
The problem of the rate region of the vector Gaussian multiple description with individual and central quadratic distortion constraints is studied. We have two main contributions. First, a lower bound on the rate region is derived. The bound is obtained by lower-bounding a weighted sum rate for each supporting hyperplane of the rate region. Second, the rate region for the scenario of the scalar Gaussian source is fully characterized by showing that the lower bound is tight. The optimal weighted sum rate for each supporting hyperplane is obtained by solving a single maximization problem. This is contrary to existing results, which require solving a min-max optimization problem. © 2010 IEEE.
Zhang, G, Klejsa, J & Kleijn, WB 2009, 'Analysis of K-channel multiple description quantization', Data Compression Conference Proceedings, pp. 53-62.
This paper studies the tight rate-distortion bound for K-channel symmetric multiple-description coding for a memoryless Gaussian source. We find that the product of a function of the individual side distortions (for single received descriptions) and the central distortion (for K received descriptions) is asymptotically independent of the redundancy among the descriptions. Using this property, we analyze the asymptotic behaviors of two different practical multiple-description lattice vector quantizers (MDLVQ). Our analysis includes the treatment of a MDLVQ system from a new geometric viewpoint, which results in an expression for the side distortions using the normalized second moment of a sphere of higher dimensionality than the quantization space. The expression of the distortion product derived from the lower bound is then applied as a criterion to assess the performance losses of the considered MDLVQ systems. © 2009 IEEE.
Zhang, G & Kleijn, WB 2008, 'Autoregressive model-based speech packet-loss concealment', ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 4797-4800.View/Download from: Publisher's site
We study packet-loss concealment for speech based on autoregressive modeling using a rigorous minimum mean square error (MMSE) approach. The effect of the model estimation error on predicting the missing segment is studied and an upper bound on the mean square error is derived. Our experiments show that the upper bound is tight when the estimation error is less than the signal variance. We also consider the usage of perceptual weighting on prediction to improve speech quality. A rigorous argument is presented to show that perceptual weighting is not useful in this context. We create simple and practical MMSE-based systems using two signal models: a basic model capturing the short-term correlation and a more sophisticated model that also captures the long-term correlation. Subjective quality comparison tests show that the proposed MMSE-based system provides state-of-the-art performance. ©2008 IEEE.
Zhang, G, Lundin, H & Kleijn, WB 2008, 'Band control policy of playout scheduling for voice over IP', European Signal Processing Conference.
We study adaptive-playout scheduling for VoIP using the framework of stochastic impulse control theory. A Wiener process is introduced to model the fluctuation of the buffer length in the absence of control. In this context, the control signal consists of length units that correspond to inserting or dropping a pitch cycle. We define an optimality criterion that has an adjustable trade-off between average buffing delay and average control length (the length of the pitch cycles added plus the length of the pitch cycles dropped). The clock-drift effect is treated in a unified manner within this framework. A band control policy is shown to be optimal. The algorithm does not require knowledge of the clock drift. It maintains the buffer length within a band region by imposing impulse control (inserted or dropped pitch cycles) whenever the bounds of the band are reached. Our experiments show that the proposed method outperforms a popular reference method. copyright by EURASIP.
Zhang, G & Wong, KYK 2006, 'Motion estimation from spheres', Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1238-1243.View/Download from: Publisher's site
This paper addresses the problem of recovering epipolar geometry from spheres. Previous works have exploited epipolar tangencies induced by frontier points on the spheres for motion recovery. It will be shown in this paper that besides epipolar tangencies, N2 point features can be extracted from the apparent contours of the N spheres when N > 2. An algorithm for recovering the fundamental matrices from such point features and the epipolar tangencies from 3 or more spheres is developed, with the point features providing a homography over the view pairs and the epipolar tangencies determining the epipoles. In general, there will be two solutions to the locations of the epipoles. One of the solutions corresponds to the true camera configuration, while the other corresponds to a mirrored configuration. Several methods are proposed to select the right solution. Experiments on using 3 and 4 spheres demonstrate that our algorithm can be carried out easily and can achieve a high precision. © 2006 IEEE.
Zhang, G, Zhang, H & Wong, KYK 2006, '1D camera geometry and its application to circular motion estimation', BMVC 2006 - Proceedings of the British Machine Vision Conference 2006, pp. 67-76.
This paper describes a new and robust method for estimating circular motion geometry from an uncalibrated image sequence. Under circular motion, all the camera centers lie on a circle, and the mapping of the plane containing this circle to the horizon line in the image can be modelled as a 1D projection. A 2 × 2 homography is introduced in this paper to relate the projections of the camera centers in two 1D views. It is shown that the two imaged circular points and the rotation angle between the two views can be derived directly from the eigenvectors and eigenvalues of such a homography respectively. The proposed 1D geometry can be nicely applied to circular motion estimation using either point correspondences or silhouettes. The method introduced here is intrinsically a multiple view approach as all the sequence geometry embedded in the epipoles is exploited in the computation of the homography for a view pair. This results in a robust method which gives accurate estimated rotation angles and imaged circular points. Experimental results are presented to demonstrate the simplicity and applicability of the new method.
Zhang, H, Zhang, G & Wong, KYK 2005, 'Auto-calibration and motion recovery from silhouettes for turntable sequences', BMVC 2005 - Proceedings of the British Machine Vision Conference 2005.View/Download from: Publisher's site
This paper addresses the problem of structure and motion from silhouettes for turntable sequences. Previous works have exploited corresponding points induced by epipolar tangencies to estimate the image invariants under turntable motion and recover the epipolar geometry. In these approaches, however, camera intrinsics are needed in order to obtain Euclidean motion and reconstruction. This paper proposes a novel approach to precisely estimate the image invariants and the rotation angles in the absence of the camera intrinsics, and to perform auto-calibration. By exploiting a special parameterization of the epipoles, it is shown that the imaged circular points can be formulated in terms of the image invariants. A fixed scalar k, introduced to account for the different scales in the homogeneous representations of the image invariants used in the parameterizations, is found crucial in both calibration and motion estimation. Given the image invariants, namely the horizon, the imaged rotation axis and its orthogonal vanishing point, this scalar can be determined from the epipoles in an image triplet. A robust method for estimating k is proposed and the rotation angles can be recovered using this estimated value of k. All the estimated variables are then refined using bundle-adjustment and auto-calibration is performed using the imaged circular points, the imaged rotation axis and the associated vanishing point. This allows the recovery of the full camera positions and orientations, and hence Euclidean reconstruction. Experimental results demonstrate the simplicity of this novel approach and the high precision in the estimated motion and reconstruction.
Zhang, H, Zhang, G & Wong, KYK 2005, 'Camera calibration with spheres: Linear approaches', Proceedings - International Conference on Image Processing, ICIP, pp. 1150-1153.View/Download from: Publisher's site
This paper addresses the problem of camera calibration from spheres. By studying the relationship between the dual images of spheres and that of the absolute conic, a linear solution has been derived from a recently proposed non-linear semi-definite approach. However, experiments show that this approach is quite sensitive to noise. In order to overcome this problem, a second approach has been proposed, where the orthogonal calibration relationship is obtained by regarding any two spheres as a surface of revolution. This allows a camera to be fully calibrated from an image of three spheres. Besides, a conic homography is derived from the imaged spheres, and from its eigenvectors the orthogonal invariants can be computed directly. Experiments on synthetic and real data show the practicality of such an approach. © 2005 IEEE.
Zhang, G & Li, H 2018, 'Effectiveness of Scaled Exponentially-Regularized Linear Units (SERLUs)'.
Recently, self-normalizing neural networks (SNNs) have been proposed with the
intention to avoid batch or weight normalization. The key step in SNNs is to
properly scale the exponential linear unit (referred to as SELU) to inherently
incorporate normalization based on central limit theory. SELU is a
monotonically increasing function, where it has an approximately constant
negative output for large negative input. In this work, we propose a new
activation function to break the monotonicity property of SELU while still
preserving the self-normalizing property. Differently from SELU, the new
function introduces a bump-shaped function in the region of negative input by
regularizing a linear function with a scaled exponential function, which is
referred to as a scaled exponentially-regularized linear unit (SERLU). The
bump-shaped function has approximately zero response to large negative input
while being able to push the output of SERLU towards zero mean statistically.
To effectively combat over-fitting, we develop a so-called shift-dropout for
SERLU, which includes standard dropout as a special case. Experimental results
on MNIST, CIFAR10 and CIFAR100 show that SERLU-based neural networks provide
consistently promising results in comparison to other 5 activation functions
including ELU, SELU, Swish, Leakly ReLU and ReLU.
Zhang, G, Kleijn, WB & Heusdens, R 2017, 'On Relationship between Primal-Dual Method of Multipliers and Kalman Filter'.
Recently the primal-dual method of multipliers (PDMM), a novel distributed
optimization method, was proposed for solving a general class of decomposable
convex optimizations over graphic models. In this work, we first study the
convergence properties of PDMM for decomposable quadratic optimizations over
tree-structured graphs. We show that with proper parameter selection, PDMM
converges to its optimal solution in finite number of iterations. We then apply
PDMM for the causal estimation problem over a statistical linear state-space
model. We show that PDMM and the Kalman filter have the same update
expressions, where PDMM can be interpreted as solving a sequence of quadratic
optimizations over a growing chain graph.
We provide a method for designing an optimal index assignment for scalar
K-description coding. The method stems from a construction of translated scalar
lattices, which provides a performance advantage by exploiting a so-called
staggered gain. Interestingly, generation of the optimal index assignment is
based on a lattice in K-1 dimensional space. The use of the K-1 dimensional
lattice facilitates analytic insight into the performance and eliminates the
need for a greedy optimization of the index assignment. It is shown that that
the optimal index assignment is not unique. This is illustrated for the
two-description case, where a periodic index assignment is selected from
possible optimal assignments and described in detail. The new index assignment
is applied to design of a K-description quantizer, which is found to outperform
a reference K-description quantizer at high rates. The performance advantage
due to the staggered gain increases with increasing redundancy among the