We discuss several methods for accelerating the convergence of the iterative solution of nonlinear equation systems commonly in use and point to interrelations between them. In particular we investigate two of the most sophisticated schemes, namely the Anderson mixing and the Broyden update, both generalized to the consideration of arbitrarily many previous iterations. For the Broyden method we give a new derivation which is much simpler than that recently proposed by Vanderbilt and Louie (1984). We show that if the additional parameters invented by these authors in order to increase flexibility are used to optimize the convergence of the iteration process they in fact cancel out. In addition we prove that in this (optimal) case the Anderson mixing and the Broyden update as applied to the inverse Jacobian are fully identical. Thus we come to the conclusion that neither of these schemes is superior. Moreover, we show that Broyden update of the inverse Jacobian is superior to updating the Jacobian itself. Finally we propose an extension of the Anderson mixing which avoids the numerical difficulties all these methods are faced with.
 

Journal of Computational Physics, 124 271-85, 1996.