Matrix And Tensor Decomposition

This book presents the algorithms used to provide recommendations by exploiting matrix factorization and tensor decomposition techniques. It highlights well-known decomposition methods for recommender systems, such as Singular Value Decomposition (SVD), UV-decomposition, Non-negative Matrix Factorization (NMF), etc. and describes in detail the pros and cons of each method for matrices and tensors. This book provides a detailed theoretical mathematical background of matrix/tensor factorization techniques and a step-by-step analysis of each method on the basis of an integrated toy example that runs throughout all its chapters and helps the reader to understand the key differences among methods. It also contains two chapters, where different matrix and tensor methods are compared experimentally on real data sets, such as Epinions, GeoSocialRec, Last.fm, BibSonomy, etc. and provides further insights into the advantages and disadvantages of each method. The book offers a rich blend of theory and practice, making it suitable for students, researchers and practitioners interested in both recommenders and factorization methods. Lecturers can also use it for classes on data mining, recommender systems and dimensionality reduction methods.

The second volume will deal with a presentation of the main matrix and tensor decompositions and their properties of uniqueness, as well as very useful tensor networks for the analysis of massive data. Parametric estimation algorithms will be presented for the identification of the main tensor decompositions. After a brief historical review of the compressed sampling methods, an overview of the main methods of retrieving matrices and tensors with missing data will be performed under the low rank hypothesis. Illustrative examples will be provided.

This book presents the algorithms used to provide recommendations by exploiting matrix factorization and tensor decomposition techniques. It highlights well-known decomposition methods for recommender systems, such as Singular Value Decomposition (SVD), UV-decomposition, Non-negative Matrix Factorization (NMF), etc. and describes in detail the pros and cons of each method for matrices and tensors. This book provides a detailed theoretical mathematical background of matrix/tensor factorization techniques and a step-by-step analysis of each method on the basis of an integrated toy example that runs throughout all its chapters and helps the reader to understand the key differences among methods. It also contains two chapters, where different matrix and tensor methods are compared experimentally on real data sets, such as Epinions, GeoSocialRec, Last.fm, BibSonomy, etc. and provides further insights into the advantages and disadvantages of each method. The book offers a rich blend of theory and practice, making it suitable for students, researchers and practitioners interested in both recommenders and factorization methods. Lecturers can also use it for classes on data mining, recommender systems and dimensionality reduction methods.

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMF’s various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD). NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets. It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining. As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models. Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors’ own recently developed techniques in the subject area. Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms. Provides a comparative analysis of the different methods in order to identify approximation error and complexity. Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book. The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia.

The second volume will deal with a presentation of the main matrix and tensor decompositions and their properties of uniqueness, as well as very useful tensor networks for the analysis of massive data. Parametric estimation algorithms will be presented for the identification of the main tensor decompositions. After a brief historical review of the compressed sampling methods, an overview of the main methods of retrieving matrices and tensors with missing data will be performed under the low rank hypothesis. Illustrative examples will be provided.

This open access book focuses on processing, modeling, and visualization of anisotropy information...--

The authors of this monograph survey recent progress in using spectral methods including matrix and tensor decomposition techniques to learn many popular latent variable models. With careful implementation, tensor-based methods can run efficiently in practice, and in many cases they are the only algorithms with provable guarantees on running time and sample complexity. The focus is on a special type of tensor decomposition called CP decomposition, and the authors cover a wide range of algorithms to find the components of such tensor decomposition. They also discuss the usefulness of this decomposition by reviewing several probabilistic models that can be learned using such tensor methods. The second half of the monograph looks at practical applications. This includes using Tensorly, an efficient tensor algebra software package, which has a simple python interface for expressing tensor operations. It also has a flexible back-end system supporting NumPy, PyTorch, TensorFlow, and MXNet. Spectral Learning on Matrices and Tensors provides a theoretical and practical introduction to designing and deploying spectral learning on both matrices and tensors. It is of interest for all students, researchers and practitioners working on modern day machine learning problems.

Introduces cutting-edge research on machine learning theory and practice, providing an accessible, modern algorithmic toolkit.

Tensor network is a fundamental mathematical tool with a huge range of applications in physics, such as condensed matter physics, statistic physics, high energy physics, and quantum information sciences. This open access book aims to explain the tensor network contraction approaches in a systematic way, from the basic definitions to the important applications. This book is also useful to those who apply tensor networks in areas beyond physics, such as machine learning and the big-data analysis. Tensor network originates from the numerical renormalization group approach proposed by K.G. Wilson in 1975. Through a rapid development in the last two decades, tensor network has become a powerful numerical tool that can efficiently simulate a wide range of scientific problems, with particular success in quantum many-body physics. Varieties of tensor network algorithms have been proposed for different problems. However, the connections among different algorithms are not well discussed or reviewed. To fill this gap, this book explains the fundamental concepts and basic ideas that connect and/or unify different strategies of the tensor network contraction algorithms. In addition, some of the recent progresses in dealing with tensor decomposition techniques and quantum simulations are also represented in this book to help the readers to better understand tensor network. This open access book is intended for graduated students, but can also be used as a professional book for researchers in the related fields. To understand most of the contents in the book, only basic knowledge of quantum mechanics and linear algebra is required. In order to fully understand some advanced parts, the reader will need to be familiar with notion of condensed matter physics and quantum information, that however are not necessary to understand the main parts of the book. This book is a good source for non-specialists on quantum physics to understand tensor network algorithms and the related mathematics.

This book is a printed edition of the Special Issue "Decomposability of Tensors" that was published in Mathematics

This book proposes applications of tensor decomposition to unsupervised feature extraction and feature selection. The author posits that although supervised methods including deep learning have become popular, unsupervised methods have their own advantages. He argues that this is the case because unsupervised methods are easy to learn since tensor decomposition is a conventional linear methodology. This book starts from very basic linear algebra and reaches the cutting edge methodologies applied to difficult situations when there are many features (variables) while only small number of samples are available. The author includes advanced descriptions about tensor decomposition including Tucker decomposition using high order singular value decomposition as well as higher order orthogonal iteration, and train tenor decomposition. The author concludes by showing unsupervised methods and their application to a wide range of topics. Allows readers to analyze data sets with small samples and many features; Provides a fast algorithm, based upon linear algebra, to analyze big data; Includes several applications to multi-view data analyses, with a focus on bioinformatics.

A pioneering monograph on tensor methods applied to distributional problems arising in statistics, this work begins with the study of multivariate moments and cumulants. An invaluable reference for graduate students and professional statisticians. 1987 edition.

Tensors for Data Processing: Theory, Methods and Applications presents both classical and state-of-the-art methods on tensor computation for data processing, covering computation theories, processing methods, computing and engineering applications, with an emphasis on techniques for data processing. This reference is ideal for students, researchers and industry developers who want to understand and use tensor-based data processing theories and methods. As a higher-order generalization of a matrix, tensor-based processing can avoid multi-linear data structure loss that occurs in classical matrix-based data processing methods. This move from matrix to tensors is beneficial for many diverse application areas, including signal processing, computer science, acoustics, neuroscience, communication, medical engineering, seismology, psychometric, chemometrics, biometric, quantum physics and quantum chemistry. Provides a complete reference on classical and state-of-the-art tensor-based methods for data processing Includes a wide range of applications from different disciplines Gives guidance for their application

Tensors are ubiquitous in the sciences. The geometry of tensors is both a powerful tool for extracting information from data sets, and a beautiful subject in its own right. This book has three intended uses: a classroom textbook, a reference work for researchers in the sciences, and an account of classical and modern results in (aspects of) the theory that will be of interest to researchers in geometry. For classroom use, there is a modern introduction to multilinear algebra and to the geometry and representation theory needed to study tensors, including a large number of exercises. For researchers in the sciences, there is information on tensors in table format for easy reference and a summary of the state of the art in elementary language. This is the first book containing many classical results regarding tensors. Particular applications treated in the book include the complexity of matrix multiplication, P versus NP, signal processing, phylogenetics, and algebraic statistics. For geometers, there is material on secant varieties, G-varieties, spaces with finitely many orbits and how these objects arise in applications, discussions of numerous open questions in geometry arising in applications, and expositions of advanced topics such as the proof of the Alexander-Hirschowitz theorem and of the Weyman-Kempf method for computing syzygies.

This book presents the algorithms used to provide recommendations by exploiting matrix factorization and tensor decomposition techniques. It highlights well-known decomposition methods for recommender systems, such as Singular Value Decomposition (SVD), UV-decomposition, Non-negative Matrix Factorization (NMF), etc. and describes in detail the pros and cons of each method for matrices and tensors. This book provides a detailed theoretical mathematical background of matrix/tensor factorization techniques and a step-by-step analysis of each method on the basis of an integrated toy example that runs throughout all its chapters and helps the reader to understand the key differences among methods. It also contains two chapters, where different matrix and tensor methods are compared experimentally on real data sets, such as Epinions, GeoSocialRec, Last.fm, BibSonomy, etc. and provides further insights into the advantages and disadvantages of each method. The book offers a rich blend of theory and practice, making it suitable for students, researchers and practitioners interested in both recommenders and factorization methods. Lecturers can also use it for classes on data mining, recommender systems and dimensionality reduction methods.

Understand the benefits of robust statistics for signal processing using this unique and authoritative text.

With solid theoretical foundations and numerous potential applications, Blind Signal Processing (BSP) is one of the hottest emerging areas in Signal Processing. This volume unifies and extends the theories of adaptive blind signal and image processing and provides practical and efficient algorithms for blind source separation: Independent, Principal, Minor Component Analysis, and Multichannel Blind Deconvolution (MBD) and Equalization. Containing over 1400 references and mathematical expressions Adaptive Blind Signal and Image Processing delivers an unprecedented collection of useful techniques for adaptive blind signal/image separation, extraction, decomposition and filtering of multi-variable signals and data. Offers a broad coverage of blind signal processing techniques and algorithms both from a theoretical and practical point of view Presents more than 50 simple algorithms that can be easily modified to suit the reader's specific real world problems Provides a guide to fundamental mathematics of multi-input, multi-output and multi-sensory systems Includes illustrative worked examples, computer simulations, tables, detailed graphs and conceptual models within self contained chapters to assist self study Accompanying CD-ROM features an electronic, interactive version of the book with fully coloured figures and text. C and MATLAB user-friendly software packages are also provided MATLAB is a registered trademark of The MathWorks, Inc. By providing a detailed introduction to BSP, as well as presenting new results and recent developments, this informative and inspiring work will appeal to researchers, postgraduate students, engineers and scientists working in biomedical engineering, communications, electronics, computer science, optimisations, finance, geophysics and neural networks.

Sketching as a Tool for Numerical Linear Algebra highlights the recent advances in algorithms for numerical linear algebra that have come from the technique of linear sketching, whereby given a matrix, one first compressed it to a much smaller matrix by multiplying it by a (usually) random matrix with certain properties. Much of the expensive computation can then be performed on the smaller matrix, thereby accelerating the solution for the original problem. It is an ideal primer for researchers and students of theoretical computer science interested in how sketching techniques can be used to speed up numerical linear algebra applications.

Tensor is a natural representation for multi-dimensional data, and tensor computation can avoid possible multi-linear data structure loss in classical matrix computation-based data analysis. This book is intended to provide non-specialists an overall understanding of tensor computation and its applications in data analysis, and benefits researchers, engineers, and students with theoretical, computational, technical and experimental details. It presents a systematic and up-to-date overview of tensor decompositions from the engineer's point of view, and comprehensive coverage of tensor computation based data analysis techniques. In addition, some practical examples in machine learning, signal processing, data mining, computer vision, remote sensing, and biomedical engineering are also presented for easy understanding and implementation. These data analysis techniques may be further applied in other applications on neuroscience, communication, psychometrics, chemometrics, biometrics, quantum physics, quantum chemistry, etc. The discussion begins with basic coverage of notations, preliminary operations in tensor computations, main tensor decompositions and their properties. Based on them, a series of tensor-based data analysis techniques are presented as the tensor extensions of their classical matrix counterparts, including tensor dictionary learning, low rank tensor recovery, tensor completion, coupled tensor analysis, robust principal tensor component analysis, tensor regression, logistical tensor regression, support tensor machine, multilinear discriminate analysis, tensor subspace clustering, tensor-based deep learning, tensor graphical model and tensor sketch. The discussion also includes a number of typical applications with experimental results, such as image reconstruction, image enhancement, data fusion, signal recovery, recommendation system, knowledge graph acquisition, traffic flow prediction, link prediction, environmental prediction, weather forecasting, background extraction, human pose estimation, cognitive state classification from fMRI, infrared small target detection, heterogeneous information networks clustering, multi-view image clustering, and deep neural network compression.

This monograph provides a systematic and example-rich guide to the basic properties and applications of tensor network methodologies, and demonstrates their promise as a tool for the analysis of extreme-scale multidimensional data. It demonstrates the ability of tensor networks to provide linearly or even super-linearly, scalable solutions.