Principal component analysis (PCA) is a fundamental tool used for the analysis of datasets with widespread applications across machine learning, engineering, and imaging. The first part of the talk is dedicated to solving Robust PCA from subsampled measurements, which is the inverse problem posed over the set that is the additive combination of the low-rank and the sparse set. Here we develop guarantees using the restricted isometry property that show that rank-r plus sparsity-s matrices can be recovered by computationally tractable methods from p=O(r(m+n-r)+s)log(mn/s) linear measurements. The second part of the talk is focused on finding an efficient way to perform large-scale optimization constrained to the set of orthogonal matrices used in PCA and for training of neural networks. We propose the landing method, which does not enforce the orthogonality exactly in every iteration, instead, it controls the distance to the constraint using computationally inexpensive matrix-vector products and enforces the exact orthogonality only in the limit. We show the practical efficiency of the proposed methods on video separation, direct exoplanet detection, online PCA, and for robust training of neural networks.