Autoencoders are neural networks which project data to and from a lower dimensional latent space, the projection being learned via training on the data. While these networks produce impressive results, there is as yet little understanding of the internal mechanisms which allow autoencoders to produce such results. In this work, we aim to describe how an autoencoder is able to process certain fundamental image attributes. We analysed two of these attributes in particular : size and position. For the former, we study the case of binary images of disks, and describe the encoding and decoding processes. In the case of position, we describe how the encoder can extract the position of a Dirac impulse. Finally, we present ongoing work into an approach to create a PCA-like autoencoder, that is to say an autoencoder which presents similar characteristics to the PCA in terms of the interpretability of the latent space. We shall show preliminary experimental results on synthetic data.