Coherent diffractive imaging (CDI) is revolutionizing the physical and biological science fields by first measuring the diffraction patterns of nano-crystals or non-crystalline samples and then inverting them to high-resolution images. The well-known phase problem is solved by the combination of coherent illumination and iterative computational algorithms. In particular, ptychography – a powerful scanning CDI method – has found wide applications with synchrotron radiation, high harmonic generation, electron and optical microscopy. However, iterative algorithms are not only computationally expensive, but also require practitioners to get algorithmic training to optimize the parameters and obtain satisfactory results. These difficulties have thus far prevented CDI from being accessible to an even broader user community. Here we demonstrated deep learning CDI using convolutional neural networks (CNNs) trained only by simulated data. The CNNs are subsequently used to directly retrieve the phase images of monolayer graphene, twisted hexagonal boron nitride and a Au nanoparticle from experimental electron diffraction patterns without any iteration. Quantitative analysis shows that the phase images recovered by the CNNs have comparable quality to those reconstructed by a conventional iterative method and the resolution of the phase images by the CNNs is in the range of 0.71-0.53 Å. Looking forward, we expect that deep learning CDI could become an important tool for real-time, atomic-scale imaging of a wide range of samples across different disciplines.