Silicon photonics enables massive fabrication of integrated photonic devices at low cost, due to its compatibility with complementary metal oxide semiconductor (CMOS) process. It has emerged as one of the most important technologies for data communication and computations, as integrated electronic circuits are reaching their limits as well as incurring high energy costs. Recently, it has also shown the potential for other applications such as sensing and light detection and ranging (LiDAR). For miscellaneous silicon photonic devices, their versatile functions mainly come from design of device geometry. Intuitively, the design of single component with specific function is fulfilled by physics-based methods like analytical models, prior practical experiences and scientific intuitions. However, the physics-based methods are laborious and have high requirements on designers’ experience. Furthermore, the performance evaluation of light-material interaction with complicated geometries has to rely on numerical electromagnetic (EM) simulations, as the computation for irregular geometry is often non-intuitive. To make the design process more efficient, inverse design approaches have been introduced, which are assisted by various iterative optimization methods and deep neural networks (DNNs).