This paper presents an efficient memory access interface architecture for H.264/AVC encoder. In the implementation of H.264/AVC encoder, the bandwidth compression of frame memory becomes a challenging issue due to some bandwidth intensive coding tools, such as multiple frames motion estimation, deblocking filter and INTRA mode decision. In this work, by analyzing the memory access patterns of each coding function module of H.264/AVC, an efficient memory access method for the Direct Memory Access (DMA) module is proposed. The proposed method carefully designed an efficient memory mapping method to decrease the memory response delay. Simulation results show that over 50% memory access cycles can be saved by using proposed method.