Healthy aging comes with a decline in episodic memory - our ability to encode and retrieve events in their spatial and temporal contexts. A key circuit that supports this process involves the prefrontal cortex and medial temporal lobes. Structural and functional impairments to this neural system occur in parallel to the cognitive deficits that present in late-life. In a sample of 208 individuals, we observed that cortical thickness in the prefrontal cortex was related to episodic memory performance in older adults, and volumes of medial temporal lobe structures such as the hippocampus were related to memory performance in younger adults. Further, age-related differences in memory performance were associated with increased functional connectivity between the hippocampus and prefrontal cortex. These findings highlight how this critical circuit differentially supports episodic memory in younger and older adults.