Macrophages play a critical role in the initiation and progression of various solid tumors. However, their prognostic significance in pancreatic ductal adenocarcinoma (PDAC) is poorly understood. This study investigated the distribution patterns of macrophages in PDAC and possible association with the overall survival (OS). We found significant differences in macrophage density (identified by CD68 and CD163 immunopositivity; p < 0.001 for both) between primary cancer and paired adjacent normal tissues. Most macrophages in cancerous pancreatic tissues were located in the stroma rather than the islets (p = 0.032 and p < 0.001). We also demonstrated that a high total macrophage density (characterized by CD68 immunopositivity) correlated with an absence of jaundice before surgery (p = 0.03) and that a high density of M2 macrophages (characterized by CD163 immunopositivity) in the stroma strongly correlated with the tumors located in the tail and body of the pancreas (p = 0.04). In addition, OS was shorter in patients with high-density M2 macrophage infiltration than in those with low-density M2 macrophage infiltration (p = 0.012). Moreover, multivariate analysis revealed that dense M2 macrophage infiltration into the stroma was an independent prognostic factor for PDAC patients (p = 0.02).