Breast cancer is the most common malignancy in women of the Western world. Even though a large percentage of breast cancer patients show pathological complete remission after standard treatment regimes, approximately 30–40% are non-responsive and ultimately develop metastatic disease. To generate a good preclinical model of invasive breast cancer, we have taken a tissue-specific approach to somatically inactivate p53 and E-cadherin, the cardinal cell-cell adhesion receptor that is strongly associated with tumor invasiveness. In breast cancer, E-cadherin is found mutated or otherwise functionally silenced in invasive lobular carcinoma (ILC), which accounts for 10–15% of all breast cancers. We show that mammary-specific stochastic inactivation of conditional E-cadherin and p53 results in impaired mammary gland function during pregnancy through the induction of anoikis resistance of mammary epithelium, resulting in loss of epithelial organization and a dysfunctional mammary gland. Moreover, combined inactivation of E-cadherin and p53 induced lactation-independent development of invasive and metastatic mammary carcinomas, which showed strong resemblance to human pleomorphic ILC. Dissemination patterns of mouse ILC mimic the human malignancy, showing metastasis to the gastrointestinal tract, peritoneum, lung, lymph nodes and bone. Our results confirm that loss of E-cadherin contributes to both mammary tumor initiation and metastasis, and establish a preclinical mouse model of human ILC that can be used for the development of novel intervention strategies to treat invasive breast cancer.