Insulin resistance is a major risk factor for type 2 diabetes. AMP-activated protein kinase (AMPK) is a drug target in the improvement of insulin sensitivity. Several insulin-sensitizing medicines are able to activate AMPK through inhibition of mitochondrial functions. These drugs, such as metformin and STZ, inhibit ATP synthesis in mitochondria to raise AMP/ATP ratio in the process of AMPK activation. However, chemicals that activate AMPK directly or by activating its upstream kinases have not been approved for treatment of type 2 diabetes in humans. In an early study, we reported that berberine inhibited oxygen consumption in mitochondria, and increased AMP/ATP ratio in cells. The observation suggests an indirect mechanism for AMPK activation by berberine. Berberine stimulates glycolysis for ATP production that offsets the cell toxicity after mitochondria inhibition. The study suggests that mitochondrial inhibition is an approach for AMPK activation. In this review article, literature is critically reviewed to interpret the role of mitochondria function in the mechanism of insulin resistance, which supports that mitochondria inhibitors represent a new class of AMPK activator. The inhibitors are promising candidates for insulin sensitizers. This review provides a guideline in search for small molecule AMPK activators in the drug discovery for type 2 diabetes.