Due to the durability and persistence of reservoirs of HIV-1-infected cells, combination antiretroviral therapy (ART) is insufficient in eradicating infection. Achieving HIV-1 cure or sustained remission without ART treatment will require the enhanced and persistent effective antiviral immune responses. Chimeric Antigen Receptor (CAR) T-cells have emerged as a powerful immunotherapy and show promise in treating HIV-1 infection. Persistence, trafficking, and maintenance of function remain to be a challenge in many of these approaches, which are based on peripheral T cell modification. To overcome many of these issues, we have previously demonstrated successful long-term engraftment and production of anti-HIV CAR T cells in modified hematopoietic stem cells (HSCs) in vivo. Here we report the development and in vivo testing of second generation CD4-based CARs (CD4CAR) against HIV-1 infection using a HSCs-based approach. We found that a modified, truncated CD4-based CAR (D1D2CAR) allows better CAR-T cell differentiation from gene modified HSCs, and maintains similar CTL activity as compared to the full length CD4-based CAR. In addition, D1D2CAR does not mediate HIV infection or stimulation mediated by IL-16, suggesting lower risk of off-target effects. Interestingly, stimulatory domains of 4-1BB but not CD28 allowed successful hematopoietic differentiation and improved anti-viral function of CAR T cells from CAR modified HSCs. Addition of 4-1BB to CD4 based CARs led to faster suppression of viremia during early untreated HIV-1 infection. D1D2CAR 4-1BB mice had faster viral suppression in combination with ART and better persistence of CAR T cells during ART. In summary, our data indicate that the D1D2CAR-41BB is a superior CAR, showing better HSC differentiation, viral suppression and persistence, and less deleterious functions compared to the original CD4CAR, and should continue to be pursued as a candidate for clinical study.