The role of the endocannabinoid system in stress-related psychiatric symptoms has been investigated in many animal and human studies. Although most of these studies consistently report long-lasting effects of prolonged stress and trauma on the endocannabinoid system, the nature and direction of these changes are controversial. We reviewed the available preclinical and clinical studies investigating the endocannabinoid system alterations long after chronic stress and trauma. We propose that the effects of prolonged stress or trauma on the endocannabinoid system are different based on the developmental age of subjects at the time of experiencing the trauma and its repetitiveness and accumulative effects. The current literature consistently demonstrates decreased levels of endocannabinoid ligands and receptors if the trauma occurs in childhood, whereas decreased levels of endocannabinoid ligands and increased levels of cannabinoid receptors are reported when trauma has happened in adulthood. It is important to note that these changes are region-specific in the brain and also there are important sex differences, which are beyond the scope of this review.