The new section III in DSM-5 suggests pathological personality traits and impairments in personality functioning such as empathy to be used for identifying personality disorders (PDs). Previous research has also theoretically and empirically advocated that psychopathology is related to the general Five Factor Model (FFM) of personality traits. The objective of the present study was to investigate the hierarchical structure of the 10 DSM PD categories using the FFM count technique (Miller et al., 2008), and to conceptualize PDs with empathy dimensions. We measured PDs and 4 dimensions of empathy (emphatic concern, perspective-taking, fantasy, and distress) in a medium-sized community sample. The results showed that higher order factors such as externalizing and internalizing could be applied to PDs based on FFM scores. PD could furthermore be conceptualized using two of the empathy dimensions, low emphatic concern and high distress, and specific PD categories could be conceptualized by using distinct dimensions of empathy (e.g., histrionic PD with high fantasy, or dependent PD with high distress). The discussion concludes that PDs based on self-reported FFM show conceptual validity, and that the presence of symptoms of PDs potentially may be screened in the community population by using empathy measures.