Mental rotation (MR) is a cognitive skill whose neural dynamics are still a matter of debate as previous neuroimaging studies produced controversial results. In order to investigate the underlying neurophysiology of MR, hemodynamic responses from the prefrontal cortex of fifteen healthy subjects were recorded with functional near-infrared spectroscopy (fNIRS) during a novel MR task that had three categorical difficulty levels. Hemodynamic activity strength (HAS) parameter, which reflects the ratio of brain activation during the task to the baseline activation level, was used to assess the prefrontal cortex activation localization and strength. Behavioral data indicated that the MR requiring conditions are more difficult than the condition that did not require MR. The right dorsolateral prefrontal cortex (DLPFC) was found to be active in all conditions and to be the dominant region in the easiest task while more complex tasks showed widespread bilateral prefrontal activation. A significant increase in left DLPFC activation was observed with increasing task difficulty. Significantly higher right DLPFC activation was observed when the incongruent trials were contrasted against the congruent trials, which implied the possibility of a robust error or conflict-monitoring process during the incongruent trials. Our results showed that right the DLPFC is a core region for the processing of MR tasks regardless of the task complexity and that the left DLPFC is involved to a greater extent with increasing task complexity, as suggested by previous neuroimaging literature.