[MarkSweep]

An example of text segmentation library for Japanese:

https://en.m.wikipedia.org/wiki/MeCab

Like you said, it does come with a dictionary to work properly.