Some additional information: France is actually a leading country when it comes to Japanese Studies, to the point where high-level knowledge of French is a relevant skill in pursuing Japanese studies elsewhere, as many papers were historically only available in French.
I studies Japanese a bit in France a long time ago and was told that spoken French includes all the sounds of spoken Japanese so that a French speaker should have no problem pronouncing Japanese, apart from the accent.
This is in contrast to the endless trouble French speakers have with the English 'th'...
The stereotypical French speaker who has trouble with English [h] would definitely have the same problem with the Japanese [h], [ç] and [ɸ]. If they can't manage the tongue placement for English [θ], how are they supposed to handle Japanese [ɕ] and [ʑ]? And that's just the consonants.
Usually when someone thinks that language X has all the sounds of language Y, it's a speaker of language X who hasn't yet realized that their ears are too attuned to language X to recognize the sounds of language Y as something different. Don't trust your ears!
It's only the letter H and R (as transcribed in romaji) that are not immediate for a French speaker but they are not difficult. Certainly much simpler than 'th' in English.
(Source: not my ears but Japanese textbooks)
How would a French speaker pronounce し [ɕi] and じ [ʑi]? My guess would be « chi » [ʃi] and « ji » [ʒi]. Close enough to be understood, but different enough to be noticeable.
Japanese prononciation is not difficult for native French speakers. Off the top of my head, the main “difficulties” are the /r/ and the /h/ (which is always silent in French). That said, I am not sure it is much more difficult for native English speakers.