I would never understand it as "but you said you would do it", I would say "would" is safe to use with non-natives. But it may depend on the native language of the other person. There are some really weird languages out there!
The other day Duolingo did show me you can say "и A и B" ("и" being "and") to say "both A and B". Start a sentence with "and", a conjunction, without anything before to... "conjunt"... I still can't parse it without raising an exception.
After re-reading I probably see the issue, but you never said "I will do X" which changes to would in reported speech:
I will do X -> You said you would do X.
Did they understood "I would do X" as "I will do X"?