You can try to summarize memories tersely and point the agent to longer markdown files, but who knows if it will read it at the right time and only then.