Yes, for the sake of your visitors. If your site is only http, then middleboxes can use your site load to opportunistically attack the visitor.
Of course, the probability that a visitor would not visit any other non-TLS site that day is low, but that probability is getting lower with every passing day.