Another reason to use HJKL is that over a 1200bauds line, it is faster to send a single byte than the 3 bytes generated by the arrow keys (^[[D^[[A^[[B^[[C)
The ADM3A was a "dumb" terminal, constructed using a board full of 7400-series TTL ICs with no microcontroller. Probably to simplify the design, it wasn't capable of sending any multi-character escape sequences. (The ANSI escape sequences hadn't been standardized at that point yet either.)