Unfortunately multi-GPU training doesn't scale linearly yet [1] so it's often a better call to get a larger card then two smaller ones, at least for the single-model case.
(non-TF) Keras has notoriously bad multi-GPU support though (and was generally not well optimized. Case in point, the latest version just re-exports/forwards to tf.keras).
Looking at something like https://lambdalabs.com/deep-learning/gpu-benchmarks or https://github.com/tensorpack/benchmarks/tree/master/other-w..., multi-gpu scaling on 2080tis seems pretty darn close to linear. Plus, there are benefits to having more than one accelerator handy on a local workstation. For one, it's much easier to have multiple experiments running simultaneously or to run parallel training (e.g. hyperparameter search or RL episodes). Given that only the uber-expensive enterprise cards have proper virtualization/time sharing, trying this workflow on a Titan RTX will most likely be suboptimal unless you always run models that can make use of most of the memory and compute (no RNNs, no Neural ODEs, etc.)
Looking at something like https://lambdalabs.com/deep-learning/gpu-benchmarks or https://github.com/tensorpack/benchmarks/tree/master/other-w..., multi-gpu scaling on 2080tis seems pretty darn close to linear. Plus, there are benefits to having more than one accelerator handy on a local workstation. For one, it's much easier to have multiple experiments running simultaneously or to run parallel training (e.g. hyperparameter search or RL episodes). Given that only the uber-expensive enterprise cards have proper virtualization/time sharing, trying this workflow on a Titan RTX will most likely be suboptimal unless you always run models that can make use of most of the memory and compute (no RNNs, no Neural ODEs, etc.)