Why'd you run them in kubernetes? Seems like extreme overkill for launching a short lived container for an integration test. What could kubernetes possibly add to that?
Because we are a big company and would like to utilize resources better.
We also want homogeneity in tech when possible (we already heavily use kubernetes, we don't want to keep docker hosts anymore).
Teams of testers need to be accounted in terms of resource quotas and RBAC.
What exactly do you see as an overkill in wanting to run short-lived containers in kubernetes rather than in docker (if we already have kubernetes and "cook" it ourselves)?
That reasoning seems more like one from policy/cargo cult rather than reasoning specific to your org. For something short lived and meant to be isolated I wouldn't want to subject them to even more infrastructural dependencies outside their control.
It's overkill because these containers typically have a lifetime counted in single digit seconds, and it takes kubernetes not only more time but also more compute resources to decide where to allocate the pod than to actually just run the thing.
We also want homogeneity in tech when possible (we already heavily use kubernetes, we don't want to keep docker hosts anymore).
Teams of testers need to be accounted in terms of resource quotas and RBAC.
What exactly do you see as an overkill in wanting to run short-lived containers in kubernetes rather than in docker (if we already have kubernetes and "cook" it ourselves)?