| > "If we spend all this time and money labelling footage, how well is this going to work? How much footage do we need to label?" Start by labeling some data yourself. If you need to scale things up, you're going to need very clear rubrics for how things should be labeled and you're not going to be able to make them without having labeled some data yourself. Definitely think about what the easiest form of your task is. Labeling bounding boxes is time intensive, labeling whether there are 2 or more supervisors on the floor should be a lot easier, and you can easily label a bunch of frames all at once. You're going to need to figure out what tooling you will need for labeling, is this available out of the box, or will you need something custom? Label X data points yourself and do some transfer learning. Label another X data points and see how much better things get. The rough rule of thumb is performance increases logarithmically with data[1]. After you have a few points on the curve about how much better things get from more data, fit a logarithmic curve and make a prediction of how much data you will need, though be prepared that you might be off by a factor of 10. As others have mentioned, it's worth thinking about false positive/negative tradeoffs and how much you care about either. If the numbers you're extrapolating to aren't satisfactory, then yeah, you need to keep messing around with your training until you bend the curve enough that it seems like you'll get there with labeled data. [1] https://ai.googleblog.com/2017/07/revisiting-unreasonable-ef... |