[bryanrasmussen]

and extract segments of images where the object are in the image as I understand it?

A segment then is a collection of images that follow each other in time?

So if you have a video comprised of img1, img2, img3, img4 and object shows in img1 and img2 and img4

Can you catch that as a sequence img1, img2, img3, img4 and can you also catch just the object img1, img2, img4 but get some sort of information that there is a break between img2 and img4 - number of images break etc.?

On edit: Or am I totally off about the segment possibilities and what it means?

Or can you only catch img1 and img2 as a sequence?

[nsonha]

I'm not in the field and what SAM does is immediately apparent when you view the home page. Did you not even give it a glance?

[bryanrasmussen]

Yes I did give it a glance, polite and clever HN member, it showed an object in a sequence of images extracted from video, and evidently followed the object from sequence.

Perhaps however my interpretation of what happens here is way off, which is why I asked in an obviously incorrect and stupid way that you have pointed out to me without clarifying exactly why it was incorrect and stupid.

So anyway there is the extraction of the object I referred to, but also seeming to follow the object through sequence of scenes?

https://github.com/facebookresearch/segment-anything-2/raw/m...

So it seems to me that they identify the object and follow it for a contiguous sequence. Img1, img2, img3, img4, is my interpretation incorrect here?

But what I am wondering is - what happens if the object is not in img3 - like perhaps two people talking and shifting viewpoint from person talking to person listening. Person talking is in img1, img2, img4. Can you get that sequence or is it just img1, img2 the sequence.

It says "We extend SAM to video by considering images as a video with a single frame." which I don't know what that means, does it mean that they concatenated all the video frames into a single image and identified the object in them, in which case their example still shows contiguous images without the object ever disappearing so my question still pertains.

So anyway my conclusion is what said when addressing me was wrong, to quote: "what SAM does is immediately apparent when you view the home page" because I (the you addressed) viewed the homepage I wondered about some things? Obviously wrong things that you have identified as being wrong.

And thus my question is: If what SAM does is immediately apparent when you view the home page can you point out where my understanding has failed?

On edit: grammar fixes for last paragraph / question.

[nsonha]

> A segment then is a collection of images that follow each other in time?

A segment is a visually distinctive... segment of image, segmentation is basically splitting an image into objects: https://segment-anything.com, as such it has nothing to do with time or video.

Now SAM 2 is about video, so they seem to add object tracking (that is attributing same object to the same segment throughout frames)

The videos in the main article demonstrate that it can track objects in and out of frame (the one with bacteria or the one with boy going around the tree). However they do acknowledge this part of the algorithm can produce incorrect result sometimes (example with the horses).

The answer to your question is img1, img2, img4, as there is no reason to believe that it can only track objects in contiguous sequence.

[bryanrasmussen]

Thanks!