Magic Mask shenanigans

First of all, if I have a Fusion inside Resolve question, about something that is inherently Fusion, should I post in this category or in the Resolve one?

I'm having a bit of trouble getting the Magic Mask to work as intended, at least according to what the controls say. For example, the track forward and backward one frame button will take me to wherever it wants at times. Mostly only the forward one is kind of reliable, but for example right now I was in frame 1468 and pressing forward one frame took me to frame 1289. Pressing the backward one frame button usually takes me to the very beginning, as if I was pressing the other button below that is there for that purpose. Is this how it's supposed to be?

Granted, I'm using it on a difficult piece of footage that wasn't shot with this in mind. It's from a Canon DSLR and the shutter speed is rather low, so it's not exactly BMPCC 6K BRAW. Added to this, it's a closeup of someone's hand drawing on the ground with a stick, and the stick is slightly darker than the dirt, so it's not an easy task for Magic Mask, and I have to manually draw add and subtract on many frames, almost all of them.

Also, the cache doesn't work very well. I've had to redo the tracking countless times because sometimes I see a frame where the hand is gone, or something else that wasn't there before. I mean, it doesn't save every single frame until that frame is overwritten manually because of an add or subtract, or some other change.

So I'm wondering if Magic Mask is the right tool for this. I was going to animate a polygon mask, but this is a hand with a stick going sideways back and forth, so it would take forever to keyframe all that.

Is there any other tool in either Resolve or Fusion for rotoscoping that would do a faster and better job?

What is the intention of the mask? i.e. why are you trying to mask the hand and/or the stick?
Does it need to be extracted to a different background or do you have other purposes for it?

Because contrary to the name Magic Mask isn't really magical. It's pretty darn impressive, but it's magic by the virtue of being trained on other footage. So it will work better with things that are more likely to have been in the training footage. A hand drawing in the dirt with a stick isn't one of those.
So depending on what your goal is, actual rotoscoping might be quicker/easier than just yelling at your computer 'I want that part, no not that part, yes, that part, but leave out that part' over and over again which is basically what adding/subtracting strokes is.

Is there any chance you can show us the shot? Also: how long is the shot? The longer it is, the more difficult it will be to track consistently over time.

What is the intention of the mask? i.e. why are you trying to mask the hand and/or the stick?
Does it need to be extracted to a different background or do you have other purposes for it?

Because contrary to the name Magic Mask isn't really magical. It's pretty darn impressive, but it's magic by the virtue of being trained on other footage. So it will work better with things that are more likely to have been in the training footage. A hand drawing in the dirt with a stick isn't one of those.
So depending on what your goal is, actual rotoscoping might be quicker/easier than just yelling at your computer 'I want that part, no not that part, yes, that part, but leave out that part' over and over again which is basically what adding/subtracting strokes is.

Is there any chance you can show us the shot? Also: how long is the shot? The longer it is, the more difficult it will be to track consistently over time.

Well, it works fairly well, but let's make it simple, if you had to rotoscope anything in DaVinci Resolve/Fusion, what would be the best way to do it, besides a keyframed polygon mask and Magic mask?

There is no besides. Those are basically the two best options, but it always depends on the subject and the shot. If you can track the shot to make it more stable, then roto becomes easier. Roto for masks i.e. local color correction are easier than masks for subject extractions. There is not one simple answer, that's why I was asking more questions.

There is no besides. Those are basically the two best options, but it always depends on the subject and the shot. If you can track the shot to make it more stable, then roto becomes easier. Roto for masks i.e. local color correction are easier than masks for subject extractions. There is not one simple answer, that's why I was asking more questions.

What about something called "Roto Assist"? It has some preferences in the Fusion settings (the ones inside Resolve), and if I right click on the polygon line and then go to the polyline submenu I can enable or disable it, but I haven't seen anything happen as I keep animating points in the polygon.

I don't think anybody has ever figured out what rotoassist does and how it should be used.

What about something called "Roto Assist"? It has some preferences in the Fusion settings (the ones inside Resolve), and if I right click on the polygon line and then go to the polyline submenu I can enable or disable it, but I haven't seen anything happen as I keep animating points in the polygon.

Roto Assist is like magnetic lasso for high contrast points. To assist with roto of well defined and high contrast edges.

Roto Assist: Enable the Roto Assist button when you begin painting with the Polyline Stroke tool. The polyline points snap to the closest edge as you click to add points to the shape. A cyan outline indicates the points that have snapped to an edge. There are three main Roto Assist options selectable through the drop down menu:

— Multiple Points: When enabled, a single click on a high contrast edge adds multiple points to define the entire edge, instead of having to add each point individually. This is a one time only click. The second click reverts to single point edge detection.

— Distance 8: Opens a dialog where you can set the pixel range within which searching for an edge takes place.

— Reset: Used for resetting the snap attribute of all snapped points. After resetting, the points become unavailable for tracking.