waveform sync doesn't sync these clips that do match

This is the rarer case: these clips do match and sound like they match to my ears - including the audio slate - but Resolve doesn't agree. The quality sounds fine so I'm surprised. Here are the clips:

https://drive.google.com/open?id=13p8hx ... p=drive_fs

Very weird. The only thing I can think of is that there is a very distinct voice from the camera operator that is not picked up sufficiently by the audio recorder which may make the two audio sources appear to be different.

I also have near-zero luck with audio sync by waveform.

I mainly deal with dialogue audio and my workflow runs through the externally recorded audio through a effects chain to clean it up, EQ, compression, De-esser, etc.

Even with repeated "clap" spikes through the clip and external audio to help sync, I can never get resolve to sync. I get there's likely a lot of waveform changes to the external audio track due to the post-processing, but visually in the waveforms, they are almost exact.

Maybe we could have an option to configure how loose of a match we will accept for the waveform sync. Like some type of acceptable threshold deviation. Like if we have a couple of clap spikes in the audio along with 96% match of waveform, I don't see why it can't do a best effort sync.


Another thing I'd love is an automatic elastic wave compensation option with synced tracks.

In some cases, I have long running video and audio clips. To the best of my ability in sample rate and video fps and drop frame settings to match the video source, after about 6+ minutes, I get a bit of audio drift that causes the lip sync to look weird.

The idea again that if there were perhaps beginning and end clap spikes in the audio we're trying to sync, and the waveform sync could be intelligent to look for matches across slightly different waveform durations/stretches, it could automatically sync to a user defined source track, then do the elastic wave stretch of the other track to automatically adjust for any timesync drift.

But I don’t hear such a voice.
Seems worth the devs taking a look since they were just in there working on related syncing issues.


Sent from my iPhone using Tapatalk

But I don’t hear such a voice.
Sent from my iPhone using Tapatalk

Lol - so perhaps the voice I'm hearing is from my neighbor's dog...

(the voice is there and it is also caught by the audio recorder, but at a hugely attenuated level that probably isn't detected by the audio sync algorithm).

Listen to director (right after 313, take 3.... clap ... the camera operator says "camera set" as best as I can hear but it is barely audible in the recorder audio but quite audible in the camera audio... Also, there is a lot of mechanical noise in the recorder audio that isn't in the camera audio. While there are portions of the audio that are definitely the same, there is too much 'noise' and other differences. I feel like the audio sync algorithm should be tweakable to allow for a noise threshold or something along those lines...

Anyway, I agree that it is worth BMD testing this pair to determine why they don't sync.

I stand corrected! Clearly you’ve listened more closely than I.


Sent from my iPhone using Tapatalk

Might be interesting to try if this syncs with PluralEyes. I think they have a fully functional time-limited demo.

The sound quality there is very different.

True enough. I will try PluralEyes. I actually own a copy. Too bad it can’t make actual synced clips in Resolve!


Sent from my iPhone using Tapatalk

This is the rarer case: these clips do match and sound like they match to my ears - including the audio slate - but Resolve doesn't agree. The quality sounds fine so I'm surprised. Here are the clips:

https://drive.google.com/open?id=13p8hx ... p=drive_fs

Hi joe, i have heard two of those files. Resolve is a machine, and for machine those 2 files are not same.
In my ear, those 2 file are not same. Just acent "Action" from woman then after that the atmosfere are not same.
So to sync this, use your ear.

action.jpg (172.81 KiB) Viewed 3343 times

Ok no prob! Seems there’s quite a consensus on this one. Though I am curious what PluralEyes will do. I just wanted to make sure this wasn’t not matching because of a bug.


Sent from my iPhone using Tapatalk

I'm quite curious what you'll find, since in my experience PluralEyes is the best.

And...
drum roll please...



PluralEyes doesn't sync it either! Resolve is completely vindicated!

I also have near-zero luck with audio sync by waveform.

I mainly deal with dialogue audio and my workflow runs through the externally recorded audio through a effects chain to clean it up, EQ, compression, De-esser, etc.

Even with repeated "clap" spikes through the clip and external audio to help sync, I can never get resolve to sync. I get there's likely a lot of waveform changes to the external audio track due to the post-processing, but visually in the waveforms, they are almost exact.

They've recently fixed a bunch of bugs in this area - perhaps after my rather persistent requesting and - far more important - providing substantial reproducible test cases. You might want to give it another try in 18b6.

I also have near-zero luck with audio sync by waveform.
Maybe we could have an option to configure how loose of a match we will accept for the waveform sync. Like some type of acceptable threshold deviation. Like if we have a couple of clap spikes in the audio along with 96% match of waveform, I don't see why it can't do a best effort sync.


Another thing I'd love is an automatic elastic wave compensation option with synced tracks.

In some cases, I have long running video and audio clips. To the best of my ability in sample rate and video fps and drop frame settings to match the video source, after about 6+ minutes, I get a bit of audio drift that causes the lip sync to look weird.

The idea again that if there were perhaps beginning and end clap spikes in the audio we're trying to sync, and the waveform sync could be intelligent to look for matches across slightly different waveform durations/stretches, it could automatically sync to a user defined source track, then do the elastic wave stretch of the other track to automatically adjust for any timesync drift.

These enhancements would certainly be nice.

Another that I'd love to see is the ability for PluralEyes to create actual synced clips rather than just aligned clips in the timeline. Or better still, to add a feature to take all the clips in a timeline and turn them into Synced clips.

See here for more detail on this: viewtopic.php?f=33&t=161857

These files don't sync because they are different.
The video file audio cuts out part way through and the audio file goes on longer than the audio on the video file. No waveform sync will deal with this kind of mismatch. Waveform sync absolutely requires the audio to be the same in each file. A partial mismatch will make the process fail.

Are you scolding me?
Clearly the audio from different mics won’t be identical. So no, sync doesn’t require them to be identical.

As far as the difference in length, yes it looks like you’re right they are different. The proxy file’s audio got chopped by a different bug in Resolve.

I’m not perfect. Neither are you. Let’s try our best to be kind to each other.


Sent from my iPhone using Tapatalk

Merely stating a fact. There is nothing personal in my post.
I made no disparaging comment or mentioned ANYTHING other that a technical issue that I have experience with. I would expect those clips to fail to sync on most software.

It is important for the content to be the same even if from different microphones. Partial files are a problem for audio sync on most editing software.
If you posted damaged files for others to test then you will not get a meaningful result.
Was it the original or proxy that was not able to sync?

"I’m not perfect. Neither are you. Let’s try our best to be kind to each other."

Please show me where I was unkind or made a comment directed at you personally.

Perhaps I was in error in interpreting your tone.
Sometimes it’s hard to tell in writing. I apologize if I mistook your intent.

Several posts ago I said:

Ok no prob! Seems there’s quite a consensus on this one. Though I am curious what PluralEyes will do. I just wanted to make sure this wasn’t not matching because of a bug.

So I was surprised that it was being brought up again.

As to just stating fact - the sound does not have to be identical. That’s part of the magic of waveform matching. It decides what’s important and what’s not.

I’ve never written one of these so I don’t know the specifics - but clearly there’s some fuzzy matching going on and I wanted to be sure the match was intentionally failing as opposed to tickling another bug.

If you’ve been following this you’re likely aware that there have been several bugs fixed in this area.

Peace.


Sent from my iPhone using Tapatalk

No problem. My reference to "identical" content was not implying identical sources but identical coverage of the specific area that requires sync. If the start and end points of the audio are substantially different the algorithm may detect them as different sources. It will depend upon exactly how much overlap of content there is. As I don't have the original clip to test I assumed you were trying to sync the file that had audio missing.
Waveform sync has indeed had many issues in Resolve so I get where you are coming from.
My sincere apologies if you thought I was being difficult.

These files don't sync because they are different.
The video file audio cuts out part way through and the audio file goes on longer than the audio on the video file. No waveform sync will deal with this kind of mismatch. Waveform sync absolutely requires the audio to be the same in each file. A partial mismatch will make the process fail.

I agree with you Peter. Yes, these file are different for machine
In my eyes, these file different too. In my ear, the second one is really weak, and the first one more atmosphere there.
In this situation, using editor's ear is best thing.

action.jpg (172.81 KiB) Viewed 3076 times

These files don't sync because they are different.

Maybe the thread title should be changed to:

Waveform Sync Doesn't Sync Clips That Don't Match

Just came in here because I'm having a similar problem to Joe and I'm not appreciating some of you keyboard warriors' condescending tone. Any proper editor will know that camera/reference-audio and master-audio are almost always worlds apart and it is the specific job of an audio matching algorithm in an editing software to bridge that gap and find similarities. If I have the time to record perfect audio right into camera, what the hell am I audio matching for? At that stage I'd expect a timecode sync. Like Joe, I've gone and taken a few clips where Davinci was being useless to both PluralEyes and Premiere's internal audio match. Unlike in Joe's case, both worked. Davinci's audio match is a fig-leaf item that's in there to make the editing panel seem more sophisticated than it is at this stage. Not a problem for a FREE software, but I'd rather it be treated with more honesty and especially less schoolteacherly mannerisms in here.

If I have the time to record perfect audio right into camera, what the hell am I audio matching for?

Better mic preamps and lower noise floor in the audio recorder, mostly.

Just came in here because I'm having a similar problem to Joe and I'm not appreciating some of you keyboard warriors' condescending tone.
…
[description of how PluralEyes and Premiere do better audio matching than Resolve does]
…
but I'd rather it be treated with more honesty and especially less schoolteacherly mannerisms in here.

Thanks Simon. This is, unfortunately, a common tone in this group. I appreciate your mentioning that it doesn’t have to be that way. BMD seems fine with it though as they don’t try to steer or moderate such things.

PS Resolve’s sync did get much better in v18 and I have managed to sync 95% of two features with it.

"schoolteacherly" - I like that term

Would be cool if there was what I would call an adjustment or 'edit' interface to audio syncing when a definitive match can't be determined - something that would identify partial matches and let the user select the best match and maybe do a slip adjustment. Could even be done as a helper function in the manual sync panel. Now that's just crazy talk.