Waveform problems, one more time )))

I suppose Davinci Resolve needs to optimise their approach to waveform generation.
For example, files for waveform the same file in FCP need about several hundreds kilobytes, less than 1 Mb. It the same time Davinci use about 10 Mb for hold each channel of audio waveform for the same video file. My cache folder for audio stored some GB of ones arleady.

I have a project included a few hundred videos and I can't works smoothly because DVR each time, I open the project started generate waveforms. I was watching for that process counting the amount of PDL files, I summarised how many PDL files DVR had to create counted all the audio channels of each files.
I decided to gave some time to generate waveforms, and activate this process used the Cut Page Source Tape, to forced DVR creating waveform for each files in my project.
But when I see waveforms in the timeline or in Source Viewer, every time after the reload my project, DVR didn't show the waveform already generated for the clips and timelines and begun generate something else on my disks.
I very want to give a chance to Davinci to get my favourite NLE but every time it conforts me with annoying behavior )))
I use the forum as the way to ground out the mental stress )))) I just want to work and create a video but don't fight with tons of bugs.

each time I open the project started generate waveforms.

That's not normal.

What kind of Library are you working in?

Local Library

OK.

Internal drives only? No network, NAS, externals?

external RAID 5 for media files, the Cache is on my SSD internal
If Davinci has several PFL files for video clip in my Bin and each PFL has size of some Mb, Is it mean that if I'll put the video clip in the Source VIewer in audio wave mode I could see waveforms, right? But It wasn't.

Alright.

You might try a test. Start a new project using media from an internal drive. Don't use the RAID. Play around with the footage, then close Resolve and reopen.

Are the waveforms being rebuilt again?

I'll copy the some files from RAID to internal SSD than try to test them with a new project. It's the good idea. Thanks. I'll report back in two days.

But any case the storage space of waveforms is too large )))) 10 Mb of FCP vs 5 Gb of DVR for the same media file set is too much

How to manage PFL files?
How to delete unnecessary PFL?
What will happening when after some time of working with several projects and media files my cache will be a huge size?
Should I go into Cache folder and delete PFL files in manual mode?
What will if I rename folder with my source files and use Change Source Folder - will DVR begin generate all the waveforms for that new named folder again in the Cache folder next to old generated PFL?

How to manage PFL files?
How to delete unnecessary PFL?
What will happening when after some time of working with several projects and media files my cache will be a huge size?
Should I go into Cache folder and delete PFL files in manual mode?
What will if I rename folder with my source files and use Change Source Folder - will DVR begin generate all the waveforms for that new named folder again in the Cache folder next to old generated PFL?

1) I consider them temporary files. Used when the source files are used, then discarded.
2) There's no automated way. I have my source files in project specific folders, so when I'm done with a project I delete the PFL files.
3) You will get an out of space error from your OS. Other stuff will break as well, there's loads of processes in windows/osx that wants cache/temporary files and if they can't then they break in horrible ways.
4) When you don't need them any more, yes. If you delete PFL files Davinci needs it will recreate them.
5) I don't know. I assume so, yes. Mind you the manual way of a) moving source files, b) move PFL files, c) change source directory in Davinci, will work too

You can move the "Cache Files Location" (in Project Settings) to a folder with lots of space, it does not need to be on the main OS drive. I had mine on a network path for a while (for example J:\CacheClip ), it's a file type that is read on first encounter with the source file in the timeline, then not read again, so the speed of the drive doesn't matter

Thank you. The idea to store on a network drive is useful. Although I thought the cache drive have to fastest drive.

How to manage PFL files?

I do this by putting them on a 10 TB internal drive dedicated to Gallery/Cache/Proxies.

Would be great if Blackmagic add some tools for manage the different cache pieces right within DVR

Thank you. The idea to store on a network drive is useful. Although I thought the cache drive have to fastest drive.

There's different types of cache:
The video cache/proxy/optimized media (for the purposes of understanding cache, those are identical, in reality there's subtle differences I will not cover) requires fast drives.
The PFL files are waveform cache that (as I wrote above) are read once and are small compared to the source footage. I would argue that any current drive, no matter how slow can read the ~~13 MB per hour file (eg 1 hour is 13 MB, 2 hours is 26MB, 3 hours is 39MB etc). If that is a limiting factor, then I feel very sorry for you, that is an indication of massive problems.
1 Gbps network, which is minimum for any network video work, transfers 125 MB/s so the waveform cache takes ~~1/10 second per hour of footage.

And yes, I agree. I wish for a PFL generator I can let rip over large volumes of source files without having Davinci open.

But why PFL so big? 13MB for one hour per each audio track of source media. Why, do the FCP for example, could use hundreds times less waveform file size? )))

I created a new project, then imported all media files from the project I wrote above.
Then DVR generated some new waveforms. I checked all waveforms for all files. They all were in place.
Then I open my old project and DVR started to do something else - it was generating already generated waveforms.
I made the print of my file system - these files was in place and had 11 MB size.
Now these files reset and have about 100 КB ((((
Seemed the waveforms just gone )))
What can it be?

UPD Then I returned to my new project - and of course found than some file waveforms was gone

So is DVR removing PFL files some way in the different projects

I checked all waveforms for all files. They all were in place.

OK.

Setting aside file size (which really is a separate issue), are those waveforms still there when you re-open the new project?

I have open panel with search results - I watch for PFL files in my Cache folders with size less than 500 kb - and I see that there aren't any files. Then as I open old project - the files starting appear - I match them with file print screen and I can see the files with 11 MB now again 116 bytes

I checked all waveforms for all files. They all were in place.

OK.

Setting aside file size (which really is a separate issue), are those waveforms still there when you re-open the new project?

No, they are gone, because previous project remove some of them and started re-generate them again

I copied all my folder to net disk.
There I found some generated PFL files.
I close DVR and copied these generated files to my SSD cache which is active one.
Then I open project and just copied files are gone ))))

I waited for DVR generated all waveforms in my problem project. Then I test all my files for waveforms. Everything was alright. Then I made list of my files and match them with previous list, made after new project generation process with Diff utylite - there wasn't any difference. But when I reopen project - again some files started generate. And matching showed difference in the files sizes

And DVR can't show waveforms for generated files until it doesn't generate new files.

OK.

Something weird is going on with your system. What OS? What version of Resolve?

OK.

Something weird is going on with your system. What OS? What version of Resolve?

The last Sonoma, 19b2. I’ve been thinking I even didn’t try a simple universal way of fixing everything like reboot of my station ))) maybe is there something that will gone after rebooting?

When one more time Davinci have finished creating of waveforms for my project , I started to work with edit, then I imported two new files. The waveforms were created, but they didn’t appear in Davinci. I can’t see them in timeline, in Source Viewer Audio mode, metadata-file section. Nowhere )))) That problem is only for these two new files.
I didn’t restart project yesterday, because I want to work a bit for my project at last ))) l gonna try to restart today and see what will happen that time ))

After reboot nothing happened.
Then I delete the problem PFL and regenerate it (Aside note : DVR do not generate something else, but I did't open yet project using the same media files)
The waveform appeared in Viewer Audio Mode, but timeline clips have no it. Even after match frame and replace back again the same range from the same source media. I see audio in Viewer but can't see it on the timeline on that clip )))
UPD It was subsublip. When I change match frame to match frame to source - and overwrite clip back - the waveform appears at last.

I think Davinci has some problem with managing subclips. I faced the problem mostly when subclips is using in workflow. Maybe peoples who do not used subclips in their everyday workflow do not faced with my issues )))

I am one of those who never uses subclips.

And since you're in the 19 Beta, you should be reporting bugs in that forum for the developers.