Cloud Store 20TB - Erratic Behaviour After Hard Reboot

What happened: All connected clients simultaneously lost access to Cloud Store. Only the second time this has happened since new, and no amount of attempts to reconnect would work, so I reluctantly did a hard reset by pulling the power cord. Why didn’t you guys put a soft-shutdown power button on the thing? It freaks me out just yanking it from the outlet!

So - Cloud Store powers up, all the clients reconnect but for several hours the access speed seemed sluggish. I look at the HDMI attached screen and there’s now some weird stuff going on.

Issues:

3x Macs are connected to the Cloud Store, all presently have access to the directories, but the User Activity window displays either only one client, 2 clients, or nothing connected at all, yet at all times it will show a transfer speed for two connections.

10G connections (using direct cables into each 10G Mac Ethernet port) are reported in User Activity as being 1G connections, they sometimes change to 10G, then revert back to 1G again.

Blackmagic Cloud Store Setup app repeatedly loses connection (even while Finder still has active access to the Cloud Store), and it will not reconnect until my Mac is restarted.

Backups seem sporadic. When I powered up after losing connection, Cloud Store initially reported 5.5TB of media needed writing to my USB-C backup drive, but by the end of the day this had climbed to 11.8TB - as if it hadn’t been backing up for months? I went into the office on the Saturday to check up on this and it was finally reporting 100% backup. Aside: Status updates (on-screen and in app) leave a lot to be desired. Was Cloud Store completing a backup, starting a fresh backup, or was it verifying an existing backup after a hard reboot? The usability is great for my team - but as an admin I find Cloud Store user-friendly to the point of being frustratingly vague about what it’s doing.

Cloud Store also no longer has Internet access, even plugging directly to a router with DHCP the Cloud Store no longer sees the outside world.

So - it’s started behaving erratically with 12TB of active projects on it - which is not good for my sleep!

cloud-store-issues.jpg (617.91 KiB) Viewed 10529 times

Hi again! Myself and my IT people have been deep diving into why the Cloud Store has been behaving weirdly, and a quick call to Blackmagic UK support confirmed what we were beginning to suspect: There's no way to configure the IP settings of the 1GB and 10GB ports independently of each other, and likely no way/no plans to do so. This has been throwing traffic out of whack on our network, and also throttling/disconnecting Cloud Store clients. This is potentially a HUGE ISSUE when trying to integrate Cloud Store into the network of a facility or larger IT infrastructure!

Detailed description from IT admin below:

Device is advertised as 4x10gb + 2x1gb Ethernet ports, however there is no way to configure ports individually.

Target optimal network:

4x Client MAC. Each have two NIC, 10gb & 1gb. 10gb for dedicated Blackmagic connection and 1gb Local lan (via 1gb switch) for internet/office connectivity.

The local lan (192.168.0.0/24) has DHCP/DNS service running and utilises a Named router on local lan subnet for Internet.

The Blackmagic only has ability to define one set of network credentials via DHCP or Static.
For the Blackmagic to access the Internet it has to be assigned an address (DHCP or Static) for the 1gb Local Lan which it connects to via a 1gb Port.

Because the Blackmagic has no ability to manually configure ports individually, connecting the Client MAC to the local LAN & directly to the Blackmagic results in both NIC being on the same Local Lan subnet.
Traffic to/from Blackmagic/Client can not be guaranteed to use the 10gb port and subsequently internet/office traffic cannot be restricted to use the 10gb port.

The only way to ensure 10gb connection to the Blackmagic and that the Blackmagic has internet connection, is to disable the 1gb MAC port and solely use the 10gb connection between the Clients and the Blackmagic.

I am having similar issues with the cloud store. It will only connect by the 5g connection or the 1g connect. I did an RMA and they sent it back to me saying it is working. As soon as i hooked it back up it worked for 2 minutes and then not traffic on the 10g connection. I have tried everything. new 6A cables, hooking up directly to the Mac. Nothing over the 10g connection. this thing worked great for the first day, and now its not worth the price. Its an overpriced storage bin currently that only one computer can access. I hope I can return it to BH or Blackmagic at this point. if i find a solution before that I will post but since BMD themselves have not addressed this I highly doubt it. Good luck.

Yeah, it's a shame, because I can't knock the performance at all - when it's up and running it doesn't even break a sweat serving multiple streams of 4K ProRes 4444 to all my users simultaneously.

But, my IT guys (seriously high-end developers) couldn't believe there wasn't a published schematic, a true enterprise-level console, a maintenance port, backup verification, provision for error reporting etc. They were like, "So, it's providing this level of critical operation, but there's not even a service running to send an email if the RAID goes degraded? It's just a dumb NAS box?"

They've been baffled why something at this price would be built around network hardware with a lack of configurability that can make it extremely difficult to integrate - and trying to get more than one Cloud Store on the network would be a nightmare.

The Cloud Store's 'plug and play' simplicity is its achilles heel!

If I hadn't people to call on who understand TCP/IP at a scientific level and were able to model the corporate network traffic and figure out why the Cloud Store was acting up, we'd not have been able to reach the (albeit imperfect) solution we did.

Has this been fixed with the latest SW update ?