Handling a Digital Crisis

Michael R Sheehan


January 19, 2022

Ask yourself some tough questions – how well can your business withstand a digital crisis?

On January 9, 2022, Jackson Hospital in Marianna, Florida USA had to face the reality that the charting system used by the emergency room was unreachable due to ransomware. The IT Department of the 100-bed facility made the call to turn off all systems to stop the spread.

All electronic records and systems were abandoned for a time and employees were forced to keep the hospital operational with pen and paper. Jackson Hospital was experiencing a Digital Crisis.

How much downtime, economic loss, or system loss can you tolerate for both critical and non-critical systems? What would be the impact if your business systems were down for an hour, a day, even a week?

What if you are a biotech or pharmaceutical performing research or manufacturing? What if you were the local hospital providing the only ER for over 200 square miles like Jackson Hospital? What about data security, data entry, or other recordings while down? How do you address regulatory issues like HIPAA and HITECH during the digital crisis?

How about ransomware attacks? Denial of Service attacks? What if the host that runs your cloud services is having a major outage?

How do you calculate lost customers, decreased productivity, or missed transactions? Do you have business continuity, digital disaster prevention and recovery plans in place?

Knowing your vulnerabilities is the very first step to overcoming a digital crisis. Like your car’s GPS, you cannot navigate where you want to go unless you first know where you are.

What are the next steps? Well, that depends on your business, but all businesses have elements in common.

  • Know the digital resources that your company is leveraging

  • Know and secure your perimeter, regardless of scope of that perimeter, cloud included

  • Evaluate which business continuity solution is right for you

  • Harness the power of the Cloud

  • Leverage virtualization to save effort, time, and expense

  • Implement a recovery system that works in minutes, not hours

  • Document continuity processes and procedures ahead of time

  • Test the solution regularly, look for gaps in process, execution, and success rate

A 2016 report by the Ponemon Institute found that human error was the second most common cause of system failure – and therefore business downtime – accounting for around 22% of all incidents. That human error does have impact to your digital company.

On March 2, 2017, most AWS services were brought down for hours, because an Amazon employee debugging code in the billing system made a mistake. This caused most AWS customers to experience problems and outages. Outages that are completely out of the control of those customers.

The vast array of recent outage examples (review our posts Facebook Outage Part 1, Part 2, Part 3 and Amazon AWS Outage) demonstrates that companies commonly have unfit solutions in place. These solutions lack processes including oversight of change control, dependency review of external SaaS resources, and the ability to function when a crisis occurs.

Usually, it boils down to failures in two major areas: training and/or planning. This means that, in most circumstances, the negative effects of an outage are preventable.

Organizations should be focusing their efforts on:

Investing in People

  • This includes training, checklists, updated documentation, rehearsals, and specific crisis tasks assigned to each employee

Investing in Prevention Services

  • Whether hardware/software solutions, or written and practiced protocols, even regular digital perimeter checks (this includes Cloud services and solutions) can prevent errors

Establishing & Rehearsing Recovery Plans

  • Practice makes perfect, and if you have rehearsed the plan, you can act swiftly when required without stopping to ask questions or wait for authorization.

Establishing & Maintaining Oversight Procedures

  • Many of the recent outages occurred because of lax attention to protocols already in place. Without regular oversight, they are useless.

Performing Regular Dependency Audits on your Company’s Digital Assets

  • How many processes will fail if one of them fails? How often do you test the processes for recovery purposes? Where is your digital ‘weakest link?’

Investing in Documentation Strategies to Store & Distribute Plans, Processes & Educational Material

  • Outdated and invalid processes (and the documentation your employees rely on for those processes) can be the single most expensive mistake your company can make if you allow it.

It is important to note that hackers and bad actors do not want to crash backbone hosting organizations, like Facebook, AWS, Microsoft or Google. They need them to carry their malware. But they exploit these loopholes in the hopes of gaining further access, planting seeds, unlocking back doors, etc.

Those that host your cloud resources have done their due diligence, but they are not responsible for your companies’ security while using their services, you are. Ultimately, it rests upon you to safeguard your content, your data, and your business continuity.

Based in Dedham, MA, ConaLogix was founded in 2018 as a fractional CIO and advisory resource for the pharmaceutical, life science, and biotech industries. We provide C-level Information Technology services on a virtual basis, assisting with architecture, integration, and testing. Using a unique, customized approach, the ConaLogix team collaborates with entrepreneurs and scientific core teams to support their vision, while guiding the most efficient development model that benefits data management and core requirements

What Are You Doing With Your Data?

Parents Guide to Online Learning at Home: Blog Image
Parents Guide - Online Learning at Home

Please download your guide now!