PERFORMANCE AND RELIABILITY OF HEALTHCARE SYSTEMS (TECHNICAL)

Technology today is offering a wealth of opportunities for the care industry, but with that also comes a depth of challenges. Not least the ever-moving target of new technologies adapted by vendors, employees and other care providers. For critical medical devices we can still turn to familiar certifications for standards of quality. A well established and extensive range of ISO standards will not only provide the quality levels necessary, but go into detail on how to achieve it, for example device calibration, auditing and testing, while at the same time pulling in national standards, such as ANSI patient safety standards in the U.S. or OHSAS in the U.K.

The HIPAA acts are a relatively new addition, (since the mid-1990s) and introduce a change in direction towards the protection of data in its logical format, as against the traditional point of view of managing systems. This works well in the current environment, where technologies are changing faster than advisory groups can process requests to update standards. Technology is becoming increasingly integrated, physical components are now typically virtual, and processor microcode is becoming software programs on generic commodity hardware.

The upshot of this is the current approach in the industry is data centric and is starting to turn away from the layered provision model that evolved technology into what it is today.

Now data is viewed as the content that the systems are built around, to the extent that data that isn’t attributed to the User, is seen as meta data and another wrapper around the payload. Relevant data is tied to the business use case, and the core reason why the systems are built.

Where the “User” is a medical patient at a clinic, HIPAA’s orientation around Protected Health Information works well in this context.

At Infocare project planning takes an approach of HIPAA compliance from inception, the team are educated on HIPAA HITECH and Privacy Rule across the product workflow. Processes are ISO 9001 and 27001 compliant end to end from the creative Agile teams in Development across to ITIL controlled production deployments.

Just like other enacted regulations (such as Sarbanes–Oxley etc.), the focus in HIPAA for I.T. is on data security. For compliant companies such as Infocare, that security is well documented, from the technical implementations to the work practices, however, standard medical regulations do not address performance and reliability in any meaningful way.

In clinics across the world, administrators have the same experience of “the system” on a go slow or “hanging for a minute”. It’s not tied to any particular vendor or application, and the possible source or cause of the issue is wide ranging. It could be the user’s computer, the office wi-fi or “server side”, to name just a few of the many plausible explanations.

Not only does this affecting work momentum but there’s the bigger questions, like how safe is the system at the back end from to the next power outage or storm, how long will it take to be brought back into operation or what will be lost by any incident.

In 2019 an internet backbone outage in the U.S. caused 911 services to go down for two days. Amongst many other unrelated critical outages that year, all Facebook services were down for a day, and global Google e-mail was down for half a day.

For Infocare the challenge is to figure out how to make security and performance work together from the systems design stage onwards throughout the product lifecycle, bearing in mind a high level of security & compliance is the first stipulation. Developers are constantly “smart thinking” new ways to make the product run smoother, faster, and improve the user experience. The product then undergoes extensive testing in the client’s environment. But even with this level of confidence in-house, Infocare still need to manage the integrated environment and take into account the impact external systems will have on the product, for example fluctuating quality in a clinic’s internet connection or a personal computer with performance issues.

A suite of monitoring tools with visual dashboards are the eyes and ears for the backend engineering team. Everything from hard disks to networks to software components are monitored, creating an extensive range of monitored end points that are then categorized on the services affected and if the error is critical enough to page the on-call engineers.

Predictive analysis in the system assesses if an outage is expected, for example if the volume on a network pipe is higher than typical and growing. Intelligent infrastructure tools are used to repair failing services by automatically selecting the appropriate action, in what is described as a “self-healing” network. System performance is also monitored for example new service instances can be spun up during high demand, while at the same time latency and other potential issues are monitored across the network.

Infocare architecture is “Disaster Recovery” tested for fail-over in a multi-site environment, and a combination of high availability solutions act as a multiplier against outages, where services can reside in multiple locations. In the meantime, the architecture is designed to make the user experience seamless during outages, for example traffic is switched over to alternative internet connections or new service instances spun up to scale with demand. This visibility over the end to end system needs to work not just for the backroom experts, but also from the perspective of the end user. For example, if a User action fails, the root causes needs to be pin pointed.

Reverse engineering can be used where there is not 100% visibility, for example test tools measure the application performance from several locations, but onsite staff and remove support can analyze the end to end performance from local I.T. equipment and bandwidth etc. upstream as far as the hosted services. So, what can hospitals and clinics do to address performance & reliability today?
1. Review Service Level Agreements with I.T. vendors. Look for performance metrics and uptime.
2. What is the staff’s experience with the vendor’s product and quality of support? Does the vendor take ownership and go beyond their remit to exceed at managing the customer experience?
3. Does the vendor show end to end connectivity: do the support staff seem trained and empowered, and are the account managers able to speak confidently about the technology and how its managed?

These points will build up a picture of the quality of the service that will be provided.

Cormac Trant
Operations Manager
E: cormac.trant@infocarehealth.com