Healthcare Data Migration Benefits from Cloud Tools

June 29, 2017 - Every organization will undergo major and minor healthcare data migrations as clinical data needs to be moved to more advanced and scalable storage solutions.

Understanding the data so the migration is done correctly the first time is the biggest challenge healthcare organizations face with data migration, Informatica Chief Healthcare Strategist Richard Cramer told HITInfrastructure.com.

“Healthcare organizations need to take data migration seriously,” Cramer advised. “Data migration needs to be done with rigor and discipline or it will not turn out well. If data migration is not done well, the consequences are extreme. Bad data is going to be migrated and fixing it takes a long time.”

The healthcare industry faces further challenges with data migration because healthcare data is more complex and follows strict compliance regulations. Entities need to fully understand the data they’re moving. Clinical data often contains different source applications, complex data, and data quality problems.

“Healthcare data is fragmented over dozens or even hundreds of applications at most organizations,” Cramer said. “It is not atypical for some of larger organizations to have 800 to 1000 applications. The data that’s in those applications is typically of suspect quality.”

READ MORE: Healthcare Cloud Data Migration Challenges Organizations

“There is an absolute epidemic in applications and it’s prevalent in healthcare. Businesses will use a field in an application that is completely different than what that field was originally for,” he continued. “When you look at that from a data migration perspective, the first thing that needs to happen is the source needs to be understood and the data in that source may be different than what the label on the field says it is.”

Organizations need to figure out how the data is going to be profiled. Data problems need to be identified before migration and then need to be resolved and documented.

Once the data is understood and data quality problems are identified, entities are faced with getting the data out of the source application. This can be challenging because many applications are not built for data to be easily migrated out. Apps need to be connected and data needs to be transformed.

“A common example is moving data out of a first generation EHR and into a new EHR. There are structural differences between how the data is sorted in the old application vs the new,” Cramer explained. “You have the semantic differences between how the data is stored in the old and the new. Do both apps have the same definition of what an encounter is? Organizations need to create the business rules and transform the data so it’s correctly loaded into the new system.”

“Once you’ve done that, you need to physically load it into the new system,” he continued. “The final step is validating that the data loaded was the data that was intended to be loaded. That end-to-end process of steps is a very disciplined practice.”

READ MORE: Overcoming Healthcare Hybrid Cloud Storage Challenges

Larger healthcare data migrations have piqued interest in the cloud because of its scalability and elasticity.

“Cloud is no longer a dirty word in healthcare,” Cramer stated. “A few years ago, that wasn’t the case. There is no organization anymore that says they will never do cloud.”

“The important thing is the idea of a hybrid data integration platform,” he continued. “Organizations can no longer rely on the fact that everything will either be on-premise or in the cloud. They need to look at data integration and data migration as a component of that and be completely agnostic toward what is cloud and what is on-premise. Organizations don’t need to care if the source system, the target system or the data migration solution is on-premise or in the cloud.”

Organizations can benefit greatly from standing their data migration solution up in the cloud. The scalability of the cloud allows entities to take advantage of all the tools needed to migrate data correctly and successfully, and to turn it on or off as needed.

Hybrid cloud solutions work well to accommodate the over performance needed during a large data migration.

“The system the data is moving to needs to be sized for the ingestion of the migrating data,” Cramer explained. “Part of the challenge with data migration is that you have a system that you’re building and it’s designed to support everyday operations. When a decade or more of data is being migrated into that system, it’s pumping a lot of data into that system. The amount of performance required to get that data in in a reasonable amount of time calls for excess capacity in that system.”

“This is where the elasticity of the cloud works out very well,” he added. “The application can be put up on a really powerful platform, it can suck all the data in quickly. Then the platform can be downsized to support everyday operational needs.” 

Organizations not taking into consideration data quality, and the over performance needed to perform the migration and continue to support everyday workflow, which can cause migration problems.

“The place where organizations find out they did data migration wrong is when they’re doing acceptance testing of a new application and putting the data in the app,” Cramer explained. “Competence can be questioned when the data is discovered to be wrong during acceptance testing; the data is being looked at and it’s wrong.”

“When data is discovered to be wrong at the end of the process during acceptance testing, organizations are months away from fixing it simply because of the work required to move the data. Then there’s the problem of figuring out what went wrong.”

“The cost of getting it wrong is extraordinary, which is why there must be discipline and knowledge to invest in the infrastructure to do it right and as quickly as possible.”

Healthcare organizations will face many more migration projects than they anticipate, which makes it important to invest in a scalable solution that includes the tools needed to migrate data successfully the first time.”

“The fact is organizations never really done migrating data; there is always another system to migrate,” said Cramer. “The beauty of cloud for migration projects is that you can spin up all the tools needed to do it right in the cloud and if it really doesn’t need to happen again it can be turned off.”