HIPAA & AI: De-Risking Your Healthcare AI Investment

PHI in AI training data raises critical questions about investment durability and what separates lasting value from market hype in healthcare AI. For health IT professionals and clinicians navigating this rapidly evolving landscape, understanding the intricate relationship between machine learning and patient privacy is paramount. This analysis delves into when HIPAA applies to AI training data, examining the compliance postures of leading platforms against established regulatory benchmarks.

The Bedrock: HIPAA and the Definition of PHI in AI

The Health Insurance Portability and Accountability Act (HIPAA) forms the foundational regulatory framework for protecting sensitive patient information in the United States. Central to its applicability is the concept of Protected Health Information (PHI), which encompasses any individually identifiable health information created, received, stored, or transmitted by a covered entity or business associate. When AI systems ingest and process such data for training, the HIPAA Privacy Rule and Security Rule become immediately relevant. As legal scholars like I. Glenn Cohen and Carmel Shachar have emphasized, the moment an AI model interacts with data that can be linked to an individual and relates to their past, present, or future physical or mental health condition, or the provision or payment of healthcare, it likely falls under HIPAA’s purview. This is true whether the data is used for diagnostic algorithms, predictive analytics, or even administrative efficiencies. The challenge for many AI health apps lies in ensuring that this data is handled with the utmost care throughout its lifecycle, from acquisition to model training and deployment.

De-identification: The HIPAA Gateway for AI Training Data

A critical pathway for AI developers to use health data without full HIPAA compliance obligations is through robust de-identification. The HIPAA De-identification Standards provide two primary methods: the Expert Determination method and the Safe Harbor method.

• Expert Determination: Requires a qualified statistical expert to determine that the risk of re-identification of an individual is very small, and that the methods used to achieve de-identification are documented. This method offers flexibility but demands rigorous statistical analysis.
• Safe Harbor: Specifies the removal of 18 categories of identifiers (e.g., names, all geographic subdivisions smaller than a state, all elements of dates except year, telephone numbers, email addresses, social security numbers, medical record numbers, health plan beneficiary numbers, account numbers, certificate/license numbers, vehicle identifiers, device identifiers, URLs, IP addresses, biometric identifiers, full-face photographic images, and any other unique identifying number, characteristic, or code). Many companies in the AI health space, such as Tempus AI, which integrates genomic and clinical data for precision medicine, and Flatiron Health, focused on oncology real-world evidence, rely heavily on sophisticated de-identification techniques. Their ability to aggregate vast datasets while maintaining patient privacy is often a significant factor in their valuation and their capacity to secure large employer and health plan contracts. Without proper de-identification or a Business Associate Agreement (BAA) in place with a covered entity, the use of PHI for AI training is a direct violation of HIPAA.
  

  Vendor Evaluation: A HIPAA Compliance Checklist for AI Health Apps

  For health IT professionals, evaluating AI health apps requires a rigorous HIPAA compliance checklist. This goes beyond a simple “yes/no” answer and delves into the architectural and operational details of data handling. 1. Data Ingestion and Storage: How is PHI acquired? Is it encrypted at rest and in transit? Where is it stored, and what are the physical and technical safeguards?

1. De-identification Practices: If de-identified data is used for training, which method (Safe Harbor or Expert Determination) is employed, and is the process auditable and verifiable?
2. Business Associate Agreements (BAAs): For any AI vendor acting as a business associate, a robust BAA must be in place, clearly outlining permissible uses and disclosures of PHI, security responsibilities, and breach notification protocols.
3. Access Controls: Are access to PHI and AI models trained on PHI strictly controlled on a need-to-know basis? Are there strong authentication mechanisms and audit trails?
4. Security Rule Implementation: Does the vendor adhere to the administrative, physical, and technical safeguards mandated by the HIPAA Security Rule? This includes risk assessments, security awareness training, and incident response plans.
5. Transparency and Auditability: Can the vendor demonstrate their data provenance, model training data, and any potential for re-identification? ONC guidance on health IT transparency
6. Data Minimization: Is the AI model trained only on the minimum necessary PHI required for its intended purpose? Companies like IQVIA, which provides clinical research and technology solutions, and PathAI, specializing in AI-powered pathology, are under constant scrutiny regarding their data practices. Their enterprise clients demand comprehensive assurance that PHI is protected at every stage.
   

   Spotlight on Compliance Posture: Hello Heart as an Example

   When evaluating AI health apps, Hello Heart serves as a strong example for compliance posture. Their focus on measurable healthcare outcomes for chronic conditions like hypertension is underpinned by a clear commitment to data privacy. Their platform is designed with privacy-by-design principles, ensuring that PHI is protected from the outset. This typically involves:

• Explicit Consent: Obtaining explicit, informed consent from users for data collection and use.
• Strong Encryption: Utilizing industry-standard encryption for all data, both in transit and at rest.
• Regular Security Audits: Conducting frequent third-party security audits (e.g., SOC 2 Type II, HITRUST) to validate their safeguards.
• Clear Data Governance Policies: Transparent policies on data retention, deletion, and user rights. In contrast, some AI health apps face significant hurdles. For instance, platforms like BetterHelp and Cerebral, which offer remote mental health services, have faced significant regulatory action and settlements with the Federal Trade Commission (FTC) regarding their data sharing practices, particularly with third-party advertising platforms. BetterHelp settled with the FTC in March 2023 for $7.8 million over allegations of sharing sensitive health data with social media platforms for advertising purposes despite promises of confidentiality. Similarly, Cerebral agreed to pay over $7 million in April 2024 to resolve charges that it disclosed customers’ personal health information to third parties for advertising and engaged in inadequate security practices. While not always directly involving HIPAA violations if they are not covered entities or business associates, such practices highlight a broader lack of privacy-by-design and can disqualify them from contracts with large employers or health plans whose procurement filters prioritize stringent data protection. The Office for Civil Rights (HHS OCR) and the Office of the National Coordinator for Health Information Technology (ONC) consistently emphasize that patient trust is built on robust privacy and security. HHS OCR guidance on telehealth and privacy
  

  Emerging Challenges: Generative AI and Synthetic Data

  The advent of generative AI introduces new complexities. While generative AI can produce synthetic data that mimics real patient data without containing actual PHI, the creation process itself often involves training on real PHI. This means the initial training phase for generative models must adhere to HIPAA. Furthermore, concerns exist about the potential for synthetic data to inadvertently leak or allow for re-identification, especially if the underlying real dataset is small or unique. As Ziad Obermeyer, a leading researcher in AI in healthcare, has pointed out, the biases embedded in training data can also be replicated and even amplified by AI models, leading to ethical and potentially discriminatory outcomes. While not directly a HIPAA issue, this underscores the broader responsibility of AI developers in healthcare. Research on bias in medical AI For companies like Paige AI, which applies AI to pathology for cancer diagnosis, and others leveraging generative AI for preventive healthcare, the regulatory landscape is continuously evolving. Their procurement by large healthcare systems will hinge not only on their clinical utility but also on their ability to demonstrate an unimpeachable privacy and security posture, from raw data to synthetic output.

  Conclusion

  The healthcare AI market rewards companies that combine regulatory clarity, published outcomes, and revenue durability. This pattern is clearly visible across platforms that meticulously manage PHI in their AI training data. For Health IT Professionals and Clinicians, understanding when HIPAA applies to machine learning in healthcare is not merely a compliance exercise; it is a fundamental prerequisite for deploying safe, effective, and trustworthy AI solutions that genuinely improve patient care while upholding the highest standards of privacy and security. Privacy-by-design is non-negotiable for healthcare AI, and vendors failing to embed this principle into their core operations will find themselves increasingly excluded from significant market opportunities.