Real-Time Clinical Intelligence Engine - Part II

Overview & Clinical Context

Clinical dashboards inside modern Electronic Health Records (EHRs) frequently fail to change outcomes due to “alert fatigue.” Clinicians are routinely inundated with passive notifications, leading to critical, time-sensitive warnings being dismissed. This project introduces a production-grade, end-to-end Clinical Decision Support (CDS) Hooks system designed to passively monitor adult ICU patients during their first six hours of admission and alert clinicians to early-stage sepsis risk.

Instead of requiring a physician to open a specific tab or manually calculate risk scores, this system operates entirely in the background. When specific clinical events occur within the EHR (such as an encounter opening or a lab value updating), the EHR automatically triggers an asynchronous HTTPS request to our external machine learning service. The service parses the raw data, evaluates an optimized XGBoost predictive model, runs a local game-theoretic interpretability model (SHAP), logs an immutable transaction record to a secure database for auditing, and returns a formatted “CDS Card” directly into the user interface of the clinician.

+----------------+       1. Hook Trigger       +--------------------+
|  Hospital EHR  | --------------------------> |  Cloud Run Engine  |
|                | <-------------------------- |  (FastAPI Service) |
+----------------+       6. Return Card        +--------------------+
        ^                                                |
        | 2. Fetch Prefetch Data                         | 3. Execute Model
        v                                                v
+----------------+                             +--------------------+
| FHIR R4 Server |                             |  XGBoost Binary    |
+----------------+                             |  Inference Core    |
                                               +--------------------+
                                                         |
                                                         | 4. Compute SHAP
                                                         v
+----------------+     5. Immutable Log Audit  +--------------------+
| Firestore DB   | <-------------------------- | SHAP Explainer     |
+----------------+                             +--------------------+

The engineering cycle is divided into distinct execution phases:

• Phase 0 (Infrastructure Setup): Provisioning the cloud environment, setting up local development pathways, establishing secure database links, and defining code repository webhooks.
• Phase 1 (Clinical Data Pipeline): Ingesting complex, unevenly sampled time-series data from the MIMIC-IV database, cleaning physiological anomalies, grouping time windows, and executing class-stratified medians to generate an analytical matrix.
• Phase 2 (Algorithmic Modeling): Training an XGBoost binary mortality classifier, adjusting parameters to accommodate a micro-scale validation cohort safely, and exporting cryptographic explainer configurations.
• Phase 3 (Algorithmic Governance): Running automated fairness evaluations across age, gender, and racial brackets using point-estimate methodologies to produce an FDA-aligned Good Machine Learning Practice (GMLP) asset.
• Phase 4 (Containerized API Deployment): Encapsulating the analytical weights, features, and schema structures into a high-performance FastAPI wrapper containerized via Docker and deployed automatically using continuous integration channels to a serverless compute cluster.
• Phase 5 (End-to-End Validation): Simulating standard real-world medical anomalies by feeding realistic synthetic health bundles against the active production service to guarantee algorithmic execution.
• Phase 6 (Portfolio Deployment): Binding the interactive user layout to web hosting services to demonstrate system utility transparently.

Phase 0: Foundations & Cloud Environments

Phase 0 establishes the underlying infrastructure. A dedicated Google Cloud Project (cytos-web) was created and linked to an isolated Firebase Native mode instance (cds-sepsis). Access permissions were restricted by generating an encrypted Google Service Account key file, mapping roles directly to document read/write parameters. The working environment was stabilized by linking local Google Drive persistent storage structures to automated development notebooks, preventing environmental drift across processing sessions.

Phase 1: Ingestion & Clinical Feature Engineering

The system extracts structured components from the MIMIC-IV clinical dataset. Time-series observations are restricted exclusively to the first six hours following ICU admission. Raw values are passed through a deterministic pipeline that filters physiological outliers (e.g., removing impossible blood pressure or oxygen saturation readings) and applies a class-stratified median imputation technique to fill missing observations based on broader cohort behavior.

Critical Processing Implementation

import pandas as pd
import numpy as np

def clean_and_impute_clinical_features(df_raw, median_reference_path=None):
    """
    Cleans physical measurement boundaries and applies class-stratified medians
    to ensure feature consistency across unstructured ICU observations.
    """
    df = df_raw.copy()
    
    # Define physiological boundary constraints
    boundaries = {
        'vital_heart_rate': (30, 220),
        'vital_sbp': (40, 260),
        'vital_spo2': (50, 100),
        'vital_temperature': (30, 45)
    }
    
    # Truncate out-of-bounds measurements to NaN
    for feature, (low, high) in boundaries.items():
        if feature in df.columns:
            df.loc[(df[feature] < low) | (df[feature] > high), feature] = np.nan
            
    # Calculate or load stratification parameters
    numerical_features = [col for col in df.columns if col.startswith('vital_') or col.startswith('lab_')]
    
    if median_reference_path is None:
        # Compute medians relative to the target ground truth to preserve class signaling
        medians = df.groupby('mortality')[numerical_features].median()
    else:
        medians = pd.read_csv(median_reference_path).set_index('mortality')
        
    # Apply stratified fill parameters
    for mortality_status in df['mortality'].unique():
        mask = df['mortality'] == mortality_status
        df.loc[mask, numerical_features] = df.loc[mask, numerical_features].fillna(medians.loc[mortality_status])
        
    return df, medians

Analytical Visualization

During data compilation, the feature distributions are visually mapped to check for scaling anomalies before introducing arrays to the optimization algorithms.

Phase 2: Predictive Modeling & Cryptographic Explainability

Phase 2 builds the machine learning core. An extreme gradient boosted trees classifier (XGBClassifier) is configured to process the clinical vectors. Because the demonstration scale involves restricted patient volumes (N=7 total validation cohort), standard cross-validation mechanisms are bypassed using rigid stratified parameter splits to ensure compilation stability.

To satisfy clinical safety constraints and remove the typical “black box” barriers associated with neural architectures, a game-theoretic local explanation model (shap.TreeExplainer) is initialized directly alongside the model tree structures. This saves a local mathematical baseline mapping how every individual clinical metric impacts the final prediction percentage.

Core Optimization Script

import xgboost as xgb
import shap
import joblib
import json

def train_and_serialize_inference_system(X_train, y_train, X_test, output_dir):
    """
    Optimizes the gradient boosted tree parameters and exports the unified
    prediction model and local SHAP explanation frameworks.
    """
    # Configure model constraints optimized for sparse clinical arrays
    model = xgb.XGBClassifier(
        n_estimators=50,
        max_depth=3,
        learning_rate=0.1,
        eval_metric='logloss',
        random_state=42
    )
    
    # Fit the clinical vectors
    model.fit(X_train, y_train)
    
    # Initialize game-theoretic explainer layer
    explainer = shap.TreeExplainer(model)
    
    # Export structural artifacts for runtime serialization
    joblib.dump(model, f"{output_dir}/sepsis_model.pkl")
    joblib.dump(explainer, f"{output_dir}/shap_explainer.pkl")
    
    # Save the strictly ordered features schema index
    with open(f"{output_dir}/feature_names.json", "w") as f:
        json.dump(list(X_train.columns), f)
        
    return model, explainer

Model Evaluation Metrics

After training, the model is evaluated against the test partition to establish baseline classification capabilities and reliability.

Clinical Explainability (SHAP)

To ensure physicians can trust the AI’s recommendation, the system generates visual explanations detailing exactly why a specific risk score was assigned.