DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks

Survival analysis, often referred to as time-to-event analysis, is a critical tool in many domains, including economics, engineering, and especially medicine. This type of analysis is used to predict the time until the occurrence of an event of interest, such as equipment failure or death due to a specific disease. Traditional approaches to survival analysis, such as the Cox proportional hazards model, have been widely used for decades. However, these methods often rely on strong parametric assumptions that may not hold in real-world scenarios, particularly when dealing with competing risks—situations where multiple possible events could occur.

In the paper titled "DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks," the authors Changhee Lee, William R. Zame, Jinsung Yoon, and Mihaela van der Schaar present a novel approach to survival analysis that leverages the power of deep learning. DeepHit is a model that directly learns the distribution of survival times, without making any assumptions about the underlying stochastic processes. This blog will provide a comprehensive overview of the DeepHit model, including its architecture, the loss function used for training, and the results from experiments that demonstrate its effectiveness.

---

Background on Survival Analysis and Competing Risks

Survival Analysis

Survival analysis, also known as time-to-event analysis, is a statistical method used to analyze the expected duration of time until one or more events happen. This type of analysis is crucial in various fields, such as:

• Medicine: To determine the time until a patient experiences a specific event, such as death, relapse, or recovery.
• Engineering: To predict the time until a machine component fails.
• Economics: To analyze the duration until an economic event, like bankruptcy or market entry, occurs.

One of the most common models in survival analysis is the Cox proportional hazards model, which models the hazard function—a function that estimates the risk of an event occurring at time , given that the event has not occurred before time . The Cox model assumes that the hazard ratio is constant over time, which implies a proportionality of hazards across different covariates.

Competing Risks

In real-world scenarios, individuals or systems may be subject to more than one type of event, each of which could terminate the observation period. These scenarios are known as competing risks. For instance, in medical studies, a patient might die from different causes (e.g., cardiovascular disease or cancer). The presence of competing risks complicates survival analysis because the occurrence of one event (e.g., death from cardiovascular disease) precludes the occurrence of another event (e.g., death from cancer).

Traditional survival models often assume independence between risks or simplify the problem by considering only one event and treating others as censoring. This can lead to biased or incomplete analysis.

DeepHit: A Novel Approach

The DeepHit model is designed to address the limitations of traditional survival analysis methods by using deep learning to directly model the joint distribution of survival times and events, without relying on strong parametric assumptions.

Model Structure

The DeepHit model consists of a neural network with the following structure:

1. Shared Sub-Network: This network is responsible for capturing the common features across all competing risks. It takes as input the covariates (features) and produces a latent representation that is shared across all potential events.

2. Cause-Specific Sub-Networks: Each event (or cause) has its own sub-network, which takes as input both the shared latent representation and the original covariates . These sub-networks are designed to capture the unique characteristics of each event. The output of each cause-specific sub-network is a probability distribution over the time-to-event for that specific cause.

   • Residual Connections: To prevent the loss of important information, DeepHit uses residual connections that feed the original covariates directly into the cause-specific sub-networks, alongside the shared latent representation.

3. Softmax Layer: The outputs of the cause-specific sub-networks are combined in a softmax layer, which produces the final probability distribution over the time-to-event for all possible events.

   • This probability distribution is represented as a vector , where is the estimated probability that the event occurs at time .

Mathematical Formulation

The DeepHit model aims to learn the joint distribution of the time-to-event and the event type, denoted by , where:

• is the time of the event.
• is the type of event.
• represents the covariates for a given instance (e.g., patient).

Loss Function

To train the model, a custom loss function is designed to handle the unique challenges of survival data, such as right-censoring.

1. Log-Likelihood Loss : This component is the log-likelihood of the observed data, modified to handle censored data.

   • The first term captures the log-probability of the observed event occurring at time for uncensored instances.
   • The second term adjusts for censored instances by considering the probability that the event occurs after the observed time .

2. **Ranking Loss :
   risk. The idea is that if a patient (or subject) experiences an event earlier than another patient, then the predicted risk for the first patient should be higher. The ranking loss penalizes the model if this order is incorrect.

   Here:

   • is a weighting factor for the loss associated with each cause.
   • is an indicator function that equals 1 if the event for patient occurs earlier than for patient and both patients are comparable (i.e., , ).
   • is a convex loss function, often chosen as , which encourages the model to separate the predicted risks according to the observed event times.

The total loss function is the sum of these two components:

This combined loss ensures that the model accurately predicts both the timing and the type of events, while appropriately handling censored data and encouraging correct ranking of risk.

DeepHit Model in Action

Training the Model

DeepHit is trained using backpropagation and gradient descent. The network's parameters are adjusted to minimize the total loss , thereby improving the accuracy of the predicted survival times and event types.

The training process involves feeding the network batches of patient data, each containing covariates, observed times, and event types (or censoring indicators). The network outputs the probability distribution over survival times for each possible event, and the loss is computed based on the discrepancy between the predictions and the actual data.

Evaluation Metrics

The performance of the DeepHit model is evaluated using the time-dependent concordance index (Ctd-index). This metric extends the traditional concordance index (C-index) to account for the time-varying nature of survival data. It measures the agreement between predicted and actual event times across the entire time horizon.

For each pair of patients, the Ctd-index compares their predicted survival times and the observed survival times. If the model consistently predicts higher risk (shorter survival time) for the patient who experiences the event earlier, the Ctd-index will be close to 1, indicating high predictive accuracy.

Experimental Results

The authors of the DeepHit paper conducted extensive experiments to compare the performance of DeepHit with traditional and state-of-the-art survival models, including:

• Cox Proportional Hazards Model (cs-Cox): A cause-specific version of the Cox model, which treats other events as censoring.
• Fine-Gray Model: A model that directly handles competing risks by modeling the cumulative incidence function (CIF).
• Deep Multi-task Gaussian Process (DMGP): A recent nonparametric Bayesian model for survival analysis with competing risks.

Datasets Used

1. SEER Dataset: A dataset from the Surveillance, Epidemiology, and End Results (SEER) program, focusing on patients with breast cancer and cardiovascular disease (CVD) as competing risks.
2. UNOS Dataset: The United Network for Organ Sharing (UNOS) database, which contains data on patients who underwent heart transplants.
3. METABRIC Dataset: A dataset from the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC), which includes gene expression profiles and clinical features of breast cancer patients.
4. Synthetic Dataset: A dataset generated to simulate two competing risks with known underlying stochastic processes, allowing for controlled comparisons.

A detailed discritpion of the datasets is provided in the folllowing table 1

Table 1: Descriptive Statistics of Real-World Datasets for Competing Risks

Key Findings

1. Performance in Competing Risks:

   • SEER Dataset: DeepHit significantly outperformed the Fine-Gray and cs-Cox models, particularly in predicting the risk of death from breast cancer. The improvements were statistically significant, demonstrating the model's ability to handle non-linear and non-proportional relationships between covariates and survival times.
   • Synthetic Dataset: DeepHit excelled in capturing the complex, non-linear relationships present in the synthetic data, outperforming all other models, including DMGP.

2. Performance in Single Risk Scenarios:

   • UNOS and METABRIC Datasets: Even in scenarios with a single risk, DeepHit provided superior predictive performance compared to traditional models like the Cox model, Random Survival Forests (RSF), and deep learning-based models like DeepSurv.

In Table 2, The cause specific hazard concordance is compared for the SEER dataset, Table 3 shows the comparison for the synthetic dataset and Table 4 shows the comparison for the single event datasets.

Table 2: Comparison of Cause-Specific Ctd-Index Performance (Mean and 95% Confidence Interval) Tested on the SEER Dataset

* indicates p-value < 0.001
‡ indicates p-value < 0.05

Table 3: Comparison of Cause-Specific Ctd-Index Performance (Mean and 95% Confidence Interval) Tested on the SYNTHETIC Dataset

* indicates p-value < 0.001

Table 4: Comparison of Cause-Specific Ctd-Index Performance (Mean and 95% Confidence Interval) Tested on Single Event Datasets

* indicates p-value < 0.001
† indicates p-value < 0.01
‡ indicates p-value < 0.05

Mathematical Insights into DeepHit

Joint Distribution Estimation

DeepHit’s core innovation is its ability to estimate the joint distribution of survival times and events. This is mathematically challenging because it requires simultaneously modeling the probability of different events over time. The neural network architecture allows for the flexible modeling of these distributions by learning complex relationships directly from data.

For each patient , the network predicts a vector , where each element represents the probability . The softmax layer ensures that these probabilities sum to 1 across all possible events and times, satisfying the requirements of a probability distribution.

Cumulative Incidence Function (CIF)

A key concept in competing risks is the cumulative incidence function (CIF), which represents the probability that a particular event occurs before or at a given time , given the covariates . Mathematically, it is defined as:

DeepHit estimates the CIF by summing the predicted probabilities over time:

This allows DeepHit to compute the CIF for each cause, which is critical for making accurate predictions in the presence of competing risks.

Conclusion: The Future of Survival Analysis

DeepHit represents a significant advancement in survival analysis, offering a powerful tool for researchers and practitioners dealing with complex, real-world data. Its ability to handle multiple competing risks, model non-linear relationships, and provide accurate predictions even in the presence of censoring makes it a versatile and robust choice for survival analysis tasks.

The model’s performance in diverse datasets highlights its generalizability and potential for widespread application in medicine, engineering, and beyond. As deep learning continues to evolve, models like DeepHit are likely to become increasingly important, providing insights and predictions that were previously unattainable with traditional methods.

For those looking to implement or extend DeepHit, the authors have provided a solid foundation, but there are numerous avenues for further research, including exploring different network architectures, loss functions, and applications in new domains.

In summary, DeepHit is not just a step forward for survival analysis—it is a leap into the future, where deep learning enables us to tackle the most challenging predictive modeling problems with unprecedented accuracy and flexibility.

References

• Cox, D. R. (1972). Regression models and life tables (with discussion). Journal of the Royal Statistical Society. Series B, 34, 187-220.
• Fine, J. P., & Gray, R. J. (1999). A proportional hazards model for the subdistribution of a competing risk. Journal of the American Statistical Association, 94(446), 496-509.
• Harrell, F. E., Califf, R. M., Pryor, D. B., Lee, K. L., & Rosati, R. A. (1982). Evaluating the yield of medical tests. Journal of the American Medical Association, 247(18), 2543-2546.
• Lee, C., Zame, W. R., Yoon, J., & van der Schaar, M. (2018). DeepHit: A deep learning approach to survival analysis with competing risks. Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI-18), 2314-2321. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16749

---