A branching process model of heterogeneous DNA damages caused by radiotherapy in in vitro cell cultures

Thierry Bastogne (1, 2) , Jean-Louis Marchand (3, 2) , Sophie Pinel (1) , Pierre Vallois (3, 2)

1 CRAN - Centre de Recherche en Automatique de Nancy
2 BIGS - Biology, genetics and statistics
3 IECL - Institut Élie Cartan de Lorraine

This paper deals with the dynamic modeling and simulation of cell damage heterogeneity and associated mutant cell phenotypes in the therapeutic responses of cancer cell populations submitted to a radiotherapy session during in vitro assays. Each cell is described by a finite number of phenotypic states with possible transitions between them. The population dynamics is then given by an age-dependent multi-type branching process. From this representation, we obtain formulas for the average size of the global survival population as well as the one of subpopulations associated with 10 mutation phenotypes. The proposed model has been implemented into Matlab© and the numerical results corroborate the ability of the model to reproduce four major types of cell responses: delayed growth, anti-proliferative, cytostatic and cytotoxic.

Mots clés: Branching process; cancer cells; heterogeneity; radiotherapy

Domaines: Cancer

Thierry Bastogne, Jean-Louis Marchand, Sophie Pinel, Pierre Vallois. A branching process model of heterogeneous DNA damages caused by radiotherapy in in vitro cell cultures. Mathematical Biosciences, 2017, 294, pp.100-109. ⟨10.1016/j.mbs.2017.09.006⟩. ⟨hal-01669004⟩