Fast detector simulation

Project goal

We are using artificial intelligence (AI) techniques to simulate the response of the particle detectors to collision events. Specifically, we are developing deep neural networks — in particular, generative adversarial networks (GANs) — to do this. Such tools will play a significant role in helping the research community cope with the vastly increased computing demands of the High Luminosity LHC (HL-LHC).

Once properly trained and optimised, generative models can simulate a variety of particles, energies, and detectors in just a fraction of the time required by classical simulation, which is based on detailed Monte Carlo methods. Our objective is to tune and integrate these new tools in the experiments’ existing simulation frameworks.

R&D topic
Machine learning and data analytics
Project coordinator(s)
Sofia Vallecorsa
Team members
Florian Rhem, Gurlukh Khattak, Krisitna Jaruskova
Collaborator liaison(s)
Intel: Claudio Bellini, Andrea Luiselli, Saletore Vikram, Hans Pabst, Adel Chaibi, Eric Petit. | SURFsara BV: Valeriu Codreanu, Maxwell Cai, Damian Podareanu. Barcelona Suepercomputing Center: John Osorio Rios, Adrià Armejach Marc Casas

Collaborators

Project background

Simulating the response of detectors to particle collisions — under a variety of conditions — is an important step on the path to new physics discoveries. However, this work is very computationally expensive. Over half of the computing workload of the Worldwide LHC Computing Grid (WLCG) is the result of this single activity.  

We are exploring an alternative approach, referred to as ‘fast simulation’, which trades some level of accuracy for speed. Fast-simulation strategies have been developed in the past, using different techniques (e.g. look-up tables or parametrised approaches). However, the latest developments in machine learning (particularly in relation to deep neural networks) make it possible to develop fast-simulation tools that are both more flexible and more accurate than those developed in the past.

Recent progress

Most of the work in 2019 focused on the acceleration of the training process using a data-parallel approach. In 2020 we turned our attention to the optimisation and acceleration of the inference process. Industry is developing new hardware platforms that promise large acceleration factors for the training and inference processes related to deep neural networks (e.g. Intel XE). In most cases, low-precision data representation (e.g. half-precision floating points or half-precision integers) is one of the key strategies for achieving significant acceleration. Given this, we have carefully studied the effect of low-precision data representation on the 3DGAN model. We obtained a 1.8x speedup by running inference using a half-precision integer representation, compared to using single-precision float points. We verified that the precision of physics results is conserved with this approach. We also verified that using a mixed-precision approach for training (dynamically switching between single-precision and half-precision floating points) converges to stable results.

Next steps

The work done so far on 3DGAN can be considered as an initial R&D phase. Our focus now is on moving from the prototyping stage to production, deployment and integration within the simulation software. To achieve this goal, it is essential to optimise resources, stabilising the training process and improving model convergence. At the same time, it is also important to perform systematic studies on model generalisation and robustness, as well as on results interpretability and reproducibility.

Validating the performance of a generative model is not an easy task. In particular, evaluating the number of missing modes (as well as their properties) is critical for ensuring that the simulated data are a good representation of the underlying theoretical models, thus meaning that they can be safely used to evaluate detector performance and model their response.

Building on the work done to optimise the 3DGAN discriminator network, the plan is to design a convolutional neural network (CNN) able to analyse the GAN-generated images and to act as a feature extractor. The CNN output can then be analysed by an XGBoost-based analyser, solving the final classification or regression problem.

Motivated by the issue of missing modes, we also intend to develop and optimise a boosting approach to improve the convergence of the 3DGAN model.

 

Publications

    F. Carminati et al., A Deep Learning tool for fast detector simulation. Poster presented at the 18th International Supercomputing Conference 2018, Frankfurt, 2018. First prize awarded for best research poster. cern.ch/go/D9sn
    G. Khattak, Training Generative Adversarial Models over Distributed Computing System (2018), revised selected papers. cern.ch/go/8Ssz
    D. Anderson, F. Carminati, G. Khattak, V. Loncar, T. Nguyen, F. Pantaleo, M. Pierini, S. Vallecorsa, J-R. Vlimant, A. Zlokapa, Large scale distributed training applied to Generative Adversarial Networks for calorimeter Simulation. Presented at the 23rd international Conference on Computing in High Energy and Nuclear Physics (CHEP 2018). Proceedings in publication.
    F. Carminati, G. Khattak, S. Vallecorsa, 3D convolutional GAN for fast simulation. Presented at the 23rd international Conference on Computing in High Energy and Nuclear Physics (CHEP 2018). Proceedings in publication.
    F. Carminati, S. Vallecorsa, G. Khattak, V. Codreanu, D. Podareanu, H. Pabst , V. Saletore, Distributed Training of Generative Adversarial Networks for Fast Detector Simulation. ISC 2018 Workshops, LNCS 11203, pp. 487–503, 2018. cern.ch/go/wLP6
    G. Khattak, S. Vallecorsa, F. Carminati, Three Dimensional Energy Parametrized Generative Adversarial Networks for Electromagnetic Shower Simulation. 2018 25th IEEE International Conference on Image Processing (ICIP), Geneva, Pages 3913-3917, 2018. cern.ch/go/7PHp
    G. Khattak, S. Vallecorsa, F. Carminati, D. Moise, Data-Parallel Training of Generative Adversarial Networks on HPC Systems for HEP Simulations. 2018 IEEE 25th International Conference on High Performance Computing (HiPC), Geneva, Pages 162-171, 2018. cern.ch/go/kTX9
    F. Carminati et al., Calorimetry with Deep Learning: Particle Classification, Energy Regression, and Simulation for High-Energy Physics, NIPS 2017. cern.ch/go/7vc8
    F. Carminati et al., Three dimensional Generative Adversarial Networks for fast simulation, ACAT 2017. cern.ch/go/BN6r
    F. Rehm, Reduced Precision Strategies for Deep Learning: A High Energy Physics Generative Adversarial Network Use Case, 10th international Conference on Pattern Recognition Applications and Methods 2021, Vienna, Pages 251 - 258, 2021. cern.ch/go/v7wF
    F. Rehm, Validation of Deep Convolutional Generative Adversarial Networks for High Energy Physics Calorimeter Simulations, Combining Artificial Intelligence and Machine Learning with Physical Sciences, California, 2021. cern.ch/go/zFp7
    J. O. Rios, A. Armejach, G. Khattak, E. Petit, S. Vallecorsa, M. Casas, Mixed-Precision Arithmetic for 3DGAN to Simulate High Energy Physics Detectors. Published at the ICMLA, 2020.
    F. Rehm, S. Vallecorsa, K. Borras, D. Krücker, Physics Validation of Novel Convolutional 2D Architectures for Speeding Up High Energy Physics Simulations. Published at 25th International Conference on Computing in High-Energy and Nuclear Physics, Geneva, 2021. cern.ch/go/kJg7
    F. Rehm, S. Vallecorsa, K. Borras, D. Krücker, Benchmark of Generative Adversarial Networks for Fast HEP Calorimeter Simulations. Published at GRID2021, Dubna, 2021. cern.ch/go/KS8h

Presentations

    D. Brayford, S. Vallecorsa, A. Atanasov, F. Baruffa, W. Riviera, Deploying AI Frameworks on Secure HPC Systems with Containers. Presented at 2019 IEEE High Performance Extreme Computing Conference (HPEC), Waltham, 2019, pp. 1-6.
    G. R. Khattak, S. Vallecorsa, F. Carminati, G. M. Khan, Particle Detector Simulation using Generative Adversarial Networks with Domain Related Constraints. Presented at 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), Boca Raton, 2019, pp. 28-33.
    F. Carminati, S. Vallecorsa, G. Khattak, 3D convolutional GAN for fast simulation (5 March). Presented at IXPUG Spring Conference, Bologna, 2018. cern.ch/go/9TqS
    F. Carminati, G. Khattak, S. Vallecorsa, Three-dimensional energy parametrized adversarial networks for electromagnetic shower simulation (7 October). Presented at 2018 IEEE International Conference on Image Processing, Athens, 2018. cern.ch/go/lVr8
    F. Carminati, V. Codreanu, G. Khattak, H. Pabst, D. Podareanu, V. Saletore, S. Vallecorsa, Fast Simulation with Generative Adversarial Networks (12 November). Presented at The International Conference for High Performance Computing, Networking, Storage, and Analysis, Dallas, 2018. cern.ch/go/Z6Wg
    F. Carminati, V. Codreanu, G. Khattak, H. Pabst, D. Podareanu, V. Saletore, S. Vallecorsa, Fast Simulation with Generative Adversarial Networks (12 November). Presented at The International Conference for High Performance Computing, Networking, Storage, and Analysis, Dallas, 2018. cern.ch/go/Z6Wg
    F. Carminati, S. Vallecorsa, G. Khattak, 3D convolutional GAN for fast simulation, IXPUG Spring Conference 2018. cern.ch/go/9TqS
    F. Carminati, G. Khattak, S. Vallecorsa, Three-dimensional energy parametrized adversarial networks for electromagnetic shower simulation (7 October). Presented at 2018 IEEE International Conference on Image Processing, Athens, 2018. cern.ch/go/lVr8
    F. Carminati, G. Khattak, D. Moise, S. Vallecorsa, Data-parallel Training of Generative Adversarial Networks on HPC Systems for HEP Simulations (18 December). Presented at 25th IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC, Bengaluru, 2018.
    F. Carminati, S. Vallecorsa, G. Khattak, 3D convolutional GAN for fast simulation (5 March). Presented at IXPUG Spring Conference, Bologna, 2018. cern.ch/go/9TqS
    F. Carminati, G. Khattak, D. Moise, S. Vallecorsa, Data-parallel Training of Generative Adversarial Networks on HPC Systems for HEP Simulations (18 December). Presented at 25th IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC, Bengaluru, 2018.
    S. Vallecorsa, Machine Learning for Fast Simulation 2017 (June 24), Presented at ISC High Performance, Frankfurt, 2017. cern.ch/go/k6sV
    E. Orlova, Deep learning for fast simulation: development for distributed computing systems (15 August), Presented at CERN openlab summer students’ lightning talks, Geneva, 2017. cern.ch/go/NW9k
    A. Gheata, GeantV (Intel Code Modernisation) (21 September), Presented at CERN openlab Open Day, Geneva, 2017. cern.ch/go/gBS6
    S. Vallecorsa, GANs for simulation (May 2017), Fermilab, Talk at DS@HEP workshop, 2017. cern.ch/go/m9Bl
    S. Vallecorsa, GeantV – Adapting simulation to modern hardware (June 2017), Talk at PASC 2017 conference, Lugano, 2017. cern.ch/go/cPF8
    S. Vallecorsa, Machine Learning-based fast simulation for GeantV (June2017), Talk at LPCC workshop, CERN, 2017.cern.ch/go/QqD7
    S. Vallecorsa, Generative models for fast simulation (August 2017), Plenary talk at ACAT conference, Seattle, 2017.cern.ch/go/gl7l
    S. Vallecorsa, Three dimensional Generative Adversarial Networks for fast simulation, ACAT 2017. cern.ch/go/jz6C
    S. Vallecorsa et al., Tutorial on "3D convolutional GAN implementation in Neon'', Intel HPC Developers Conference 2017. cern.ch/go/ZtZ7
    F. Rehm, Reduced Precision Strategies for Deep Learning: 3DGAN Use Case (March 2021), 10th international Conference on Pattern Recognition Applications and Methods, Vienna, 2021. cern.ch/go/9ZRr
    F. Rehm, Validation of GANs for High Energy Physics Simulations (February 2021), Combining Artificial Intelligence and Machine Learning with Physical Sciences, California, 2021. cern.ch/go/8Pfc
    F. Rehm, Reduced Precision Strategies in Deep Learning: A 3D GAN use case (October 2020), Accelerated Artificial Intelligence for Big-Data Experiments Conference, Urbana-Champaign, 2020. cern.ch/go/T6wj
    F. Rehm, Reduced Precision Strategies for Deep Learning: 3DGAN Use Case (October 2020), 4th Inter-experiment Machine Learning Workshop CERN, Geneva, 2021. cern.ch/go/Hq6j
    J. O. Rios, A. Armejach, G. Khattak, E. Petit, S. Vallecorsa, M. Casas, Mixed-Precision Arithmetic for 3DGAN to Simulate High Energy Physics Detectors (13 October). Presented at the 2020 IXPUG Annual Meeting, US, 2020. cern.ch/go/7PXj
    F. Rhem, Reduced Precision Strategies for Deep Learning: A GAN use case from High Energy Physics (19 October). Presented at the FS 2020 Accelerated Artificial Intelligence for Big-Data Experiments Conference, Urbana-Champaign, 2020. cern.ch/go/T6wj
    F. Rhem, S. Vallecorsa, Reduced Precision Strategies for Deep Learning: 3DGAN Use Case (21 October ). Presented at the 4th IML Machine Learning Workshop CERN, Geneva, 2020. cern.ch/go/Tnf8
    F. Rehm, S. Vallecorsa, K. Borras, D. Krücker, Physics Validation of Novel Convolutional 2D Architectures for Speeding Up High Energy Physics Simulations (19 May). Presented at vCHEP2021, Geneva, 2021.
    F. Rehm, S. Vallecorsa, K. Borras, D. Krücker, Benchmark of Generative Adversarial Networks for Fast HEP Calorimeter Simulations (6 July). Presented at GRID2021, Dubna, 2021. cern.ch/go/6PSl
    F. Rehm, S. Vallecorsa, V. Saletore, H. Pabst, A. Chaibi, K. Borras, D. Krücker, Deep Learning for Accelerating High Energy Physics Simulations (18 March). Presented at DPG-Frühjahrstagung, Germany, 2021. cern.ch/go/nN7S
    F. Rehm, S. Vallecorsa, K. Borras, D. Krücker, V. Saletore, H. Pabst, A. Chaibi, (Quantum) Machine Learning for Calorimeter Simulations (29 April). Presented at DESY-CMS meeting, 2021. cern.ch/go/8nLg