Helpt een gokverslaafde AI-inktvis de volgende pandemie in te dijken?

Lennert
Saerens

Beeld je even in, het is eind 2019 en het coronavirus is langzaam maar zeker bezig aan hun wereldwijde opmars. Het is al laat, maar als epidemioloog ben je de laatste ontwikkelingen op de voet aan het volgen wanneer je plots wordt opgeschrikt door het geluid van je telefoon. Na een kort moment van twijfel besluit je om toch op te nemen. Aan de andere kant van de lijn weerklinkt een onbekende stem met een duidelijke boodschap: Vanwege je expertise word je gevraagd als mede-besluitvormer over de preventiestrategieën die zullen worden ingezet om het virus in België in te dijken.

Hoofdpijn

Tijdens de daaropvolgende jaren is de spanning tijdens elke vergadering keer op keer te snijden. Miljoenen mensen vertrouwen op je vermogen om knopen door te hakken, en kijken kritisch toe hoe je, samen met je collega’s, het land doorheen de crisis probeert te leiden. Je tracht zo goed en zo kwaad je kan de verschillende strategieën tegenover elkaar af te wegen, maar er zijn schijnbaar honderd verschillende objectieven om rekening mee te houden. Niet enkel de medische impact van de strategie moet in overweging genomen worden, maar ook de economische en sociale impact moeten worden meegenomen in de vergelijking. Je zou van minder hoofdpijn krijgen.

Epidemiologische modellen als pijnstiller

Verschillende strategieën uitproberen is dus de boodschap! Het zou natuurlijk volledig onverantwoord zijn om zomaar willekeurige preventiestrategieën te testen binnen de brede bevolking. De kosten en de risico’s voor de volksgezondheid zouden veel te groot zijn, de experimenten te beperkt, en dan hebben we het nog niet eens gehad over ethische bedenkingen. Wanneer besluitvormers voor dit soort uitdaging komen te staan, baseren ze zich op de resultaten van computersimulaties met epidemiologische modellen. Deze simulaties bieden de besluitvormers inzichten in de impact van verschillende preventiestrategieën. Op die manier kunnen de afwegingen tussen verschillende strategieën met elkaar vergeleken worden. Zo kan de ene strategie bijvoorbeeld een enorm positieve impact hebben op het aantal hospitalisaties maar een grote economische kost met zich meebrengen, terwijl een andere strategie dan weer een minder grote positieve impact heeft op het aantal hospitalisaties maar ook op sociaal vlak minder lijden veroorzaakt. Het is dus van cruciaal belang om de optimale strategieën, en de onderlinge afwegingen, snel en efficiënt te identificeren, zodat de besluitvormers een geïnformeerde beslissing kunnen nemen.

Prijzige paracetamol

Probleem opgelost zou je denken. Simuleer elke strategie voldoende om een goed beeld te krijgen van haar effecten, en vergelijken maar. Spijtig genoeg is de realiteit een pak complexer. Simulaties met epidemiologische modellen gebruiken namelijk enorm veel computerkracht. Hierdoor zijn ze erg duur en tijdrovend. Voeg hieraan toe dat elke strategie vaak genoeg gesimuleerd moet worden om zeker te zijn van haar effecten, dat er talloze strategieën bestaan om te overwegen, en dat er rekening gehouden moet worden met strakke deadlines. De nood aan een efficiënt gebruik van het beperkt budget aan dure simulaties wordt al snel duidelijk.

Op dit moment wordt het beschikbare simulatiebudget gelijk verdeeld over elk van de preventiestrategieën. Concreet betekent dit dat elke strategie even vaak gesimuleerd wordt, ongeacht haar effectiviteit. Deze methode is echter inefficiënt aangezien grote delen van het simulatiebudget gebruikt worden voor het uittesten van suboptimale strategieën. Verder is er ook geen wetenschappelijke consensus over het aantal simulaties dat nodig is per strategie, waardoor standaard grote budgetten gealloceerd moeten worden. Hierdoor is het voor kleinere studies, die vaak onderzoek doen naar minder bekende ziektes, onmogelijk om dit soort epidemiologische modellen te gebruiken en gaan veel inzichten verloren. 

Laat AI het budget beheren

Om de problemen als gevolg van deze inefficiëntie tegen te gaan, onderzocht ik in mijn thesis het potentieel van een specifiek soort AI-algoritmen genaamd Multi-objective Multi-armed Bandits (MOMABs). Je kan je dit soort algoritmes inbeelden als een soort inktvis die de verschillende gokautomaten in een casino uitprobeert. Onze denkbeeldige gokker heeft aan elke tentakel een gokautomaat en wilt zo optimaal mogelijk achterhalen welke machine hen de grootste winst geeft. In deze analogie komt iedere poging met een gokmachine overeen met het simuleren van een preventiestrategie, en is het maximaal aantal pogingen gelijk aan het simulatiebudget. Deze algoritmes die toelaten optimale gokmachines te identificeren, kunnen dus gebruikt worden om optimale preventiestrategieën te vinden. Op die manier lijken ze de ideale kandidaat om het simulatiebudget op een efficiënte manier te benutten.

Het dengue casino

In mijn thesis onderzocht ik preventiestrategieën om dengue in te dijken. Sinds begin 2024 rapporteerde de WHO meer dan twaalfmiljoen gevallen en stierven meer dan 7800 mensen aan de gevolgen van dengue. Een eerste besmetting met het dengue-virus heeft zelden zware gevolgen, maar een tweede opeenvolgende infectie gaat vaak gepaard met levensbedreigende symptomen. Vaccinatie is een veelbelovende strategie voor het onderdrukken van dengue-epidemieën, maar is ook niet zonder haar uitdagingen. Dit alles tezamen maakt van dengue een interessante en relevante setting om te bestuderen. Er werd gebruik gemaakt van gloednieuw epidemiologisch model dat de simulatie van de effecten van een breed scala aan vaccinatiestrategieën toelaat. Zo kan een onderscheid gemaakt worden tussen het prioritair vaccineren van volwassenen of kinderen, maar ook tussen individuen die al eens besmet waren met dengue of niet. Drieënvijftig vaccinatiestrategieën werden geselecteerd voor evaluatie met behulp van vijf verschillende MOMAB-algoritmes. Vier van deze algoritmes zijn aanpassingen van bestaande technieken, en als vijfde ontwikkelden we een spiksplinternieuwMOMAB-algoritme. Dit nieuw algoritme staat toe om kennis over het onderliggende proces mee te nemen in het beheer van het budget.

Resultaten

Uit de resultaten blijkt dat drie van de geteste MOMAB-algoritmes het simulatiebudget veel efficiënter gebruiken dan de huidige techniek waarbij het budget gelijk verdeeld wordt. Voor verschillende groottes van het simulatiebudget identificeren deze algoritmes de optimale strategieën drie keer vaker dan de huidige techniek.

Hierdoor kunnen kleinere budgetten gebruikt worden en kunnen besluitvormers sneller, en met een beetje minder stress, aan de slag met de resultaten van de computersimulaties. Of misschien hadden ze toch beter de telefoon niet opgenomen.

Bibliografie

Bibliography

[1] The bugs book: A practical introduction to bayesian analysis. CHANCE, 26:56 – 59, 2013.

[2] Nicole L. Achee, Fred Gould, T. Alex Perkins, Robert C. Reiner, Jr., Amy C. Morrison,

Scott A. Ritchie, Duane J. Gubler, Remy Teyssou, and Thomas W. Scott. A critical

assessment of vector control for dengue prevention. PLOS Neglected Tropical Diseases,

9(5):1–19, 05 2015.

[3] B. Adams, E. C. Holmes, C. Zhang, M. P. Mammen, S. Nimmannitya, S. Kalayanarooj,

and M. Boots. Cross-protective immunity can account for the alternating epidemic pattern

of dengue virus serotypes circulating in bangkok. Proceedings of the National Academy of

Sciences, 103(38):14234–14239, 2006.

[4] Shipra Agrawal and Navin Goyal. Analysis of thompson sampling for the multi-armed

bandit problem, 2012.

[5] Ma´ıra Aguiar, Vizda Anam, Konstantin B. Blyuss, Carlo Delfin S. Estadilla, Bruno V.

Guerrero, Dami´an Knopoff, Bob W. Kooi, Akhil Kumar Srivastav, Vanessa Steindorf,

and Nico Stollenwerk. Mathematical models for dengue fever epidemiology: A 10-year

systematic review. Physics of Life Reviews, 40:65 – 92, 2022.

[6] Maira Aguiar and Nico Stollenwerk. Mathematical models of dengue fever epidemiology:

multi-strain dynamics, immunological aspects associated to disease severity and vaccines.

Communication in Biomathematical Sciences, 1(1):1–12, Dec. 2017.

[7] Ma´ıra Aguiar, Sebastien Ballesteros, Bob W. Kooi, and Nico Stollenwerk. The role of

seasonality and import in a minimalistic multi-strain dengue model capturing differences

between primary and secondary infections: Complex dynamics and its implications for

data analysis. Journal of Theoretical Biology, 289:181–196, 2011.

[8] Ma´ıra Aguiar, Bob W. Kooi, Filipe Rocha, Peyman Ghaffari, and Nico Stollenwerk. How

much complexity is needed to describe the fluctuations observed in dengue hemorrhagic

fever incidence data? Ecological Complexity, 16:31–40, 2013. Modelling ecological pro-

cesses: proceedings of MATE 2011.

[9] Ma´ıra Aguiar and Nico Stollenwerk. Dengvaxia Efficacy Dependency on Serostatus: A

Closer Look at More Recent Data. Clinical Infectious Diseases, 66(4):641–642, 10 2017.

[10] Ma´ıra Aguiar and Nico Stollenwerk. Dengvaxia: age as surrogate for serostatus. The

Lancet Infectious Diseases, 18(3):245, 2018.

[11] Ma´ıra Aguiar and Nico Stollenwerk. The impact of serotype cross-protection on vaccine

trials: Denvax as a case study. Vaccines, 8(4), 2020.

144BIBLIOGRAPHY 145

[12] Ma´ıra Aguiar, Nico Stollenwerk, and Scott B. Halstead. The impact of the newly licensed

dengue vaccine in endemic countries. PLOS Neglected Tropical Diseases, 10(12):1–23, 12

2016.

[13] Ma´ıra Aguiar, Nico Stollenwerk, and Scott B. Halstead. The risks behind dengvaxia rec-

ommendation. The Lancet Infectious Diseases, 16(8):882–883, 2016.

[14] Aguiar, M., Kooi, B.W., Martins, J., and Stollenwerk, N. Scaling of stochasticity in dengue

hemorrhagic fever epidemics. Math. Model. Nat. Phenom., 7(3):1–11, 2012.

[15] Aguiar, Ma´ıra, Kooi, Bob, and Stollenwerk, Nico. Epidemiology of dengue fever: A model

with temporary cross-immunity and possible secondary infection shows bifurcations and

chaotic behaviour in wide parameter regions. Math. Model. Nat. Phenom., 3(4):48–70,

2008.

[16] Edward J. Allen, Linda J. S. Allen, Armando Arciniega, and Priscilla E. Greenwood.

Construction of equivalent stochastic differential equation models. Stochastic Analysis and

Applications, 26:274 – 297, 2008.

[17] L. Alonso-Palomares, M. Moreno-Garc´ıa, H. Lanz-Mendoza, and M. Salazar. Molecular

basis for arbovirus transmission by aedes aegypti mosquitoes. Intervirology, 61:255 – 264,

2019.

[18] David F. Anderson. Incorporating postleap checks in tau-leaping. The Journal of chemical

physics, 128 5:054103, 2007.

[19] David F. Anderson. A modified next reaction method for simulating chemical systems with

time dependent propensities and delays. The Journal of chemical physics, 127 21:214107,

2007.

[20] N. Anggriani, H. Tasman, M.Z. Ndii, A.K. Supriatna, E. Soewono, and E Siregar. The

effect of reinfection with the same serotype on dengue transmission dynamics. Applied

Mathematics and Computation, 349:62–80, 2019.

[21] Jean-Yves Audibert, S´ebastien Bubeck, and R´emi Munos. Best arm identification in multi-

armed bandits. In Annual Conference Computational Learning Theory, 2010.

[22] Peter Auer, Nicol`o Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed

bandit problem. Machine Learning, 47:235–256, 2002.

[23] Frank G. Ball, Tom Britton, Thomas A. House, Valerie Isham, Denis Mollison, Lorenzo

Pellis, and Gianpaolo Scalia Tomba. Seven challenges for metapopulation models of epi-

demics, including households models. Epidemics, 10:63–7, 2015.

[24] Shahera Banu, A. Choudhury, and S. Tong. Dengue: Emergence, determinants and climate

change. pages 237–248, 2016.

[25] Nicole E. Basta, Dennis L. Chao, M. Elizabeth Halloran, Laura Matrajt, and Ira M.

Longini. Strategies for pandemic and seasonal influenza vaccination of schoolchildren in

the united states. American Journal of Epidemiology, 170:679 – 686, 2009.

[26] Lora Billings, Ira B. Schwartz, Leah B. Shaw, Marie McCrary, Donald S. Burke, and

Derek A.T. Cummings. Instabilities in multiserotype disease models with antibody-

dependent enhancement. Journal of Theoretical Biology, 246(1):18–27, 2007.BIBLIOGRAPHY 146

[27] Shibadas Biswal et al. Efficacy of a tetravalent dengue vaccine in healthy children aged 4–16

years: a randomised, placebo-controlled, phase 3 trial. The Lancet, 395(10234):1423–1433,

2020.

[28] Shibadas Biswal, Humberto Reynales, Xavier Saez-Llorens, Pio Lopez, Charissa Borja-

Tabora, Pope Kosalaraksa, Chukiat Sirivichayakul, Veerachai Watanaveeradej, Luis

Rivera, Felix Espinoza, LakKumar Fernando, Reynaldo Dietze, Kleber Luz, Rivaldo

Venˆancio da Cunha, Jos´e Jimeno, Eduardo L´opez-Medina, Astrid Borkowski, Manja Brose,

Martina Rauscher, Inge LeFevre, Svetlana Bizjajeva, Lulu Bravo, and Derek Wallace. Ef-

ficacy of a tetravalent dengue vaccine in healthy children and adolescents. New England

Journal of Medicine, 381(21):2009–2019, 2019. PMID: 31693803.

[29] Shibadas Biswal, Humberto Reynales, Xavier Saez-Llorens, Pio Lopez, Charissa Borja-

Tabora, Pope Kosalaraksa, Chukiat Sirivichayakul, Veerachai Watanaveeradej, Luis

Rivera, Felix Espinoza, LakKumar Fernando, Reynaldo Dietze, Kleber Luz, Rivaldo

Venˆancio da Cunha, Jos´e Jimeno, Eduardo L´opez-Medina, Astrid Borkowski, Manja Brose,

Martina Rauscher, Inge LeFevre, Svetlana Bizjajeva, Lulu Bravo, and Derek Wallace. Ef-

ficacy of a tetravalent dengue vaccine in healthy children and adolescents. New England

Journal of Medicine, 381(21):2009–2019, 2019. PMID: 31693803.

[30] Wolfgang Bock and Yashika Jayathunga. Optimal control of a multi-patch dengue model

under the influence of wolbachia bacterium. Mathematical Biosciences, 315:108219, 2019.

[31] Kobporn Boonnak, Kaitlyn M. Dambach, Gina C. Donofrio, Boonrat Tassaneetrithep,

and Mary A. Marovich. Cell type specificity and host genetic polymorphisms influ-

ence antibody-dependent enhancement of dengue virus infection. Journal of Virology,

85(4):1671–1683, 2011.

[32] Oliver J. Brady, Peter W. Gething, Samir Bhatt, Jane P. Messina, John S. Brownstein,

Anne G. Hoen, Catherine L. Moyes, Andrew W. Farlow, Thomas W. Scott, and Simon I.

Hay. Refining the global spatial limits of dengue virus transmission by evidence-based

consensus. PLOS Neglected Tropical Diseases, 6(8):1–15, 08 2012.

[33] S´ebastien Bubeck, R´emi Munos, and Gilles Stoltz. Pure exploration in multi-armed bandits

problems. In International Conference on Algorithmic Learning Theory, 2009.

[34] S´ebastien Bubeck, R´emi Munos, and Gilles Stoltz. Pure exploration in finitely-armed and

continuous-armed bandits. Theor. Comput. Sci., 412:1832–1852, 2011.

[35] D. S. Burke, A. Nisalak, D. E. Johnson, and R. M. Scott. A prospective study of dengue

infections in bangkok. The American Journal of Tropical Medicine and Hygiene, 38(1):172–

180, Jan 1988.

˚

[36] Anne Tuiskunen B¨ack and

Ake Lundkvist. Dengue viruses – an overview. Infection Ecology

& Epidemiology, 3(1):19839, 2013. PMID: 24003364.

[37] Xiaodong Cai. Exact stochastic simulation of coupled chemical reactions with delays. The

Journal of chemical physics, 126 12:124108, 2007.

[38] Yang Cao, Daniel T. Gillespie, and Linda R. Petzold. Efficient step size selection for the

tau-leaping simulation method. The Journal of chemical physics, 124 4:044109, 2006.

[39] Maria Rosario Capeding, Ngoc Huu Tran, Sri Rezeki S Hadinegoro, Hussain Imam

HJ Muhammad Ismail, Tawee Chotpitayasunondh, Mary Noreen Chua, Chan Quang Lu-

ong, Kusnandi Rusmil, Dewa Nyoman Wirawan, Revathy Nallusamy, Punnee Pitisut-BIBLIOGRAPHY 147

tithum, Usa Thisyakorn, In-Kyu Yoon, Diane van der Vliet, Edith Langevin, Thelma

Laot, Yanee Hutagalung, Carina Frago, Mark Boaz, T Anh Wartel, Nadia G Tornieporth,

Melanie Saville, and Alain Bouckenooghe. Clinical efficacy and safety of a novel tetrava-

lent dengue vaccine in healthy children in asia: a phase 3, randomised, observer-masked,

placebo-controlled trial. The Lancet, 384(9951):1358–1365, 2014.

[40] Clara Champagne and Bernard Cazelles. Comparison of stochastic and deterministic frame-

works in dengue modelling. Mathematical Biosciences, 310:1–12, 2019.

[41] Dennis L. Chao, M. Elizabeth Halloran, Valerie Obenchain, Ira M. Longini, and Angela R.

McLean. Flute, a publicly available stochastic influenza epidemic simulation model. PLoS

Computational Biology, 6, 2010.

[42] Dennis L. Chao, Scott B. Halstead, M. Elizabeth Halloran, and Ira M. Longini. Controlling

dengue with vaccines in thailand. PLoS Neglected Tropical Diseases, 6, 2012.

[43] Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. In Neural

Information Processing Systems, 2011.

[44] Hannah E. Clapham, Derek A. T. Cummings, and Michael A. Johansson. Immune status

alters the probability of apparent illness due to dengue virus infection: Evidence from

a pooled analysis across multiple cohort and cluster studies. PLOS Neglected Tropical

Diseases, 11(9):1–12, 09 2017.

[45] Laurent Coudeville and Geoff P. Garnett. Transmission dynamics of the four dengue

serotypes in southern vietnam and the potential impact of vaccination. PLOS ONE,

7(12):1–11, 12 2012.

[46] Derek Cummings, Ira Schwartz, Lora Billings, Leah Shaw, and Samuel Burke. Dynamic

effects of antibody-dependent enhancement on the fitness of viruses. Proceedings of the

National Academy of Sciences of the United States of America, 102:15259–64, 11 2005.

[47] Gabriel de Oliveira Ramos, Bruno C. da Silva, Roxana Radulescu, Ana L. C. Bazzan, and

Ann Now´e. Toll-based reinforcement learning for efficient equilibria in route choice. Knowl.

Eng. Rev., 35:e8, 2020.

[48] Wanwisa Dejnirattisai, Amonrat Jumnainsong, Naruthai Onsirisakul, Patricia Fitton, Siri-

jitt Vasanawathana, Wannee Limpitikul, Chunya Puttikhunt, Carolyn Edwards, Thaneeya

Duangchinda, Sunpetchuda Supasa, Kriangkrai Chawansuntati, Prida Malasit, Juthathip

Mongkolsapaya, and Gavin Screaton. Cross-reacting antibodies enhance dengue virus in-

fection in humans. Science, 328(5979):745–748, 2010.

[49] Sheng-Qun Deng, Xian Yang, Yong Wei, Jia-Ting Chen, Xiao-Jun Wang, and Hong-Juan

Peng. A review on dengue vaccine development. Vaccines, 8(1), 2020.

[50] Odo Diekmann, Hans Heesterbeek, and Tom Britton. Mathematical tools for understanding

infectious disease dynamics. 2012.

[51] Madalina M. Drugan and Ann Nowe. Designing multi-objective multi-armed bandits algo-

rithms: A study. In The 2013 International Joint Conference on Neural Networks (IJCNN),

pages 1–8, 2013.

[52] M˘ad˘alina M. Drugan. Covariance matrix adaptation for multiobjective multiarmed bandits.

IEEE Transactions on Neural Networks and Learning Systems, 30(8):2493–2502, 2019.BIBLIOGRAPHY 148

[53] M˘ad˘alina M Drugan, Ann Now´e, and Bernard Manderick. Pareto upper confidence bounds

algorithms: An empirical study. In 2014 IEEE Symposium on Adaptive Dynamic Program-

ming and Reinforcement Learning (ADPRL), pages 1–8, 2014.

[54] Gabriele Eichfelder. Adaptive scalarization methods in multiobjective optimization. In

Vector Optimization, 2008.

[55] Stephen Eubank, V. S. Anil Kumar, Madhav V. Marathe, Aravind Srinivasan, and Nan

Wang. Structure of social contact networks and their impact on epidemics. In Discrete

Methods in Epidemiology, 2004.

[56] Eyal Even-Dar, Shie Mannor, and Y. Mansour. Action elimination and stopping conditions

for the multi-armed bandit and reinforcement learning problems. J. Mach. Learn. Res.,

7:1079–1105, 2006.

[57] Jorge A. Falc´on-Lezama, Ruth A. Mart´ınez-Vega, Pablo A. Kuri-Morales, Jos´e Ramos-

Casta˜neda, and Ben Adams. Day-to-day population movement and the management of

dengue epidemics. Bulletin of Mathematical Biology, 78(10):2011–2033, Oct 2016.

[58] Neil Ferguson, Roy Anderson, and Sunetra Gupta. The effect of antibody-dependent en-

hancement on the transmission dynamics and persistence of multiple-strain pathogens.

Proceedings of the National Academy of Sciences, 96(2):790–794, 1999.

[59] Neil M. Ferguson, Derek A. T. Cummings, Simon Cauchemez, Christophe Fraser, Steven

Riley, Aronrag Cooper Meeyai, Sopon Iamsirithaworn, and Donald S. Burke. Strategies for

containing an emerging influenza pandemic in southeast asia. Nature, 437:209–214, 2005.

[60] Neil M. Ferguson, Isabel Rodr´ıguez-Barraquer, Ilaria Dorigatti, Luis Mier-y Teran-Romero,

Daniel J. Laydon, and Derek A. T. Cummings. Supplementary materials for benefits and

risks of the sanofi-pasteur dengue vaccine: Modeling optimal deployment, September 2016.

Science 353, 1033 (2016). DOI: 10.1126/science.aaf9590.

[61] Neil M. Ferguson, Isabel Rodr´ıguez-Barraquer, Ilaria Dorigatti, Luis Mier y Teran-Romero,

Daniel J. Laydon, and Derek A. T. Cummings. Benefits and risks of the sanofi-pasteur

dengue vaccine: Modeling optimal deployment. Science, 353(6303):1033–1036, 2016.

[62] Max Souza Filipe Rocha, Ma´ıra Aguiar and Nico Stollenwerk. Time-scale separation and

centre manifold analysis describing vector-borne disease dynamics. International Journal

of Computer Mathematics, 90(10):2105–2125, 2013.

[63] Diana B. Fischer, Yale Arbo, and Scott B. Halstead. Observations related to pathogenesis

of dengue hemorrhagic fever. v. examination of agspecific sequential infection rates using

a mathematical model. The Yale Journal of Biology and Medicine, 42:329 – 349, 1970.

[64] Laura Fumanelli, Marco Ajelli, Stefano Merler, Neil M. Ferguson, and Simon Cauchemez.

Model-based comprehensive analysis of school closure policies for mitigating influenza epi-

demics and pandemics. PLoS Computational Biology, 12, 2016.

[65] Timothy C. Germann, Kai Kadau, Ira M. Longini, and Catherine Macken. Mitigation

strategies for pandemic influenza in the united states. Proceedings of the National Academy

of Sciences of the United States of America, 103 15:5935–40, 2006.

[66] Jayanta Kumar Ghosh, Uttam Ghosh, and Susmita Sarkar. Qualitative analysis and opti-

mal control of a two-strain dengue model with its co-infections. International Journal of

Applied and Computational Mathematics, 6(6):161, Oct 2020.BIBLIOGRAPHY 149

[67] Robert Gibbons, Siripen Kalanarooj, Richard Jarman, Ananda Nisalak, David Vaughn,

Timothy Endy, Mammen Mammen, and Anon Srikiatkhachorn. Analysis of repeat hospi-

tal admissions for dengue to estimate the frequency of third or fourth dengue infections

resulting in admissions and dengue hemorrhagic fever, and serotype sequences. The Amer-

ican journal of tropical medicine and hygiene, 77:910–3, 12 2007.

[68] Michael A. Gibson and Jehoshua Bruck. Efficient exact stochastic simulation of chemical

systems with many species and many channels. Journal of Physical Chemistry A, 104:1876–

1889, 2000.

[69] Daniel T. Gillespie. Exact stochastic simulation of coupled chemical reactions. The Journal

of Physical Chemistry, 81:2340–2361, 1977.

[70] Robert J. Glass, Laura M. Glass, Walter E. Beyeler, and H. Jason Min. Targeted social

distancing designs for pandemic influenza. Emerging Infectious Diseases, 12:1671 – 1681,

2006.

[71] N.L. Gonz´alez Morales, M. N´u˜nez-L´opez, J. Ramos-Casta˜neda, and J.X. Velasco-

Hern´andez. Transmission dynamics of two dengue serotypes with vaccination scenarios.

Mathematical Biosciences, 287:54–71, 2017. 50th Anniversary Issue.

[72] Aditya Gopalan, Shie Mannor, and Y. Mansour. Thompson sampling for complex online

problems. In International Conference on Machine Learning, 2013.

[73] Adriana O Guilarde, Marilia D Turchi, Joao Bosco Siqueira Jr, Valeria C. R Feres, Benigno

Rocha, Jose E Levi, Vanda A. U. F Souza, Lucy Santos Vilas Boas, Claudio S Pannuti,

and Celina M. T Martelli. Dengue and Dengue Hemorrhagic Fever among Adults: Clin-

ical Outcomes Related to Viremia, Serotypes, and Antibody Response. The Journal of

Infectious Diseases, 197(6):817–824, 03 2008.

[74] Maria G. Guzman, Duane J. Gubler, Alienys Izquierdo, Eric Martinez, and Scott B. Hal-

stead. Dengue infection. Nature Reviews Disease Primers, 2(1):16055, 2016.

[75] Maria G. Guzman, Scott B. Halstead, Harvey Artsob, Philippe Buchy, Jeremy Far-

rar, Duane J. Gubler, Elizabeth Hunsperger, Axel Kroeger, Harold S. Margolis, Eric

Mart´ınez, Michael B. Nathan, Jose Luis Pelegrino, Cameron Simmons, Sutee Yoksan, and

Rosanna W. Peeling. Dengue: a continuing global threat. Nature Reviews Microbiology,

8(12):S7–S16, 2010.

[76] Maria G. Guzman and Eva Harris. Dengue. The Lancet, 385(9966):453–465, 2015.

[77] Maria G. Guzm´an and Gustavo Kouri. Dengue: an update. The Lancet Infectious Diseases,

2(1):33–42, 2002.

[78] Sri Rezeki Hadinegoro, Jose Luis Arredondo-Garc´ıa, Maria Rosario Capeding, Carmen

Deseda, Tawee Chotpitayasunondh, Reynaldo Dietze, H.I. Hj Muhammad Ismail, Hum-

berto Reynales, Kriengsak Limkittikul, Doris Maribel Rivera-Medina, Huu Ngoc Tran,

Alain Bouckenooghe, Danaya Chansinghakul, Margarita Cort´es, Karen Fanouillere, Remi

Forrat, Carina Frago, Sophia Gailhardou, Nicholas Jackson, Fernando Noriega, Eric Plen-

nevaux, T. Anh Wartel, Betzana Zambrano, and Melanie Saville. Efficacy and long-term

safety of a dengue vaccine in regions of endemic disease. New England Journal of Medicine,

373(13):1195–1206, 2015. PMID: 26214039.

[79] S. B. Halstead. Immune enhancement of viral infection. Progress in Allergy, 31:301–364,

1982.BIBLIOGRAPHY 150

[80] Scott B Halstead. Neutralization and antibody-dependent enhancement of dengue viruses.

volume 60 of Advances in Virus Research, pages 421–467. Academic Press, 2003.

[81] Scott B. Halstead. Critique of World Health Organization Recommendation of a Dengue

Vaccine. The Journal of Infectious Diseases, 214(12):1793–1795, 08 2016.

[82] Scott B. Halstead, Leah C. Katzelnick, Philip K. Russell, Lewis Markoff, Maira Aguiar,

Leonila R. Dans, and Antonio L. Dans. Ethics of a partially effective dengue vaccine:

Lessons from the philippines. Vaccine, 38(35):5572–5576, 2020.

[83] llkka Hanski. Metapopulation Ecology. Oxford University Press, 03 1999.

[84] W. K. Hastings. Monte carlo sampling methods using markov chains and their applications.

Biometrika, 57:97–109, 1970.

[85] Ross-William S. Hendron and Michael B. Bonsall. The interplay of vaccination and vector

control on small dengue networks. Journal of Theoretical Biology, 407:349–361, 2016.

[86] Joshua T. Herbeck, John E. Mittler, Geoffrey S. Gottlieb, and James I. Mullins. An hiv

epidemic model based on viral load dynamics: Value in assessing empirical trends in hiv

virulence and community viral load. PLoS Computational Biology, 10, 2014.

[87] Hippocrates. Hippocrates. Heinemann; Putnam, London; New York, 1923. With an English

translation by W.H.S. Jones and Paul Potter.

[88] K. Hu, C. Thoens, S. Bianco, S. Edlund, M. Davis, J. Douglas, and J.H. Kaufman. The

effect of antibody-dependent enhancement, cross immunity, and vector population on the

dynamics of dengue fever. Journal of Theoretical Biology, 319:62–74, 2013.

[89] Paul Jaccard. Etude de la distribution florale dans une portion des alpes et du jura. Bulletin

de la Societe Vaudoise des Sciences Naturelles, 37:547–579, 01 1901.

[90] Kevin G. Jamieson and Ameet Talwalkar. Non-stochastic best arm identification and

hyperparameter optimization. ArXiv, abs/1502.07943, 2015.

[91] Junchen Jin and Xiaoliang Ma. A multi-objective agent-based control approach with appli-

cation in intelligent traffic signal system. IEEE Trans. Intell. Transp. Syst., 20(10):3900–

3912, 2019.

[92] K.M. Ariful Kabir and Jun Tanimoto. Cost-efficiency analysis of voluntary vaccination

against n-serovar diseases using antibody-dependent enhancement: A game approach.

Journal of Theoretical Biology, 503:110379, 2020.

[93] Anmol Kagrecha, Jayakrishnan Nair, and Krishna Jagannathan. Constrained regret mini-

mization for multi-criterion multi-armed bandits, 2023.

[94] Parastu Kasaie, Stephen A. Berry, Maunank S. Shah, Eli S. Rosenberg, Karen W. Hoover,

Thomas L. Gift, Harrell Chesson, Jeff Pennington, Danielle German, Colin P. Flynn, Chris

Beyrer, and David W. Dowdy. Impact of providing preexposure prophylaxis for human

immunodeficiency virus at clinics for sexually transmitted infections in baltimore city: An

agent-based model. Sexually Transmitted Diseases, 45(12), 2018.

[95] Matt J. Keeling and Pejman Rohani. Modeling Infectious Diseases in Humans and Animals.

Princeton University Press, Princeton, 2008.BIBLIOGRAPHY 151

[96] Alexander Kensert, Pieter Libin, Gert Desmet, and Deirdre Cabooter. Deep reinforce-

ment learning for the direct optimization of gradient separations in liquid chromatography.

Journal of Chromatography A, 1720:464768, 2024.

`

[97] William Ogilvie Kermack and

A. G. Mckendrick. A contribution to the mathematical

theory of epidemics. Proceedings of The Royal Society A: Mathematical, Physical and

Engineering Sciences, 115:700–721, 1927.

[98] Aaron A. King, Matthieu Domenech de Cell`es, Felicia M. G. Magpantay, and Pej-

man Rohani. Avoidable errors in the modelling of outbreaks of emerging pathogens,

with special reference to ebola. Proceedings of the Royal Society B: Biological Sciences,

282(1806):20150347, May 2015.

[99] Srisakul C. Kliks, Ananda Nisalak, Walter E. Brandt, Larry Wahl, and Donald S. Burke.

Antibody-dependent enhancement of dengue virus growth in human monocytes as a risk

factor for dengue hemorrhagic fever. The American Journal of Tropical Medicine and

Hygiene, 40(4):444 – 451, 1989.

[100] Gerhart Knerer, Christine S. M. Currie, and Sally C. Brailsford. Impact of combined vector-

control and vaccination strategies on transmission dynamics of dengue fever: a model-based

analysis. Health Care Management Science, 18(2):205–217, Jun 2015.

[101] Gerhart Knerer, Christine S. M. Currie, and Sally C. Brailsford. The economic impact and

cost-effectiveness of combined vector-control and dengue vaccination strategies in thailand:

results from a dynamic transmission model. PLOS Neglected Tropical Diseases, 14(10):1–

32, 10 2020.

[102] Di´ana Knipl and Seyed M. Moghadas. The potential impact of vaccination on the dynamics

of dengue infections. Bulletin of Mathematical Biology, 77(12):2212–2230, Dec 2015.

[103] Tadeusz J. Kochel, Douglas M. Watts, Alfonso S. Gozalo, Daniel F. Ewing, Kevin R. Porter,

and Kevin L. Russell. Cross-serotype neutralization of dengue virus in aotus nancymae

monkeys. The Journal of Infectious Diseases, 191(6):1000–1004, March 2005.

[104] Bob W. Kooi, Ma´ıra Aguiar, and Nico Stollenwerk. Bifurcation analysis of a family of multi-

strain epidemiology models. Journal of Computational and Applied Mathematics, 252:148–

158, 2013. Selected papers on Computational and Mathematical Methods in Science and

Engineering (CMMSE).

[105] Bob W. Kooi, Ma´ıra Aguiar, and Nico Stollenwerk. Analysis of an asymmetric two-strain

dengue model. Mathematical Biosciences, 248:128–139, 2014.

[106] Elena Krasheninnikova, Javier Garc´ıa, Roberto Maestre, and Fernando Fern´andez. Rein-

forcement learning for pricing strategy optimization in the insurance industry. Eng. Appl.

Artif. Intell., 80:8–19, 2019.

[107] Volodymyr Kuleshov and Doina Precup. Algorithms for multi-armed bandit problems.

arXiv preprint arXiv:1402.6028, 2014.

´

[108] B. Lefebvre, Rojina Karki, Renaud Misslin, K. Nakhapakorn,

E. Daud´e, and R. Paul. Im-

portance of public transport networks for reconciling the spatial distribution of dengue and

the association of socio-economic factors with dengue risk in bangkok, thailand. Interna-

tional Journal of Environmental Research and Public Health, 19, 2022.BIBLIOGRAPHY 152

[109] Katerina Lepenioti, Minas Pertselakis, Alexandros Bousdekis, Andreas Louca, Fenareti

Lampathaki, Dimitris Apostolou, Gregoris Mentzas, and Stathis Anastasiou. Machine

learning for predictive and prescriptive analytics of operational data in smart manufac-

turing. In Sophie Dupuy-Chessa and Henderik A. Proper, editors, Advanced Information

Systems Engineering Workshops - CAiSE 2020 International Workshops, Grenoble, France,

June 8-12, 2020, Proceedings, volume 382 of Lecture Notes in Business Information Pro-

cessing, pages 5–16. Springer, 2020.

[110] Pieter Libin. Guiding the mitigation of epidemics with reinforcement learning. PhD thesis,

2020.

[111] Pieter Libin, Timothy Verstraeten, Diederik M. Roijers, Jelena Grujic, Kristof Theys,

Philippe Lemey, and Ann Now´e. Bayesian best-arm identification for selecting influenza

mitigation strategies, 2018.

[112] Pieter Libin, Timothy Verstraeten, Kristof Theys, Diederik M. Roijers, Peter Vrancx,

and Ann Now´e. Efficient evaluation of influenza mitigation strategies using preventive

bandits. In Gita Sukthankar and Juan A. Rodr´ıguez-Aguilar, editors, Autonomous Agents

and Multiagent Systems - AAMAS 2017 Workshops, Visionary Papers, S˜ao Paulo, Brazil,

May 8-12, 2017, Revised Selected Papers, volume 10643 of Lecture Notes in Computer

Science, pages 67–85. Springer, 2017.

[113] Pieter J. K. Libin, Arno Moonens, Timothy Verstraeten, Fabian Perez-Sanjines, Niel Hens,

Philippe Lemey, and Ann Now´e. Deep reinforcement learning for large-scale epidemic

control. In Yuxiao Dong, Georgiana Ifrim, Dunja Mladenic, Craig Saunders, and Sofie Van

Hoecke, editors, Machine Learning and Knowledge Discovery in Databases. Applied Data

Science and Demo Track - European Conference, ECML PKDD 2020, Ghent, Belgium,

September 14-18, 2020, Proceedings, Part V, volume 12461 of Lecture Notes in Computer

Science, pages 155–170. Springer, 2020.

[114] Fredrik Liljeros, Christofer R Edling, Luis A. Nunes Amaral, Harry Eugene Stanley, and

˚

Yvonne

Aberg. The web of human sexual contacts. Nature, 411:907–908, 2001.

[115] Jos´e Louren¸co and Mario Recker. Viral and epidemiological determinants of the invasion

dynamics of novel dengue genotypes. PLOS Neglected Tropical Diseases, 4(11):1–12, 11

2010.

[116] Jos´e Louren¸co and Mario Recker. Natural, persistent oscillations in a spatial multi-strain

disease system with application to dengue. PLOS Computational Biology, 9(10):1–11, 10

2013.

[117] Sandra B. Maier, Xiao Huang, Eduardo Massad, Marcos Amaku, Marcelo N. Burattini,

and David Greenhalgh. Analysis of the optimal vaccination age for dengue in brazil with

a tetravalent dengue vaccine. Mathematical Biosciences, 294:15–32, 2017.

[118] Patrick Mannion, Karl Mason, Sam Devlin, Jim Duggan, and Enda Howley. Multi-objective

dynamic dispatch optimisation using multi-agent reinforcement learning: (extended ab-

stract). In Adaptive Agents and Multi-Agent Systems, 2016.

[119] Maia Martcheva. An Introduction to Mathematical Epidemiology. Texts in Applied Math-

ematics. Springer New York, NY, New York, NY, 1 edition, 2015.

[120] Renaud Marti, Zhichao Li, T. Catry, E. Roux, M. Mangeas, P. Handschumacher, J. Gau-

dart, A. Tran, L. Demagistri, J. Faure, J. J. Carvajal, Bruna Drumond, Lei Xu, V. Her-BIBLIOGRAPHY 153

breteau, H. Gurgel, N. Dessay, and P. Gong. A mapping review on urban landscape factors

of dengue retrieved from earth observation data, gis techniques, and survey questionnaires.

Remote. Sens., 12:932, 2020.

[121] Nico Stollenwerk Ma´ıra Aguiar and Bob W. Kooi. Torus bifurcations, isolas and chaotic

attractors in a simple dengue fever model with ade and temporary cross immunity. Inter-

national Journal of Computer Mathematics, 86(10-11):1867–1877, 2009.

[122] Kaisa Miettinen. Nonlinear multiobjective optimization. In International Series in Oper-

ations Research and Management Science, 1998.

[123] Arti Mishra and Sunita Gakkhar. Non-linear dynamics of two-patch model incorporating

secondary dengue infection. International Journal of Applied and Computational Mathe-

matics, 4(1):19, Nov 2017.

[124] Arti Mishra and Sunita Gakkhar. Modeling of dengue with impact of asymptomatic in-

fection and ade factor. Differential Equations and Dynamical Systems, 28(3):745–761, Jul

2020.

[125] Kristof Van Moffaert, Madalina M. Drugan, and Ann Now´e. Scalarized multi-objective

reinforcement learning: Novel design techniques. In Proceedings of the 2013 IEEE Sym-

posium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013,

IEEE Symposium Series on Computational Intelligence (SSCI), 16-19 April 2013, Singa-

pore, pages 191–199. IEEE, 2013.

[126] Amy C Morrison, Emily Zielinski-Gutierrez, Thomas W Scott, and Ronald Rosenberg.

Defining challenges and proposing solutions for control of the virus vector aedes aegypti.

PLOS Medicine, 5(3):1–5, 03 2008.

[127] Jo¨el Mossong, Niel Hens, Mark Jit, Philippe Beutels, Kari Auranen, Rafael Mikolajczyk,

Marco Massari, Stefania Salmaso, Gianpaolo Scalia Tomba, Jacco Wallinga, Janneke Hei-

jne, Malgorzata Sadkowska-Todys, Magdalena Rosinska, and W. John Edmunds. Social

contacts and mixing patterns relevant to the spread of infectious diseases. PLOS Medicine,

5(3):1–1, 03 2008.

[128] David Murillo, Susan A. Holechek, Anarina L. Murillo, Fabio Sanchez, and Carlos Castillo-

Chavez. Vertical transmission in a two-strain model of dengue fever. Letters in Biomathe-

matics, 1(2):249–271, January 2014.

[129] Allan H. Murphy. The finley affair: A signal event in the history of forecast verification.

Weather and Forecasting, 11(1):3 – 20, 1996.

[130] M. Z. NDII, D. ALLINGHAM, R. I. HICKSON, and K. GLASS. The effect of wolbachia on

dengue dynamics in the presence of two serotypes of dengue: symmetric and asymmetric

epidemiological characteristics. Epidemiology and Infection, 144(13):2874–2882, 2016.

[131] Meksianis Z. Ndii, R.I. Hickson, David Allingham, and G.N. Mercer. Modelling the trans-

mission dynamics of dengue in the presence of wolbachia. Mathematical Biosciences,

262:157–166, 2015.

[132] Ian Osband, Daniel Russo, and Benjamin Van Roy. (more) efficient reinforcement learning

via posterior sampling. In Neural Information Processing Systems, 2013.

[133] A. PANDEY and J. MEDLOCK. The introduction of dengue vaccine may temporarily

cause large spikes in prevalence. Epidemiology and Infection, 143(6):1276–1286, 2015.BIBLIOGRAPHY 154

[134] Howard Raiffa and Robert Schlaifer. Applied Statistical Decision Theory. Studies in Man-

agerial Economics. Division of Research, Graduate School of Business Administration,

Harvard University, Boston, 1961.

[135] Peter Rashkov and Bob W. Kooi. Complexity of host-vector dynamics in a two-strain

dengue model. Journal of Biological Dynamics, 15(1):35–72, 2021. PMID: 33357025.

[136] David A. Rasmussen, Oliver Ratmann, and Katia Koelle. Inference for nonlinear epidemi-

ological models using genealogies and time series. PLoS Computational Biology, 7, 2011.

[137] Mario Recker, Konstantin Blyuss, Cameron Simmons, Hien Tinh Tran, Bridget Wills,

Jeremy Farrar, and Sunetra Gupta. Immunological serotype interactions and their effect

on the epidemiological pattern of dengue. Proceedings. Biological sciences / The Royal

Society, 276:2541–8, 04 2009.

[138] Nicholas G. Reich, Sourya Shrestha, Aaron A. King, Pejman Rohani, Justin Lessler, Siripen

Kalayanarooj, In-Kyu Yoon, Robert V. Gibbons, Donald S. Burke, and Derek A. T. Cum-

mings. Interactions between serotypes of dengue highlight epidemiological impact of cross-

immunity. Journal of The Royal Society Interface, 10(86):20130414, 2013.

[139] Nicholas G. Reich, Sourya Shrestha, Aaron A. King, Pejman Rohani, Justin Lessler, Siripen

Kalayanarooj, In-Kyu Yoon, Robert V. Gibbons, Donald S. Burke, and Derek A. T. Cum-

mings. Interactions between serotypes of dengue highlight epidemiological impact of cross-

immunity. Journal of the Royal Society Interface, 10(86):20130414, Sep 2013.

[140] Mathieu Reymond. On incorporating prior knowledge about the decision maker in multi-

objective reinforcement learning. PhD thesis, Vrije Universiteit Brussel, 2023.

[141] Mathieu Reymond, Conor F. Hayes, Lander Willem, Roxana Radulescu, Steven Abrams,

Diederik M. Roijers, Enda Howley, Patrick Mannion, Niel Hens, Ann Now´e, and Pieter

Libin. Exploring the pareto front of multi-objective COVID-19 mitigation policies using

reinforcement learning. Expert Syst. Appl., 249:123686, 2024.

[142] Luis Rivera, Shibadas Biswal, Xavier S´aez-Llorens, Humberto Reynales, Eduardo L´opez-

Medina, Charissa Borja-Tabora, Lulu Bravo, Chukiat Sirivichayakul, Pope Kosalaraksa,

Luis Martinez Vargas, Delia Yu, Veerachai Watanaveeradej, Felix Espinoza, Rey-

naldo Dietze, LakKumar Fernando, Pujitha Wickramasinghe, Edson Duarte MoreiraJr,

Asvini D Fernando, Dulanie Gunasekera, Kleber Luz, Rivaldo Venˆancioda Cunha, Mar-

tina Rauscher, Olaf Zent, Mengya Liu, Elaine Hoffman, Inge LeFevre, Vianney Tricou,

Derek Wallace, MariaTheresa Alera, and for the TIDES study group Borkowski, Astrid.

Three-year Efficacy and Safety of Takeda’s Dengue Vaccine Candidate (TAK-003). Clinical

Infectious Diseases, 75(1):107–117, 10 2021.

[143] Luis Rivera, Shibadas Biswal, Xavier S´aez-Llorens, Humberto Reynales, Eduardo L´opez-

Medina, Charissa Borja-Tabora, Lulu Bravo, Chukiat Sirivichayakul, Pope Kosalaraksa,

Luis Martinez Vargas, Delia Yu, Veerachai Watanaveeradej, Felix Espinoza, Rey-

naldo Dietze, LakKumar Fernando, Pujitha Wickramasinghe, Edson Duarte MoreiraJr,

Asvini D Fernando, Dulanie Gunasekera, Kleber Luz, Rivaldo Venˆancioda Cunha, Mar-

tina Rauscher, Olaf Zent, Mengya Liu, Elaine Hoffman, Inge LeFevre, Vianney Tricou,

Derek Wallace, MariaTheresa Alera, and for the TIDES study group Borkowski, Astrid.

Three-year Efficacy and Safety of Takeda’s Dengue Vaccine Candidate (TAK-003). Clinical

Infectious Diseases, 75(1):107–117, 10 2021.BIBLIOGRAPHY 155

[144] Christian P. Robert. The bayesian choice : from decision-theoretic foundations to compu-

tational implementation. 2007.

[145] Filipe Rocha, Ma´ıra Aguiar, Max Souza, and Nico Stollenwerk. Understanding the effect

of vector dynamics in epidemic models using center manifold analysis. AIP Conference

Proceedings, 1479(1):1319–1322, 09 2012.

[146] Filipe Rocha, Lu´ıs Mateus, Urszula Skwara, Ma´ıra Aguiar, and Nico Stollenwerk. Under-

standing dengue fever dynamics: a study of seasonality in vector-borne disease models.

International Journal of Computer Mathematics, 93(8):1405–1422, 2016.

[147] Diederik Roijers, Luisa Zintgraf, and Ann Nowe. Interactive thompson sampling for multi-

objective multi-armed bandits. pages 18–34, 09 2017.

[148] Adrian Rosebrock. Intersection over union (iou) for object detection, 2016. Accessed:

2024-06-17.

[149] Alan L. Rothman. Dengue: defining protective versus pathologic immunity. The Journal

of Clinical Investigation, 113(7):946–951, 4 2004.

[150] Alan L. Rothman. Cellular immunology of sequential dengue virus infection and its role

in disease pathogenesis. Current Topics in Microbiology and Immunology, 338(1):83 – 98,

2009. Cited by: 98.

[151] Daniel Russo. Simple bayesian algorithms for best arm identification. In Annual Conference

Computational Learning Theory, 2016.

[152] Daniel Russo and Benjamin Van Roy. Eluder dimension and the sample complexity of

optimistic exploration. In Neural Information Processing Systems, 2013.

[153] Ilya O. Ryzhov, Warren B. Powell, and Peter I. Frazier. The knowledge gradient algorithm

for a general class of online learning problems. Operations Research, 60(1):180–195, 2012.

[154] A B SABIN. The dengue group of viruses and its family relationships. Bacteriological

Reviews, 14(3):225–232, September 1950.

[155] A B Sabin. Research on dengue during world war ii. The American Journal of Tropical

Medicine and Hygiene, 1(1):30–50, Jan 1952.

[156] Nadhirat Sangkawibha, Suntharee Rojanasuphot, Sompop Ahandrik, Sukho Viriyapongse,

Sujarti Jatanasen, Viraj Salitul, Boonluan Phanthumachinda, and Scott B. Halstead. Risk

factors in dengue shock syndrome: A prospective epidemiologic study in rayong, thailand:

I. the 1980 outbreak. American Journal of Epidemiology, 120(5):653–669, 11 1984.

[157] Boris V. Schmid and Mirjam E E Kretzschmar. Determinants of sexual network structure

and their impact on cumulative network measures. PLoS Computational Biology, 8, 2012.

[158] Steven L. Scott. A modern bayesian look at the multi-armed bandit. Applied Stochastic

Models in Business and Industry, 26:639–658, 2010.

[159] Afrina Andriani Sebayang, Hilda Fahlena, Vizda Anam, Dami´an Knopoff, Nico Stollen-

werk, Ma´ıra Aguiar, and Edy Soewono. Modeling dengue immune responses mediated by

antibodies: A qualitative study. Biology, 10(9), 2021.

[160] Eunha Shim. Optimal dengue vaccination strategies of seropositive individuals. Mathe-

matical Biosciences and Engineering, 16(3):1171–1189, 2019.BIBLIOGRAPHY 156

[161] Beatriz Sierra, Ana B. Perez, Katrin Vogt, Gissel Garcia, Kathrin Schmolke, Eglys Aguirre,

Mayling Alvarez, Florian Kern, Gustavo Kour´ı, Hans-Dieter Volk, and Maria G. Guzman.

Secondary heterologous dengue infection risk: Disequilibrium between immune regulation

and inflammation. Cellular Immunology, 262(2):134–140, 2010.

[162] Alexander Slepoy, Aidan P. Thompson, and Steven J. Plimpton. A constant-time kinetic

monte carlo algorithm for simulation of large biochemical reaction networks. The Journal

of chemical physics, 128 20:205101, 2008.

[163] Aleksandrs Slivkins et al. Introduction to multi-armed bandits. Foundations and Trends®

in Machine Learning, 12(1-2):1–286, 2019.

[164] Saranya Sridhar, Alexander Luedtke, Edith Langevin, Ming Zhu, Matthew Bonaparte,

Tifany Machabert, Stephen Savarino, Betzana Zambrano, Annick Moureau, Alena Khro-

mava, Zoe Moodie, Ted Westling, Cesar Mascare˜nas, Carina Frago, Margarita Cort´es,

Danaya Chansinghakul, Fernando Noriega, Alain Bouckenooghe, Josh Chen, Su-Peing Ng,

Peter B. Gilbert, Sanjay Gurunathan, and Carlos A. DiazGranados. Effect of dengue

serostatus on dengue vaccine safety and efficacy. New England Journal of Medicine,

379(4):327–340, 2018. PMID: 29897841.

[165] Ashley L. St. John and Abhay P. S. Rathore. Adaptive immune responses to primary and

secondary dengue virus infections. Nature Reviews Immunology, 19(4):218–230, 2019.

[166] N. Stollenwerk, M. Aguiar, S. Ballesteros, J. Boto, B. Kooi, and L. Mateus. Dynamic noise,

chaos and parameter estimation in population biology. Interface Focus, 2(2):156–169, 2012.

[167] Auliya Abdurrohim Suwantika, Woro Supadmi, Mohammad Ali, and Rizky Abdulah. Cost-

effectiveness and budget impact analyses of dengue vaccination in indonesia. PLoS Ne-

glected Tropical Diseases, 15(8):e0009664, 2021.

[168] T. T. Tanimoto. An Elementary Mathematical Theory of Classification and Prediction.

International Business Machines Corporation, 1958. Accessed: 2024-06-18.

[169] Quirine A. ten Bosch, Brajendra K. Singh, Muhammad R. A. Hassan, Dave D. Chadee, and

Edwin Michael. The role of serotype interactions and seasonality in dengue model selection

and control: Insights from a pattern matching approach. PLOS Neglected Tropical Diseases,

10(5):1–25, 05 2016.

[170] William R. Thompson. On the likelihood that one unknown probability exceeds another

in view of the evidence of two samples. Biometrika, 25:285–294, 1933.

[171] Sherry Towers. Sir infectious disease model with age classes, 2012. URL:

http://sherrytowers.com/2012/12/11/sir-model-with-age-classes/ (Access Date: Access

Date).

[172] T. Tsheten, A. Clements, D. Gray, R. Adhikary, and K. Wangdi. Clinical features and out-

comes of covid-19 and dengue co-infection: a systematic review. BMC Infectious Diseases,

21, 2021.

[173] Peter Vamplew, Richard Dazeley, Adam Berry, Rustam Issabekov, and Evan Dekker. Em-

pirical evaluation methods for multiobjective reinforcement learning algorithms. Machine

Learning, 84:51–80, 2011.

[174] David W. Vaughn, Sharone Green, Siripen Kalayanarooj, Bruce L. Innis, Suchitra Nim-

mannitya, Saroj Suntayakorn, Timothy P. Endy, Boonyos Raengsakulrach, Alan L. Roth-BIBLIOGRAPHY 157

man, Francis A. Ennis, and Ananda Nisalak. Dengue Viremia Titer, Antibody Response

Pattern, and Virus Serotype Correlate with Disease Severity. The Journal of Infectious

Diseases, 181(1):2–9, 01 2000.

[175] Joannes Vermorel and Mehryar Mohri. Multi-armed bandit algorithms and empirical eval-

uation. In European conference on machine learning, pages 437–448. Springer, 2005.

[176] Luis Villar, Gustavo Horacio Dayan, Jos´e Luis Arredondo-Garc´ıa, Doris Maribel Rivera,

Rivaldo Cunha, Carmen Deseda, Humberto Reynales, Maria Selma Costa, Javier Osvaldo

Morales-Ram´ırez, Gabriel Carrasquilla, Luis Carlos Rey, Reynaldo Dietze, Kleber Luz,

Enrique Rivas, Maria Consuelo Miranda Montoya, Margarita Cort´es Supelano, Betzana

Zambrano, Edith Langevin, Mark Boaz, Nadia Tornieporth, Melanie Saville, and Fernando

Noriega. Efficacy of a tetravalent dengue vaccine in children in latin america. New England

Journal of Medicine, 372(2):113–123, 2015. PMID: 25365753.

[177] Eryu Wang, Haolin Ni, Renling Xu, Alan D. T. Barrett, Stanley J. Watowich, Duane J.

Gubler, and Scott C. Weaver. Evolutionary relationships of endemic/epidemic and sylvatic

dengue viruses. Journal of Virology, 74(7):3227–3234, 2000.

[178] Wei-Kung Wang, Day-Yu Chao, Chuan-Liang Kao, Han-Chung Wu, Yung-Ching Liu,

Chien-Ming Li, Shih-Chung Lin, Shih-Ting Ho, Jyh-Hsiung Huang, and Chwan-Chuen

King. High levels of plasma dengue viral load during defervescence in patients with dengue

hemorrhagic fever: Implications for pathogenesis. Virology, 305(2):330–338, 2003.

[179] Weijia Wang and Mich`ele Sebag. Hypervolume indicator and dominance reward based

multi-objective monte-carlo tree search. Machine Learning, 92:403 – 429, 2013.

[180] Daniela Weiskopf and Alessandro Sette. T-cell immunity to infection with dengue virus in

humans. Frontiers in Immunology, 5, 2014.

[181] E.G. Westaway, M.A. Brinton, S.Ya. Gaidamovich, M.C. Horzinek, A. Igarashi,

L. K¨a¨ari¨ainen, D.K. Lvov, J.S. Porterfield, P.K. Russell, and D.W. Trent. Flaviviridae.

Intervirology, 24(4):183–192, 07 2008.

[182] Paul S. Wikramaratna, Cameron P. Simmons, Sunetra Gupta, and Mario Recker. The

effects of tertiary and quaternary infections on the epidemiology of dengue. PLOS ONE,

5(8):1–8, 08 2010.

[183] Paul S. Wikramaratna, Cameron P. Simmons, Sunetra Gupta, and Mario Recker. The

effects of tertiary and quaternary infections on the epidemiology of dengue. PLOS ONE,

5(8):1–8, 08 2010.

[184] Annelies Wilder-Smith. Dengue vaccine development by the year 2020: challenges and

prospects. Current Opinion in Virology, 43:71–78, 2020. Viral elimination * Special Section:

Defeat Dengue.

[185] Lander Willem, Sean Stijven, Ekaterina Vladislavleva, Jan Broeckhove, Philippe Beutels,

and Niel Hens. Active learning to understand infectious disease models and improve policy

making. PLoS Computational Biology, 10, 2014.

[186] Lander Willem, Frederik Verelst, Joke Bilcke, Niel Hens, and Philippe Beutels. Lessons

from a decade of individual-based models for infectious disease transmission: a systematic

review (2006-2015). BMC Infectious Diseases, 17, 2017.BIBLIOGRAPHY 158

[187] Hannah Woodall and Ben Adams. Partial cross-enhancement in models for dengue epi-

demiology. Journal of Theoretical Biology, 351:67–73, 2014.

[188] World Health Organization. Meeting of the strategic advisory group of experts on immu-

nization, april 2016: conclusions and recommendations. Weekly Epidemiological Record,

91(21):266–284, 5 2016. Accessed: 2024-01-24.

[189] World Health Organization. Dengue vaccine: Who position paper, july 2016 – recommen-

dations. Vaccine, 35(9):1200–1201, 2017.

[190] World Health Organization. Dengue vaccines: Who position paper – september 2018.

Weekly Epidemiological Record, 93(35):457–476, 9 2018. Accessed: 2024-01-24.

[191] World Health Organization. Dengue and severe dengue, March 2023. Accessed: 2024-01-24.

[192] World Health Organization. Dengue - global situation, May 2024. [Accessed: 2024-07-21].

[193] Joseph T. Wu, Joseph T. Wu, Steven Riley, Christophe Fraser, and Gabriel M. Leung.

Reducing the impact of the next influenza pandemic using household-based public health

interventions. Hong Kong medical journal = Xianggang yi xue za zhi, 15 Suppl 9:38–41,

2006.

[194] Ling Xue, Hongyu Zhang, Wei Sun, and Caterina Scoglio. Transmission dynamics of

multi-strain dengue virus with cross-immunity. Applied Mathematics and Computation,

392:125742, 2021.

[195] Luis Mier y Teran-Romero, Ira B. Schwartz, and Derek A.T. Cummings. Breaking the

symmetry: Immune enhancement increases persistence of dengue viruses in the presence

of asymmetric transmission rates. Journal of Theoretical Biology, 332:203–210, 2013.

[196] Saba Yahyaa. The exploration vs exploitation trade-off in bandit problems: An empirical

study. 04 2012.

[197] Saba Yahyaa, Madalina Drugan, and Bernard Manderick. Multivariate normal distribution

based multi-armed bandits pareto algorithm. 01 2014.

[198] Saba Yahyaa, Madalina M. Drugan, and Bernard Manderick. Scalarized and Pareto Knowl-

edge Gradient for Multi-objective Multi-armed Bandits, pages 99–116. Springer Interna-

tional Publishing, Cham, 2015.

[199] Saba Yahyaa and Bernard Manderick. Thompson sampling for multi-objective multi-armed

bandits problem. 04 2015.

[200] Saba Q. Yahyaa, Madalina M. Drugan, and Bernard Manderick. Annealing-pareto multi-

objective multi-armed bandit algorithm. In 2014 IEEE Symposium on Adaptive Dynamic

Programming and Reinforcement Learning (ADPRL), pages 1–8, 2014.

[201] Saba Q. Yahyaa, Madalina M. Drugan, and Bernard Manderick. Knowledge gradient for

multi-objective multi-armed bandit algorithms. In B´eatrice Duval, H. Jaap van den Herik,

St´ephane Loiseau, and Joaquim Filipe, editors, ICAART 2014 - Proceedings of the 6th

International Conference on Agents and Artificial Intelligence, Volume 1, ESEO, Angers,

Loire Valley, France, 6-8 March, 2014, pages 74–83. SciTePress, 2014.

[202] Saba Q. Yahyaa, Madalina M. Drugan, and Bernard Manderick. Linear scalarized knowl-

edge gradient in the multi-objective multi-armed bandits problem. In 22th European Sym-BIBLIOGRAPHY 159

posium on Artificial Neural Networks, ESANN 2014, Bruges, Belgium, April 23-25, 2014,

2014.

[203] Logan Michael Yliniemi and Kagan Tumer. Multi-objective multiagent credit assignment

in reinforcement learning and nsga-ii. Soft Computing, 20:3869 – 3887, 2016.

[204] Wu Zeng, Yara A Halasa-Rappel, Nicolas Baurin, Laurent Coudeville, and Donald S Shep-

ard. Cost-effectiveness of dengue vaccination in ten endemic countries. Vaccine, 36(3):413–

420, 2018.

[205] Eckart Zitzler and Lothar Thiele. Multiobjective evolutionary algorithms: a comparative

case study and the strength pareto approach. IEEE Trans. Evol. Comput., 3:257–271,

1999.

[206] Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M. Fonseca, and Viviane Grunert

da Fonseca. Performance assessment of multiobjective optimizers: an analysis and review.

IEEE Trans. Evol. Comput., 7:117–132, 2003.

[207] Masrour Zoghi, Shimon Whiteson, Remi Munos, and Maarten de Rijke. Relative upper

confidence bound for the k-armed dueling bandit problem, 2013.

Download scriptie (116.81 MB)
Universiteit of Hogeschool
Vrije Universiteit Brussel
Thesis jaar
2024
Promotor(en)
Prof. Dr. Pieter Libin