Self-Improved Retrosynthetic Planning

Junsu Kim1 (presenter), Sungsoo Ahn2, Hankook Lee1, and Jinwoo Shin1

1Korea Advanced Institute of Science and Technology (KAIST)2Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)

• Goal: finding a series of chemically valid reactions starting from target molecule until reaching the building block molecules.• In a backward and recursive manner.

• Crucial in drug discovery and material design.

Background: Retrosynthetic Planning

Retrosynthetic planning

Building blocks

Intermediate molecule

Target molecule

Single-stepretrosynthesis model

Search algorithm

• The main challenge of retrosynthetic planning is twofold:• (a) Finding an accurate single-step retrosynthesis model.

• Why? To guarantee chemical validity of searched pathways.

• (b) Designing an efficient search algorithm.

• Why? The search space is huge due to the vast number of possible chemical reactions.

Background: Retrosynthetic Planning

Building blocks

Intermediate molecule

Target molecule

• Most of existing works are two-stage framework.• Stage 1. Train single-step retrosynthesis model parameterized by deep neural networks (DNN).

• Stage 2. Run search algorithms with the trained DNN-based single-step retrosynthesis model.

Background: Retrosynthetic Planning

Single-step retrosynthesis model

(Fixed) Single-stepretrosynthesis model

Search algorithm

Product Reactants

• The existing two-stage frameworks are suboptimal.• DNN-based single-step retrosynthesis model only considers the requirement (a), not (b).

• (a) Reaction pathways should be represented by real-world reactions.

• (b) Reaction pathways should be executable using “building block” molecules.

• Motivated by this, we propose an end-to-end framework.• Directly training the DNNs towards generating reaction pathways with both properties (a) and (b).

Our approach

• Key idea: self-improving procedure that trains backward reaction model(i.e., single-step retrosynthesis model) to imitate successful pathways found by itself.

Our approach

• Step A. Gather reactions from reaction pathways found by search algorithm combined with the backward reaction model.

Our approach

• Step B. Discard unrealistic reactions using a reference backward reaction model. • Reference backward reaction model determines whether a reaction resembles real-world reactions.

Our approach

• Step C. Augment reactions via a forward reaction model (i.e., single-step synthesis model).

Our approach

• Step D. Train the backward reaction model to imitate the generated reactions.

Our approach

• Step A. Gather reactions from reaction pathways found by search algorithm combined with the backward reaction model.

Our approach (in detail)

Detailed description of Step A

Retrosynthetic planning algorithm

Backward reaction model

Search algorithm

Reaction collection

C

<latexit sha1_base64="sSHMS5xtQKqOAG4ZgvOiCUDl0mE=">AAAB8nicbVBNS8NAEN3Urxq/qh69LBbBU0lEUA9isRePFewHpKFstpt26WYTdidCCf0ZXjwo0qv/w7sX8d+4aXvQ1gcDj/dmmDcTJIJrcJxvq7Cyura+Udy0t7Z3dvdK+wdNHaeKsgaNRazaAdFMcMkawEGwdqIYiQLBWsGwlvutR6Y0j+UDjBLmR6QvecgpASN5nYjAgBKR1cbdUtmpOFPgZeLOSfnmw75OJl92vVv67PRimkZMAhVEa891EvAzooBTwcZ2J9UsIXRI+swzVJKIaT+bRh7jE6P0cBgrUxLwVP09kZFI61EUmM48ol70cvE/z0shvPQzLpMUmKSzRWEqMMQ4vx/3uGIUxMgQQhU3WTEdEEUomC/Z5gnu4snLpHlWcc8rV/dOuXqLZiiiI3SMTpGLLlAV3aE6aiCKYvSEXtCrBdaz9WZNZq0Faz5ziP7Aev8B1r+Uog==</latexit>

Gather reactions

Searchsuccessful pathway

given target molecule T

T

Reaction pathway

• Step B. Discard unrealistic reactions using a reference backward reaction model. • Reference backward reaction model determines whether a reaction resembles real-world reactions.

Our approach (in detail)

Detailed description of Step B

Reference backward reaction model

pass

reject

likelihood > ✏

<latexit sha1_base64="0jAFROTOBNvS/mLRo/nKW6CNiqc=">AAACB3icbVDLSgMxFM3UV62vqktBgkVwVWakoG6k6MZlBfuAtpRMetuGZiZDckcsQ3du/BU3LhRx6y+4829M21lo64HA4Zxzk9zjR1IYdN1vJ7O0vLK6ll3PbWxube/kd/dqRsWaQ5UrqXTDZwakCKGKAiU0Ig0s8CXU/eH1xK/fgzZChXc4iqAdsH4oeoIztFInf9hCeEDERIqhvWSgVHdML2kLIiPkJFBwi+4UdJF4KSmQFJVO/qvVVTwOIEQumTFNz42wnTCNgksY51qxgYjxIetD09KQBWDayXSPMT22Spf2lLYnRDpVf08kLDBmFPg2GTAcmHlvIv7nNWPsnbcTEUYxQshnD/ViSVHRSSm0KzRwlCNLGNfC/pXyAdOMo60uZ0vw5ldeJLXTolcqXtyWCuWrtI4sOSBH5IR45IyUyQ2pkCrh5JE8k1fy5jw5L8678zGLZpx0Zp/8gfP5A7gimdw=</latexit>

likelihood ✏

<latexit sha1_base64="n3zs+sxjSyg4mJf01oyFgRM3Q0g=">AAACCnicbVDLSgMxFM3UV62vqks30SK4KjNSUHdFNy4r2Ad0Ssmkt21oZjImd8QydO3GX3HjQhG3foE7/8b0sdDWA4HDOecmuSeIpTDout9OZml5ZXUtu57b2Nza3snv7tWMSjSHKldS6UbADEgRQRUFSmjEGlgYSKgHg6uxX78HbYSKbnEYQytkvUh0BWdopXb+0Ed4QMRUioG9pK9UZ0R9CXfUh9gIOc4U3KI7AV0k3owUyAyVdv7L7yiehBAhl8yYpufG2EqZRsEljHJ+YiBmfMB60LQ0YiGYVjpZZUSPrdKhXaXtiZBO1N8TKQuNGYaBTYYM+2beG4v/ec0Eu+etVERxghDx6UPdRFJUdNwL7QgNHOXQEsa1sH+lvM8042jby9kSvPmVF0nttOiVihc3pUL5clZHlhyQI3JCPHJGyuSaVEiVcPJInskreXOenBfn3fmYRjPObGaf/IHz+QNlDJta</latexit>

• Step C. Augment reactions via a forward reaction model (i.e., single-step synthesis model).• Based on the knowledge; there can be multiple products resulting from the same reactant-set.

Our approach (in detail)

Detailed description of Step C

Original reaction

Augmented reaction

• Step D. Train the backward reaction model by maximizing the log-likelihood of the reactions in .

Our approach (in detail)

C [ C0

<latexit sha1_base64="CjQBqiODcStjLaW0X5U6RyWA3dY=">AAACBnicbVDLSsNAFL3xWesr6lKEwSK6KokU1F2xG5cV7AOaUCbTSTt08mBmIpSQlRt/xY0LRdz6De78GydtQG09cOFwzr3ce48XcyaVZX0ZS8srq2vrpY3y5tb2zq65t9+WUSIIbZGIR6LrYUk5C2lLMcVpNxYUBx6nHW/cyP3OPRWSReGdmsTUDfAwZD4jWGmpbx45AVYjgnnayJBDkhj9CKdZ36xYVWsKtEjsglSgQLNvfjqDiCQBDRXhWMqebcXKTbFQjHCalZ1E0hiTMR7SnqYhDqh00+kbGTrRygD5kdAVKjRVf0+kOJByEni6M79Rznu5+J/XS5R/6aYsjBNFQzJb5CccqQjlmaABE5QoPtEEE8H0rYiMsMBE6eTKOgR7/uVF0j6v2rXq1W2tUr8u4ijBIRzDGdhwAXW4gSa0gMADPMELvBqPxrPxZrzPWpeMYuYA/sD4+AblRZjG</latexit>

• Our framework achieves the state-of-the-art performance.• Ours improves success rate from 86.84% to 96.32%.

• Search time, length and cost of searched pathways are improved.

• Backward reaction model maintains its reliability.

Experiments

• We propose an end-to-end framework based on self-improved model adaptation to improve retrosynthetic planning.

• We also propose an additional reaction augmentation scheme.

• Our work reduces the gap between supervised learning of single-step retrosynthesis models and the goal of retrosynthetic planning.

Conclusion