sim-real joint reinforcement transfer for 3d indoor...

Sim-Real Joint Reinforcement Transfer for 3D Indoor Navigation

Fengda Zhu† Linchao Zhu§ Yi Yang§‡

†UTS-SUSTech Joint Research Centre, Southern University of Science and Technology§CAI, University of Technology Sydney ‡Baidu Research

{zhufengdaaa,zhulinchao7}@gmail.com [email protected]

Abstract

There has been an increasing interest in 3D indoor nav-

igation, where a robot in an environment moves to a target

according to an instruction. To deploy a robot for naviga-

tion in the physical world, lots of training data is required

to learn an effective policy. It is quite labour intensive to

obtain sufficient real environment data for robots training

while synthetic data is much easier to construct by render-

ing. Though it is promising to utilize the synthetic environ-

ments to facilitate navigation training in the real world, real

environment are heterogeneous from synthetic environment

in two aspects. First, the visual representations of the two

environments have significant variances. Second, the house

plans of the two environments are rather different. There-

fore, two types of information, i.e., visual representation

and policy behavior, need to be adapted in the reinforce-

ment model. The learning procedure of visual representa-

tion and that of policy behavior are presumably reciprocal.

We propose to jointly adapt visual representation and pol-

icy behavior to leverage the mutual impacts of environment

and policy. Specifically, our method employs an adversarial

feature adaptation model for visual representation transfer

and a policy mimic strategy for policy behavior imitation.

The experimental results show that our method outperforms

the baseline by 21.73% without any additional human an-

notations.

1. Introduction

Autonomous indoor navigation is a problem in which the

robot navigates to a target according to an instruction within

a building such as house, office and yard. This task will

benefit many applications where a robot takes over human

being’s job, such as house cleaning, package delivery and

patrolling. Solving indoor navigation in a 3D environment

Part of this work was done when Yi Yang was visiting Baidu Research

during his Professional Experience Program.

Corresponding author: Yi Yang.

Rich Synthetic Environment

Insufficient Real Environment

... ...

train<latexit sha1_base64="z3p2E60EO1lWmwgJXfB9OdNKJyM=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWj5qKtSgWnPr7kJkHbwCalCoNah+9YcJy2KukElqTM9zUwxyqlEwyWeVfmZ4StmEjnjPoqIxN0G+WHZGLqwzJFGi7VNIFu7viZzGxkzj0HbGFMdmtTY3/6v1MoxuglyoNEOu2PKjKJMEEzK/nAyF5gzl1AJlWthdCRtTTRnafCo2BG/15HVoX9U9yw/XteZtEUcZzuAcLsGDBjThHlrgAwMBz/AKb45yXpx352PZWnKKmVP4I+fzB/3tjso=</latexit><latexit sha1_base64="z3p2E60EO1lWmwgJXfB9OdNKJyM=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWj5qKtSgWnPr7kJkHbwCalCoNah+9YcJy2KukElqTM9zUwxyqlEwyWeVfmZ4StmEjnjPoqIxN0G+WHZGLqwzJFGi7VNIFu7viZzGxkzj0HbGFMdmtTY3/6v1MoxuglyoNEOu2PKjKJMEEzK/nAyF5gzl1AJlWthdCRtTTRnafCo2BG/15HVoX9U9yw/XteZtEUcZzuAcLsGDBjThHlrgAwMBz/AKb45yXpx352PZWnKKmVP4I+fzB/3tjso=</latexit><latexit sha1_base64="z3p2E60EO1lWmwgJXfB9OdNKJyM=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWj5qKtSgWnPr7kJkHbwCalCoNah+9YcJy2KukElqTM9zUwxyqlEwyWeVfmZ4StmEjnjPoqIxN0G+WHZGLqwzJFGi7VNIFu7viZzGxkzj0HbGFMdmtTY3/6v1MoxuglyoNEOu2PKjKJMEEzK/nAyF5gzl1AJlWthdCRtTTRnafCo2BG/15HVoX9U9yw/XteZtEUcZzuAcLsGDBjThHlrgAwMBz/AKb45yXpx352PZWnKKmVP4I+fzB/3tjso=</latexit>

train<latexit sha1_base64="z3p2E60EO1lWmwgJXfB9OdNKJyM=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWj5qKtSgWnPr7kJkHbwCalCoNah+9YcJy2KukElqTM9zUwxyqlEwyWeVfmZ4StmEjnjPoqIxN0G+WHZGLqwzJFGi7VNIFu7viZzGxkzj0HbGFMdmtTY3/6v1MoxuglyoNEOu2PKjKJMEEzK/nAyF5gzl1AJlWthdCRtTTRnafCo2BG/15HVoX9U9yw/XteZtEUcZzuAcLsGDBjThHlrgAwMBz/AKb45yXpx352PZWnKKmVP4I+fzB/3tjso=</latexit><latexit sha1_base64="z3p2E60EO1lWmwgJXfB9OdNKJyM=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWj5qKtSgWnPr7kJkHbwCalCoNah+9YcJy2KukElqTM9zUwxyqlEwyWeVfmZ4StmEjnjPoqIxN0G+WHZGLqwzJFGi7VNIFu7viZzGxkzj0HbGFMdmtTY3/6v1MoxuglyoNEOu2PKjKJMEEzK/nAyF5gzl1AJlWthdCRtTTRnafCo2BG/15HVoX9U9yw/XteZtEUcZzuAcLsGDBjThHlrgAwMBz/AKb45yXpx352PZWnKKmVP4I+fzB/3tjso=</latexit><latexit sha1_base64="z3p2E60EO1lWmwgJXfB9OdNKJyM=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWj5qKtSgWnPr7kJkHbwCalCoNah+9YcJy2KukElqTM9zUwxyqlEwyWeVfmZ4StmEjnjPoqIxN0G+WHZGLqwzJFGi7VNIFu7viZzGxkzj0HbGFMdmtTY3/6v1MoxuglyoNEOu2PKjKJMEEzK/nAyF5gzl1AJlWthdCRtTTRnafCo2BG/15HVoX9U9yw/XteZtEUcZzuAcLsGDBjThHlrgAwMBz/AKb45yXpx352PZWnKKmVP4I+fzB/3tjso=</latexit>

transfer<latexit sha1_base64="WHnxvpk2BCaIfZpCtx96sHWgCEE=">AAAB73icbZBNS8NAEIYn9avWr6pHL8EieCqJCHosevFYwX5AG8pmO2mXbjZxdyKU0j/hxYMiXv073vw3btsctPWFhYd3ZtiZN0ylMOR5305hbX1jc6u4XdrZ3ds/KB8eNU2SaY4NnshEt0NmUAqFDRIksZ1qZHEosRWObmf11hNqIxL1QOMUg5gNlIgEZ2StNmmmTIS6V654VW8udxX8HCqQq94rf3X7Cc9iVMQlM6bjeykFE6ZJcInTUjczmDI+YgPsWFQsRhNM5vtO3TPr9N0o0fYpcufu74kJi40Zx6HtjBkNzXJtZv5X62QUXQcTodKMUPHFR1EmXUrc2fFuX2jkJMcWGNfC7uryIdOMk42oZEPwl09eheZF1bd8f1mp3eRxFOEETuEcfLiCGtxBHRrAQcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPZC6QLw==</latexit><latexit sha1_base64="WHnxvpk2BCaIfZpCtx96sHWgCEE=">AAAB73icbZBNS8NAEIYn9avWr6pHL8EieCqJCHosevFYwX5AG8pmO2mXbjZxdyKU0j/hxYMiXv073vw3btsctPWFhYd3ZtiZN0ylMOR5305hbX1jc6u4XdrZ3ds/KB8eNU2SaY4NnshEt0NmUAqFDRIksZ1qZHEosRWObmf11hNqIxL1QOMUg5gNlIgEZ2StNmmmTIS6V654VW8udxX8HCqQq94rf3X7Cc9iVMQlM6bjeykFE6ZJcInTUjczmDI+YgPsWFQsRhNM5vtO3TPr9N0o0fYpcufu74kJi40Zx6HtjBkNzXJtZv5X62QUXQcTodKMUPHFR1EmXUrc2fFuX2jkJMcWGNfC7uryIdOMk42oZEPwl09eheZF1bd8f1mp3eRxFOEETuEcfLiCGtxBHRrAQcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPZC6QLw==</latexit><latexit sha1_base64="WHnxvpk2BCaIfZpCtx96sHWgCEE=">AAAB73icbZBNS8NAEIYn9avWr6pHL8EieCqJCHosevFYwX5AG8pmO2mXbjZxdyKU0j/hxYMiXv073vw3btsctPWFhYd3ZtiZN0ylMOR5305hbX1jc6u4XdrZ3ds/KB8eNU2SaY4NnshEt0NmUAqFDRIksZ1qZHEosRWObmf11hNqIxL1QOMUg5gNlIgEZ2StNmmmTIS6V654VW8udxX8HCqQq94rf3X7Cc9iVMQlM6bjeykFE6ZJcInTUjczmDI+YgPsWFQsRhNM5vtO3TPr9N0o0fYpcufu74kJi40Zx6HtjBkNzXJtZv5X62QUXQcTodKMUPHFR1EmXUrc2fFuX2jkJMcWGNfC7uryIdOMk42oZEPwl09eheZF1bd8f1mp3eRxFOEETuEcfLiCGtxBHRrAQcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPZC6QLw==</latexit>

action<latexit sha1_base64="cU1FzO+DRJJixbY48Gi4zS77RPI=">AAAB7XicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoMuiG5cV7APaoWTSTBubx5BkhDL0H9y4UMSt/+POvzHTzkJbD4QczrmXe++JEs6M9f1vr7S2vrG5Vd6u7Ozu7R9UD4/aRqWa0BZRXOluhA3lTNKWZZbTbqIpFhGnnWhym/udJ6oNU/LBThMaCjySLGYEWye1Mcm/QbXm1/050CoJClKDAs1B9as/VCQVVFrCsTG9wE9smGFtGeF0VumnhiaYTPCI9hyVWFATZvNtZ+jMKUMUK+2etGiu/u7IsDBmKiJXKbAdm2UvF//zeqmNr8OMySS1VJLFoDjlyCqUn46GTFNi+dQRTDRzuyIyxtpl4AKquBCC5ZNXSfuiHjh+f1lr3BRxlOEETuEcAriCBtxBE1pA4BGe4RXePOW9eO/ex6K05BU9x/AH3ucPtiWPNA==</latexit><latexit sha1_base64="cU1FzO+DRJJixbY48Gi4zS77RPI=">AAAB7XicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoMuiG5cV7APaoWTSTBubx5BkhDL0H9y4UMSt/+POvzHTzkJbD4QczrmXe++JEs6M9f1vr7S2vrG5Vd6u7Ozu7R9UD4/aRqWa0BZRXOluhA3lTNKWZZbTbqIpFhGnnWhym/udJ6oNU/LBThMaCjySLGYEWye1Mcm/QbXm1/050CoJClKDAs1B9as/VCQVVFrCsTG9wE9smGFtGeF0VumnhiaYTPCI9hyVWFATZvNtZ+jMKUMUK+2etGiu/u7IsDBmKiJXKbAdm2UvF//zeqmNr8OMySS1VJLFoDjlyCqUn46GTFNi+dQRTDRzuyIyxtpl4AKquBCC5ZNXSfuiHjh+f1lr3BRxlOEETuEcAriCBtxBE1pA4BGe4RXePOW9eO/ex6K05BU9x/AH3ucPtiWPNA==</latexit><latexit sha1_base64="cU1FzO+DRJJixbY48Gi4zS77RPI=">AAAB7XicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoMuiG5cV7APaoWTSTBubx5BkhDL0H9y4UMSt/+POvzHTzkJbD4QczrmXe++JEs6M9f1vr7S2vrG5Vd6u7Ozu7R9UD4/aRqWa0BZRXOluhA3lTNKWZZbTbqIpFhGnnWhym/udJ6oNU/LBThMaCjySLGYEWye1Mcm/QbXm1/050CoJClKDAs1B9as/VCQVVFrCsTG9wE9smGFtGeF0VumnhiaYTPCI9hyVWFATZvNtZ+jMKUMUK+2etGiu/u7IsDBmKiJXKbAdm2UvF//zeqmNr8OMySS1VJLFoDjlyCqUn46GTFNi+dQRTDRzuyIyxtpl4AKquBCC5ZNXSfuiHjh+f1lr3BRxlOEETuEcAriCBtxBE1pA4BGe4RXePOW9eO/ex6K05BU9x/AH3ucPtiWPNA==</latexit>


Synthetic Model<latexit sha1_base64="PG92w/kLv9ojFjVdyYawM1/19Lk=">AAAB+XicbZDLSgMxFIYz9VbrbdSlm2ARXJUZEXRZdONGqGgv0A4lkzltQzPJkGQKw9A3ceNCEbe+iTvfxrSdhbYeCHz8/znJyR8mnGnjed9OaW19Y3OrvF3Z2d3bP3APj1papopCk0ouVSckGjgT0DTMcOgkCkgccmiH49uZ356A0kyKJ5MlEMRkKNiAUWKs1Hfdx0yYERhGe/heRsD7btWrefPCq+AXUEVFNfruVy+SNI1BGMqJ1l3fS0yQE2Xv5DCt9FINCaFjMoSuRUFi0EE+33yKz6wS4YFU9giD5+rviZzEWmdxaDtjYkZ62ZuJ/3nd1Ayug5yJJDUg6OKhQcqxkXgWA46YAmp4ZoFQxeyumI6IItTYsCo2BH/5y6vQuqj5lh8uq/WbIo4yOkGn6Bz56ArV0R1qoCaiaIKe0St6c3LnxXl3PhatJaeYOUZ/yvn8AVcjk3M=</latexit><latexit sha1_base64="PG92w/kLv9ojFjVdyYawM1/19Lk=">AAAB+XicbZDLSgMxFIYz9VbrbdSlm2ARXJUZEXRZdONGqGgv0A4lkzltQzPJkGQKw9A3ceNCEbe+iTvfxrSdhbYeCHz8/znJyR8mnGnjed9OaW19Y3OrvF3Z2d3bP3APj1papopCk0ouVSckGjgT0DTMcOgkCkgccmiH49uZ356A0kyKJ5MlEMRkKNiAUWKs1Hfdx0yYERhGe/heRsD7btWrefPCq+AXUEVFNfruVy+SNI1BGMqJ1l3fS0yQE2Xv5DCt9FINCaFjMoSuRUFi0EE+33yKz6wS4YFU9giD5+rviZzEWmdxaDtjYkZ62ZuJ/3nd1Ayug5yJJDUg6OKhQcqxkXgWA46YAmp4ZoFQxeyumI6IItTYsCo2BH/5y6vQuqj5lh8uq/WbIo4yOkGn6Bz56ArV0R1qoCaiaIKe0St6c3LnxXl3PhatJaeYOUZ/yvn8AVcjk3M=</latexit><latexit sha1_base64="PG92w/kLv9ojFjVdyYawM1/19Lk=">AAAB+XicbZDLSgMxFIYz9VbrbdSlm2ARXJUZEXRZdONGqGgv0A4lkzltQzPJkGQKw9A3ceNCEbe+iTvfxrSdhbYeCHz8/znJyR8mnGnjed9OaW19Y3OrvF3Z2d3bP3APj1papopCk0ouVSckGjgT0DTMcOgkCkgccmiH49uZ356A0kyKJ5MlEMRkKNiAUWKs1Hfdx0yYERhGe/heRsD7btWrefPCq+AXUEVFNfruVy+SNI1BGMqJ1l3fS0yQE2Xv5DCt9FINCaFjMoSuRUFi0EE+33yKz6wS4YFU9giD5+rviZzEWmdxaDtjYkZ62ZuJ/3nd1Ayug5yJJDUg6OKhQcqxkXgWA46YAmp4ZoFQxeyumI6IItTYsCo2BH/5y6vQuqj5lh8uq/WbIo4yOkGn6Bz56ArV0R1qoCaiaIKe0St6c3LnxXl3PhatJaeYOUZ/yvn8AVcjk3M=</latexit>

Real Model<latexit sha1_base64="bWpQFK8yHxaS6jTbxXjvV0Y5aI8=">AAAB8nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGJtIQ1ls5m0SzfZsLsRSunP8OJBEa/+Gm/+G7dtDtr6wsLDOzPszBtmgmvjut9OaWV1bX2jvFnZ2t7Z3avuHzxqmSuGLSaFVJ2QahQ8xZbhRmAnU0iTUGA7HF5P6+0nVJrL9MGMMgwS2k95zBk11vLvkYouuZURil615tbdmcgyeAXUoFCzV/3qRpLlCaaGCaq177mZCcZUGc4ETirdXGNG2ZD20beY0gR1MJ6tPCEn1olILJV9qSEz9/fEmCZaj5LQdibUDPRibWr+V/NzE18GY55mucGUzT+Kc0GMJNP7ScQVMiNGFihT3O5K2IAqyoxNqWJD8BZPXobHs7pn+e681rgq4ijDERzDKXhwAQ24gSa0gIGEZ3iFN8c4L8678zFvLTnFzCH8kfP5A659kNk=</latexit><latexit sha1_base64="bWpQFK8yHxaS6jTbxXjvV0Y5aI8=">AAAB8nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGJtIQ1ls5m0SzfZsLsRSunP8OJBEa/+Gm/+G7dtDtr6wsLDOzPszBtmgmvjut9OaWV1bX2jvFnZ2t7Z3avuHzxqmSuGLSaFVJ2QahQ8xZbhRmAnU0iTUGA7HF5P6+0nVJrL9MGMMgwS2k95zBk11vLvkYouuZURil615tbdmcgyeAXUoFCzV/3qRpLlCaaGCaq177mZCcZUGc4ETirdXGNG2ZD20beY0gR1MJ6tPCEn1olILJV9qSEz9/fEmCZaj5LQdibUDPRibWr+V/NzE18GY55mucGUzT+Kc0GMJNP7ScQVMiNGFihT3O5K2IAqyoxNqWJD8BZPXobHs7pn+e681rgq4ijDERzDKXhwAQ24gSa0gIGEZ3iFN8c4L8678zFvLTnFzCH8kfP5A659kNk=</latexit><latexit sha1_base64="bWpQFK8yHxaS6jTbxXjvV0Y5aI8=">AAAB8nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGJtIQ1ls5m0SzfZsLsRSunP8OJBEa/+Gm/+G7dtDtr6wsLDOzPszBtmgmvjut9OaWV1bX2jvFnZ2t7Z3avuHzxqmSuGLSaFVJ2QahQ8xZbhRmAnU0iTUGA7HF5P6+0nVJrL9MGMMgwS2k95zBk11vLvkYouuZURil615tbdmcgyeAXUoFCzV/3qRpLlCaaGCaq177mZCcZUGc4ETirdXGNG2ZD20beY0gR1MJ6tPCEn1olILJV9qSEz9/fEmCZaj5LQdibUDPRibWr+V/NzE18GY55mucGUzT+Kc0GMJNP7ScQVMiNGFihT3O5K2IAqyoxNqWJD8BZPXobHs7pn+e681rgq4ijDERzDKXhwAQ24gSa0gIGEZ3iFN8c4L8678zFvLTnFzCH8kfP5A659kNk=</latexit>

Figure 1. This figure illustrates the general framework about

transferring knowledge from synthetic model (top) to real model

(down). In testing, the two models receive images and predict ac-

tions (turn left, turn right or forward) to navigate in environments.

is the basis of these mobile robotic applications in the real-

world scenario.

Earlier works adopt imitation learning methods includ-

ing behavior cloning [3] and DAGGER [30] for 3D nav-

igation tasks. These approaches train a robot to emulate

an expert, such as the shortest path in indoor navigation.

These approaches fail into over-optimization, which sup-

press a larger set of close-optimal solution. Deep rein-

forcement learning approaches based on actor-critic, e.g.,

A3C [25] and UNREAL [20] are widely used in recent re-

searches [27, 44, 40, 34]. Advantage of these end-to-end ap-

proaches is that it discretizes the agent and state space [48]

and explores explicit map representations for planning [24].

There has been impressive progress on deep reinforce-

ment learning (RL) for many tasks, such as Atari video

games, GO [26, 35], robot control [10], self-driving [33]

and navigation [40]. However, it is difficult to apply rein-

forcement learning to the physical environment since sam-

pling a large number of episodes for training a robot is time-

consuming and even impossible. Thus, recent researches

focus on the RL model to learn policy in a simulated en-

vironment rather than in the physical world to resolve the

problem [40, 34]. There are two types of environments

that can be used to train a robot. The first one is rendered

synthetic environment. A represent work is SUNCG [36],

111388

which produces a 3D voxel representation from a single-

view depth map observation. The second one is recon-

structed environment such as Matterport3D [5], which con-

sists of real images captured with a Matterport camera.

One problem of only using the Matterport3D dataset for

training, however, is the lack of diversity of scenes. the

SUNCG dataset [36] consists of over 45,622 synthetic in-

door 3D scenes with customizable layout and texture, while

Matterport3D [5] contains only 90 houses. Lack of training

houses and scenes can significantly hamper performance

because of overfitting. Models trained on the SUNCG

dataset, on the contrary, is more capable of generalizing to

unseen scenes.

In this paper we propose a joint framework, namely Joint

Reinforcement Transfer (JRT), to resolve the problem of

lacking training data in the real environment. Our premise is

that synthetic training data is much easier and cheaper to ob-

tain than real data. As shown in Figure 1, our algorithm in-

tegrates adversarial feature adaption and policy mimic into

a joint framework. The joint learning of the adversarial fea-

ture adaption and policy mimic makes them mutually ben-

eficial and reciprocal. In this way, the feature adaption and

policy mimic are tightly correlated. The adversarial feature

adaptation not only generates better representation well fits

real environment, but is also more suitable for policy imita-

tion.

It is intuitive to transfer visual feature from synthetic do-

main to real domain, so that we can directly adopt knowl-

edge learned from synthetic environment without changing

the policy function. This kind of method, also called Do-

main Adaptation, is widely applied in numerous tasks [22,

32, 43, 23, 9, 38]. Inspired by the idea of Adversarial Dis-

criminative Domain Adaptation, we use the adversarial loss

to supervise our transfer training process. Compared to im-

age translation methods like CycleGan [45], which directly

translates the visual image with GAN [13, 15, 4, 47], our

approach drops the redundant steps that generates target im-

ages and extracting feature for target domain. Therefore,

our model is pimplier in framework and less parameter in

model so that be easy to converge.

In addition, the policy could be tuned under the supervi-

sion of teacher trained on synthetic environment. Our mo-

tivation is two-fold. First, the layout of real environment

is not exactly the same as the synthetic environment. As

shown in Figure 2, houses of real environment has more

rooms and much more connections between rooms, which

means more complicated structure. Second, some knowl-

edge learned in synthetic scene, such as finding doors and

bypassing obstacles is also helpful in real environment nav-

igation. Thus, we propose our Joint Reinforcement Trans-

fer(JRT) to integrate the two training stages together.

Our experimental results show that this joint method out-

performs baseline with finetune by 21.73% without any ad-

SUNCG houses

Matterport3D houses

Figure 2. This figure shows the different house plans of the

SUNCG dataset and Matterport3D dataset. Houses in SUNCG are

smaller and simpler in layout. Meanwhile, houses in Matterport3D

have more rooms and are more complicated in house plan.

ditional human annotations. Also, By qualitatively visual-

izing the results of the two adaptation methods, we shows

the two parts are mutually enhanced to benefit real-world

navigation.

2. Related Work

The proposed work is related to transferring reinforce

policy trained on virtual environment to real environment.

In this section, we briefly review several methods on 3D

Navigation, Indoor Environment, Domain Adaptation and

Policy Transfer with high relation to our topic.

3D Indoor Environment There has been a rising interest

in indoor reinforce environment. House3D [40] is a manu-

ally created large scale environment. AI2-THOR [21] is an

interactable synthetic indoor environment. Agent can inter-

active with some interactable objects such as open a drawer

or pick up a statue. CHALET [42] is another interactable

synthetic indoor environment with larger interactive action

space. Recent works tend to focus on simulated environ-

ment based on real imagery. However, the scale of these

datasets are quite small compared with virtual datasets. Ac-

tive Vision dataset [42] consists of dense scans of 16 differ-

ent houses. And Matterport3D [5] is a larger, multi-layer

environment. Minos is a cross domain environment which

consist both House3D and Matterport3D. MINOS provides

similar setting of both environments, which is convenient

for our environmental transferring experiment.

Domain Adaptation The problem of domain adaptation

has been widely studied and arises in different visual appli-

cation scenarios such as image classification [22, 32], object

detection [43] and action recognition [11]. Prior work such

as [19, 14] used Maximum Mean Discrepancy(MMD) [29]

loss to minimize the difference between the source and tar-

get feature distributions. MMD computes the difference of

data distributions in the two domains. Parameter adaptation,

11389

another kind of early method used in [43, 9, 17], adapt the

classifier like SVM trained on the source domain.

In contrast, Recent works introduce adversarial ap-

proach into domain adaptation. Generative Adversarial Net-

work(GAN) [13], lets generator G and discriminator D

compete against each other, is widely applied because of

its powerful ability for learning and generalizing from data

distribution. Gradient Reversal [12] directly optimizes the

mapping by reversing the gradient of discriminator. To go

one step further, Adversarial Discriminative Domain Adap-

tation [38] uses a target generative model optimized by

adversarial loss to learn source feature distribution. This

method can better model the difference in low level features.

There are several work [16, 41] focus on Synthetic-to-Real

visual image translation and a benchmark called VisDA [28]

has been proposed recently.

Policy Transfer Some works have explicitly studied ac-

tion policy transferring in reinforcement learning senario in

different way. Hinton et al. [18] proposes the method of

Network Distillation, using student model to learn the out-

put distribution of teacher model. Policy Distillation [31]

employs this method to transfer knowledge between two

environment based on DQN algorithm. Target-driven Vi-

sual Navigation [48] is a model for better generalize to new

goals rather than new environmental scene. Semantic Target

Driven Navigation [27] learns policy only based on detec-

tion and segmentation result, so that model can be easily

transferred to unseen environments.

3. Model

3.1. Baseline Setup

We demonstrate our approach of reinforce model trans-

fer based on room goal navigation task. Before our ap-

proach, we will first present our general model under the

framework of standard reinforcement setting.

According to partially observable Markov decision pro-

cess (POMDP), we can formulate our problem as (S, A, O,

P, R). The agent starts from a initial state s ∈ S, which

is the pose of the agent, a position direction pair. The ac-

tion a ∈ A is a discrete set of predefined actions. Ob-

servation space is the union of multi-modal sensory input

space such as RGB image, depth image and force O ={Orgb, Odepth, Oforce...}. Probabilistic state-action tran-

sition function is represented as p(si+1 | si, ai). Reward

R(s, a) is a function related to the Euclidean distance to the

goal and normalized time [34].

LSTM A3C LSTM A3C is a general model of asyn-

chronous advantage actor-critic algorithm. Compared to

Feedforward A3C whose policy only based on the current

observation, LSTM A3C introduce temporal experience to

achieve better performance.

LSTM A3C starts with a convolution network used as

visual embedding module

f = CNN(O)

f stands for visual feature of current observation. The vi-

sual feature sequence from the step 0 to current step t is

written as fseq = {f0, f1, ..., ft}.

Also we have a goal g to indicate which room agent is

required to go. g is encoded into a semantic feature vector

as gemb by a embedding layer:

gemb = Gemb(g)

Then visual feature of each step and goal embedding are

concatenated to fed to (Long-Short Term Memory) LSTM

units for temporal encoding

ht = LSTM([ft, gemb], ht−1)

where ft is the visual feature of current step and ht is the

LSTM output vector, encoding the historical information

from step 0 to t. Note that we simplify the propagation of

memory cells for convenience.

Follow the architecture of actor-critic, we have a policy

module a = π(a|ht) to predict the distribution of ongoing

actions and value v = V (ht) to predict the value of this

state. Loss functions of A3C baseline is defined as

LA3C = −logπ(a|ht; θ)(Rt − V (ht; θ))−H(π(a|ht; θ))

where term H(π(st; θ)) represents the entropy of policy.

We maximize this term to encourage exploration.

UNREAL UNREAL is an advanced version of A3C with

several unsupervised auxiliary tasks providing wider train-

ing signals without additional training data. This improve-

ment is general so that it can augment many reinforce tasks

including navigation in this paper. We will prove that our

approach can also be applied to UNREAL model in experi-

ment and achieve the state-of-the-art on our setting.

Re-Formulation In the following sections, we will pro-

pose two method, Feature Adaptation and Policy Transfer.

For simplify notation, we re-define the model as two parts,

which consist of a mapping function M and a policy func-

tion P .

Mapping function M is fed with observation(only RGB

images in our experiment) x ∼ X and output visual embed-

ding f ∼ F . We define X as RGB image distribution and

F as feature distribution. Xs, Xr ⊆ X represent image dis-

tribution of synthetic domain and real domain respectively.

Similarly, we use Fs, Fr ⊆ F to represent synthetic visual

feature and real visual feature.

Policy function P takes f as input and predict discrete

action with softmax activation. Our representation of re-

inforce model is written as

π(a|s; θ) = softmax(P (M(o; θM ); θP ))

We use Lpolicy to represent the united form of actor-

critic loss for both A3C and UNREAL.

11390

Sufficient

Synthetic Data

Environments

Ms<latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="bNAs+I2C0tbb1QKxkboFH4kMSXc=">AAAB4XicbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScONGqOC0hXYomfS2Dc1khuSOUIY+gxsXivhS7nwb05+Fth4IfJyTkHtPnClpyfe/vdLW9s7uXnm/clA9PDqunVRbNs2NwFCkKjWdmFtUUmNIkhR2MoM8iRW248ndPG8/o7Ey1U80zTBK+EjLoRScnBU+9As769fqfsNfiG1CsII6rNTs1756g1TkCWoSilvbDfyMooIbkkLhrNLLLWZcTPgIuw41T9BGxWLYGbtwzoANU+OOJrZwf78oeGLtNIndzYTT2K5nc/O/rJvT8CYqpM5yQi2WHw1zxShl883ZQBoUpKYOuDDSzcrEmBsuyPVTcSUE6ytvQuuqETh+9KEMZ3AOlxDANdzCPTQhBAESXuAN3j3tvXofy7pK3qq3U/gj7/MHyz+NbA==</latexit><latexit sha1_base64="/Wwuq/7N+zHMVRFZUI1Jjq3tMaE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSL/ZY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaWNza3unvFvZ2z84PKoen7RNkmnGfZbIRHdDargUivsoUPJuqjmNQ8k74eR2Xu88cW1Eoh5xmvIgpiMlIsEoWsu/H+RmNqjW3Lq7EFkHr4AaFGoNql/9YcKymCtkkhrT89wUg5xqFEzyWaWfGZ5SNqEj3rOoaMxNkC+WnZEL6wxJlGj7FJKF+3sip7Ex0zi0nTHFsVmtzc3/ar0Mo0aQC5VmyBVbfhRlkmBC5peTodCcoZxaoEwLuythY6opQ5tPxYbgrZ68Du2rumf5wa01b4o4ynAG53AJHlxDE+6gBT4wEPAMr/DmKOfFeXc+lq0lp5g5hT9yPn8A8XqOvw==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit><latexit sha1_base64="iG6KdByPiDBnI3MrGm5FbXLHWMY=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZjtpl242YXcjlNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u0nVJon8tFMUgxiOpQ84owaa/n3/VxP+9WaW3fnIqvgFVCDQs1+9as3SFgWozRMUK27npuaIKfKcCZwWullGlPKxnSIXYuSxqiDfL7slJxZZ0CiRNknDZm7vydyGms9iUPbGVMz0su1mflfrZuZ6DrIuUwzg5ItPooyQUxCZpeTAVfIjJhYoExxuythI6ooMzafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBAYdneIU3RzovzrvzsWgtOcXMMfyR8/kD8rqOww==</latexit>

Insufficient

Real DataMr

<latexit sha1_base64="FzcIoKmF3CNv7HrUAi2ltwq2dTA=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cW1Eoh5xkvIgpkMlIsEoWsu/7+d62q/W3Lo7F1kFr4AaFGr2q1+9QcKymCtkkhrT9dwUg5xqFEzyaaWXGZ5SNqZD3rWoaMxNkM+XnZIz6wxIlGj7FJK5+3sip7Exkzi0nTHFkVmuzcz/at0Mo+sgFyrNkCu2+CjKJMGEzC4nA6E5QzmxQJkWdlfCRlRThjafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBgYBneIU3RzkvzrvzsWgtOcXMMfyR8/kD8TWOwg==</latexit><latexit sha1_base64="FzcIoKmF3CNv7HrUAi2ltwq2dTA=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cW1Eoh5xkvIgpkMlIsEoWsu/7+d62q/W3Lo7F1kFr4AaFGr2q1+9QcKymCtkkhrT9dwUg5xqFEzyaaWXGZ5SNqZD3rWoaMxNkM+XnZIz6wxIlGj7FJK5+3sip7Exkzi0nTHFkVmuzcz/at0Mo+sgFyrNkCu2+CjKJMGEzC4nA6E5QzmxQJkWdlfCRlRThjafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBgYBneIU3RzkvzrvzsWgtOcXMMfyR8/kD8TWOwg==</latexit><latexit sha1_base64="FzcIoKmF3CNv7HrUAi2ltwq2dTA=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJFqGDaQhvKZrtpl242YXcilNDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cW1Eoh5xkvIgpkMlIsEoWsu/7+d62q/W3Lo7F1kFr4AaFGr2q1+9QcKymCtkkhrT9dwUg5xqFEzyaaWXGZ5SNqZD3rWoaMxNkM+XnZIz6wxIlGj7FJK5+3sip7Exkzi0nTHFkVmuzcz/at0Mo+sgFyrNkCu2+CjKJMGEzC4nA6E5QzmxQJkWdlfCRlRThjafig3BWz55FVoXdc/yw2WtcVPEUYYTOIVz8OAKGnAHTfCBgYBneIU3RzkvzrvzsWgtOcXMMfyR8/kD8TWOwg==</latexit>

Feature Adaption Policy Transfer

D<latexit sha1_base64="jzHNDnkZCeZMuLp27f4bdcTa35g=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY1IPHFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxrezevsJleaxfDCTBP2IDiUPOaPGWo27frniVt25yCp4OVQgV71f/uoNYpZGKA0TVOuu5ybGz6gynAmclnqpxoSyMR1i16KkEWo/my86JWfWGZAwVvZJQ+bu74mMRlpPosB2RtSM9HJtZv5X66YmvPYzLpPUoGSLj8JUEBOT2dVkwBUyIyYWKFPc7krYiCrKjM2mZEPwlk9ehdZF1bPcuKzUbvI4inACp3AOHlxBDe6hDk1ggPAMr/DmPDovzrvzsWgtOPnMMfyR8/kDl2uMyA==</latexit><latexit sha1_base64="jzHNDnkZCeZMuLp27f4bdcTa35g=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY1IPHFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxrezevsJleaxfDCTBP2IDiUPOaPGWo27frniVt25yCp4OVQgV71f/uoNYpZGKA0TVOuu5ybGz6gynAmclnqpxoSyMR1i16KkEWo/my86JWfWGZAwVvZJQ+bu74mMRlpPosB2RtSM9HJtZv5X66YmvPYzLpPUoGSLj8JUEBOT2dVkwBUyIyYWKFPc7krYiCrKjM2mZEPwlk9ehdZF1bPcuKzUbvI4inACp3AOHlxBDe6hDk1ggPAMr/DmPDovzrvzsWgtOPnMMfyR8/kDl2uMyA==</latexit><latexit sha1_base64="jzHNDnkZCeZMuLp27f4bdcTa35g=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY1IPHFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxrezevsJleaxfDCTBP2IDiUPOaPGWo27frniVt25yCp4OVQgV71f/uoNYpZGKA0TVOuu5ybGz6gynAmclnqpxoSyMR1i16KkEWo/my86JWfWGZAwVvZJQ+bu74mMRlpPosB2RtSM9HJtZv5X66YmvPYzLpPUoGSLj8JUEBOT2dVkwBUyIyYWKFPc7krYiCrKjM2mZEPwlk9ehdZF1bPcuKzUbvI4inACp3AOHlxBDe6hDk1ggPAMr/DmPDovzrvzsWgtOPnMMfyR8/kDl2uMyA==</latexit>

fr<latexit sha1_base64="wTyPFmciYqun4MjfUawa0kIhNPA=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn40yPVsUK25dXchsg5eATUo1BpUv/rDhGUxV8gkNabnuSkGOdUomOSzSj8zPKVsQke8Z1HRmJsgXyw7IxfWGZIo0fYpJAv390ROY2OmcWg7Y4pjs1qbm//VehlGN0EuVJohV2z5UZRJggmZX06GQnOGcmqBMi3sroSNqaYMbT4VG4K3evI6tK/qnuWH61rztoijDGdwDpfgQQOacA8t8IGBgGd4hTdHOS/Ou/OxbC05xcwp/JHz+QMXjI7b</latexit><latexit sha1_base64="wTyPFmciYqun4MjfUawa0kIhNPA=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn40yPVsUK25dXchsg5eATUo1BpUv/rDhGUxV8gkNabnuSkGOdUomOSzSj8zPKVsQke8Z1HRmJsgXyw7IxfWGZIo0fYpJAv390ROY2OmcWg7Y4pjs1qbm//VehlGN0EuVJohV2z5UZRJggmZX06GQnOGcmqBMi3sroSNqaYMbT4VG4K3evI6tK/qnuWH61rztoijDGdwDpfgQQOacA8t8IGBgGd4hTdHOS/Ou/OxbC05xcwp/JHz+QMXjI7b</latexit><latexit sha1_base64="wTyPFmciYqun4MjfUawa0kIhNPA=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn40yPVsUK25dXchsg5eATUo1BpUv/rDhGUxV8gkNabnuSkGOdUomOSzSj8zPKVsQke8Z1HRmJsgXyw7IxfWGZIo0fYpJAv390ROY2OmcWg7Y4pjs1qbm//VehlGN0EuVJohV2z5UZRJggmZX06GQnOGcmqBMi3sroSNqaYMbT4VG4K3evI6tK/qnuWH61rztoijDGdwDpfgQQOacA8t8IGBgGd4hTdHOS/Ou/OxbC05xcwp/JHz+QMXjI7b</latexit>

fs<latexit sha1_base64="nik/Uu3gbbScV0bKvWUFKLb0xgU=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn40yM1sUK25dXchsg5eATUo1BpUv/rDhGUxV8gkNabnuSkGOdUomOSzSj8zPKVsQke8Z1HRmJsgXyw7IxfWGZIo0fYpJAv390ROY2OmcWg7Y4pjs1qbm//VehlGN0EuVJohV2z5UZRJggmZX06GQnOGcmqBMi3sroSNqaYMbT4VG4K3evI6tK/qnuWH61rztoijDGdwDpfgQQOacA8t8IGBgGd4hTdHOS/Ou/OxbC05xcwp/JHz+QMZEY7c</latexit><latexit sha1_base64="nik/Uu3gbbScV0bKvWUFKLb0xgU=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn40yM1sUK25dXchsg5eATUo1BpUv/rDhGUxV8gkNabnuSkGOdUomOSzSj8zPKVsQke8Z1HRmJsgXyw7IxfWGZIo0fYpJAv390ROY2OmcWg7Y4pjs1qbm//VehlGN0EuVJohV2z5UZRJggmZX06GQnOGcmqBMi3sroSNqaYMbT4VG4K3evI6tK/qnuWH61rztoijDGdwDpfgQQOacA8t8IGBgGd4hTdHOS/Ou/OxbC05xcwp/JHz+QMZEY7c</latexit><latexit sha1_base64="nik/Uu3gbbScV0bKvWUFKLb0xgU=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn40yM1sUK25dXchsg5eATUo1BpUv/rDhGUxV8gkNabnuSkGOdUomOSzSj8zPKVsQke8Z1HRmJsgXyw7IxfWGZIo0fYpJAv390ROY2OmcWg7Y4pjs1qbm//VehlGN0EuVJohV2z5UZRJggmZX06GQnOGcmqBMi3sroSNqaYMbT4VG4K3evI6tK/qnuWH61rztoijDGdwDpfgQQOacA8t8IGBgGd4hTdHOS/Ou/OxbC05xcwp/JHz+QMZEY7c</latexit>

Ladv<latexit sha1_base64="VkdzUba83z+QxeVfhekCZi2Q4RQ=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OLBQwX7AW0ok82mXbrZhN1NoYT+CC8eFPHq7/Hmv3Hb5qDVFxYe3plhZ94gFVwb1/1ySmvrG5tb5e3Kzu7e/kH18Kitk0xR1qKJSFQ3QM0El6xluBGsmyqGcSBYJxjfzuudCVOaJ/LRTFPmxziUPOIUjbU694Mcw8lsUK25dXch8he8AmpQqDmofvbDhGYxk4YK1Lrnuanxc1SGU8FmlX6mWYp0jEPWsygxZtrPF+vOyJl1QhIlyj5pyML9OZFjrPU0DmxnjGakV2tz879aLzPRtZ9zmWaGSbr8KMoEMQmZ305Crhg1YmoBqeJ2V0JHqJAam1DFhuCtnvwX2hd1z/LDZa1xU8RRhhM4hXPw4AoacAdNaAGFMTzBC7w6qfPsvDnvy9aSU8wcwy85H99uhI+e</latexit><latexit sha1_base64="VkdzUba83z+QxeVfhekCZi2Q4RQ=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OLBQwX7AW0ok82mXbrZhN1NoYT+CC8eFPHq7/Hmv3Hb5qDVFxYe3plhZ94gFVwb1/1ySmvrG5tb5e3Kzu7e/kH18Kitk0xR1qKJSFQ3QM0El6xluBGsmyqGcSBYJxjfzuudCVOaJ/LRTFPmxziUPOIUjbU694Mcw8lsUK25dXch8he8AmpQqDmofvbDhGYxk4YK1Lrnuanxc1SGU8FmlX6mWYp0jEPWsygxZtrPF+vOyJl1QhIlyj5pyML9OZFjrPU0DmxnjGakV2tz879aLzPRtZ9zmWaGSbr8KMoEMQmZ305Crhg1YmoBqeJ2V0JHqJAam1DFhuCtnvwX2hd1z/LDZa1xU8RRhhM4hXPw4AoacAdNaAGFMTzBC7w6qfPsvDnvy9aSU8wcwy85H99uhI+e</latexit><latexit sha1_base64="VkdzUba83z+QxeVfhekCZi2Q4RQ=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OLBQwX7AW0ok82mXbrZhN1NoYT+CC8eFPHq7/Hmv3Hb5qDVFxYe3plhZ94gFVwb1/1ySmvrG5tb5e3Kzu7e/kH18Kitk0xR1qKJSFQ3QM0El6xluBGsmyqGcSBYJxjfzuudCVOaJ/LRTFPmxziUPOIUjbU694Mcw8lsUK25dXch8he8AmpQqDmofvbDhGYxk4YK1Lrnuanxc1SGU8FmlX6mWYp0jEPWsygxZtrPF+vOyJl1QhIlyj5pyML9OZFjrPU0DmxnjGakV2tz879aLzPRtZ9zmWaGSbr8KMoEMQmZ305Crhg1YmoBqeJ2V0JHqJAam1DFhuCtnvwX2hd1z/LDZa1xU8RRhhM4hXPw4AoacAdNaAGFMTzBC7w6qfPsvDnvy9aSU8wcwy85H99uhI+e</latexit>

Lmimic<latexit sha1_base64="R2DV6liEtBy4V5OluDoxEB5t9mo=">AAAB8HicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YsHDxVsrbRLyaZpG5pkl2RWKEt/hRcPinj153jz35i2e9DWFwIP78yQmTdKpLDo+99eYWV1bX2juFna2t7Z3SvvHzRtnBrGGyyWsWlF1HIpNG+gQMlbieFURZI/RKPraf3hiRsrYn2P44SHig606AtG0VmPt91MCSXYpFuu+FV/JrIMQQ4VyFXvlr86vZilimtkklrbDvwEw4waFEzySamTWp5QNqID3naoqeI2zGYLT8iJc3qkHxv3NJKZ+3sio8rasYpcp6I4tIu1qflfrZ1i/zLMhE5S5JrNP+qnkmBMpteTnjCcoRw7oMwItythQ2ooQ5dRyYUQLJ68DM2zauD47rxSu8rjKMIRHMMpBHABNbiBOjSAgYJneIU3z3gv3rv3MW8tePnMIfyR9/kDAACQhg==</latexit><latexit sha1_base64="R2DV6liEtBy4V5OluDoxEB5t9mo=">AAAB8HicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YsHDxVsrbRLyaZpG5pkl2RWKEt/hRcPinj153jz35i2e9DWFwIP78yQmTdKpLDo+99eYWV1bX2juFna2t7Z3SvvHzRtnBrGGyyWsWlF1HIpNG+gQMlbieFURZI/RKPraf3hiRsrYn2P44SHig606AtG0VmPt91MCSXYpFuu+FV/JrIMQQ4VyFXvlr86vZilimtkklrbDvwEw4waFEzySamTWp5QNqID3naoqeI2zGYLT8iJc3qkHxv3NJKZ+3sio8rasYpcp6I4tIu1qflfrZ1i/zLMhE5S5JrNP+qnkmBMpteTnjCcoRw7oMwItythQ2ooQ5dRyYUQLJ68DM2zauD47rxSu8rjKMIRHMMpBHABNbiBOjSAgYJneIU3z3gv3rv3MW8tePnMIfyR9/kDAACQhg==</latexit><latexit sha1_base64="R2DV6liEtBy4V5OluDoxEB5t9mo=">AAAB8HicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YsHDxVsrbRLyaZpG5pkl2RWKEt/hRcPinj153jz35i2e9DWFwIP78yQmTdKpLDo+99eYWV1bX2juFna2t7Z3SvvHzRtnBrGGyyWsWlF1HIpNG+gQMlbieFURZI/RKPraf3hiRsrYn2P44SHig606AtG0VmPt91MCSXYpFuu+FV/JrIMQQ4VyFXvlr86vZilimtkklrbDvwEw4waFEzySamTWp5QNqID3naoqeI2zGYLT8iJc3qkHxv3NJKZ+3sio8rasYpcp6I4tIu1qflfrZ1i/zLMhE5S5JrNP+qnkmBMpteTnjCcoRw7oMwItythQ2ooQ5dRyYUQLJ68DM2zauD47rxSu8rjKMIRHMMpBHABNbiBOjSAgYJneIU3z3gv3rv3MW8tePnMIfyR9/kDAACQhg==</latexit>

Ps<latexit sha1_base64="dJY6EYPHAoP8ebyYJ/8dJ5qTKEs=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA220m7dLMJuxuhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEquDau++2UNja3tnfKu5W9/YPDo+rxSVsnmWLos0QkqhtSjYJL9A03ArupQhqHAjvh5G5e7zyh0jyRj2aaYhDTkeQRZ9RYy28Ncj0bVGtu3V2IrINXQA0KtQbVr/4wYVmM0jBBte55bmqCnCrDmcBZpZ9pTCmb0BH2LEoaow7yxbIzcmGdIYkSZZ80ZOH+nshprPU0Dm1nTM1Yr9bm5n+1XmaimyDnMs0MSrb8KMoEMQmZX06GXCEzYmqBMsXtroSNqaLM2HwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDDs/wCm+OdF6cd+dj2VpyiplT+CPn8wf3Uo7G</latexit><latexit sha1_base64="dJY6EYPHAoP8ebyYJ/8dJ5qTKEs=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA220m7dLMJuxuhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEquDau++2UNja3tnfKu5W9/YPDo+rxSVsnmWLos0QkqhtSjYJL9A03ArupQhqHAjvh5G5e7zyh0jyRj2aaYhDTkeQRZ9RYy28Ncj0bVGtu3V2IrINXQA0KtQbVr/4wYVmM0jBBte55bmqCnCrDmcBZpZ9pTCmb0BH2LEoaow7yxbIzcmGdIYkSZZ80ZOH+nshprPU0Dm1nTM1Yr9bm5n+1XmaimyDnMs0MSrb8KMoEMQmZX06GXCEzYmqBMsXtroSNqaLM2HwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDDs/wCm+OdF6cd+dj2VpyiplT+CPn8wf3Uo7G</latexit><latexit sha1_base64="dJY6EYPHAoP8ebyYJ/8dJ5qTKEs=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA220m7dLMJuxuhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEquDau++2UNja3tnfKu5W9/YPDo+rxSVsnmWLos0QkqhtSjYJL9A03ArupQhqHAjvh5G5e7zyh0jyRj2aaYhDTkeQRZ9RYy28Ncj0bVGtu3V2IrINXQA0KtQbVr/4wYVmM0jBBte55bmqCnCrDmcBZpZ9pTCmb0BH2LEoaow7yxbIzcmGdIYkSZZ80ZOH+nshprPU0Dm1nTM1Yr9bm5n+1XmaimyDnMs0MSrb8KMoEMQmZX06GXCEzYmqBMsXtroSNqaLM2HwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDDs/wCm+OdF6cd+dj2VpyiplT+CPn8wf3Uo7G</latexit>

Pr<latexit sha1_base64="LNUr66W3+KVoZBgjIZHdsYapHMQ=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA220m7dLMJuxuhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEquDau++2UNja3tnfKu5W9/YPDo+rxSVsnmWLos0QkqhtSjYJL9A03ArupQhqHAjvh5G5e7zyh0jyRj2aaYhDTkeQRZ9RYy28NcjUbVGtu3V2IrINXQA0KtQbVr/4wYVmM0jBBte55bmqCnCrDmcBZpZ9pTCmb0BH2LEoaow7yxbIzcmGdIYkSZZ80ZOH+nshprPU0Dm1nTM1Yr9bm5n+1XmaimyDnMs0MSrb8KMoEMQmZX06GXCEzYmqBMsXtroSNqaLM2HwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDDs/wCm+OdF6cd+dj2VpyiplT+CPn8wf1zY7F</latexit><latexit sha1_base64="LNUr66W3+KVoZBgjIZHdsYapHMQ=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA220m7dLMJuxuhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEquDau++2UNja3tnfKu5W9/YPDo+rxSVsnmWLos0QkqhtSjYJL9A03ArupQhqHAjvh5G5e7zyh0jyRj2aaYhDTkeQRZ9RYy28NcjUbVGtu3V2IrINXQA0KtQbVr/4wYVmM0jBBte55bmqCnCrDmcBZpZ9pTCmb0BH2LEoaow7yxbIzcmGdIYkSZZ80ZOH+nshprPU0Dm1nTM1Yr9bm5n+1XmaimyDnMs0MSrb8KMoEMQmZX06GXCEzYmqBMsXtroSNqaLM2HwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDDs/wCm+OdF6cd+dj2VpyiplT+CPn8wf1zY7F</latexit><latexit sha1_base64="LNUr66W3+KVoZBgjIZHdsYapHMQ=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA220m7dLMJuxuhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEquDau++2UNja3tnfKu5W9/YPDo+rxSVsnmWLos0QkqhtSjYJL9A03ArupQhqHAjvh5G5e7zyh0jyRj2aaYhDTkeQRZ9RYy28NcjUbVGtu3V2IrINXQA0KtQbVr/4wYVmM0jBBte55bmqCnCrDmcBZpZ9pTCmb0BH2LEoaow7yxbIzcmGdIYkSZZ80ZOH+nshprPU0Dm1nTM1Yr9bm5n+1XmaimyDnMs0MSrb8KMoEMQmZX06GXCEzYmqBMsXtroSNqaLM2HwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDDs/wCm+OdF6cd+dj2VpyiplT+CPn8wf1zY7F</latexit>

Xs<latexit sha1_base64="WoJ3gE95vHwoN7vp9RbHALE8ZHE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn53kJvZoFpz6+5CZB28AmpQqDWofvWHCctirpBJakzPc1MMcqpRMMlnlX5meErZhI54z6KiMTdBvlh2Ri6sMyRRou1TSBbu74mcxsZM49B2xhTHZrU2N/+r9TKMboJcqDRDrtjyoyiTBBMyv5wMheYM5dQCZVrYXQkbU00Z2nwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+dj2VpyiplT+CPn8wcDoY7O</latexit><latexit sha1_base64="WoJ3gE95vHwoN7vp9RbHALE8ZHE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn53kJvZoFpz6+5CZB28AmpQqDWofvWHCctirpBJakzPc1MMcqpRMMlnlX5meErZhI54z6KiMTdBvlh2Ri6sMyRRou1TSBbu74mcxsZM49B2xhTHZrU2N/+r9TKMboJcqDRDrtjyoyiTBBMyv5wMheYM5dQCZVrYXQkbU00Z2nwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+dj2VpyiplT+CPn8wcDoY7O</latexit><latexit sha1_base64="WoJ3gE95vHwoN7vp9RbHALE8ZHE=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn53kJvZoFpz6+5CZB28AmpQqDWofvWHCctirpBJakzPc1MMcqpRMMlnlX5meErZhI54z6KiMTdBvlh2Ri6sMyRRou1TSBbu74mcxsZM49B2xhTHZrU2N/+r9TKMboJcqDRDrtjyoyiTBBMyv5wMheYM5dQCZVrYXQkbU00Z2nwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+dj2VpyiplT+CPn8wcDoY7O</latexit>

Xr<latexit sha1_base64="N3RKwg3098hhU44tgCDjUuSYZVI=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn53kOvZoFpz6+5CZB28AmpQqDWofvWHCctirpBJakzPc1MMcqpRMMlnlX5meErZhI54z6KiMTdBvlh2Ri6sMyRRou1TSBbu74mcxsZM49B2xhTHZrU2N/+r9TKMboJcqDRDrtjyoyiTBBMyv5wMheYM5dQCZVrYXQkbU00Z2nwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+dj2VpyiplT+CPn8wcCHI7N</latexit><latexit sha1_base64="N3RKwg3098hhU44tgCDjUuSYZVI=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn53kOvZoFpz6+5CZB28AmpQqDWofvWHCctirpBJakzPc1MMcqpRMMlnlX5meErZhI54z6KiMTdBvlh2Ri6sMyRRou1TSBbu74mcxsZM49B2xhTHZrU2N/+r9TKMboJcqDRDrtjyoyiTBBMyv5wMheYM5dQCZVrYXQkbU00Z2nwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+dj2VpyiplT+CPn8wcCHI7N</latexit><latexit sha1_base64="N3RKwg3098hhU44tgCDjUuSYZVI=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjBdMW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cdvmoK0vLDy8M8POvGEqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3fzeueJayMS9YjTlAcxHSkRCUbRWn53kOvZoFpz6+5CZB28AmpQqDWofvWHCctirpBJakzPc1MMcqpRMMlnlX5meErZhI54z6KiMTdBvlh2Ri6sMyRRou1TSBbu74mcxsZM49B2xhTHZrU2N/+r9TKMboJcqDRDrtjyoyiTBBMyv5wMheYM5dQCZVrYXQkbU00Z2nwqNgRv9eR1aF/VPcsP17XmbRFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+dj2VpyiplT+CPn8wcCHI7N</latexit>



Lidt<latexit sha1_base64="lgUu9zJV/IDnjVx7uuOx8kcQj/o=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OLBQwX7AW0om82mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94glcKg6347pbX1jc2t8nZlZ3dv/6B6eNQ2SaYZb7FEJrobUMOlULyFAiXvpprTOJC8E4xvZ/XOE9dGJOoRJyn3YzpUIhKMorU694NchDgdVGtu3Z2LrIJXQA0KNQfVr36YsCzmCpmkxvQ8N0U/pxoFk3xa6WeGp5SN6ZD3LCoac+Pn83Wn5Mw6IYkSbZ9CMnd/T+Q0NmYSB7Yzpjgyy7WZ+V+tl2F07edCpRlyxRYfRZkkmJDZ7SQUmjOUEwuUaWF3JWxENWVoE6rYELzlk1ehfVH3LD9c1ho3RRxlOIFTOAcPrqABd9CEFjAYwzO8wpuTOi/Ou/OxaC05xcwx/JHz+QN3so+k</latexit><latexit sha1_base64="lgUu9zJV/IDnjVx7uuOx8kcQj/o=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OLBQwX7AW0om82mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94glcKg6347pbX1jc2t8nZlZ3dv/6B6eNQ2SaYZb7FEJrobUMOlULyFAiXvpprTOJC8E4xvZ/XOE9dGJOoRJyn3YzpUIhKMorU694NchDgdVGtu3Z2LrIJXQA0KNQfVr36YsCzmCpmkxvQ8N0U/pxoFk3xa6WeGp5SN6ZD3LCoac+Pn83Wn5Mw6IYkSbZ9CMnd/T+Q0NmYSB7Yzpjgyy7WZ+V+tl2F07edCpRlyxRYfRZkkmJDZ7SQUmjOUEwuUaWF3JWxENWVoE6rYELzlk1ehfVH3LD9c1ho3RRxlOIFTOAcPrqABd9CEFjAYwzO8wpuTOi/Ou/OxaC05xcwx/JHz+QN3so+k</latexit><latexit sha1_base64="lgUu9zJV/IDnjVx7uuOx8kcQj/o=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OLBQwX7AW0om82mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94glcKg6347pbX1jc2t8nZlZ3dv/6B6eNQ2SaYZb7FEJrobUMOlULyFAiXvpprTOJC8E4xvZ/XOE9dGJOoRJyn3YzpUIhKMorU694NchDgdVGtu3Z2LrIJXQA0KNQfVr36YsCzmCpmkxvQ8N0U/pxoFk3xa6WeGp5SN6ZD3LCoac+Pn83Wn5Mw6IYkSbZ9CMnd/T+Q0NmYSB7Yzpjgyy7WZ+V+tl2F07edCpRlyxRYfRZkkmJDZ7SQUmjOUEwuUaWF3JWxENWVoE6rYELzlk1ehfVH3LD9c1ho3RRxlOIFTOAcPrqABd9CEFjAYwzO8wpuTOi/Ou/OxaC05xcwx/JHz+QN3so+k</latexit>

“Navigate to kitchen”

Gemb<latexit sha1_base64="KsqH4Uw8ILlH+s/LFSEGIaiSWNI=">AAAB7nicbVDLSgMxFL1TX7W+qi7dBEvBVZkRoS6LLnRZwT6gHUomvdOGZjJDkhHK0I9w40IRt36PO//GtJ2Fth4IHM7J5Z57gkRwbVz32ylsbG5t7xR3S3v7B4dH5eOTto5TxbDFYhGrbkA1Ci6xZbgR2E0U0igQ2Akmt3O/84RK81g+mmmCfkRHkoecUWOlzt0gwyiYDcoVt+YuQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxZxZ6RqlSEJY2WfNGSh/p7IaKT11MYi1YiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5reTIVfIjJhaQpniNithY6ooM7ahki3BWz15nbQva57lD1eVxk1eRxHO4BwuwIM6NOAemtACBhN4hld4cxLnxXl3PpZfC04+cwp/4Hz+AFxAj5I=</latexit><latexit sha1_base64="KsqH4Uw8ILlH+s/LFSEGIaiSWNI=">AAAB7nicbVDLSgMxFL1TX7W+qi7dBEvBVZkRoS6LLnRZwT6gHUomvdOGZjJDkhHK0I9w40IRt36PO//GtJ2Fth4IHM7J5Z57gkRwbVz32ylsbG5t7xR3S3v7B4dH5eOTto5TxbDFYhGrbkA1Ci6xZbgR2E0U0igQ2Akmt3O/84RK81g+mmmCfkRHkoecUWOlzt0gwyiYDcoVt+YuQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxZxZ6RqlSEJY2WfNGSh/p7IaKT11MYi1YiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5reTIVfIjJhaQpniNithY6ooM7ahki3BWz15nbQva57lD1eVxk1eRxHO4BwuwIM6NOAemtACBhN4hld4cxLnxXl3PpZfC04+cwp/4Hz+AFxAj5I=</latexit><latexit sha1_base64="KsqH4Uw8ILlH+s/LFSEGIaiSWNI=">AAAB7nicbVDLSgMxFL1TX7W+qi7dBEvBVZkRoS6LLnRZwT6gHUomvdOGZjJDkhHK0I9w40IRt36PO//GtJ2Fth4IHM7J5Z57gkRwbVz32ylsbG5t7xR3S3v7B4dH5eOTto5TxbDFYhGrbkA1Ci6xZbgR2E0U0igQ2Akmt3O/84RK81g+mmmCfkRHkoecUWOlzt0gwyiYDcoVt+YuQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxZxZ6RqlSEJY2WfNGSh/p7IaKT11MYi1YiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5reTIVfIjJhaQpniNithY6ooM7ahki3BWz15nbQva57lD1eVxk1eRxHO4BwuwIM6NOAemtACBhN4hld4cxLnxXl3PpZfC04+cwp/4Hz+AFxAj5I=</latexit>

Lpolicy<latexit sha1_base64="KIzjSffF4wyxoGE1fzpj1XIlW5E=">AAAB8XicbZC7SgNBFIbPxluMt6ilzWAQrMKuCFoGbSwsIpgYTJYwO5lNhsxlmZkVliVvYWOhiK1vY+fbOEm20MQfBj7+cw5zzh8lnBnr+99eaWV1bX2jvFnZ2t7Z3avuH7SNSjWhLaK40p0IG8qZpC3LLKedRFMsIk4fovH1tP7wRLVhSt7bLKGhwEPJYkawddbjbT9PFGckm/SrNb/uz4SWISigBoWa/epXb6BIKqi0hGNjuoGf2DDH2jLC6aTSSw1NMBnjIe06lFhQE+azjSfoxDkDFCvtnrRo5v6eyLEwJhOR6xTYjsxibWr+V+umNr4McyaT1FJJ5h/FKUdWoen5aMA0JZZnDjDRzO2KyAhrTKwLqeJCCBZPXob2WT1wfHdea1wVcZThCI7hFAK4gAbcQBNaQEDCM7zCm2e8F+/d+5i3lrxi5hD+yPv8AexSkRE=</latexit><latexit sha1_base64="KIzjSffF4wyxoGE1fzpj1XIlW5E=">AAAB8XicbZC7SgNBFIbPxluMt6ilzWAQrMKuCFoGbSwsIpgYTJYwO5lNhsxlmZkVliVvYWOhiK1vY+fbOEm20MQfBj7+cw5zzh8lnBnr+99eaWV1bX2jvFnZ2t7Z3avuH7SNSjWhLaK40p0IG8qZpC3LLKedRFMsIk4fovH1tP7wRLVhSt7bLKGhwEPJYkawddbjbT9PFGckm/SrNb/uz4SWISigBoWa/epXb6BIKqi0hGNjuoGf2DDH2jLC6aTSSw1NMBnjIe06lFhQE+azjSfoxDkDFCvtnrRo5v6eyLEwJhOR6xTYjsxibWr+V+umNr4McyaT1FJJ5h/FKUdWoen5aMA0JZZnDjDRzO2KyAhrTKwLqeJCCBZPXob2WT1wfHdea1wVcZThCI7hFAK4gAbcQBNaQEDCM7zCm2e8F+/d+5i3lrxi5hD+yPv8AexSkRE=</latexit><latexit sha1_base64="KIzjSffF4wyxoGE1fzpj1XIlW5E=">AAAB8XicbZC7SgNBFIbPxluMt6ilzWAQrMKuCFoGbSwsIpgYTJYwO5lNhsxlmZkVliVvYWOhiK1vY+fbOEm20MQfBj7+cw5zzh8lnBnr+99eaWV1bX2jvFnZ2t7Z3avuH7SNSjWhLaK40p0IG8qZpC3LLKedRFMsIk4fovH1tP7wRLVhSt7bLKGhwEPJYkawddbjbT9PFGckm/SrNb/uz4SWISigBoWa/epXb6BIKqi0hGNjuoGf2DDH2jLC6aTSSw1NMBnjIe06lFhQE+azjSfoxDkDFCvtnrRo5v6eyLEwJhOR6xTYjsxibWr+V+umNr4McyaT1FJJ5h/FKUdWoen5aMA0JZZnDjDRzO2KyAhrTKwLqeJCCBZPXob2WT1wfHdea1wVcZThCI7hFAK4gAbcQBNaQEDCM7zCm2e8F+/d+5i3lrxi5hD+yPv8AexSkRE=</latexit>

reward<latexit sha1_base64="IUpmDw3zvnHW2+kwnHkxV06s0+M=">AAAB7XicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsJu3azW7Y3Sgl9D948aCIV/+PN/+N2zYHbX1h4eGdGXbmDVPOtPG8b2dldW19Y7O0Vd7e2d3brxwctrTMFMUmlVyqTkg0ciawaZjh2EkVkiTk2A5HN9N6+xGVZlLcm3GKQUIGgsWMEmOtlsInoqJ+perVvJncZfALqEKhRr/y1YskzRIUhnKiddf3UhPkRBlGOU7KvUxjSuiIDLBrUZAEdZDPtp24p9aJ3Fgq+4RxZ+7viZwkWo+T0HYmxAz1Ym1q/lfrZia+CnIm0sygoPOP4oy7RrrT092IKaSGjy0Qqpjd1aVDogg1NqCyDcFfPHkZWuc13/LdRbV+XcRRgmM4gTPw4RLqcAsNaAKFB3iGV3hzpPPivDsf89YVp5g5gj9yPn8AwRqPOw==</latexit><latexit sha1_base64="IUpmDw3zvnHW2+kwnHkxV06s0+M=">AAAB7XicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsJu3azW7Y3Sgl9D948aCIV/+PN/+N2zYHbX1h4eGdGXbmDVPOtPG8b2dldW19Y7O0Vd7e2d3brxwctrTMFMUmlVyqTkg0ciawaZjh2EkVkiTk2A5HN9N6+xGVZlLcm3GKQUIGgsWMEmOtlsInoqJ+perVvJncZfALqEKhRr/y1YskzRIUhnKiddf3UhPkRBlGOU7KvUxjSuiIDLBrUZAEdZDPtp24p9aJ3Fgq+4RxZ+7viZwkWo+T0HYmxAz1Ym1q/lfrZia+CnIm0sygoPOP4oy7RrrT092IKaSGjy0Qqpjd1aVDogg1NqCyDcFfPHkZWuc13/LdRbV+XcRRgmM4gTPw4RLqcAsNaAKFB3iGV3hzpPPivDsf89YVp5g5gj9yPn8AwRqPOw==</latexit><latexit sha1_base64="IUpmDw3zvnHW2+kwnHkxV06s0+M=">AAAB7XicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsJu3azW7Y3Sgl9D948aCIV/+PN/+N2zYHbX1h4eGdGXbmDVPOtPG8b2dldW19Y7O0Vd7e2d3brxwctrTMFMUmlVyqTkg0ciawaZjh2EkVkiTk2A5HN9N6+xGVZlLcm3GKQUIGgsWMEmOtlsInoqJ+perVvJncZfALqEKhRr/y1YskzRIUhnKiddf3UhPkRBlGOU7KvUxjSuiIDLBrUZAEdZDPtp24p9aJ3Fgq+4RxZ+7viZwkWo+T0HYmxAz1Ym1q/lfrZia+CnIm0sygoPOP4oy7RrrT092IKaSGjy0Qqpjd1aVDogg1NqCyDcFfPHkZWuc13/LdRbV+XcRRgmM4gTPw4RLqcAsNaAKFB3iGV3hzpPPivDsf89YVp5g5gj9yPn8AwRqPOw==</latexit>

Figure 3. An overview of our Joint Reinforcement Transfer (JRT). Our model contains two RL models: synthetic model with Ms and

Ps and real model with Mr and Pr . We use red line and blue line to represent the testing procedure of the two models. Black arrows

stand for forward and dotted lines stand for backward in the training procedure. fs and fr are feature embeddings of synthetic image Xs

and real image Xr , respectively. For feature adaptation, we apply adversarial loss Ladv on top of discriminator D. Additionally, we have

identity loss Lidt to regularize the learned semantic embedding. In policy transfer, we have the mimic loss Lmimic trained with policy

loss Lpolicy .

3.2. Adversarial Feature Adaptation

For adapting model trained from simulated environment

to real environment, one intuitive idea is mapping visual

observation to a latent space with same distribution. Thus

the following policy network can be easily applied to real

environment since its input distribution will not change sig-

nificantly.

Motivated by this, we consider adversarial learning

method to adapt the mapping function for real images. Note

that we use A3C to demonstrate our approach for simplic-

ity. Following the notation of [38], we assume Xs as im-

age drawn from a distribution of synthetic image domain

psyn(x, y) and Ys as its label. Similarly, we have real im-

age Xr and label Yr from preal(x, y). We mark the mapping

function for synthetic images as Ms. Thus we have

fs = Ms(xs)

Here fs also follows a synthetic feature distribution repre-

sented as fs ∼ Fs. Our goal is to train a real mapping

function Mr for visual embedding fr ∼ Fr, where fs and

fr belongs to the same distribution.

fr = Mr(sr)

fs, fr ∈ S, Fr = Fs

To build our adversarial learning procedure, we need a

binary discriminator D to distinguish whether a feature f

is mapped from Xs or Xr. Label lk equal to 1 is when

f is mapped from Xs and equal to 0 otherwise. Thus the

classifier D is optimized by cross entropy loss below

Lcls(Xs, Xr) =− Exs∼Xslog(D(Ms(xs)))

− Exr∼Xrlog(1−D(Mr(xr)))

We design our adversarial loss to optimize Mr so that

Mr and C can compete each other

Ladv = Exr∼Xrlog(D(Mr(xr)))

The optimization of Mr, which equal to maximizing

Exr∼Xrlog(D(Mr(xr))), will force the distribution of Fr

getting closer to Fs, so that discriminator D is more diffi-

cult to distinguish Fs and Fr. By alternatively optimizing

Mr and D, The visual embedding fs and fr will finally be-

long to same distribution.

However, Ms(xs) and Mr(xr) can still be mapped to

different semantic vector in latent space. It is due to adver-

sarial method above does not ensure fs to have the same

semantic representation as fr. Inspired by the technique

of [37], we used identity mapping loss to regularize the

mapping function Mr to be an approximate function of

identity mapping when input is a synthetic image xs. We

use L2 loss because our regularization is applied on feature

space. This approach suppose that mapping function Mr

has good generalization ability and synthetic image shares

exact the same semantic space as real image. Thus we pro-

pose our identity mapping loss:

11391

Lidt = ‖Ms(Xs)−Mr(Xs)‖2

By introducing L2 weight norm for both Mr and D re-

spectively, our full objective of feature adaption is written

as:

Ltotal = Ladv + Lcls + λ1Lidt + λ2Lnorm

Although Ladv and Lcls are in same magnitude, we need

to balance regularization terms by λ1 and λ2. How perfor-

mance influenced by loss weight are fully discussed in ex-

periment section.

We use unpaired images from synthetic environment and

real environment to train our model. As shown in Fig-

ure 3, our gradient are only computed from adversarial loss

Ladv and identity loss Lidt. Instead of reinforcement train-

ing procedure, we train model following the framework of

ADDA [38]. Parameter of synthetic mapping function Ms

are fixed while Mr and discriminator D is updated per step.

We sample batches of images every training step, compute

gradient from two losses and then update Mr and D.

In testing, Mr is connected with policy function Pr.

Without Policy Transfer method, we can just simply copy

the parameter from Ps to Pr and achieve a significant per-

formance improvement from the adaptation of Mr.

3.3. Policy Mimic

Previously we have a powerful original policy network

trained on large scale synthetic data and a mapping function

to adapt visual embedding to real environment. However,

due to the huge gap between synthetic environment and real

environment, adapting visual embedding only is not enough

for reinforce model transfer.

Therefore we introduce an auxiliary approach to trans-

fer our reinforce policy. We find it is necessary to combine

transferred policy with knowledge gained from real envi-

ronment. Since our task is to predict next action by clas-

sification, we introduce Policy Distillation method [31], to

transfer knowledge leaned from synthetic environment.

Our baseline trained on synthetic environment acts as a

teacher model T . A student model learns knowledge from

T is written as S. Both model is fed by real environment

data. And the action probability p predicted by T is served

as soft label to train S, which is also called mimic. Student

model S is trained with a log-likelihood loss to predict the

same action distribution:

p = softmax(Ps(Ms(xr)))

Lmimic =− Exr∼Xr[p · log(softmax(Pr(fr))))

− (1− p) · (1− log(softmax(Pr(fr))))]

Different from [31], which student model only super-

vised by abest = argmax(q) our student model S learns

the full distribution of p from teacher T . This loss function

enables student learn more knowledge such radios of very

small probabilities from soft labels, as demonstrated in [18].

We optimize Lmimic together with Lpolicy in real envi-

ronment. By balancing the weights between reinforce loss

and policy transfer loss, our full objective of Policy Transfer

is defined as

Ltotal = Lpolicy + λLmimic

Note that loss weight λ can be different when Lmimic is

trained together with UNREAL [20] since UNREAL have

several auxiliary tasks which make the magnitude of Lpolicy

slightly larger.

Unlike Feature Adaptation, we follow the reinforcement

learning framework of [25], training model in real environ-

ment only. As shown in Figure 3, synthetic model and real

model take the same input from real environment Xr. Mr

is pretrained by adaptation method last section. Parameters

of Ms, Ps and Mr is fixed and only Pr is tuned by mimic

loss Lmimic and policy loss Lpolicy .

In testing, as the blue line shown in Figure 3, Pr is con-

nected at the bottom of Mr. Tuning policy function Pr can

get another significant performance improvement.

4. Experiments

4.1. Datasets and Baseline Setup

Our setting is mainly based on MINOS dataset [34], for

MINOS provide unified interfaces for both synthetic and

real environments. Same configuration of scenes, multi-

modal sensory inputs and action space facilitate implemen-

tation.

Synthetic Environment SUNCG [36] is a large scale envi-

ronment consists of over 45,622 synthetic indoor 3D scenes.

It renders image observation with 3D occupancy and se-

mantic labels for a scene from a single depth map. Fol-

lowing the setting of MINOS, we manually select a subset

of 360 single-floor houses fit for room navigation. A large

range of houses in SUNCG, however, are not appropriate

for our task. For some scenes, rooms are not connected with

each other and other scenes are not even houses. SUNCG

scenes are split into 207/76/77 for training, validation and

testing respectively.

Real Environment Matterport3D [5] is a multi-layer real-

like environment with 90 scenes. We follow the same split-

ting strategy as original dataset. It has 61 rooms for training

and 18 rooms for testing. We sample 10 episode for each

test house for total 180 episode as our test set.

Baselines We consider two algorithms, A3C and UNREAL,

which trained on SUNCG and Matterport3D from scratch

as our baseline models. For each baseline model, we adopt

11392

Figure 4. The left figure shows the testing results of real baseline

on real environment. The right figure shows the testing results of

synthetic baseline on synthetic and real environment respectively.

RMSProp solver with 4 asynchronous threads. Agent is

trained for 13.2M total steps, corresponding to 1 day of ex-

perience on a Quadro P5000 GPU device.

There are three widely used types of goals for indoor

navigation: PointGoal, ObjectGoal and RoomGoal. Point-

Goal instructs agent to go to a precise relative point, repre-

sented as (x, y). ObjectGoal makes agent to find a object

in house. RoomGoal, as the most complex task, requires

agent to find a specified room house. RoomGoal is quite

difficult not only because texture of furnitures are in di-

verse color but also difference of its shapes and positions

can make distinct observations. Moreover, to navigate ef-

ficiently enough, agents have to learn how rooms are con-

nected, such as a bedroom is always connected with a bath-

room and a kitchen is often connected with a living room.

As a result, we adopt RoomGoal as goal type to prove our

idea.

We test our baseline models on the sampled test

episodes. Usually, performance of a navigation model is

evaluated by success rate [2] report as percentage. Re-

cent paper [1] proposes a new measurement called Success

weighted by (normalized inverse) Path Length(SPL). This

metric requires model to navigate to goal as possible when

taking the optimal path. We evaluate our models on both

metrics.

To show our baseline models are sufficiently trained, we

test the success rate of our baselines for every 30K iter-

ations. Figure 4 indicates both baselines converged after

8 × 106 iterations. However, synthetic baseline perform

badly when tested on real environment and its performance

continually drop after 8× 106 iterations. This performance

drop shows there is domain gap between synthetic and real

environments.

4.2. Adversarial Feature Adaption Experiments

For unpaired adversarial training, we sample 10,000

RGB images from synthetic and real environment respec-

tively. We adopt Adam optimizer with initial learning rate

= 0.0001, β1 = 0.9, β2 = 0.999. Each training batch with

method % success rate % SPL

sim baseline 24.13% ± 2.12% 22.46% ± 1.52%

real baseline 32.67% ± 0.73% 27.60% ± 2.33%

sim+FT 36.80% ± 2.64% 27.95% ± 1.18%

sim+FA 56.27% ± 2.94% 38.74% ± 0.85%

sim+PM 47.64% ± 2.25% 33.15% ± 2.37%

sim+FA+PM 58.53% ± 2.96% 42.25% ± 2.23%Table 1. Success rate for different baselines and feature adaptation

results. Here FT stands for finetune, FA means adversarial feature

adaptation and PM means policy mimic.

size 64 consists of 32 synthetic images and 32 real images.

This training process is easy to converge so that we test all

the model at 1000 iterations.

Note that we use initialize the parameter of Mr with Ms

because Ms content some semantic encoding. Without a

good starting point, model is easily collapse and its perfor-

mance could be even lower than baseline.

Comparison against baselines In Table 1, we com-

pare three different baselines with our feature adaptation

method. Among three baselines the best performance is

achieved by fine-tuning on real environment after train-

ing on reinforcement synthetic environment. This base-

line gets highest success rate because it is trained on both

environment so that get more information. However, our

method outperforms the best baseline greatly by 19.47%,

even though baseline with finetune is trained with more in-

formation.

Note that baseline with fine-tune access more informa-

tion than feature adaptation since fine-tune on real environ-

ment will not only get the image input but also get the re-

ward supervision. Even though std is large since our scale of

testing data is small, huge increasing of mean value show-

ing feature adaptation method works on reinforcement pol-

icy transfer.

Analysis of the identity loss In Table 2, we compare the

influence of different identity loss weight to adaption per-

formance. We find that identity loss plays a important role

in adversarial training. Without identity loss, which means

its loss weight equals 0, performance will drop by 7.74%.

When identity weight goes even larger, mapping func-

tion for real environment Mr tend to act much more like

Ms. It makes feature distribution Fr far away with Fs,

which may led policy function Pr make wrong decision.

When identity weight continue increasing, Performance of

the whole model will drop and eventually be equal to syn-

thetic baseline. Thus we adopt the hyper-parameter identity

loss weight = 0.0005 in the following experiments.

4.3. Policy Mimic Experiments

Following the Policy Mimic training described in the

Section 3.3, we only update policy function Pr in real envi-

ronment.

11393

idt ablation % success rate

idt=0 48.53% ± 2.93%

idt=5 ∗ e−6 54.00% ± 2.23%

idt=5 ∗ e−4 56.27% ± 3.94%

idt=5 ∗ e−2 53.07% ± 1.55%Table 2. Ablation study: success rate for different weight of iden-

tity loss

method % success rate

UNREAL baseline 24.13% ± 2.12%

UNREAL baseline+FA 56.27% ± 2.94%

UNREAL baseline+FA+PM 58.50% ± 2.96%

A3C baseline 16.00% ± 2.02%

A3C baseline+FA 34.80% ± 0.88%

A3C baseline+FA+PM 50.00% ± 1.88%Table 3. Success rate for different baselines, feature adapta-

tion(FA) and policy mimic(PM) results

mimic ablation % success rate

UNREAL baseline 24.13% ± 2.12%

UNREAL baseline+FT 36.80% ± 2.64%

mimic weight=0 36.00% ± 3.55%

mimic weight=0.1 58.53% ± 2.96%

mimic weight=0.2 34.53% ± 1.95%

mimic weight=0.5 22.80% ± 1.85%Table 4. Ablation study: success rate for different weight for

mimic loss. Here FT means baseline with finetune.

The environmental configuration and model hyper-

parameter is same as baseline when trained in real envi-

ronment. We use Mr trained in Adversarial Feature Adap-

tation method and initialize Pr by Ps. Similar to feature

adaptation, if we just random initialize our policy function,

the result could be much lower. To prove our policy mimic

method is general, we test our method on both A3C model

and UNREAL model.

Comparison against baselines Table 3 summarizes our re-

sults and compares our policy mimic method with feature

adaptation only and baseline.

UNREAL baseline is higher than A3C baseline for 8%

since the auxiliary task provides additional training signals.

However, after feature adaptation, the performance gap is

increased to nearly 22%. A reasonable explanation is ad-

ditional training signals are mainly beneficial to policy net-

work. Thus feature transfer can be a greater help for UN-

REAL model compared to A3C model.

Finally, it is the policy mimic method that reduces the

performance gap. Even though the teacher model for policy

mimic of A3C is A3C baseline trained on synthetic envi-

ronment, model can still absorb useful knowledge learned

from synthetic data.

Analysis of the mimic loss Table 4 reports our ablation

experiment upon mimic loss. Compared with ablation ex-

periment for weight of identity loss, weight of mimic loss

seems to be more sensitive. Performance will drop rapidly

if mimic weight be slight deviate the optimal value. When

mimic loss weight is 0, it is equivalent to finetuning model

on real environment. Compared to finetune baseline above,

since mapping function Mr has been transferred, the per-

formance is slightly higher.

The reason why mimic loss is rather important is that the

policy function can overfit easily in the real environmental

without rich training data. Thus imitating behavior of model

trained on synthetic model can solve this difficulty.

4.4. Visualization

In this section, we evaluate our methods from qualitative

aspects.

We run different models on unseen real environments in

the test set. Meanwhile, we record agents navigating trajec-

tories, RGB input sequences and some intermediate results.

By comparing these results, we testify that our methods are

able to optimize intermediate procedure and therefore, im-

prove final performance.

Visual Attention We now attempt to analysis how adver-

sarial feature adaptation influence mapping function.

We plot sequences of heatmap for last convolution layer

combining with RGB inputs. Figure 5 compares three dif-

ferent models with similar RGB images input.

It is obvious that baseline trained on synthetic data is not

able to embedding real environment data. The model can

not focus on specific targets. On the contrary, its attention

is scattered on the whole map. Finally, it can not go to target

room but miss the door by keep turning right.

After adversarial training even though without identity

loss, model learns to focus on some targets like cabinet,

wall and window. However, these targets have minor effect

on navigation. It means model can learn how to distinguish

objects by adversarial adaptation. By paying too much at-

tention on the wall at right, this model misses to cross the

door either.

With identity loss augmenting adversarial training,

model not only learns to recognize objects, but also know

the semantic difference between targets. We find that this

model pay more attention on door like edges and be able to

find the real door.

Policy Behavior Based on model after adversarial adaption,

we now analysis the policy behavior in some complex cases.

Figure 6 shows a scene where model need to cross a door

not being able to distinguish from RGB information easily.

Model without policy mimic does not have a stable navi-

gation behavior. It repeatedly turns left or right in a random

way. It seems that model is overfit on a local minimum,

which prohibit its exploration.

In contrast, model with policy mimic gains a more sta-

ble navigation. It keeps turning following one direction to

11394

Baseline trained on synthetic environment

Adversarial Feature Adaptation

Adversarial Feature Adaptation Lidt = 0<latexit sha1_base64="sY0jPbIfJuH/8pcWSwdin7mCkBw=">AAAB8nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL148FDB2kIaymazbZduNmF3IpTQn+HFgyJe/TXe/Ddu2xy09YWFh3dm2Jk3TKUw6LrfTmlldW19o7xZ2dre2d2r7h88miTTjLdYIhPdCanhUijeQoGSd1LNaRxK3g5HN9N6+4lrIxL1gOOUBzEdKNEXjKK1/LteLiKckCvi9qo1t+7ORJbBK6AGhZq96lc3SlgWc4VMUmN8z00xyKlGwSSfVLqZ4SllIzrgvkVFY26CfLbyhJxYJyL9RNunkMzc3xM5jY0Zx6HtjCkOzWJtav5X8zPsXwa5UGmGXLH5R/1MEkzI9H4SCc0ZyrEFyrSwuxI2pJoytClVbAje4snL8HhW9yzfn9ca10UcZTiCYzgFDy6gAbfQhBYwSOAZXuHNQefFeXc+5q0lp5g5hD9yPn8AHlqQeQ==</latexit><latexit sha1_base64="sY0jPbIfJuH/8pcWSwdin7mCkBw=">AAAB8nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL148FDB2kIaymazbZduNmF3IpTQn+HFgyJe/TXe/Ddu2xy09YWFh3dm2Jk3TKUw6LrfTmlldW19o7xZ2dre2d2r7h88miTTjLdYIhPdCanhUijeQoGSd1LNaRxK3g5HN9N6+4lrIxL1gOOUBzEdKNEXjKK1/LteLiKckCvi9qo1t+7ORJbBK6AGhZq96lc3SlgWc4VMUmN8z00xyKlGwSSfVLqZ4SllIzrgvkVFY26CfLbyhJxYJyL9RNunkMzc3xM5jY0Zx6HtjCkOzWJtav5X8zPsXwa5UGmGXLH5R/1MEkzI9H4SCc0ZyrEFyrSwuxI2pJoytClVbAje4snL8HhW9yzfn9ca10UcZTiCYzgFDy6gAbfQhBYwSOAZXuHNQefFeXc+5q0lp5g5hD9yPn8AHlqQeQ==</latexit><latexit sha1_base64="sY0jPbIfJuH/8pcWSwdin7mCkBw=">AAAB8nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL148FDB2kIaymazbZduNmF3IpTQn+HFgyJe/TXe/Ddu2xy09YWFh3dm2Jk3TKUw6LrfTmlldW19o7xZ2dre2d2r7h88miTTjLdYIhPdCanhUijeQoGSd1LNaRxK3g5HN9N6+4lrIxL1gOOUBzEdKNEXjKK1/LteLiKckCvi9qo1t+7ORJbBK6AGhZq96lc3SlgWc4VMUmN8z00xyKlGwSSfVLqZ4SllIzrgvkVFY26CfLbyhJxYJyL9RNunkMzc3xM5jY0Zx6HtjCkOzWJtav5X8zPsXwa5UGmGXLH5R/1MEkzI9H4SCc0ZyrEFyrSwuxI2pJoytClVbAje4snL8HhW9yzfn9ca10UcZTiCYzgFDy6gAbfQhBYwSOAZXuHNQefFeXc+5q0lp5g5hD9yPn8AHlqQeQ==</latexit>

Lidt = 5 ∗ e−4<latexit sha1_base64="ZHvpSv7v7UNEnOj+1CEcQ6g1/cQ=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3AwWQQRLIhXdCEU3LlxUsBdoY5hMpu3QyYWZiVBD8FXcuFDEre/hzrdx2mah1R8GPv5zDufM78WcSWVZX0Zhbn5hcam4XFpZXVvfMDe3mjJKBKENEvFItD0sKWchbSimOG3HguLA47TlDS/H9dY9FZJF4a0axdQJcD9kPUaw0pZr7ly7KfNVhs7RCTpE9C49qmauWbYq1kToL9g5lCFX3TU/u35EkoCGinAsZce2YuWkWChGOM1K3UTSGJMh7tOOxhAHVDrp5PoM7WvHR71I6BcqNHF/TqQ4kHIUeLozwGogZ2tj879aJ1G9MydlYZwoGpLpol7CkYrQOArkM0GJ4iMNmAimb0VkgAUmSgdW0iHYs1/+C83jiq35plquXeRxFGEX9uAAbDiFGlxBHRpA4AGe4AVejUfj2Xgz3qetBSOf2YZfMj6+Aec9k48=</latexit><latexit sha1_base64="ZHvpSv7v7UNEnOj+1CEcQ6g1/cQ=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3AwWQQRLIhXdCEU3LlxUsBdoY5hMpu3QyYWZiVBD8FXcuFDEre/hzrdx2mah1R8GPv5zDufM78WcSWVZX0Zhbn5hcam4XFpZXVvfMDe3mjJKBKENEvFItD0sKWchbSimOG3HguLA47TlDS/H9dY9FZJF4a0axdQJcD9kPUaw0pZr7ly7KfNVhs7RCTpE9C49qmauWbYq1kToL9g5lCFX3TU/u35EkoCGinAsZce2YuWkWChGOM1K3UTSGJMh7tOOxhAHVDrp5PoM7WvHR71I6BcqNHF/TqQ4kHIUeLozwGogZ2tj879aJ1G9MydlYZwoGpLpol7CkYrQOArkM0GJ4iMNmAimb0VkgAUmSgdW0iHYs1/+C83jiq35plquXeRxFGEX9uAAbDiFGlxBHRpA4AGe4AVejUfj2Xgz3qetBSOf2YZfMj6+Aec9k48=</latexit><latexit sha1_base64="ZHvpSv7v7UNEnOj+1CEcQ6g1/cQ=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3AwWQQRLIhXdCEU3LlxUsBdoY5hMpu3QyYWZiVBD8FXcuFDEre/hzrdx2mah1R8GPv5zDufM78WcSWVZX0Zhbn5hcam4XFpZXVvfMDe3mjJKBKENEvFItD0sKWchbSimOG3HguLA47TlDS/H9dY9FZJF4a0axdQJcD9kPUaw0pZr7ly7KfNVhs7RCTpE9C49qmauWbYq1kToL9g5lCFX3TU/u35EkoCGinAsZce2YuWkWChGOM1K3UTSGJMh7tOOxhAHVDrp5PoM7WvHR71I6BcqNHF/TqQ4kHIUeLozwGogZ2tj879aJ1G9MydlYZwoGpLpol7CkYrQOArkM0GJ4iMNmAimb0VkgAUmSgdW0iHYs1/+C83jiq35plquXeRxFGEX9uAAbDiFGlxBHRpA4AGe4AVejUfj2Xgz3qetBSOf2YZfMj6+Aec9k48=</latexit>

Figure 5. Heatmap for the last convolution in mapping function. Red and yellow regions are places with large values where network

mainly focus on. We can capture the relationship between scene transitions from left to right and attention on heatmaps.

Adversarial Feature Adaptation without Policy Mimic

Policy Mimic based on Adversarial Feature Adaptation

Figure 6. RGB image sequences from left to right indicates the models’ trajectories in testing.

explore scene of whole room.

This knowledge is learned from large scale synthetic

data, where model is able to be trained among diverse

scenes to avoid overfitting. The knowledge is suitable for

navigation in real environment, however, which model can

hardly learn from because of few training data.

5. Conclusion

To relieve the problem of lacking real training data, we

propose JRT to discover the knowledge learned from syn-

thetic reinforcement environment to facilitate the training

process in real environment. Specifically, we integrate ad-

versarial feature adaptation and policy network into a joint

network. We demonstrated the effectiveness of the frame-

work with extensive experiments. We also conducted ab-

lation studies of the identity loss and mimic loss to show

its superiority. Finally, our methods outperform baselines

in both qualitative and quantitative ways without any addi-

tional human annotations.

In the future, we will validate the transfer ability on other

tasks, e.g., Embodied Question Answering [8] and exploit

temporal information for feature adaptation [39, 46, 7] and

feature analysis [6].

Also, policy mimic can be further improved by consider-

ing episode-wise optimization. In addition, it is meaningful

to investigate how to transfer policy in vision and language

navigation tasks.

Acknowledgments. We thank AWS Cloud Credits for Re-

search for partly supporting this research.

11395

References

[1] P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy,

S. Gupta, V. Koltun, J. Kosecka, J. Malik, R. Mottaghi,

M. Savva, et al. On evaluation of embodied navigation

agents. arXiv preprint arXiv:1807.06757, 2018. 6

[2] P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson,

N. Sunderhauf, I. Reid, S. Gould, and A. van den Hen-

gel. Vision-and-language navigation: Interpreting visually-

grounded navigation instructions in real environments. In

Proceedings of the IEEE Conference on Computer Vision

and Pattern Recognition, pages 3674–3683, 2018. 6

[3] B. D. Argall, S. Chernova, M. Veloso, and B. Browning. A

survey of robot learning from demonstration. Robotics and

autonomous systems, 2009. 1

[4] J. Cao, Y. Guo, Q. Wu, C. Shen, J. Huang, and M. Tan.

Adversarial learning with local coordinate coding. In Inter-

national Conference on Machine Learning, pages 707–715,

2018. 2

[5] A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Nießner,

M. Savva, S. Song, A. Zeng, and Y. Zhang. Matterport3d:

Learning from rgb-d data in indoor environments. arXiv

preprint arXiv:1709.06158, 2017. 2, 5

[6] X. Chang and Y. Yang. Semisupervised feature analy-

sis by mining correlations among multiple tasks. IEEE

transactions on neural networks and learning systems,

28(10):2294–2305, 2017. 8

[7] X. Chang, Y.-L. Yu, Y. Yang, and E. P. Xing. Semantic pool-

ing for complex event analysis in untrimmed videos. IEEE

transactions on pattern analysis and machine intelligence,

39(8):1617–1632, 2017. 8

[8] A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Ba-

tra. Embodied question answering. In Proceedings of the

IEEE Conference on Computer Vision and Pattern Recogni-

tion (CVPR), volume 5, page 6, 2018. 8

[9] L. Duan, I. W. Tsang, D. Xu, and S. J. Maybank. Domain

transfer svm for video concept detection. 2009. 2, 3

[10] Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel.

Benchmarking deep reinforcement learning for continuous

control. In International Conference on Machine Learning,

pages 1329–1338, 2016. 1

[11] N. Faraji Davar, T. de Campos, D. Windridge, J. Kittler, and

W. Christmas. Domain adaptation in the context of sport

video action recognition. In Domain Adaptation Workshop,

in conjunction with NIPS, 2011. 2

[12] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle,

F. Laviolette, M. Marchand, and V. Lempitsky. Domain-

adversarial training of neural networks. The Journal of Ma-

chine Learning Research, 17(1):2096–2030, 2016. 3

[13] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu,

D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Gen-

erative adversarial nets. In Advances in neural information

processing systems, pages 2672–2680, 2014. 2, 3

[14] A. Gretton, A. J. Smola, J. Huang, M. Schmittfull, K. M.

Borgwardt, and B. Scholkopf. Covariate shift by kernel mean

matching. 2009. 2

[15] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and

A. C. Courville. Improved training of wasserstein gans. In

I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus,

S. Vishwanathan, and R. Garnett, editors, Advances in Neu-

ral Information Processing Systems 30, pages 5767–5777.

Curran Associates, Inc., 2017. 2

[16] A. Handa, V. Patraucean, V. Badrinarayanan, S. Stent, and

R. Cipolla. Understanding real world indoor scenes with

synthetic data. In Proceedings of the IEEE Conference

on Computer Vision and Pattern Recognition, pages 4077–

4085, 2016. 3

[17] Y. He, P. Liu, Z. Wang, Z. Hu, and Y. Yang. Filter prun-

ing via geometric median for deep convolutional neural net-

works acceleration. In Proceedings of the IEEE Conference

on Computer Vision and Pattern Recognition (CVPR), 2019.

3

[18] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge

in a neural network. arXiv preprint arXiv:1503.02531, 2015.

3, 5

[19] J. Huang, A. Gretton, K. M. Borgwardt, B. Scholkopf, and

A. J. Smola. Correcting sample selection bias by unlabeled

data. In Advances in neural information processing systems,

pages 601–608, 2007. 2

[20] M. Jaderberg, V. Mnih, W. M. Czarnecki, T. Schaul, J. Z.

Leibo, D. Silver, and K. Kavukcuoglu. Reinforcement learn-

ing with unsupervised auxiliary tasks. In International Con-

ference on Learning Representations (ICLR), 2016. 1, 5

[21] E. Kolve, R. Mottaghi, D. Gordon, Y. Zhu, A. Gupta, and

A. Farhadi. Ai2-thor: An interactive 3d environment for vi-

sual ai. arXiv preprint arXiv:1712.05474, 2017. 2

[22] M. Long, J. Wang, G. Ding, J. Sun, and S. Y. Philip. Transfer

feature learning with joint distribution adaptation. In 2013

IEEE International Conference on Computer Vision, pages

2200–2207. IEEE, 2013. 2

[23] Y. Luo, L. Zheng, T. Guan, J. Yu, and Y. Yang. Taking

a closer look at domain shift: Category-level adversaries

for semantics consistent domain adaptation. In The IEEE

Conference on Computer Vision and Pattern Recognition

(CVPR), 2019. 2

[24] P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard,

A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu,

et al. Learning to navigate in complex environments. arXiv

preprint arXiv:1611.03673, 2016. 1

[25] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap,

T. Harley, D. Silver, and K. Kavukcuoglu. Asynchronous

methods for deep reinforcement learning. In International

conference on machine learning(ICML), 2016. 1, 5

[26] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves,

I. Antonoglou, D. Wierstra, and M. Riedmiller. Play-

ing atari with deep reinforcement learning. arXiv preprint

arXiv:1312.5602, 2013. 1

[27] A. Mousavian, A. Toshev, M. Fiser, J. Kosecka, and J. David-

son. Visual representations for semantic target driven navi-

gation. In Proceedings of european conference on computer

vision, 2018. 1, 3

[28] X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and

K. Saenko. Visda: The visual domain adaptation challenge.

arXiv preprint arXiv:1710.06924, 2017. 3

11396

[29] J. Quinonero-Candela, M. Sugiyama, A. Schwaighofer, and

N. Lawrence. Covariate shift and local learning by distribu-

tion matching, 2008. 2

[30] S. Ross, G. Gordon, and D. Bagnell. A reduction of imi-

tation learning and structured prediction to no-regret online

learning. In Proceedings of the fourteenth international con-

ference on artificial intelligence and statistics, 2011. 1

[31] A. A. Rusu, S. G. Colmenarejo, C. Gulcehre, G. Desjardins,

J. Kirkpatrick, R. Pascanu, V. Mnih, K. Kavukcuoglu,

and R. Hadsell. Policy distillation. arXiv preprint

arXiv:1511.06295, 2015. 3, 5

[32] K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting vi-

sual category models to new domains. In European confer-

ence on computer vision, pages 213–226. Springer, 2010. 2

[33] A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani. Deep

reinforcement learning framework for autonomous driving.

Electronic Imaging, 2017(19):70–76, 2017. 1

[34] M. Savva, A. X. Chang, A. Dosovitskiy, T. Funkhouser,

and V. Koltun. Minos: Multimodal indoor simulator

for navigation in complex environments. arXiv preprint

arXiv:1712.03931, 2017. 1, 3, 5

[35] D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou,

A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton,

et al. Mastering the game of go without human knowledge.

Nature, 550(7676):354, 2017. 1

[36] S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and

T. Funkhouser. Semantic scene completion from a single

depth image. In Computer Vision and Pattern Recognition

(CVPR), 2017 IEEE Conference on, pages 190–198. IEEE,

2017. 1, 2, 5

[37] Y. Taigman, A. Polyak, and L. Wolf. Unsupervised cross-

domain image generation. arXiv preprint arXiv:1611.02200,

2016. 4

[38] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell. Adversarial

discriminative domain adaptation. In Computer Vision and

Pattern Recognition (CVPR), volume 1, page 4, 2017. 2, 3,

4, 5

[39] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, G. Liu, A. Tao, J. Kautz,

and B. Catanzaro. Video-to-video synthesis. arXiv preprint

arXiv:1808.06601, 2018. 8

[40] Y. Wu, Y. Wu, G. Gkioxari, and Y. Tian. Building generaliz-

able agents with a realistic and rich 3d environment. In Pro-

ceedings of european conference on computer vision, 2018.

1, 2

[41] J. Xu, D. Vazquez, A. M. Lopez, J. Marın, and D. Ponsa.

Learning a part-based pedestrian detector in a virtual world.

IEEE Transactions on Intelligent Transportation Systems,

15(5):2121–2131, 2014. 3

[42] C. Yan, D. Misra, A. Bennnett, A. Walsman, Y. Bisk, and

Y. Artzi. Chalet: Cornell house agent learning environment.

arXiv preprint arXiv:1801.07357, 2018. 2

[43] J. Yang, R. Yan, and A. G. Hauptmann. Cross-domain video

concept detection using adaptive svms. In Proceedings of the

15th ACM international conference on Multimedia, pages

188–197. ACM, 2007. 2, 3

[44] H. Yu, X. Lian, H. Zhang, and W. Xu. Guided feature trans-

formation (gft): A neural language grounding module for

embodied agents. In Conference on Robot Learning (CoRL),

2018. 1

[45] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-

to-image translation using cycle-consistent adversarial net-

works. In Proceedings of the ieee international conference

on computer vision, 2017. 2

[46] L. Zhu, Z. Xu, Y. Yang, and A. G. Hauptmann. Uncovering

the temporal context for video question answering. Interna-

tional Journal of Computer Vision, 124(3):409–421, 2017.

8

[47] M. Zhu, P. Pan, W. Chen, and Y. Yang. Dm-gan: Dynamic

memory generative adversarial networks for text-to-image

synthesis. In Proceedings of the IEEE Conference on Com-

puter Vision and Pattern Recognition (CVPR), 2019. 2

[48] Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-

Fei, and A. Farhadi. Target-driven visual navigation in in-

door scenes using deep reinforcement learning. In Robotics

and Automation (ICRA), 2017 IEEE International Confer-

ence on, 2017. 1, 3

11397

sim-real joint reinforcement transfer for 3d indoor...

Documents