## Entropy Based Search Algorithm for Experimental Design

Predicted outcome .... Advanced Electron Spin Resonance Technology Research Center Workshop 2008 (ACERT 2008), Ithaca ... Sivia, D. S., Skilling, J., Data Analysis: A Bayesian Tutorial, 2nd Edition, Oxford University Press, 2006. â¢ Skilling ...
Entropy Based Search Algorithm for Experimental Design Nabin K. Malakar Kevin H. Knuth University at Albany, SUNY

Outline

• • • •

Autonomous Experimental Design Maximum Entropy for Inquiry Nested Entropy Sampling for Search Application to Robotic Arm

MaxEnt 2010

The Scientific Method

MaxEnt 2010

The Scientific Method

CAN WE AUTOMATE INQUIRY?

MaxEnt 2010

Mars Exploration

Mars Exploration Rovers: Spirit and Opportunity 128 kilobits per second / 10 Megabytes per day MaxEnt 2010

Mars Exploration Rover Mission Control

Event: MER Mission Activities Date: Spirit Sol 4 Source: Kris Becker MaxEnt 2010

Time Constraints and Human Intervention

6 to 44 minute round-trip communication delay

MaxEnt 2010

Missions to Jupiter’s Moons

60 to 100 minute round-trip communication delay

MaxEnt 2010

Data: not a problem!

MaxEnt 2010

Drinking from Fire hose

No one was hurt during this shot MaxEnt 2010

Bayesian Decision Theory

MaxEnt 2010

Bayesian Decision Theory U (d e , e ) = ∑ p (θ | d e , D , e ) log p (θ | d e , D , e ) Experiment Predicted outcome

• Expected Utility:

EU (e ) = ∑ U (d e ,e ) p (d e ) , de

• The best action maximizes the expected utility

eˆ = arg max EU (e )

MaxEnt 2010

Maximization of Entropy… • Optimal Experiment: maximizes the Information Entropy of the predictive distributions ⎛ ⎞ eˆ = arg max ⎜ C − ∑ p (d e | D ,e , M ) log p (d e | D ,e M )⎟. ⎜ ⎟ d e ⎝ ⎠

• This can also be shown using the Inquiry Calculus (Knuth)

MaxEnt 2010

Estimated value of a parameter “c”

Entropic Search vs. Random Search

MaxEnt 2010

Nested Entropy Sampling INPUT: posterior samples from the inference phase SET UP: Generate a set of s sample experiments randomly and compute the entropy H for each. WHILE samples have different entropy values Select the sample s* from the set that has the least entropy, denoted H*. Generate a trial experiment strial by selecting another sample at random from . the set EXPLORE LOOP Explore by varying the parameters of strial and computing the new value of Htrial. Accept the trial if H > H* otherwise reject it. Monitor acceptance range and change exploration step size Replace s* with strial END WHILE OUTPUT: prescription of the optimal experiment.

MaxEnt 2010

MaxEnt 2010

Simulations with Parameterized Landscape ⎡ 1⎧ H ( x ′, y ′) = ∑ ak Exp ⎢ − ⎨ (( x ′ − uk′ ) k ⎣ 2⎩

A ( y ′ − vk′ ))⎛⎜⎜ k ⎝ Ck

Ck ⎞ ⎛ ( x ′ − uk′ )⎞ ⎫⎤ ⎟⎟ ⎬⎥ ⎟⎟ ⎜⎜ ′ ′ Bk ⎠ ⎝ ( y − vk )⎠ ⎭⎦

We Investigated landscapes comprised of 1- 100 Gaussians MaxEnt 2010

Illustration of NES Sampling

MaxEnt 2010

Compression Efficiency vs. Landscape Complexity

MaxEnt 2010

Compression Efficiency and Convergence Time

MaxEnt 2010

MaxEnt 2010

Experiments with Robotic Arm • Infer the position (x, y) and radius of a circle • Using light sensor measure intensity at a given position • Automated inference and inquiry engines • Programmed only with model and likelihood---no search strategy!

MaxEnt 2010

NES Simulations with Robotic Arm • Brute Force versus NES method

MaxEnt 2010

Time for Convergence

-.- Brute Force Method -.- NES Method

MaxEnt 2010

Conclusions • Implemented Entropy-Based Search – Reduces the number of measurements

• Nested Entropy Sampling – Reduces the number of inquiry computations – Compression Ratio independent of landscape complexity

• Can be adapted for other Utility Functions

MaxEnt 2010

ROBOT DEMO

MaxEnt 2010

Acknowledgements • • • • •

Prof. Ariel Caticha, Physics, University at Albany Dr. Tom Loredo, Cornell University Dr. Julian Center, Autonomous Explorations Inc. Physics Department, University at Albany The MaxEnt Organizers for support

MaxEnt 2010

References • • •

• • • • • • • • • • •

Bernardo, J. M. Expected Information as Expected Utility, The Annals of Statistics, 7, 3, 686-690, 1979. Earle, K.A., Knuth, K.H., Schneider, D.J, Budil, D.E. 2008. Information Theory and Spectral Information, Advanced Electron Spin Resonance Technology Research Center Workshop 2008 (ACERT 2008), Ithaca NY, May 2008. Knuth, K. H., Erner P. M., Frasso S. 2007, Designing intelligent instruments. In Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Saratoga Springs, NY, USA, 2007, edited by K. H. Knuth, A. Caticha, J. L. Center, A. Gi_n, C. C. Rodriguez, AIP Conference Proceedings 954, American Institute of Physics, Melville NY, pp. 203-211. Knuth, K. H. and Center, J. L., Autonomous Science Platforms and Question-Asking Machines, Cognitive Information Processing 2010 (CIP 2010), Italy, 2010. Lindley, D. V., Making Decisions Wiley-interscience, 1971. Lindley, D. V., On a Measure of the Information Provided by an Experiment, Ann. Math. Stat., 27, 4, 9861005, 1956. Loredo, T. J. 2004, Bayesian adaptive exploration. In Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Jackson Hole WY, USA, edited by G. J. Erickson & Y. Zhai, AIP Conference Proceedings 707, American Institute of Physics, New York, 2004, 330. Sebastiani, P. and Wynn, H. P., Maximum Entropy Sampling and Optimal Bayesian Experimental Design, J. Roy. Stat. Soc. B, 62, pp. 145-157, (2000). Shannon, C. E., A Mathematical Theory of Communication, Bell System Technical Journal, 27, 379423, 623656, 1948. Sivia, D. S., Skilling, J., Data Analysis: A Bayesian Tutorial, 2nd Edition, Oxford University Press, 2006. Skilling, J., Nested Sampling for General Bayesian Computation, Bayesian Analysis 4, pp. 833-860, 2006. Thrun S., Burgard W., Fox D.,. Probabilistic Robotics, MIT Press:Cambridge, 2005. Thompson, D. R., Smith, T. and Wettergreen, D., Information-Optimal Selective Data Return for Autonomous Rover Traverse Science and Survey. ICRA 2008. Fischer, R., 2004. Bayesian Experimental Design-Studies for Fusion Diagnostics. In Bayesian Inference and Maximum Entropy Methods in Science and Engineering, edited by R. Fischer, R. Preuss and U. von Toussaint, AIP Conference Proceedings, 735, pp. 76-83.

MaxEnt 2010

MaxEnt 2010