## Robot stories

Every summer before school was over, I was assigned a list of books to read. Mostly nonfiction and historical fiction, but in fourth grade there that was that first science fiction book. I often remember how that book made me feel, and marvel at the impact that it had in my life. I had read some science fiction before—Well’s *Time Traveller* and *War of the Worlds*—but this was different. This was a book with witty and thought-provoking short stories by Isaac Asimov. Each of them delivered drama, comedy, mystery and a surprise ending in about ten pages. And they had robots. And those robots had personalities, in spite of their very simple programming: *The Three Laws of Robotics*.

- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Back in the 1980s, robotics—understood as autonomous mechanical thinking—was no more than a dream. A wonderful dream that fueled many children’s imaginations and probably shaped the career choices of some. I know in my case it did.

Fast forward some thirty-odd years, when I met *Astro*: one of three research robots manufactured by the French company Aldebaran. This NAO robot found its way into the computer science classroom of Tom Simpson in Heathwood Hall Episcopal School, and quickly learned to navigate mazes, recognize some student’s faces and names, and even dance the Macarena! It did so with effortless coding: a basic command of the computer language `python`

, and some idea of object oriented programming.

I could not let this opportunity pass. I created a small undergraduate team with Danielle Talley from USC (a brilliant sophomore in computer engineering, with a minor in music), and two math majors from Morris College: my Geometry expert Fabian Maple, and a McGyver-style problem solver, Wesley Alexander. Wesley and Fabian are supported by a Department of Energy-Environmental Management grant to Morris College, which funds their summer research experience at USC. Danielle is funded by the National Science Foundation through the Louis Stokes South Carolina-Alliance for Minority Participation (LS-SCAMP).

They spent the best of their first week on this project completing a basic programming course online. At the same time, the four of us reviewed some of the mathematical tools needed to teach Astro new tricks: basic algebra and trigonometry, basic geometry, and basic calculus and statistics. The emphasis—I need to point out in case you missed it—is in the word **basic**.

### Talk the talk

The psychologist seated herself and watched Herbie narrowly as he took a chair at the other side of the table and went through the three books systematically.

At the end of half an hour, he put them down, “Of course, I know why you brought these.”

The corner of Dr. Calvin’s lip twitched, “I was afraid you would. It’s difficult to work with you, Herbie. You’re always a step ahead of me.”

“It’s the same with these books, you know, as with the others. They just don’t interest me. There’s nothing to your textbooks. Your science is just a mass of collected data plastered together by makeshift theory — and all so incredibly simple, that it’s scarcely worth bothering about.”

“It’s your fiction that interests me. Your studies of the interplay of human motives and emotions” – his mighty hand gestured vaguely as he sought the proper words.

Liar!

— Isaac Asimov

Astro can understand what I say, and maintain conversations with me. He has four microphones for voice recognition and sound localization, and two speakers for text-to-speech synthesis. But he sometimes gets confused: I say “acute”, and he understands “abide,” for example. This might sound frustrating by all standards. But mathematicians can’t be judged by any reasonable standard. Far from seeing this as a shortcoming, my team used as motivation the mechanics behind his *speech recognition* functionality, and they wrote some code to turn him into an expert Mastermind player.

Wait, what? How are these two things possibly connected?

Since Astro’s hearing is far from perfect, when he hears a word which is not clear, he gathers a short list of possible candidates. He then chooses the most likely guess based upon different considerations—using clues from context, for example. Based on similar strategies, he can be taught to be a perfect player of 5-letter-word Mastermind: Pick any five-letter word in the English language. Any word, provided it has no repeated letters. Astro chooses one random word as his first guess, and requests a score like in Mastermind.

To illustrate this process, I will show you how the game went when I chose Fabian’s favorite word: “acute.”

Astro: Is it firth? Me : NO Astro: How many letters of my guess are in your word, in the right position? How many letters of my guess are in your word, in the wrong position? Me : 1, 0 Astro: Is it foams? Me : NO Astro: How many letters of my guess are in your word, in the right position? How many letters of my guess are in your word, in the wrong position? Me : 0, 1 Astro: Is it junta? Me : NO Astro: How many letters of my guess are in your word, in the right position? How many letters of my guess are in your word, in the wrong position? Me : 1, 2 Astro: Is it acute? Me : YES Astro: Wonderful, I found it in 4 attempts!

I don’t want to get too technical here, but will mention some of the ideas. The main algorithm is based on techniques of numerical root finding and solving nonlinear equations — nothing complex: high-school level bracketing by bisection, or Newton’s method. To design better winning strategies, my team exploits the benefits of randomness. The analysis of this part is done with basic probability and statistics.

### Walk the walk

Donovan’s pencil pointed nervously. “The red cross is the selenium pool. You marked it yourself.”

“Which one is it?” interrupted Powell. “There were three that MacDougal located for us before he left.”

“I sent Speedy to the nearest, naturally; seventeen miles away. But what difference does that make?” There was tension in his voice. “There are penciled dots that mark Speedy’s position.”

And for the first time Powell’s artificial aplomb was shaken and his hands shot forward for the man.

“Are you serious? This is impossible.”

“There it is,” growled Donovan.

The little dots that marked the position formed a rough circle about the red cross of the selenium pool. And Powell’s fingers went to his brown mustache, the unfailing signal of anxiety.

Donovan added: “In the two hours I checked on him, he circled that damned pool four times. It seems likely to me that he’ll keep that up forever. Do you realize the position we’re in?”

Runaround

— Isaac Asimov

Astro moves around too. It does so thanks to a sophisticated system combining one accelerometer, one gyrometer and four ultrasonic sensors that provide him with stability and positioning within space. He also enjoys eight force-sensing resistors and two bumpers. And that is only for his legs! He can move his arms, bend his elbows, open and close his hands, or move his torso and neck (up to 25 degrees of freedom for the combination of all possible joints). Out of the box, and without much effort, he can be coded to walk around, although in a mechanical way: He moves forward a few feet, stops, rotates in place or steps to a side, etc. A very naïve way to go from `A`

to `B`

retrieving an object at `C`

, could be easily coded in this fashion as the diagram shows:

Fabian and Wesley devised a different way to code Astro taking full advantage of his inertial measurement unit. This will allow him to move around smoothly, almost like a human would. The key to their success? Polynomial interpolation and plane geometry. For advanced solutions, they need to learn about splines, curvature, and optimization. Nothing they can’t handle.

### Sing me a song

He said he could manage three hours and Mortenson said that would be perfect when I gave him the news. We picked a night when she was going to be singing Bach or Handel or one of those old piano-bangers, and was going to have a long and impressive solo.

Mortenson went to the church that night and, of course, I went too. I felt responsible for what was going to happen and I thought I had better oversee the situation.

Mortenson said, gloomily, “I attended the rehearsals. She was just singing the same way she always did; you know, as though she had a tail and someone was stepping on it.”

One Night of Song

— Isaac Asimov

Astro has excellent eyesight and understanding of the world around him. He is equipped with two HD cameras, and a bunch of computer vision algorithms, including facial and shape recognition. Danielle’s dream is to have him read from a music sheet and sing or play the song in a toy piano. She is very close to completing this project: Astro is able now to identify partitures, and extract from them the location of the pentagrams. Danielle is currently working on identifying the notes and the clefs. This is one of her test images, and the result of one of her early experiments:

Most of the techniques Danielle is using are accessible to any student with a decent command of vector calculus, and enough scientific maturity. The extraction of pentagrams and the different notes on them, for example, is performed with the Hough transform. This is a fancy term for an algorithm that basically searches for straight lines and circles by solving an optimization problem in two or three variables.

The only thing left is an actual performance. Danielle will be leading Fabian and Wes, and with the assistance of Mr. Simpson’s awesome students Erica and Robert, Astro will hopefully learn to physically approach the piano, choose the right keys, and play them in the correct order and speed. Talent show, anyone?

### Leave a Reply Cancel reply

### We have moved!

### In the news:

### Recent Posts

- Migration
- Computational Geometry in Python
- Searching (again!?) for the SS Central America
- Jotto (5-letter Mastermind) in the NAO robot
- Robot stories
- Advanced Problem #18
- Book presentation at the USC Python Users Group
- Areas of Mathematics
- More on Lindenmayer Systems
- Some results related to the Feuerbach Point
- An Automatic Geometric Proof
- Sympy should suffice
- A nice application of Fatou’s Lemma
- Have a child, plant a tree, write a book
- Project Euler with Julia
- Seked
- Nezumi San
- Ruthless Thieves Stealing a Roll of Cloth
- Which one is the fake?
- Stones, balances, matrices
- Buy my book!
- Trigonometry
- Naïve Bayes
- Math still not the answer
- Sometimes Math is not the answer
- What if?
- Edge detection: The Convolution Approach
- OpArt
- So you want to be an Applied Mathematician
- Smallest Groups with Two Eyes
- The ultimate metapuzzle
- Where are the powers of two?
- Geolocation
- Boundary operators
- The Cantor Pairing Function
- El País’ weekly challenge
- Math Genealogy Project
- Basic Statistics in sage
- A Homework on the Web System
- Apollonian gaskets and circle inversion fractals
- Toying with basic fractals
- Unusual dice
- Wavelets in sage
- Edge detection: The Scale Space Theory
- Bertrand Paradox
- Voronoi mosaics
- Image Processing with numpy, scipy and matplotlibs in sage
- Super-Resolution Micrograph Reconstruction by Nonlocal-Means Applied to HAADF-STEM
- The Nonlocal-means Algorithm
- The hunt for a Bellman Function.
- Presentation: Hilbert Transform Pairs of Wavelets
- Presentation: The Dual-Tree Complex Wavelet Transform
- Presentation: Curvelets and Approximation Theory
- Poster: Curvelets vs. Wavelets (Mathematical Models of Natural Images)
- Wavelet Coefficients
- Modeling the Impact of Ebola and Bushmeat Hunting on Western Lowland Gorillas
- Triangulations
- Mechanical Geometry Theorem Proving

### Pages

- About me
- Books
- Curriculum Vitae
- Research
- Teaching
- Mathematical Imaging
- Introduction to the Theory of Distributions
- An Introduction to Algebraic Topology
- The Basic Practice of Statistics
- MA598R: Measure Theory
- MA122—Fall 2014
- MA141—Fall 2014
- MA142—Summer II 2012
- MA241—Spring 2014
- MA242—Fall 2013
- Past Sections
- MA122—Spring 2012
- MA122—Spring 2013
- Lesson Plan—section 007
- Lesson Plan—section 008
- Review for First part (section 007)
- Review for First part (section 008)
- Review for Second part (section 007)
- Review for Third part (section 007)
- Review for the Second part (section 008)
- Review for the Fourth part (section 007)
- Review for Third and Fourth parts (section 008)

- MA122—Fall 2013
- MA141—Spring 2010
- MA141—Fall 2012
- MA141—Spring 2013
- MA141—Fall 2013
- MA141—Spring 2014
- MA141—Summer 2014
- MA142—Fall 2011
- MA142—Spring 2012
- MA241—Fall 2011
- MA241—Fall 2012
- MA241—Spring 2013
- MA242—Fall 2012
- MA242—Spring 2012
- First Midterm Practice Test
- Second Midterm-Practice Test
- Third Midterm—Practice Test
- Review for the fourth part of the course
- Blake Rollins’ code in Java
- Ronen Rappaport’s project: messing with strings
- Sam Somani’s project: Understanding Black-Scholes
- Christina Papadimitriou’s project: Diffusion and Reaction in Catalysts

- Problem Solving
- Borsuk-Ulam and Fixed Point Theorems
- The Cantor Set
- The Jordan Curve Theorem
- My oldest plays the piano!
- How many hands did Ernie shake?
- A geometric fallacy
- What is the next number?
- Remainders
- Probability and Divisibility by 11
- Convex triangle-square polygons
- Thieves!
- Metapuzzles
- What day of the week?
- Exact Expression
- Chess puzzles
- Points on a plane
- Sequence of right triangles
- Sums of terms from Fibonacci
- Alleys
- Arithmetic Expressions
- Three circles
- Pick a point
- Bertrand Paradox
- Unusual dice
- El País’ weekly challenge
- Project Euler with Julia

- LaTeX

### Categories

### Archives

- November 2014
- September 2014
- August 2014
- July 2014
- June 2014
- March 2014
- December 2013
- October 2013
- September 2013
- July 2013
- June 2013
- April 2013
- January 2013
- December 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- November 2011
- September 2011
- August 2011
- June 2011
- May 2011
- April 2011
- February 2011
- January 2011
- December 2010
- May 2010
- April 2010
- September 2008
- September 2007
- August 2007

### @eseprimo

- RT @amuellerml: For those that haven't seen it, matplotlib 2.2 has a layout engine. Update now and set figure.constrainedlayout.do to True in… 1 day ago
- RT @algebrasnotwar: Hey that’s me! twitter.com/uofsc_cte/stat… 1 day ago
- RT @Simon_Gregg: If the little white square is 1, what other numbers can you see? #numbersearch Also... Which one doesn't belong? #wodb htt… 2 days ago
- RT @stevenstrogatz: 10 Books to Spark a Love of Math in Kids of All Ages | MindShift | KQED News (from Feb 22) kqed.org/mindshift/5059… 2 days ago
- RT @Beatriz_Nino: Después de ver toda la gente que lo ha hecho en sus clases, por fin lo he probado con mis alumnos. https://t.co/g7cI6FMkNB 4 days ago
- RT @rasbt: Great tutorial on the PageRank algorithm -- this resource is a bit older, but nonetheless an excellent yet short write-up, inclu… 1 week ago
- RT @JohnDCook: Plastic bag found in the Mariana Trench. Sad, but also interesting. news.nationalgeographic.com/2018/05/plasti… 1 week ago
- RT @bit101: #MayContainGIFs day 15 https://t.co/6Fj5azjbCf 1 week ago
- RT @willhek1: Playing with maths (literally). I found this in a shop for kids. The best way to visualize the volume of an stellated rhombic… 1 week ago
- RT @Rainmaker1973: Watch these @MIT origami robotic metamorphosis exoskeletons, including a foldable robot glider buff.ly/2IDG5pb h… 1 week ago
- RT @p_trivino: Sin duda la construcción más dificil que he logrado Un polígono de n lados girando dentro de otro polígono de m lados con ta… 1 week ago
- Oh wait, *of course* there is a way to do #latex + @lilypondblog in @orgmode_bot Once again, @emacs #FTW https://t.co/RyeNe2YXdK 1 week ago
- Typing music in #LaTeX w/ #musixtex has been a fun ride. But srsly: ain't nobody gottime fo'tha. @lilypondblog #FTW… twitter.com/i/web/status/9… 1 week ago
- RT @mathematicsprof: I SOLVED THE RIEMANN! https://t.co/fLeza37895 1 week ago
- RT @juliomulero: En matemáticas, una serie es la generalización de la noción de suma aplicada a los términos de una sucesión matemática. Es… 2 weeks ago
- RT @fhuszar: Physicists, teachers or geek parents help me: My son asked me "How do we know the Earth is spherical?" and then he said "I wan… 2 weeks ago
- Someone send this to @simongerman600 please twitter.com/carlosandradas… 2 weeks ago
- RT @mathematicsprof: For convergence of sequences of real numbers, there is generally one kind of convergence, but for functions. there are… 2 weeks ago
- RT @Wikingenieria: Diferentes proyecciones de mapas mundiales que distorsionan la Tierra. Vía @CodyPhelan https://t.co/SkIxIj1dKa 2 weeks ago
- So very impressed with @overleaf v2 (in beta so far). Kudos for a well designed tool, guys. Impressed with the lo… twitter.com/i/web/status/9… 2 weeks ago

### Math updates on arXiv.org

- Gradient descent in hyperbolic space. (arXiv:1805.08207v1 [math.OC])
- Measuring Congressional District Meandering. (arXiv:1805.08208v1 [math.HO])
- Possible Worlds, Incompleteness and Undefinability. (arXiv:1805.08209v1 [math.LO])
- R(p,q)-deformed conformal Virasoro algebra. (arXiv:1805.08238v1 [math-ph])
- Geodesically Equivalent Metrics on Homogenous Spaces. (arXiv:1805.08240v1 [math.DG])
- Petrov-Galerkin Method for Fully Distributed-Order Fractional Partial Differential Equations. (arXiv:1805.08242v1 [math.NA])
- Algorithmic and algebraic aspects of unshuffling permutations. (arXiv:1805.08255v1 [cs.DS])
- Mathematics of Smoothed Particle Hydrodynamics, Part I: a Nonlocal Stokes Equation. (arXiv:1805.08261v1 [math.NA])
- Magnetostatic problems in fractal domains. (arXiv:1805.08262v1 [math-ph])
- Colouring Square-Free Graphs without Long Induced Paths. (arXiv:1805.08270v1 [math.CO])

### Computational Geometry updates on arXiv.org

- Large Scale computation of Means and Clusters for Persistence Diagrams using Optimal Transport. (arXiv:1805.08331v1 [stat.ML])
- Rule-Based Drawing, Analysis and Generation of Graphs for Mason's Mark Design. (arXiv:1805.08453v1 [cs.CG])
- Orthogonal Point Location and Rectangle Stabbing Queries in 3-d. (arXiv:1805.08602v1 [cs.CG])
- Efficient constant factor approximation algorithms for stabbing line segments with equal disks. (arXiv:1803.08341v3 [cs.CG] UPDATED)

### sagemath

- An error has occurred; the feed is probably down. Try again later.

Francisco—enjoyed the part i could comprehend, and a big WOW to the rest. Bob T

Thanks Bob! Glad you enjoyed it.