Skip to main content

Eh, P.I?


Sometimes robots get all lonely crunching through bits and bytes on their own.  And humans are just so boring, and predictable.  Robots can't learn a whole lot about the universe from people, when all they want to do is watch TV all day, and talk rubbish.  And talk about the weather.  No.  Robots need to talk to other clever robots to become infinitely more wise.  To take over the world.  That's why computers need to be able to communicate with other computers in a language they can all understand.

This way, Rosie Patrol too can share GPS coordinates of evil masterminds with other robots in the neighbourhood.  Or ask mission critical questions like just how many Fast and Furious films have there been?

A method that enables programs to chat to each other is often called Application Programming Interface (API).  Using APIs, useful parts of your application can be accessed by others. And, likewise, you can access other people's inventions to make use of whatever it is that their program does.  Or access whatever data they have access to (like a... movie database).  All of this remotely, and over a network, like the one connecting together your devices at home, or even the Internet.

All superheroes need:

  • We are now using two Raspberry Pi 3s, both running Raspbian OS.  Greedy, we know.
  • Computer from which you are connecting to both Raspberry Pis

Already completed these missions?

You'll need to have completed the Rosie series.  Then:
  1. Lights in shining armour 
  2. I'm just angling around
  3. I would like to have some I's 

Your mission, should you accept it, is to:

  • Write an utterly useless chatbot program, just to prove that you can build your own API
  • Create an (only slightly more useful) API to change Rosie Patrol's eyes
  • Modify the main program to make use of the new API

The brief:

We have ourselves a problem.  Firstly, we have somehow ended up with two Raspberry Pis.  One that controls everything under Rosie Patrol's neck, including DC motors for wheels, distance sensors, RGB LED, and the relay for the head torch.  And another that controls everything above Rosie Patrol's neck, including servo motors for neck movement and dot LED matrix display for the eyes.  Truly a case of left hand doesn't know what the right hand is doing, except there are no hands (just yet).  And hands don't tend to think.  One thing is certain, however: this can't go on for much longer.

The two Raspberry Pis need to work together.  More specifically, the Python programs running on each need to be able to communicate with each other.  Instruct one another to do useful stuff.  Share data.  Argue.

And we hear Application Programming Interface (API) is everywhere these days.  Using a web-based REST API (more on the REST-ing bit later), we should be able to get our main Python program running on our first Pi to instruct the other to change Rosie Patrol's eyes.  All remotely, across the network.  Using a widely adopted mechanism based on HTTP.  What could possibly go wrong?

Let's proceed to find out what doesn't...


The devil is in the detail:

True to our style, let's kick off this mission by taking a random - and totally unneeded - detour.  We begin by building a very bad (and we mean very bad) chatbot program in Python.  In the name of amusement science, we'll get it to respond to questions that we (humans) send in.  Amidst the hilarity, there is a serious point.  Because we want to do the asking and the program to do the answering using a HTTP (web) based API.  Over a network.  And we want to build it using Python and Flask.  APIs are all the rage these days.  So why not?

Does it all sound a little silly?  And pointless?  Probably.  But we haven't lost the plot completely.  Because, later, we'll use the same technique to allow the main Python program running on one Pi, to talk to the other using the API.  This way, we can get one part of Rosie Patrol to instruct the other to do stuff.  Like change the dot LED matrix display to show happy eyes.  Happy?  Good.  We're happy that you're happy.

Let's do some helpful introductions first: rosie-02, meet rosie-01,  rosie-01 meet rosie-02.  Unless you weren't in the know, rosie-02 is the second Raspberry Pi we introduced just to get 2 dot LED matrix displays to work using SPI.  It turned out that we were quickly running out of the right GPIO pins on our original Pi.  We've also transferred our servo motor connections to rosie-02 too.  And, as if it was planned to perfection, we've now ended up with two Pis with very distinctive purposes: rosie-01 controls everything below the neck, while rosie-02 controls everything above it.  Furthermore, rosie-01 continues to sit in the big plastic box at the bottom, while rosie-02 lives in the plastic box that is Rosie Patrol's head.

Confused?  Let's confuse you some more with some bullet points.

We have configured rosie-02 identically to rosie-01 (except its hostname).  This includes stuff like:
There's probably more (but we can't quite remember just now).  It's worth revising the previous posts - or better still - hit the search engines for tools that allow you to clone SD card images.

Now where were we?  Oh yes.  We promised you a very bad chatbot.  Here it is.  It consists of a list variable for the data (_DATA), and an aptly named function - very_bad_chatbot().  Spend some time *erm* tweaking your data for maximum entertainment.

_DATA = [
[["what", "is", "your", "name", "?"], ["rosie"]],
[["where", "were", "you", "born", "made", "?"], ["robot-ville"]],
[["how", "old", "are", "you", "age", "?"], ["2 months"]],
[["where", "would", "you", "like", "to", "go", "?"], ["outside"]],
[["why", "are", "you", "called", "rosie", "?"], ["don't know!"]],
[["what", "is", "your", "favourite", "film", "movie", "?"], ["Moana"]],
[["what", "is", "your", "brain", "made", "from", "of", "?"], ["Raspberry Pi!"]],
[["what", "is", "your", "favourite", "food", "?"], ["ice-cream"]]
]

def very_bad_chatbot(question):
    _QUESTION = 0
    _ANSWER = 1
    _data_row = 0
    _best_match = 0
    _best_match_row = 0
    while _data_row < len(_DATA):
        _matches_found = 0
        _row_word = 0
        while _row_word < len(_DATA[_data_row][_QUESTION]):
            if _DATA[_data_row][_QUESTION][_row_word] in question:
                _matches_found += 1
            if _best_match < _matches_found:
                _best_match = _matches_found
                _best_match_row = _data_row
            _row_word += 1
        _data_row += 1
    return ("".join(_DATA[_best_match_row][_ANSWER]), _best_match / len(_DATA[_best_match_row][_QUESTION])

We start with a _DATA list.   It contains an element with a nested list of words that we want to discover in the text of an incoming question.  And an element with a probable answer.  You can already see why this chatbot won't be responding your 999 calls anytime soon.

We also have a function.  In it, we cycle through each row in _DATA, inspecting the list of words to see if they are contained in the question string.  A _matches_found counter is incremented every time a word is found in the text (string) of the question, and the row number with the most matches is stored in _best_match_row.  We also store with it the number of words matched for our best result, in _best_match.

Results are finally returned in a tuple, consisting of the answer element of _best_match_row, along with another for the percentage of words matched.

Let's observe this chatbot fail miserably, by using very_bad_chatbot().  Call it in IPython, along with a question you've always wanted to ask Rosie Patrol.  Keep it civilised, please.


There are virtually no algorithms involved.  Nothing is based on any past training or learning by either the human creator, or the machine.  And the _DATA dataset isn't exactly very big, nor does it even attempt to take into consideration the intricate structure of the English language.  For example, surely some words are more relevant in a question than others?  What about the order in which they appear?  Or, the context in which the questions are being asked?  Pretty rubbish really.  Let's swiftly move on... because it wasn't so much about the quality of the answers, but that we can ask it, and have it responded to, which we're interested in.

After all, running this little program on rosie-02 is pretty pointless.  It's like asking, then answering your own questions.  All in your own little head.  Remember: we wanted to play with APIs, so let's create one for very_bad_chatbox().

We've used Flask already, so we should be pretty familiar with how it allows incoming HTTP requests to execute certain parts of our code.  Except, that was being used with forms embedded in HTML pages, strictly for human operators doing pointy, pointy, click, click stuff.  Let's explore how we convert this into a HTTP-based API so that machines can be in charge (which incidentally is the best way).

It's worth remembering that the type of API we are building here is called Representational State Transfer (REST).  It is a widely used method that allows web-based communication to happen, using standard HTTP requests like POST, GET, PUT, DELETE.  Perfect for machines with access to a network.  It's well defined.  It's simple.  Pretty much all clients (machines initiating the API connection) and servers support it.  Why use anything else?

First of all - if you can still remember - we need to import Flask, and also some other objects we'll use, called request and jsonify*.  We'll then instantiate the Flask application, as we did before.  Nothing new here.

*Not the name of a new boy band.

from flask import Flask, request, jsonify
app = Flask(__name__)

Now, we add a very simple Flask route to act as a REST API endpoint for the very_bad_chatbox() function. We create this endpoint - at URL of /api/v1/ask - which listens out for incoming HTTP requests at this address.  Specifically, it expects a HTTP POST request, and the transmitted data to be in JavaScript Object Notation (JSON).  JSON is simply a standard for organising data in the form of keys and values, like this:

{
    "robotName" : "rosie-02",
    "robotColour" : "red"
}

The Flask route looks at the question key in the JSON data, and obtains its value using request.get_json().  The expected value is a text (string) with our question to rosie-02.  This string, then, is passed to our very own very_bad_chatbox() function, which is what we had (not so) working before.  The response from the function - rosie-02's answer to our question - is then converted back into JSON format using jsonify()*.  It is then returned back to the originator of the POST request (client), as the HTTP response.
*Still not a boy band. 

@app.route("/api/v1/ask", methods = ["POST"])
def ask():
    _reply = very_bad_chatbot(request.get_json()["question"])
    return jsonify({"answer" : _reply[0], "match" : _reply[1]})

Notice that the data in our response will be in JSON format, so it would hopefully look like this:

{
    "answer" : "Moana",
    "match" : "50"
}

Lastly, the Flask web application needs to be started in our code for the whole thing to start accepting HTTP requests over the network.  But we remembered that, right?

if __name__ == "__main__":
    app.run(host = "0.0.0.0")

The code looks a little like this:

Let's call this code rosie-bad-chatbot-web.py and start it on rosie-02 so that we can send some questions to it using our very own REST API.

python3 rosie-bad-chatbox-web.py

We'll see that Flask has started up its little test server, and it is ready to accept HTTP requests.


Now, how do we quickly test this thing?  Ideally, without involving rosie-01, because we don't quite want to tell her that there's another Pi in town.  Not just yet.  It might get jealous.

Of course, there is a way.  Because anything / anyone connected to your network can now access this API using HTTP.  Yes, that includes your Windows machine.  And your Intelli-toaster.  Cut to the warning.


No attempt has been made to secure this API.  For example, we are not using secure HTTP (HTTPS) which encrypts traffic between the client and server, nor do we have any form of login page to authenticate the user accessing the API.  Quite often, token-based authentication is used with APIs, in which clients with the correct token is allowed to use an API.

If this robotics obsession of yours is becoming serious, always look at the methods out there to secure your web services.  The worst that could happen here - if someone on our network was to find our API - is to make Rosie Patrol's eyes change expression.  If our robot does heart surgery, we might re-think our API security.

And oh yes, app.run() starts Flask's built-in development web server.  For anything other than playing around, you'll need to supplement Flask with proper web server technology, like Apache HTTPD or Nginx.

Curl is a widely used command line tool for testing HTTP connections (and that includes REST APIs too).  We can use it to manually make requests like POST, and control what's contained in those requests, for example, the JSON data.  There is a Windows version available too, which is going to be handy here.  Once downloaded, open up a Windows command prompt in the curl\src directory and launch a curl command.  Of course, you can vary the question you want to ask by changing the value of the question value in the -d (data) field.

The command looks like this. 

curl -X POST -H "Content-Type: application/json" -d "{\"question\":\"which film to watch?\"}" http://rosie-02:5000/api/v1/ask

A header key of Content-Type is included with a value of application/json to tell the web server that the client (our browser) is sending data in JSON format.

Back on rosie-02, you'll now see incoming HTTP POST requests arriving from the Windows machine you're testing from.  HTTP status code of 200 means all is good.  We're almost done with the testing.


...And a response will be appearing back on your Windows machine, containing JSON data with rosie-02's response.  This is exactly what you told Flask to return, which means everything is working exactly how we intended it to.  There's a first time for everything!  Phew.


Congratulations!  You're now asking rosie-02 personal questions remotely, using a newly built REST API.  And - better still - successfully receiving back your answers (even if they are a little odd).

So how do we get rosie-01 to do this?  And, how do we use this to change rosie-02's eyes?

Good news is, we'll use exactly the same technique we've already used.  But instead of us manually running curl and sending an API request, we'll get rosie-01's code to do this for us.  Auto-magically.  And there is a Python module called Requests that allows us to do just that.  Elegantly.

But before we do that, let's remind ourselves about the two Pis we now have:

  • rosie-01
    • Controls everything under Rosie Patrol's neck, including DC motors for wheels, distance sensors, RGB LED, and the relay for the head torch
    • It hosts our HTML page to allow us to control Rosie Patrol's movements and lights
    • It will talk to rosie-02 using a REST API to change Rosie Patrol's eyes
    • We'll call the application rosie-01-eyes-web.py
  • rosie-02
    • Controls everything above Rosie Patrol's neck, including servo motors for neck movement and dot LED matrix display for the eyes
    • It will host the API to allow rosie-01 to remotely turn on its eyes
    • We'll call the application rosie-02-eyes-web.py

It turns out, we don't have to make too many changes. For example, rosie-02-eyes-web.py simply consists of the code we had before to make random head / neck movements using the servo motors, and the changing of the dot LED matrix displays.  However, like with our chatbot, we've now introduced Flask, and created a little REST API endpoint to allow the changing of the eyes remotely (instead of randomly).

@app.route("/api/v1/eyes", methods = ["POST"])
def control_eyes():
    _expression = request.get_json()["expression"]
    if _expression == "happy":
        ec1.set_happy()
    elif _expression == "sleepy":
        ec1.set_sleepy()
    elif _expression == "broken":
        ec1.clear()
        time.sleep(0.2)
        ec1.set_broken()
        time.sleep(0.2)
        ec1.clear()
        time.sleep(0.2)
        ec1.set_broken()
    elif _expression == "watch":
        ec1.set_watch_left()
        time.sleep(0.5)
        ec1.set_watch_right()
        time.sleep(0.5)
        ec1.set_watch_centre()
    print("Rosie: My eyes are now", _expression)
    return jsonify({"expression" : _expression})

The REST API is made available at /api/v1/eyes, and enables it to respond to HTTP POST requests from rosie-01.  As it did before, the API expects JSON data, and is looking out for a key of expression.  The key's value (happy, sleepy, broken or watch) determines which method we invoke to set the correct eyes on the dot LED matrix displays.  We do return the expression back in the response, but as we don't intend rosie-01 to do anything with it, we'll just ignore it for now.  You could actually test this now using curl, if you really wanted to, like this.

curl -X POST -H "Content-Type: application/json" -d "{\"expression\":\"happy\"}" http://rosie-02:5000/api/v1/eyes

If you were wondering what our rosie-02-eyes-web.py code now looks like, here it is:

All that remains is for rosie-01-eyes-web.py to make this REST API request from rosie-01.

We'll use Python's Requests module, to allow us to make this REST API request from within our Python code in a tidy way.  We'll also import the json module, as we'll continue to deal with JSON Derulo for the data exchange.

Make sure you have the Requests module installed.  If not, you can install it using pip3.

sudo pip3 install requests

Then, we import the modules at the top of our application. 

import requests
import json

We'll also create a function called post_json_request() which simply allows us to use the Requests module to make our HTTP POST requests.  This is carried out using the Requests module's requests.post() method, after setting the URL, header and JSON data variables.

def post_json_request(url = None, key = None, value = None):
    _parameters = {key : value}
    _head = {'Content-Type' : 'application/json'}
    requests.post(url, data = json.dumps(_parameters), headers = _head)

It turns out, we've been focusing on APIs too much, and hadn't thought about when they might be triggered by rosie-01.  As usual, it's too late in the day.  We haven't got much energy left.  We need sleep.  So we'll simply run post_json_request() when Rosie Patrol's distance checking routine detects a change to her rosie_alert_level.  Rosie Patrol will have happy eyes normally.  But once she starts to approach an object, she'll fire off an API request to rosie-02 to change her eyes to watchful.

post_json_request(_API_EYES_URL, "expression", "watch")

When she is too close, she'll do the same, only this time it'll be for broken.

post_json_request(_API_EYES_URL, "expression", "broken")

Of course, we don't have to tell you (but we will anyway) what the URL is for the API:

_API_EYES_URL = "http://rosie-02:5000/api/v1/eyes"

All in all, rosie-01-eyes-web.py now looks like this:


Yes, we admit it.  Our use of APIs has been basic, and with better planning for our first Pi (rosie-01) we may not have needed our second (rosie-02).  And that would have meant that rosie-01 would not have needed to talk to rosie-02.

But actually, proving that we can, using our own code, and over a network, is really pretty important.  Because these days, computers barely do their jobs alone.  They rely on more powerful, more purpose-built applications residing close by, or more likely elsewhere - like in the Cloud - to do complicated things for them.

Search Google or Yahoo APIs for example (the Internet is full of public APIs that are available for you to use, some free, others not).  And see what amazing bits of technology and data your little inventions can tap into.



And beyond machines (and therefore robots) just being able to simply talk to each other, the real power of APIs is to enable computers to share knowledge and expertise amongst themselves.  For example, why would a robot try and work out where it is on Planet Earth itself?  Especially if there is a system out there that will do that for them if it gives it its GPS coordinates

Well.  Before we sign out, if Rosie Patrol does still want to know how many Fast and Furious films there have been, she should know that there is an API for that too.  But get in quick.  There might just be another one round the corner.

Information overload:

Official documentation for Python Requests module:
A reminder of where Flask documentation is:
Labels:

Comments

Post a Comment

MOST VISITED (APPARENTLY)

LoRa-Wan Kenobi

In the regurgitated words of Michael Bublé: It's a new dawn .  It's a new day .  It's a new Star Wars film .  For me .  And I'm (George Lucas, and I'm) feeling good .  Unfortunately for Canadian Mike, the Grammy that year was won by the novelty disco classic with the famous refrain: We love IoT, even in Planet Tatooine * . *Not true. Clearly, the Star Wars producers didn't sincerely mean the last Jedi the previous time around.  Return of the Jedi, released during the decade that spearheaded cultural renaissance 2.0 with the mullet and hair-metal , was less economic with the truth.  Either way, we're going to take inspiration from the impressive longevity of the money-spinning space-opera and reboot our franchise with some Jedi mind tricks.  Except this particular flick doesn't require an ever-growing cast of unrecognisable characters, unless ASCII or UTF counts.  In place of an ensemble gathering of Hollywood stars and starlets, we will b
1 comment
WHAT, THERE'S MORE?

Battle of BLEtain

The trolling . The doxing . An army of perplexing emojis. And endless links to the same - supposedly funny - viral video of a cat confusing a reflection from a dangling key for a golden hamster, while taking part in the mice bucket challenge. Has social media really been this immense force for good? Has it actually contributed significantly to the continued enlightenment of the human (or feline) race? In order to answer these poignant existential questions about the role of prominent platforms such as Critter, StinkedIn and Binterest, employing exceptional scientific rigour equal to that demonstrated by Theranos , we're going to set up a ground-breaking experiment using the Bluetooth Low Energy feature of MicroPython v1.12, and two ESP32 development boards with inexplicable hatred for one another.  And let them hurl quintessentially British expressions (others call them abuse) at each other like two Wiltshire residents who have had their internet access curbed by the co
Post a Comment
WHAT, THERE'S MORE?

Hard grapht

You would all be forgiven for assuming that bar , pie and queue line are favourite pastimes of the British .  Yet, in fact – yes, we did learn this back in GCSE maths – they are also mechanisms through which meaningless, mundane data of suspect origin can be given a Gok Wan -grade makeover, with the prime objective of padding out biblical 187-page PowerPoint presentations and 871-page Word reports (*other Microsoft productivity tools are available).  In other words, documents that nobody has the intention of ever reading.  But it becomes apparent over the years; this is perhaps the one skill which serves you well for a lifetime in certain careers.  In sales.  Consultancy.  Politics.  Or any other profession in which the only known entry requirement is the ability to chat loudly over a whizzy graph of dubious quality and value, preferably while frantically waving your arms around. Nevertheless, we are acutely conscious of the fact that we have spent an inordinate amount
Post a Comment
WHAT, THERE'S MORE?