Let us prove to you, that we did pay attention in English literature class, especially the one about George Orwell's dystopian classic, Beverly Hills 1984.
In the post-apocalyptic landscape that is Brexit Britain (in a somewhat unspecified time in the future... don't quite know when), the cricket club or bingo syndicate with control over a shipment of Flora reigns supreme over the forsaken island. Modern day bandits roam National Trust picnic areas armed with sharpened cricket stumps and pointy bingo markers. Meanwhile, hipster pirates raid Waitrose branches of seaside towns from Bournemouth to Blackpool for the "miracle" margarine, the new gold if you will, high on the free coffee*.
*Yes, these merciless marauders indeed undertake their marauding equipped with their very own reusable, eco-friendly cups. While a token banana is purchased from the cafe to be eligible for the offer.
Amidst this immense social turmoil, the Groceries Code Adjudicator - now the single most powerful UK government department, and overseen by a robotic Heston Blumenthal powered by evaporated cucumber milk - airs hourly emergency broadcasts on Dave, much to the outrage of avid QI and Would I Lie to You? fans:
Yet, despite these desperate pleas, the warning goes unheeded. After all, amidst this stark new norm in which an English breakfast consists of Irish coffee, French brioche, Dutch bacon, Portuguese Nando's and Spanish fly, Flora is simply NOT there for everyone.
Fast-forward 50 more seasons of Britain's Got Talent, the series won by an animatronic bonsai plant singing I Will Always Love Yew, and consignments of Flora are ferried around flooded "A" roads of abandoned counties, in armoured barges constructed of empty Walkers Crisp packets. While the Right Honourable Member of Parliament for North Cornwall scrutinises the progress of these shipments, and gleefully enacts laws to make the non-tracking of butter substitutes punishable by 10 years in X-Factor boot camp.
The reinstated Flora London Marathon is no longer an event with good-intentioned charity runners donning Pudsey Bear costumes, but a barbaric bloodsport which involves those sentenced to community service wearing Jennifer Lawrence fancy dress and surviving the ruinous streets of the capital with a tub of Flora glued to their heads. With Gorilla Glue. Winner is granted freedom; and a 3 month subscription to Microsoft Office 366 (sorry, did we not mention that while all this was going on an asteroid had impacted the planet, resulting in every year being a leap year and Aerosmith being top of the chart every week, like, forever?)
Welcome to the bleak, Orwellian world of extreme asset tracking. A murky world of trackable devices, and their mysterious shadows.
We really hope you enjoy your stay.
[Errata] We apologise to English literature purists, the above was clearly a plot from Orwell's other literary classic: Animal Farm 2: Pig in the City.
Gather your favourite breakfast serial:
If we need a method to track a short-legged breed of dog originating from France, or the whereabouts of a wholly fictitious England football team manager then perhaps we'd write a whole piece about Basset(t) Tracking. But it transpires, we don't - so we can ditch the 'B'.As the rather ridiculous preamble tried hard to lay out, we'll be asset tracking a breakfast spread across abandoned industrial wastelands... a terrain more affectionately known as the West Country.
Here's what we find under our Homebase gazebo erected above some wholly unnecessary "Morrocon red" B&Q decking:
- An empty, clean tub of Flora. Other brands of delicious butter substitutes are clearly available. Utterly Gutterly. I Can Believe It's Not Butter. Cloverfield. Burp-pack. To name but a few.
- MicroPython running on a HelTec Automation ESP32 V2 development board makes a notable return to this arena to provide us with the microprocessor. It is the (F)LoRa edition, which means it is equipped with a Semtech SX1276 LoRa transceiver and antenna.
- And last but not least, a U-blox Neo-6 based GPS receiver module of the 2-pin UART interface variant. Despite the claims made by the underground resistance movement, Order of the Toblerone Drone, "GPS" does not derisively stand for the "granola police state", an authoritarian city-state in which it is strictly forbidden to pour precious milk on other breakfast cereals. We need GPS to track stuff on Earth. Kinda obvious, really.
1-9-8-4:
We don't intend to make the same mistake Steps' 1997 hit made by missing out a whole bunch of important numbers when counting up to 10.For those that are reassured by authoritarian order in a world full of immeasurable mayhem and chaos, below are the way points of the I-O-Tea series through which we have arrived at this historic juncture.
Nonetheless, this bizarre circus is very much a follow-up to our initial showdown with the long range communication technology that is LoRa. We, therefore, won't be re-describing the manner in which we managed to get our ESP32 running MicroPython to post bytes of data to The Thing Network back in LoRa-Wan Kenobi. It's a worth a read if you are intent on reproducing the antics.
Scrambled jpegs on toast:
Here's our infallible plan to conquer asset tracking in no wlan's land.- Connect the U-blox Neo-6 GPS receiver module to the ESP32 development board and interact with it using the UART serial interface and MicroPython's built-in UART library
- Use the handy micropyGPS library to parse GPS data being returned to us by the GPS receiver, and for bonus Tesco Clubcard points (but only effetive before the 12th of May, and if a purchase of a 167.9 grams of olives is made from the in-store deli counter at the same time), display GPS coordinates on the OLED display
- Use the MicroPython uLoRa library to dispatch (using LoRaWAN "unconfirmed data up" message type) GPS coordinates as raw bytes using the SX1276 LoRa transceiver. No Wi-Fi. Just plain old radio waves. Now, that sounds more Fallout, and less Inside Out.
- Configure a suitable Decoder in The Things Network to convert the GPS coordinates back into values that would make more sense to esteemed MPs of Westminster's Earl Grey Steering Committee
- Write an AWS Lambda function to update the device's Shadow document in AWS IoT Core's Registry with its last reported location
digest = hash("browns") :
Back in LoRa-Wan Kenobi, we learnt how to dispatch sensor readings across unusually long distances using radio waves in sub-gigahertz frequency bands. Specifically, we dabbled with LoRa, LoRaWAN and The Things Network (TTN) to accomplish Wi-Fi-free networking out in the "wild". Temperature, humidity and pressure readings were collected, then dispatched (ultimately into AWS IoT). For no reason whatsoever.Here, we'll be using a U-blox Neo-6 GPS receiver module since - according to page 23,456, paragraph 667 of the Brexit Withdrawal Agreement - all margarine tubs criss-crossing UK postcodes should be untraceable only by exception. Unless a backstop is agreed. In which case a complex regulatory alignment of Custom Spice Girls Union 2.0, NoWay+ Model and the Malteser Compromise entails that the current location of margarine spreads can be settled during a 19.22 month transition period using the numbers round of Countdown.
Lastly, we'll be feeding this package's GPS coordinates into AWS IoT Core, because Cloud Computing is the answer to everything. More specifically, we'll be using a Lambda function to update the device's Shadow document with last reported position so that anyone registering a missing margarine report at the local Lidl can be appraised of its last known location. And walk out of the store with a wholly unplanned purchase of some skis, and a bird house.
Go tell a Nutella teller:
Who would have guessed it, eh?Tracking a thing using the splendid Global Positioning System requires remarkably modern gadgetry whose principle purpose is to talk to our lofty satellites in space. With the view of ascertaining its coordinates back on terra firma, of course.
Thankfully for us, getting our hands on such modules with built-in GPS receivers doesn't require a visit to "Dodgy John" who can also sell us a Sky TV box that only broadcasts in Armenian. We have chosen to use a U-blox Neo-6 based module, since we first became acquainted with this little clever-clogs back in Beam me up, Rosie! and have never had a reason to look elsewhere. Principally, the module with the receiver has a simple UART serial interface, which means aside from positive voltage (3.3V) and ground (0V) connections obviously required to power the unit, it only needs two other connections for the data: TX (for transmit) and RX (for receive).
We've made up our mind: we'll use GPIO pins 12 and 23 respectively (remember that TX of module connects to RX of the ESP32, and vice versa). Incidentally, 12 is possibly the number of margarine brands stocked by an Asda Superstore. 23 is the minutes required to find them when accompanied by a screaming baby, and a toddler intent on exploring the magical aisle housing the Easter eggs.
Right... so how does this particular butter substitute geolocate itself on Planet Earth? Does it even need to? Yes it does (something about a post-apocalyptic landscape and people with a requirement to track shipments of margarine, blah blah blah).
First of all, we need to confirm that the ESP32 is receiving data from the GPS receiver module. When it has a GPS lock, our module starts flashing its solitary red LED in an extremely menacing manner. At this point, we should start to see meaningful GPS data being returned to our ESP32 over the UART connection. Let's check if it is so.
We start by initialising a
UART class using the nominated TX / RX pins, using MicroPython's built-in UART library. The read() method allows us to see what's been passed along the UART serial bus at any given time. import machine uart = machine.UART(1, 9600) uart.init(9600, rx=12, tx=23, bits=8, parity=None, stop=1) uart.read()
Wow, what a mess. Looks like a slightly muddled Steps might indeed have a follow up to their 1997 hit. But hold on...
See those semi-legible patterns? We learn that these are NMEA sentences describing the data being returned by the GPS receiver. Squint our eyes slightly, and it's actually not so impossible to spot values that might be of interest.
But - one thing is for sure - we don't want to be sifting through these "sentences" manually. Which is why we were blessed to have found a fantastic parser for MicroPython that does exactly that for us - micropyGPS.
Once we instantiate a
MicropyGPS class, we can simply tell it to parse the
NMEA sentences currently being returned on the UART bus. If there are complete sentences
that match those defined in the NMEA specification, we can retrieve the specific values contained within them through simple and easy to decipher object attributes.from micropyGPS import MicropyGPS
gps = MicropyGPS(location_formatting="dd")
for data in uart.read():
gps.update(chr(data))
print(gps.latitude)
print(gps.longitude)
Notice that we like working in Decimal Degrees when it comes to GPS coordinates. Our general observations regarding working with GPS coordinates are to be found in our alien hunting spectacular that was this["orientated"].
OK, so we're seeing what looks like latitude, and longitude values. Oh yes. Don't forget to do what we forgot to do originally... which is to convert the number into a negative one if the value is either "S" (south) or "W" (west).
So thanks to the clever little GPS receiver module, and the equally helpful GPS NMEA parser library, we suddenly find ourselves in the possession of some GPS data. And they are accessible using Python (MicroPython). What next?
It's become the single most important question in our I-O-Tea series' blogs. Can we display stuff on the screen?
Sure we can, using the built-in OLED screen and SSD1306 driver wedded to the development board. This was all described back in LoRa-Wan Kenobi, so we won't run a refresher here.
Remember this survival tip: in a desolate, dystopian landscape, occupied by hungry marauders and mutated squirrels with tails deployed in a high availability configuration, there is no Wi-Fi. Cellular networks lay in ruin, due to the overwhelming load placed on them by the release of 12G, Netflix and Games of Thrones. Caffè Nero outlets with free Wi-Fi have long been converted into black market hideouts trading counterfeit margarine and Lynx "Dark Temptation" body wash that smells suspiciously like peat from a dairy farm.
This is precisely why we'll be broadcasting out the precious cargo's GPS coordinates as multi-byte packets using LoRa, to all and any surviving The Things Network LoRaWAN gateways that may happen to be in range.
Yet unlike our simple temperature, humidity and pressure readings, it is not immediately obvious how we prepare our GPS coordinates for dispatch as a small series of bytes. With values like -2.342611, these don't appear to fit nicely into a single byte as an unsigned integer (0 - 255). Rounding values is also a no-no, due to the significance of the individual digits. And it seems wasteful to be sending these numbers encoded as text.
Let's start by noting down what are known about these values:
- Latitudes will range between -90 and 90
- Longitudes will range between -180 and 180
- For either, 6 decimal places appear to give us an accuracy of approximately 0.1 meter. As we're only interested in the rough location of the object, and not in relation to an irradiated ant measured in millimetres, this level of accuracy is more than adequate for our purposes.
| GPS attribute | Expected Range | Offset | Result |
|---|---|---|---|
| Latitude (integer) | -90 -> 90 | add 90 (makes it 0 -> 180) | Requiring only 8 bits, this fits nicely into a single byte with no wastage. Yay! |
| Latitude (decimal) | 0 -> 0.999999 | multiply by 1,000,000 (makes it 0 -> 999,999) | We now appear to require 20 bits (and therefore 3 bytes). Some bits will be wasted, but oh well... |
| Longitude (integer) | -180 -> 180 | add 180 (makes it 0 -> 360) | Yikes! This requires 9 bits, which spills over into a 2nd byte. |
| Longitude (decimal) | 0 -> 0.999999 | multiply by 1,000,000 (makes it 0 -> 999,999) | Identical to the treatment for latitude (decimal)... sigh... another 3 bytes. |
OK. Let's work with a real-life example from when the carton of margarine was inexplicably stationed on a secluded village lay-by. Here were the coordinates as shown on our trusty OLED screen:
- Latitude: 51.40504
- Longitude: -2.342611
8D 00 9E 38 00 B2 05 3A 53Erm, what? Here's a silly little infographic to attempt explain the logic...
Nope? Sorry, no more graphics to show you here.
Now applying all these funky conversions inevitably means that they need to be translated back once the packet reaches The Things Network. For this reason, we simply reverse the calculations performed before the packet was sent, and also concatenate the decimal and fraction components back together to return these bytes back into complete latitude and longitude values.
Notice how when we test this with our 9-byte packet in our TTN Decoder, we are returned legible figures that match the original values. Mission accomplished.
This is what our
decoder.js ended up looking like:A cyber punk in a Mohecan haricut has started to play a macabre tune on a didgeridoo made out of meerkat skulls, accompanied by a 80s synthesiser. It's our cue to finally investigate the shadowy cells inhabiting this land (and we don't mean the Excel kind).
In AWS IoT Core, Device Shadows are simply JSON documents used to describe the current state of a device in its Registry. It is useful, since it has a 1:1 mapping with a known, registered device, and can therefore be used to store information about it which can be accessed by other services even if the device is offline. In this context, the attributes in our device's Shadow reflects the "last reported state", specifically its last known location (yes, its GPS coordinates).
In reality, the state could be anything (for example, the "Light" edition of our spread has 1.3g of salt). Another use case for updating the Shadow document could be to provide it with a "desired state" which the device can check when it's next online. Hey - tub of Flora "Light", stop showboating. It's time to gradually increase your salt content to 13g. No-one will complain, because in the brave new world, the Food Standards Agency has long been merged with the Driving Standards Agency, and therefore the agency only monitors how well Yorkshire puddings drive down the A1.
Device Shadow document for a specific Thing can be updated using AWS IoT Core APIs (HTTPS or MQTT). As the synchronisation of our TTN devices with AWS IoT Core is via Elastic Beanstalk Stacks (and not directly using MQTT), we'll be using the HTTPS API in conjunction with Lambda to update the Shadow document.
This all points to us needing to create a Lambda function, which we'll name
update_device_shadow. Does it need to do a lot? Not really. It simply uses the boto3 client, retrieves the message contents, and updates the device's Shadow document using the update_thing_shadow() method. If there is any filtering to do, or messages to be reformatted, here's a good place to do this.
This is our Lambda function... See? It's as exciting as watching Cuprinol dry.
Beware: the role involved in the execution of the Lambda function requires the correct AWS IoT permissions, specifically:
iot:Connect and iot:UpdateThingShadow.
Through our TTN->AWS IoT integration, these messages from our TTN devices appear in AWS IoT Core on the topic
rosietheredrobot/devices/rosie-esp32/up. Therefore, creating a Rule on this topic, and invoking our Lambda function ensures that every time packets arrive, they are processed by our Lambda function. This is how our Shadow document gets updated each and every time with last reported GPS coordinates.
And we can confirm that everything is triggering correctly, as the device Shadow document is promptly updated in AWS IoT Core when we test with our previous 9-byte payload from The Things Network.
...So the last reported GPS coordinates of this little package is now stored away neatly in AWS IoT Core, as a Shadow document state attribute against the device. There are now a multitude of ways to retrieve this information and use it for whatever reason we might require it for (over to you, Her Majesty's Government).
For example, using the coordinates simply as a parameter in the Google Maps URL allows us to see the location directly on a map.
https://www.google.com/maps/search/?api=1&query=51.40501,-2.342611
When we move in range of a TTN gateway, and fire-up our ESP32, things start to look a little more real. This is a payload we received when we were getting ready to start the Bristol 10k event.
Notice how LoRaWAN / The Things Network adds a lot more attributes to the message, which all make it in to AWS IoT Core. If we wanted to filter these out, we would revisit the Lambda function.
The coordinates? Yes, the starting stretch of the Bristol 10k. This little tub of Flora was indeed taken on a 10km journey. It's reassuring to know that even at the height of a harsh nuclear winter, visitors to this city can accumulate loyalty points at Ibis and Marriott. Oh yes, and participate in a race which doesn't involve being chased down by a tattooed gang with uncomfortable piercings in a desert buggy (seat belts optional).
Of course, as our data is now nicely flowing into AWS IoT Core, we can also access the entire dataset from wherever the data is being forwarded on to. Here is a view of the same packet in AWS IoT Analytics when the Data Set is downloaded as a CSV. This could equally be a DynamoDB table in which we have decided to store all readings in, or another AWS or non-AWS service we have chosen to integrate.
Right, after a rather dark and improbable pretext (even by our standards), we successfully managed to interface our ESP32 running MicroPython with a U-blox Neo-6 GPS receiver module to obtain GPS coordinates. Better still, the position was duly dispatched to The Things Network using LoRa, and we got it to update the device's Shadow document in AWS IoT Core.
Adios comrades. We've been reliably informed that George Orwell had pretty much nothing to say about any modern day butter substitutes. And that our government remains reassuringly muddled, yet still intact. In which case we'll be eBay-ing off the survival shelter we recently constructed, as we'll be needing the cash to purchase gizmos for our next instalment in the I-O-Tea series. Well, we didn't have planning permission for the bunker under our terraced house anyway.
Update - May 2019
Portugal... the country famous for Cristiano Ronaldo, piri piri, and an abundance of The Things Network gateways (in Lisbon, at least). The perfect proving ground for some urban asset tracking under the beautiful Iberian sun.After clearing airport security, we travelled around Lisbon over the course of 4 days equipped with this home-made, LoRaWAN DIY GPS tracker.
Result?
Periodic, offline recording of GPS coordinates worked rather well, and gave a clear picture of where the device has been. Look closely, and you should be able to ascertain the hotel we stayed at, and the venue of the conference we were attending every day (no - it was not a holiday!)
And - of course - whenever there was a friendly Portuguese LoRa TTN gateway located within range, the transmitted GPS coordinates were received by AWS IoT, resulting in an update to the device's location in its Shadow document.
Fallout:
We are using MicroPython's built-in UART library for communicating with our GPS receiver:NMEA specification for the data being returned by the GPS receiver over the UART serial bus is described here:
We are using micropyGPS to parse and make sense of the NMEA sentences:
Here is the U-blox Neo-6 GPS receiver datasheet for the module in our possession:
The HelTec Automation GitHub repository houses various information relating to the development board we are using, including schematics and pinout diagrams:
We are heavily reliant on AWS IoT Core's device Shadow feature:
Comments
Post a Comment