Over the weekend, my friend Mark Omo (markomo.me) and I participated in the Hack Arizona 2016 (hackarizona.org) hackathon, creating a drone control platform we called Drone Flight Area Control (GitHub). Much to our excitement, we won not only first place overall, but best in software and third place in the Raytheon drone category. So, here's the writeup we did for the competition (also on Devpost), plus some additional notes.
Personally, I wrote the frontend, as has often been the case in projects I've worked on. It built on techniques I had used in creating an early demo for a search and rescue application, using the Google Maps API to bring an interactive and detailed map into the interface with our own data on top of it. This was actually the first functional non-standalone webapp I've made, so seeing it function as expected was very exciting for me. It was also my first time using jQuery, which I had been reluctant to learn for various reasons. However, seeing as how it is one of the leading web frameworks, learning what it is and how it works was definitely worth the time, and it definitely saved me time. The big challenge we hit was efficiently exchanging data between the frontend and backend. Originally, the plan was to use WebSockets, and that plan still makes the most sense in my mind. However, implementation was more nuanced than just "using WebSockets". Ultimately, I didn't contribute to the backend, but even if it had used WebSockets the client-side implementation was a bit over my head that late at night. The backend actually does use Python's WebSockets library for the server, but in reality all the defining aspects of WebSockets were ignored. Rather, it just responded to HTTP GET requests with JSON as would a normal HTTP server. The difficulty that arose from this, however, was getting the headers right. Generating the HTTP headers is achieved through the use of a single function that generates the appropriate headers for a desired HTTP response code (in our case, 200). However, since we were sending out JSON, we needed to add in Cross-Origin Resource Sharing data for the client browser to allow itself to interpret it. The solution ended up not being that difficult (just add the proper headers to the _gen_headers() library), but since we were both busy with other things at the time we didn't get a chance to look into how it all worked in our situation and get it implemented. Ultimately, we presented the interface using a plugin for Chrome that just disabled CORS security as a quick-fix. Despite those challenges, we ended up producing a highly functional web-based drone control platform that is essentially ready to run with real drones (we tested using SITL simulated drones), save for a few minor details on the drone end to deal with the realities of reality. We're very happy with how our software came out, and very pleased that we won the competition. So, here's the rest of the writeup:
Inspiration
Unmanned Aerial Vehicles are the future! They are one of the world's fastest growing industries, with enterprises around the world exploring their widespread deployment. The difficulty, however, lies in efficiently managing them without unnecessary loitering and wasted time.
What It Does
Drone Flight Area Control, or DFAC for short, maintains a list of drones and tasks, and sends drones out to those tasks in the most efficient order. The drones and tasks can be monitored and managed through the DFAC web interface, which allows the user to visualize the layout of the tasks, keep track of the drones and their current tasks, and add, modify, and remove tasks. The interactions with the DFAC backend, however, are limitless. Data is exchanged via HTTP GET and JSON, so there are many innovative possibilities for managing the drones and tasks. This also means DFAC can be used as a scheduling and control backend to existing applications.
Who Did What
The DFAC backend and interface was developed by Mark Omo, and the DFAC web interface was developed by James Rowley.
How I Built It
Mark: The DFAC backend is a multi-tiered, multi-threaded, socket driven system, which allows it to be infinitely extensible and scalable to hundreds of task regions and hundreds of thousands of drones. The backend is entirely python based, leveraging existing resources, and enabling quick additions and modifications. Adding new types or classes of UAVs is very easy, and seamless integration into the current tasking system allows them to execute tasks tailored to their strengths.
James: The DFAC web interface is a fully client-side webapp which can be either served as a static webpage or run from disk. It utilizes the basics web technologies, HTML, CSS, and JavaScript, as well as jQuery and the Google Maps API. jQuery is used for interaction with the DFAC backend as well as various simplifications, and the Google Maps API is used for displaying the map, the tasks and the drones, as well as for selecting a location for a new task. The interface is also scalable, and usable on mobile devices.
Challenges I Ran Into
Mark: Ensuring the application was thread safe and did not require cross thread variables was a challenge, along with ensuring the whole system meshed together and could meet the load demands of many vehicles at the same time.
James: I have never written a real-time webapp before, so getting the data to update and synchronize with the server in a timely manner was a challenge. Additionally, getting the CSS to cooperate and not send entire sections off the page while still being scalable and extensible was tough at times.
Accomplishments That I'm Proud Of
Mark: Getting the system working in the short time we were allotted was the biggest achievement in my eyes, along with keeping the server thread safe, and eliminating the possibility of the threads tripping on one another.
James: I'm definitely proud of getting the whole thing to run smoothly and interact with the DFAC backend, while being able to handle hundreds of tasks and drones scalably. Also I'm pretty happy with the overall look and feel, as well as the drone and marker graphics.
What I Learned
Mark: I learned the importance of writing agreeable system interface documents. James and I went back and forth several times on the structure and terminology used in the application interface, using only text chat to boot, so getting the interface right was a learning experience.
James: This was my first time working with jQuery, which I was a bit reluctant to pick up, but I've seen the light and learned quite a bit about what seems to be one of the most prominent web frameworks. I also picked up a bit of python from modifying the dummy server, so that's nice.
What's Next for Drone Flight Area Control
In the future, we want to build in specific applications to assist with task creation, including automatic response to traffic congestion, traffic accidents, and emergencies, automatic industrial and commercial inspection (i.e. inspecting a skyscraper or pipeline), and features targeted for search and rescue applications. In terms of specific features, we want to add result collection and presentation for tasks, a log of previous tasks and drone actions, alerts for completed tasks and drone problems, and task prioritization. For the frontend, we want to add paths/trails for the drone as well as heading of the drone.
Note: Since we submitted it that last line has changed, in my mind: heading of the drone is way too tough to pull off with the Google Maps API and an SVG icon, and I definitely want to add tags underneath the markers and drones with their numbers, plus the number of the drone that is handling a task, or the number of the task a drone is handling.
Unmanned Aerial Vehicles are the future! They are one of the world's fastest growing industries, with enterprises around the world exploring their widespread deployment. The difficulty, however, lies in efficiently managing them without unnecessary loitering and wasted time.
What It Does
Drone Flight Area Control, or DFAC for short, maintains a list of drones and tasks, and sends drones out to those tasks in the most efficient order. The drones and tasks can be monitored and managed through the DFAC web interface, which allows the user to visualize the layout of the tasks, keep track of the drones and their current tasks, and add, modify, and remove tasks. The interactions with the DFAC backend, however, are limitless. Data is exchanged via HTTP GET and JSON, so there are many innovative possibilities for managing the drones and tasks. This also means DFAC can be used as a scheduling and control backend to existing applications.
Who Did What
The DFAC backend and interface was developed by Mark Omo, and the DFAC web interface was developed by James Rowley.
How I Built It
Mark: The DFAC backend is a multi-tiered, multi-threaded, socket driven system, which allows it to be infinitely extensible and scalable to hundreds of task regions and hundreds of thousands of drones. The backend is entirely python based, leveraging existing resources, and enabling quick additions and modifications. Adding new types or classes of UAVs is very easy, and seamless integration into the current tasking system allows them to execute tasks tailored to their strengths.
James: The DFAC web interface is a fully client-side webapp which can be either served as a static webpage or run from disk. It utilizes the basics web technologies, HTML, CSS, and JavaScript, as well as jQuery and the Google Maps API. jQuery is used for interaction with the DFAC backend as well as various simplifications, and the Google Maps API is used for displaying the map, the tasks and the drones, as well as for selecting a location for a new task. The interface is also scalable, and usable on mobile devices.
Challenges I Ran Into
Mark: Ensuring the application was thread safe and did not require cross thread variables was a challenge, along with ensuring the whole system meshed together and could meet the load demands of many vehicles at the same time.
James: I have never written a real-time webapp before, so getting the data to update and synchronize with the server in a timely manner was a challenge. Additionally, getting the CSS to cooperate and not send entire sections off the page while still being scalable and extensible was tough at times.
Accomplishments That I'm Proud Of
Mark: Getting the system working in the short time we were allotted was the biggest achievement in my eyes, along with keeping the server thread safe, and eliminating the possibility of the threads tripping on one another.
James: I'm definitely proud of getting the whole thing to run smoothly and interact with the DFAC backend, while being able to handle hundreds of tasks and drones scalably. Also I'm pretty happy with the overall look and feel, as well as the drone and marker graphics.
What I Learned
Mark: I learned the importance of writing agreeable system interface documents. James and I went back and forth several times on the structure and terminology used in the application interface, using only text chat to boot, so getting the interface right was a learning experience.
James: This was my first time working with jQuery, which I was a bit reluctant to pick up, but I've seen the light and learned quite a bit about what seems to be one of the most prominent web frameworks. I also picked up a bit of python from modifying the dummy server, so that's nice.
What's Next for Drone Flight Area Control
In the future, we want to build in specific applications to assist with task creation, including automatic response to traffic congestion, traffic accidents, and emergencies, automatic industrial and commercial inspection (i.e. inspecting a skyscraper or pipeline), and features targeted for search and rescue applications. In terms of specific features, we want to add result collection and presentation for tasks, a log of previous tasks and drone actions, alerts for completed tasks and drone problems, and task prioritization. For the frontend, we want to add paths/trails for the drone as well as heading of the drone.
Note: Since we submitted it that last line has changed, in my mind: heading of the drone is way too tough to pull off with the Google Maps API and an SVG icon, and I definitely want to add tags underneath the markers and drones with their numbers, plus the number of the drone that is handling a task, or the number of the task a drone is handling.
Links:
DFAC on GitHub
DFAC Frontend on GitHub (the part I wrote)
DFAC Backend on GitHub (the part Mark wrote)
DFAC on Devpost
Hack Arizona 2016
Hack Arizona 2016 on Devpost
Mark Omo's website
DFAC on GitHub
DFAC Frontend on GitHub (the part I wrote)
DFAC Backend on GitHub (the part Mark wrote)
DFAC on Devpost
Hack Arizona 2016
Hack Arizona 2016 on Devpost
Mark Omo's website