The Secret Sauce: A Conversation with DIU and ModalAI about Becoming Blue

Advertisement

The Defense Innovation Unit (DIU) Blue UAS Framework has taken on a life of its own. A logical outgrowth of the Department of Defense’s (DOD) need for modular, interoperable and cyber secure equipment and architectures, “Blue,” as we know it, was truly born when Congress passed Section 848 of the Fiscal Year 2020 National Defense Authorization Act (NDAA). That section essentially forbade the DOD from operating or procuring Chinese uncrewed aircraft systems (UAS) or related components. 

Several years later, and two-iterations into the project, only a handful of companies can boast of being true Blue. ModalAI is one of those companies. In this interview, Chad Sweet, ModalAI’s CEO, and Matthew Borowski, the Program Manager for DIU’s Blue UAS Framework Program, provide some insights into what it takes to become blue.

Dawn Zoldi: Matthew, tell us about the origins of the Blue Framework Program.

Matthew Borowksk (DIU): The Blue Framework program really kicked off before “Blue UAS” was even a thing. It was called the “Group One UAV Architecture.” What we were looking to do was to put some components from industry to the test for an architecture that could be used to promote and develop the DOD’s small UAS. Initially, it was a concept that we were given from the Office of the Secretary of Defense (OSD). The DOD was looking for partners to help our UAS navigate in complex areas. We created a Framework to spur industry and grow a trusted industrial base to have these components available.  

Dawn Zoldi: How did you execute the Framework procedurally?

Matthew Borowksk (DIU): We used DIU's Commercial Solutions Opening (CSO) to solicit industry for components that they thought would be relevant. When we first launched that program, it was very open-ended. We wanted to understand what industry was working on, why they were working on it and how it could benefit us. At that time, any company with a small UAS component could apply. We received about 80 submissions. Out of that, we chose five companies to go on a contract to develop those components for the DOD.

Dawn Zoldi: How does the CSO process work?

Matthew Borowkski (DIU): The CSO starts with what we call an Area of Interest (AOI) or solicitation and ends with an “OT,” Other Transaction (OT) agreement. There are three phases to a CSO. It kicks off when the AOI goes out to industry on our website (diu.mil). We usually let them stay active for about two weeks. In Phase 1, the company submits a white paper or pitch deck where they outline their best solution to the problem posed in the AOI. The teams that work on these evaluations with us are pretty broad. We bring folks from different military services, OSD, FFRDCs like MIT Lincoln Laboratory and other subject matter experts to help us evaluate what we're looking at.

Dawn Zoldi: What criteria do you use to evaluate companies during Phase 1 of the CSO process?

Matthew Borowkski (DIU): In Phase 1, we're looking to see what a company has done in the marketplace, specifically what problems they have solved that may be needed in the DOD. We consider: company viability (e.g., are they on a path to sustain themselves at some point?), product novelty (e.g., is it new or do we already have this somewhere in our inventory?) and relevance (e.g., is it tied to the language in AOI?)  In the end, it really comes down to whether we want to learn more about the solution. If that the answer is yes, then we move on to Phase 2 with that company. If the answer is no, they receive a non-select letter from our agreements officer.

Dawn Zoldi: Tell us about Phase 2 of the CSO.

Matthew Borowkski (DIU): We call Phase 2 “the pitch phase.” It gets more collaborative, interactive and interesting.  We like to have in-person pitches, when possible, either at the vendor site, at a site that we host. If necessary, we can do them virtually.  Either way, we have a meeting where the company gives us their full run-through of their solution. We have a lot of time for Q&A. Ideally, we also  have some kind of demonstration or something that we can physically look at, touch and interact with. This is where we get to understand more about what the company's doing, what the solution could be and what we may want to dig into more. 

After we finish up Phase 2, there's a period of time we spend with the companies asking follow on questions. We make a decision out of those Phase 2 interactions about who we want to work with. 

Dawn Zoldi: Does something occur between Phase 2 and Phase 3?

Matthew Borowkski (DIU): Yes, the next step after Phase 2 is to issue an RFP or request for a prototype proposal. That's a document that goes out to a company and basically says the government wishes to go into a form of negotiation with you. Everything gets worked through our agreements officer, who's a warranted contracting officer. We give them a draft statement of work (SOW), which includes all the elements we might want to see. It’s not open-ended, but there is quite a bit where we're asking the company to fill in the blanks. We usually engage in about a two-week negotiation about what we will do together, how much it will cost and timelines. 

Dawn Zoldi: What occurs during Phase 3?

Matthew Borowkski (DIU): When the government decides to move forward with a SOW, we turn that into Phase 3, an OT agreement or award, which enables us to put a company on contract to do certain tasks that we pay them for, with certain milestones. I can start a program plan and then we're off to the races! One of our first OTs was with ModalAI, as part of the Group One UAV Architecture.

Dawn Zoldi: Is DIU looking for one winner per CSO?

Matthew Borowkski (DIU): No. We issue awards to as many companies as we think have technology that can solve the problem. We explore what's out there. Of course, we have a budget that we've got to adhere to and that limits the number of companies we can put on contract. But we figure out what we can do within that budget. It really comes down to  what we are seeing that's interesting. For example, if we go in there thinking we want two different data links and suddenly there's three, there’s no reason we couldn't add a third as long as the money's there.

Dawn Zoldi: What is the best tip you have for a company responding to a CSO?

Matthew Borowkski (DIU): What works really well is when you tell us, “This is what we've done. Here's what we've accomplished. Do you want to jump on board?” That works really well. The best thing you can do is really tell your story and show what you're doing. The more results you have out there, the better shot you have at getting on to an award. 

Dawn Zoldi: What’s the worst thing a company can say in response to a CSO?

Matthew Borowkski (DIU): What doesn't work well is when you tell us that you’ll do whatever we want. We don't want to be the forcing function for what you're building. We don't want to build things just for us. That does not work well for a DIU prototype. Everything we're doing is dual use. We want to leverage success in the commercial market to serve us.

Dawn Zoldi: All of this is about dual use technology. Why is that important to DIU?

Matthew Borowkski (DIU): We are fast followers of the emerging technology coming out of commercial industry. The DOD doesn't invent a lot of things. We need to rapidly adopt this technology into our missions and capability sets. If you're doing interesting things in the commercial market, we have a place for you. We have offices in Silicon Valley, Boston, Austin, Texas and now Chicago and, of course, the Pentagon. The whole concept comes down to tech that is able to be used in industry but can have a home in the military as well. 

Dawn Zoldi: So, how did the Group One UAV Architecture evolve into the Blue UAS Framework?

Matthew Borowkski (DIU): We were in the midst of the Group One UAV Architecture project when Congress passed the FY20 NDAA Section 848. We were already doing prototyping with those 5 companies, including ModalAI. Luckily, we were already on the right track because we had focused on trusted cyber secure technologies for sustainable autonomous small UAS. So, the 848 language did not greatly affect our initial agreements. But we did change the name to Blue UAS Framework 1.0 at that time. 

Dawn Zoldi: That's a great segue, Chad, because you plugged into these DIU programs at the start. Tell us how ModalAI got involved.

Chad Sweet (ModalAI): ModalAI started in 2018, right around the time of the Group One UAV Architecture. We started the company as former employees of Qualcomm, a large semiconductor company here in San Diego, California. There I led robotics R&D and helped to build a lot of highly integrated and sophisticated technology around Qualcomm's processors for unmanned aerial systems. So, when we started ModalAI, we had a nice running start. One of the first things I read was DIU’s CSO for the Group One UAV Architecture. My reaction was, “Oh my gosh! This is exactly what we want to do!” Just four days after we started ModalAI, I was pitching DIU. That led to our being involved in the Blue Framework program very early on. By mid November 2018, we were under contract. As Matthew said, we were “off to the races” with our tech, which highly leveraged what we had been doing at Qualcomm. Our first generation program was to develop a small, autonomous autopilot for the DIU. We fused our VOXL companion computer and Flight Core flight controller together into a single board configuration that we call VOXL Flight. The VOXL Flight accelerates autonomy for developers with a Qualcomm Snapdragon 821 and open-source flight controller. It condenses the computing power of what typically would be seven PCBs into a SWAP-optimized, single board form factor. VOXL Flight became the guts of that first Framework effort.

Dawn Zoldi: ModalAI contributed its autopilot to Blue UAS Framework 1.0. What other components were involved in the Framework?

Matthew Borowkski (DIU): We kind of bucketed the components into four categories. One is software protocols. We've got a lot of work in UAS software protocols, which includes ground control station software, the flight control software protocols by which they speak and communicate, plus pairing and networking. The other three buckets are very much tied to NDAA Section 848: cameras and gimbals, flight controllers plus data links and radios.

Dawn Zoldi: Can you walk us through the evolution from Blue UAS Framework 1.0 to 2.0? 

Matthew Borowkski (DIU): We started to plan for the next phase of the Blue UAS  Framework in 2019. At that point, we had the original 5 awards and a total of 9 companies, including subcontractors, that were involved. We did a refresh to the first CSO and released it in March 2020, when the world shut down with COVID. We didn't do live pitches and had to do everything virtually. OSD assisted with some of the CARES Act funding that Congress provided to assist companies during the pandemic. We expanded and filled in some of the gaps from Section 848, which was really the catalyst for Blue UAS Framework 2.0. By July 2020, we had awarded all the contracts.

Dawn Zoldi: Which companies came out on top for Blue UAS Framework 2.0? 

Matthew Borowkski (DIU): Several new companies came under 2.0. ModalAI again received an award, but this time for the development of VOXL 2, a new, next generation advanced autopilot. We also worked with Doodle Labs on the Helix Smart Radio. We worked with Immervision on their low light navigation camera. We worked with Vantage Robotics on a swappable gimbal-camera interface system and a related company, Obsidian Sensors, for a new type of thermal camera. All of this - the cameras, data links and flight controllers - tied back to Section 848. 

Dawn Zoldi: Chad, tell us about how VOXL 2 is different from your original VOXL. 

Chad Sweet (ModalAI): In short, VOXL 2 is a big computing upgrade. It’s an incredibly capable, flying supercomputer that enables navigation in highly complex environments, including GPS-denied areas. It has a newer chipset and architecture built off of the powerful Qualcomm QRB5165. A big part of our specialized expertise and knowledge involves how to uncork all the advanced capabilities of GPUs and NPUs for neural processing, sensors and DSPs for embedded flight control. All of this enables smaller, safer and smarter drones. Basically, the VOXL 2 has everything you need to fly a drone autonomously. It has inputs for six image sensors including two big ISPs for video processing. Plus you can integrate a number of computer vision sensors. After that, all that's left really are the ESCs, motors and propellers. This gets you flying while doing incredible autonomous processing. It also weighs only 16 grams. When you're fighting gravity, weight is all that really matters. VOXL 2 is smaller than a business card, so it’s a lightweight package that can power the smallest robots. It is also low power, consuming only between 5 and 7 watts on average, while flying. The competition generally weighs 10 times more and uses at least twice as much power. This is where the VOXL advantage comes in, with a lot more computing power per gram. 

Dawn Zoldi: Can you elaborate on how the DOD or DOD-related agencies are using ModalAI’s VOXL family of autopilots?

Chad Sweet (ModalAI): VOXL enables all kinds of things that the DOD cares about out in the field, including unique missions in GPS-denied environments. One really cool example is our involvement in the DARPA OFFSET program. It involved swarming robots with quad rotor and fixed wing UAS in a test environment. There were maybe 300 plus drones of various shapes and sizes operating from a central control using swarm behaviors to find an April Tag in a hidden spot. In this case, they used our Blue 1.0 reference platform, the VOXL m500 development UAS, which is open and programmable. Because it was programmable, they were able to build new and atypical use cases. 

Most of the UAS you find out there are not open in any way shape or form. We put all of the source code out there publicly. We pride ourselves on creating a MOSA, or modular open software architecture. We built it around standard frameworks like MAVLINK, which was the command and control for the swarm in OFFSET.

We work with tons of different R&D labs throughout the government and not just DOD, including the Departments of Energy and the Interior, as well as NASA JPL, to name a few.

Dawn Zoldi: Tell us about some of the commercial use cases for VOXL.

Chad Sweet (ModalAI): Cleo Robotics in Boston is using VOXL. They've built a little indoor asset inspection UAS that can fly and bump into things called the “Dronut.” It has a very unique two rotor propulsion system that fits into that circular enclosure, as opposed to the more traditional quad rotor motor configuration. We engaged with them while we were at Qualcomm, so they were an early customer of ModalAI. One of the lessons on all of this is how long it takes for these different platforms to get to market on mass scale. Our goal is to accelerate our customers' path towards these advanced capabilities. 

We also have a customer called Corvus drones that builds greenhouse inspecting drones. It flies through the greenhouse autonomously with no GPS, taking pictures of the crop and plants along the way and sends that to the cloud for analysis. That’s a fun one to talk about.

We have the sensors that can do this kind of mapping for both clients, as well as vision technology for GPS-denied operations. These things can really help get a product to the next level in a much quicker fashion. We're building our software stack off of 10 years of development at this point. It’s quite a chunk of capability to make available to our customers so that they don't have to repeat the work. That's the main value proposition of VOXL, VOXL 2 and our VOXL SDK, when you put it all together. 

Dawn Zoldi: There’s a myth out there that DIU serves as the sole clearinghouse for the DOD for UAS and related tech. What’s the reality? 

Matthew Borowkski (DIU): We are definitely not the only way commercial UAS and tech companies can work with the DOD. There are other programs out there in all of the military services. Our DOD partners also have similar programs. We are working to streamline processes for the entire DoD and helping companies supply their technology more easily.  I think we serve in a kind of de facto function as small UAS commercial experts. That's more or less just based on what we've been doing and not any kind of formal role.

Dawn Zoldi: Is there a way for companies working with these other DOD entities or its partners, but aren't working through DIU, to plug into DIU’s efforts?

Matthew Borowkski (DIU): Companies working with DOD or its partners can always ask your sponsor to come work with us. DIU is a joint organization working on accelerating commercial tech to our DoD partners, the Services, components, combatant commands and our team on the frontlines. We can have a conversation gov to gov and figure out how we could do a program that brings that kind of technology in. It’s important to know that if you have government contacts and they're confused about how to make this happen, just come talk to us. We can work directly with each other and figure out if we can make a program out of their requirements.

Dawn Zoldi: Is there a particular kind of dual-use tech that you believe is becoming increasingly important?

Matthew Borowkski (DIU): I would like to challenge the industry to go after software. A lot of people like to focus on the hardware. The hardware is important but we've solved a lot of the hardware issues. ModalAI’s VOXL 2 is ahead of his time. Work on the software piece. Software truly enables the platform to be better.

Dawn Zoldi: Are we looking at a Blue Framework 3.0 anytime soon?

Matthew Borowkski (DIU): The answer is yes. We will be sharing more in the coming months.  

Dawn Zoldi: Chad, what's next for ModelAI?

Chad Sweet (ModalAI): In the next few months we expect to have some big announcements about a smaller platform for VOXL 2 and another reference UAS that highlights some of what you can do in such a small form factor. Those are things we're working on, but it’s hard to put releases against specific dates. Rest assured we are working hard every single day to put out even more capability.

Dawn Zoldi: Chad, tell everybody how they can reach you and please provide any concluding thoughts.

Chad Sweet (ModalAI): I'm on LinkedIn, so you can find me there. Check our website. We have a lot of great content, especially for those that are interested in developing with VOXL 2. Visit:
https://modalai.com
https://docs.modalai.com
https://code.modalai.com
https://forum.modalai.com

Dawn Zoldi: Matthew, you get the last word.

Matthew Borowkski (DIU): It’s been an incredible journey working with the Blue UAS Framework companies--they do great work. It’s our hope that the Blue UAS Framework continues and that we continue to serve as the DOD’s front door first. We can't do what we do without industry doing what they do. We know how challenging it can be to get the funding and resources you need, especially in down economic times. Stay hungry and stay nimble. Keep developing. There will be a time when we'll be asking for more tech and we hope that you're ready.

Dawn of Drones Episode 77: ModalAI and DIU