Since 4 November, Palantir’s Philanthropy Engineering team has been supporting Team Rubicon’s Hurricane Sandy relief efforts. We have engineers on the ground in Brooklyn and Far Rockaway who are standing up, operating, and training volunteers on our software, while others provide reachback support from our offices across the country. This is an exciting time for Philanthropy Engineering at Palantir not only because we are supporting such an inspiring mission, but because it is the first time that we’ve deployed engineers to a disaster area to support relief efforts directly. Several of our Philanthropy Engineers have been writing up reports from the field, and we want to share them here since they best communicate what this effort is all about.
But first a little background. Team Rubicon is a non-profit relief organization that dispatches volunteers to disaster zones from its network of nearly 5,000 military veterans throughout the United States. Their mobilization in New York and New Jersey in the aftermath of Hurricane Sandy has rightfully earned the attention of local and federal government and the media alike. You can read about their efforts on the White House blog and the New York Times “At War” blog, or you can watch this recent segment on NBC Nightly News.
We have donated our software and technical expertise to help Team Rubicon streamline the process of receiving, responding to, and closing out requests for help from residents whose homes were damaged by Hurricane Sandy. The video below from this story by WNYC provides a glimpse of how our software is being used to get more residents the help they need, faster.
Dispatches from a Forward Deployed Philanthropy Engineer
Spending their nights at a base camp in a warehouse next to a rock-climbing gym in Brooklyn and their days at a makeshift forward operating base in a school bus on Rockaway Beach Boulevard, our Philanthropy Engineers have been on the ground for a week and a half now. After a first wave of engineers stood up the software and endured a nor’easter that hit the area with freezing temperatures, 60mph winds, and several inches of snow on Tuesday night, we immediately began to see the impact that we could have on Team Rubicon’s everyday operations. One of those in our second wave of Philanthropy Engineers has been keeping a journal of his time on the ground. These dispatches from the field tell the story.
Sunday 11 November
From 6:30 am to 6:00 pm, we helped Team Rubicon handle several thousand volunteers and several hundred simultaneous projects by distributing data entry and scrubbing duties to dozens of both trained volunteers and Palantirians who were monitoring the Rubicon instance. Eight mobile devices running Palantir Mobile spent the day feeding disparate data into the instance—pictures of residents standing in front of their damaged homes, phone numbers, information about the work crews and tools needed to help them, etc. This information was modeled and analyzed by a team of Palantirians located around the country and then acted on within hours by teams of volunteers. To give just one example of how critical this reachback support has been, one of Team Rubicon’s mobile operators almost ran out of battery life until a Palantirian in North Carolina noticed his battery level remotely and emailed us in NYC, whereupon we sent out a Rubicon member with an extra battery who was able to find the operator in need by using Blue Force Tracking in Palantir Mobile.
Data entry was accomplished via any and every means available: texting pictures of hand-written forms brought in by local residents; texting data from those forms via SMS to Brian in Brooklyn; entering data into Webflow on an iPad in the hands of a Rubicon volunteer offsite at the headquarters of another aid organization that wished to coordinate with Rubicon. Our networked command-and-control system, designed to work around the dial-up level of bandwidth available on the Rockaway peninsula during most of the day, was so well-oiled and managed by Brian and Zach that Alec and I were simultaneously able to work and train two Rubicon employees to a high level of proficiency with Palantir Gotham. They are now able to perform 90% of the tasks necessary to keep Rubicon functioning at optimal efficiency.
An encouraging day, but there’s more work to be done. We are in need of long, deep ontology conversations over dinner. There’s a lot to think about.
Volunteers entering data by SMS
Monday 12 November
We started the morning slightly overwhelmed as we walked into our Rockaway operations center and saw the piles of yesterday’s work orders distributed at random on the tables, chairs, and floor. Some of them were covered with handwritten notes while others were torn into patterns that seemed to evince a mysterious and forgotten organizational artform. Priority number one immediately became the acceleration of our plan to replace paper work orders with direct Rubicon-team-leader-to-Palantir-instance dispatch. This will be up and running tomorrow morning thanks to our smart, eager Rubicon trainees and our support from Palantirians across the country. Steve paid us a very welcome visit in our warehouse office this evening to deliver laptops.
To clean up our data and determine who had already had a visit from a work crew, be it Rubicon or anyone else–several groups of volunteer contractors and spontaneous aid organizations were operating in the Rockaways on Sunday–we opened Palantir Phone Bank, run by a Rubiconian sifting through the Browser app and managing three volunteers calling hundreds of open work cases to determine their current need.
As the morning wore on, our Rubicon users took over more and more of our tactical management responsibilities. Data entry, data updates, data scrubbing and dispatch were all handled by Rubiconians. Alec and I are still doing some minute-to-minute work. When our Alaskan firefighter chainsaw-wielding wood-chipping expert walked in asking for trees to attack, in five minutes Alec had him a printed map of every NYC 311 call in four square miles reporting a downed tree. But when 50 merchant marine cadets walked up looking for work, it was Tina, a Rubiconian, who, on her second day using Palantir Gotham, pulled up all the open work orders, quickly ran a histogram on crew size needed, threw all the big jobs on the map, and five minutes later handed them a package of eight 10+ person jobs all along the end of one particularly hard-hit street.
About 30 local contractors and carpenters are taking the day off tomorrow to volunteer, and they heard Rubicon might be able to help focus their efforts. It was pretty fun watching their jaws drop when I handed them a packet of nine cases within walking distance of one another topped by a map pinpointing each residence with its job type and contact info. The icons on the map were coded according to the jobs’ priority. Two red icons, for example, indicated two residents who were elderly and had no family in the area to help them. The priority coding system, which will be refined by an ontology update tonight, lets us respond to all of Rubicon’s and the local community’s highest priorities. A local synagogue, for example, which yesterday morning requested help from a 50-plus-person work crew to haul out tens of thousands of pounds of sand and destroyed furniture, is as of this afternoon ready for rebuilding.
Most exciting of all, perhaps, was our visit from celebrity chef Rocco DiSpirito, who brought us all lunch in the form of “gourmet soup” and garbage bags full of fresh bread. “This bread,” said one of his kitchen helpers in a reverent whisper as she handed Alec a couple of pieces, “is very expensive.”
A good day’s work.
Color-coded Work Requests in Rockaway as viewed in Palantir Gotham’s Map application
Tuesday 13 November
Today, it rained. Rain makes us wet. Being wet makes us cold. Being cold makes us put on gloves. Wearing gloves makes typing 44% less accurate than not wearing gloves.
The weather delayed our plan to completely replace paper with interactive Palantir case checkout kiosks, as the contingency plan of setting up the laptops under the tent was immediately made ridiculous by the horizontal water whipping through our parking lot. Our new data model still needed some fleshing out anyway, so the extra day was a good chance to do some more thorough scrubbing, prepare for the nightly correction of mistakes we’ve made, and finalize our new ontology while simultaneously entertaining a seemingly endless parade of visitors to our ever-expanding operations center. I sit in the back now, and try to stay out of the way of the Rubiconians who are actually getting work done while I field phone calls and coordinate meetings. But it’s fun as long as we can keep inventing new systems that do neat things. Like radically different ontologies that allow us to scale out to cover the entire city in a logical way, if that’s something we ever decide we want to do. It’s fixed now, and ready to go. All in a rainy day’s work.
Patrick now mans the Google Voice phone line that Alec set up, which all of the team leaders text with progress reports. Patrick reports time-sensitive ones as needed, and then periodically pastes the entire log into a notepad file which he then imports into the instance. Rubicon analysts in the front of the bus then go through that dispatch file and tag the updates to property changes on the open cases. Number of data-contributing users on the Rubicon instance: 40. Number of analysts contributing to a typical object by the time it is complete: 5–8.
Alec and I have not yet showered in New York State, which is great in some ways—-no one seems to visit our corner of the warehouse anymore, so we get way more work done around dinnertime—-but probably not so great for hygiene. We’re going to go clean up and try to get more than four hours of sleep tonight.
Oh, and we got gourmet soup. Again. Rocco DiSpirito drove up by himself, dropped it off, and left with a quiet word of thanks.
A Philanthropy Engineer shows the Rubicon instance to an NYPD officer aboard the green bus operating base
What’s Next
Our Philanthropy Engineers will remain embedded with Team Rubicon for as long as they are needed. We are incredibly proud to be supporting such an inspiring organization and we’re excited to see what more we can accomplish in the coming days and weeks.