A new report by The Intercept provides some compelling specificson collaborationbetween Palo Altos secretive big data shop Palantir and the intelligence agencies that it keeps on itsclient roster.
While somebroad strokes are known about Palantirs relationship with spy agencies, the finer points seldom see daylight. That guaranteeof secrecy no doubt aidsthe companys growth into 2017, though its cozy relationship with a presidency keen on implementing aggressive policies with big opportunities for big data certainly cant hurt, either.
Palantirs work with state-sponsored spying appears to date back to 2008, when the company demoed its software to the Government Communications Headquarters (GCHQ), the British counterpart to the National Security Agency (NSA). According to classified internal documents quoted in the report, Palantir made quite the impression (We were very impressed. You need to see it to believe it.)
Apparently,beyond kicking in early money, intelligence agencies actually worked directly with the company to develop custom software through [an] iterative collaboration between Palantir computer scientists and analysts from various intelligence agencies over the course of nearly three years.
The report goes into more detail on the development of two particular software systems, one called Kite and one known as XKEYSCORE Helper. Through Kite, implemented by GCHQ, Palantir offered deep customization:
Out of the box, Kite wasable to handle a variety of types of data (includingdates, images, geolocations, etc.), but GCHQ was free to extend it by writing custom fields for complicated types of data the agencymight need to analyze.The import tools were designed to handle a variety of use cases, including static data sets, databases that wereupdated frequently, and data stores controlled by third parties to which GCHQ was able to gain access.
Designed in apparent collaboration with Palantir, XKEYSCORE Helper sought to make data obtained through the NSAs muscular XKEYSCORE program more digestible and offered a way to port it into Palantirs more analyst-friendly interface:
This collaborative environment also produced a piece of software called XKEYSCORE Helper, a tool programmed with Palantir (and thoroughly stamped with its logo) that allowed analysts to essentially import data from the NSAs pipeline, investigate and visualize it through Palantir, and then presumably pass it to fellow analysts or Five Eyes intelligence partners.
Palantir, founded in 2004 by Alex Karp and Peter Thiel, remainssomething of an enduring mystery in Silicon Valley. In 2013, TechCrunch obtained a closely guarded list of the companys clients, revealing relationships with the CIA, DHS, FBI, NSA, Marine Corps and Air Force, among others.Though the company continues to grow, much of what it does remains under wraps, or understood mostly in abstract terms.
While the picture is far from complete, everydetail that emerges aboutPalantirs mission, software implementation and client base helps piece together the puzzle of perhaps the most politically relevant company in technology right now.