Generative AI has its pitfalls, such as fabricating information, exhibiting biases, and producing toxic content. Despite these concerns, is it possible to harness its potential safely?
Rick Caccia, CEO of WitnessAI, is optimistic.
“Securing AI models presents a real challenge, especially intriguing for AI researchers, but it’s distinct from securing their use,” explained Caccia, a former SVP of marketing at Palo Alto Networks, in an interview with TechCrunch. “Consider a sports car: a powerful engine—the AI model—won’t suffice without robust brakes and steering for control. Both are crucial for high-performance driving.”
Enterprises are expressing demand for such controls. Although companies are cautiously optimistic about the productivity boosts generative AI can offer, they remain wary of its limitations.
An IBM poll shows that 51% of CEOs are recruiting for generative AI roles newly created this year. However, only 9% of companies report that they are prepared to tackle threats related to privacy and intellectual property, according to a survey by Riskonnect.
WitnessAI’s platform intercepts interactions between employees and their organization’s custom generative AI models—not API-restricted models like OpenAI’s GPT-4, but more akin to Meta’s Llama 3—applying risk mitigation policies and safeguards.
“One of enterprise AI’s promises is democratizing data access so employees can perform better. However, too much access or data leaks pose significant risks.”
WitnessAI offers multiple modules tailored to address diverse generative AI risks. One module enforces rules preventing employees from using AI tools in unauthorized ways, such as querying unreleased earnings reports or copying internal codebases. Another module redacts sensitive information from prompts and employs techniques to defend models against prompt injection attacks.
“We believe in helping enterprises by clearly defining issues like AI adoption safety and providing targeted solutions,” said Caccia. “The CISO aims to protect business data, and WitnessAI aids by ensuring data protection, preventing prompt injection, and enforcing identity-based policies. Meanwhile, the chief privacy officer wants regulatory compliance, which we facilitate through visibility and risk reporting.”
However, there is a privacy concern: all data must pass through WitnessAI’s platform before reaching the model. While WitnessAI is transparent and offers tools for monitoring model access and prompts, it nonetheless introduces its own privacy risks.
Addressing these concerns, Caccia noted that the platform is both isolated and encrypted, mitigating data leakage risks.
“We’ve created a platform with millisecond-latency and built-in regulatory separation. Each customer gets a distinct, encrypted instance of our platform, keeping their AI activity data private—even from us,” stated Caccia.
This level of security might reassure customers, but employees may still have reservations about surveillance. Surveys indicate a general aversion to workplace monitoring, with many believing it adversely affects morale. Almost one-third of respondents in a Forbes survey admitted they might consider leaving their jobs if their employer monitored online activity.
Despite these challenges, Caccia remains confident in the strong interest for WitnessAI’s platform. The startup has a pipeline of 25 early corporate users in its proof-of-concept phase and has raised $27.5 million from Ballistic Ventures and Google’s corporate venture arm, GV.
The funding will expand WitnessAI’s team from 18 to 40 members by the year’s end. This growth is crucial for competing in the emerging model compliance and governance market, facing giants like AWS, Google, and Salesforce, as well as startups like CalypsoAI.
“We’re positioned to stay on track well into 2026, even with zero sales, but we’re already seeing twenty times the pipeline needed to meet our sales targets for this year,” Caccia said. “Though this is our initial funding and public launch, secure AI enablement and use is a nascent market, and our features are evolving to meet its demands.”
