Building Trustworthy AI: Navigating the Challenges and Future of Agentic Software with Shane Emmons

Highlights

• AI’s trust problem arises from its probabilistic, sometimes inconsistent nature, posing risks especially in sensitive domains like healthcare.
• Swept.ai tests AI agents at the task level to ensure consistency and enforce hardline policies, going beyond soft “guardrails.”
• AI security parallels cybersecurity, requiring both prevention and detection to handle adversarial attacks like jailbreak attempts.
• Shane advocates hands-on AI learning by engaging with tools like ChatGPT as a thought partner, not just for simple tasks.
• Larger language models offer greater knowledge but higher cost, while smaller, fine-tuned models provide efficiency for niche applications.
• The next 1-2 years will see AI agents automating complex workflows, delivering measurable ROI beyond basic copilot functions.
• Current AI-generated content suffers from “AI slop”—repetitive, low-quality output—which will diminish as quality controls and algorithms improve.

---

Summary

In this insightful episode of The Innovators and Investors Podcast, host Kristian Marquez interviews Shane Emmons, founder and CEO of Swept.ai. The discussion centers around the challenges and realities of artificial intelligence (AI) today, particularly the issue of trust in AI agents and agentic software. Shane explains that AI’s inherent unpredictability stems from its probabilistic nature—AI makes educated guesses based on patterns in data rather than deterministic calculations, which can sometimes lead to inconsistent or even harmful outputs. Swept.ai’s mission is to address this “trust problem” by monitoring AI agents’ performance, ensuring consistency, and enforcing strict policies to prevent undesirable behaviors, especially in high-risk areas like healthcare.

Shane elaborates on the concepts of general intelligence and superintelligence, defining superintelligence as an AI system smarter than all humans combined, a goal many labs are pursuing. He clarifies how AI prediction fundamentally differs from deterministic computation and why even simple tasks like “1+1” can sometimes yield wrong answers if the model is influenced by erroneous inputs from its training data.

Swept.ai’s approach is to evaluate AI agents at the task level rather than the model level, focusing on scoped applications to better assess reliability and consistency. Shane highlights the parallels between AI security and cybersecurity, explaining how they use detection and alert systems to combat adversarial attacks like prompt injection or jailbreak attempts, rather than relying solely on preventive measures.

The conversation also touches on Shane’s extensive background in AI and data science, his views on how users can deepen their AI understanding by engaging interactively with tools like ChatGPT and Claude Gemini, and the importance of recognizing AI’s limits. Shane discusses the trade-offs between large and small language models, noting that larger models generally possess broader knowledge but are more expensive, while smaller models can be fine-tuned for specific tasks at reduced cost and latency.

Looking ahead, Shane predicts that AI agents will soon move beyond simple chatbots to autonomously handle complex workflows, delivering real business ROI by automating repetitive tasks. He cautions that while AI-generated content currently floods the internet with “AI slop” —low-quality, repetitive material—this will eventually subside as quality improves and algorithms adapt. He stresses the importance of companies understanding their workflows thoroughly to effectively implement AI agents.

Finally, Shane shares personal reflections on his entrepreneurial journey, emphasizing the value of focusing on building strong business fundamentals before chasing venture capital, and credits mentors who shaped his leadership. He encourages businesses to deploy more AI while managing risks thoughtfully, offering Swept.ai as a resource for de-risking AI adoption.

Key Insights

• AI’s Probabilistic Nature Creates Trust Challenges: AI systems operate by predicting the most likely next output based on vast but imperfect data patterns. This stochastic process means outputs can be inconsistent or erroneous, unlike traditional deterministic algorithms. Understanding this is crucial for users and developers to set realistic expectations and implement safeguards. Swept.ai addresses this by monitoring AI agents to alert users when inconsistencies arise, which is vital for applications where errors could cause harm.
• Scoped AI Agents Enable Effective Testing and Control: Rather than attempting to test or control large language models in their entirety (which have infinite input-output possibilities), Swept.ai focuses on AI agents designed for specific tasks. By narrowing the scope, they can define expected correct behaviors and enforce policies that prevent agents from exceeding their boundaries, much like a traffic cop directing safe flow. This approach balances AI’s flexibility with necessary control.
• AI Security Requires Both Prevention and Firefighting: Just as cybersecurity cannot rely solely on prevention (e.g., phishing attacks still succeed despite training), AI systems must assume adversarial attempts like prompt injections or jailbreaks will occur. Swept.ai’s role is to detect suspicious activity and alert or block it, supplementing preventive “guardrails” which can be bypassed. This layered security mindset is essential to mitigate risks from malicious actors exploiting AI vulnerabilities.
• Practical AI Education Through Interaction: Shane emphasizes that non-experts can build AI literacy by actively using tools like ChatGPT not just for simple queries but as thinking partners—asking questions, role-playing, and exploring AI’s reasoning. This hands-on approach helps users identify AI’s strengths and weaknesses, enhancing their ability to prompt effectively and understand AI’s probabilistic responses. This is more accessible and practical than formal AI degrees for many professionals.
• Trade-offs Between Large and Small Language Models: Large models, with hundreds of billions of parameters, have richer knowledge and more nuanced capabilities but are computationally expensive. Smaller models are “quantized” versions that run faster and cheaper but may lose some accuracy or breadth. Swept.ai helps users start with large models and then fine-tune or trim them to create efficient smaller models tailored for specific tasks, optimizing both performance and cost.
• AI Agents Are Transitioning from Chatbots to Workflow Automation: Recent breakthroughs in agentic AI allow decomposition of complex tasks into subtasks handled by specialized sub-agents, improving reliability and expanding applicability beyond coding to broader business processes. Companies that deeply understand their workflows can harness agents to automate routine work, reduce oversight costs, and free human workers for higher-value activities, marking a shift toward greater AI-driven productivity.
• The Era of “AI Slop” Will Eventually Pass: The current flood of AI-generated content often lacks quality, exhibiting repetitive or unnatural patterns (termed “AI slop”) similar to the uncanny valley in CGI. This stems from attempts to game SEO and content volume rather than focusing on value. As search engines and AI platforms refine algorithms, and users demand better quality, this trend will diminish, clarifying AI’s role as a genuine productivity tool rather than a source of noise.
• Leadership Lessons: Focus on Fundamentals Before Fundraising: Shane’s reflection on his startup journey underscores the importance of building a solid business foundation prior to aggressive fundraising. Early distractions chasing venture capital can detract from product and customer development. Prioritizing fundamentals attracts investors organically and leads to sustainable growth, a lesson valuable for entrepreneurs navigating AI ventures and beyond.
• Human-AI Collaboration Requires Understanding and Oversight: Shane compares AI to an eager intern—motivated but inexperienced and lacking judgment. Successful AI integration involves setting clear policies, monitoring outputs, and recognizing AI’s limitations. This mindset helps organizations avoid overreliance on AI while maximizing its benefits, ensuring AI acts as a reliable assistant rather than a risky autonomous decision-maker.
• Superintelligence Remains a Distant but Driving Vision: The podcast clarifies the distinction between general AI (human-level intelligence) and superintelligence (AI surpassing all human intelligence combined). While superintelligence is a long-term goal pursued by major labs, current technologies focus on narrow, specialized agents. Awareness of these distinctions helps contextualize present AI capabilities and the ethical, technical challenges ahead.

Overall, this conversation offers a comprehensive, nuanced view of AI’s current state, challenges, and potential, highlighting Swept.ai’s unique role in building trust and reliability into AI deployments while encouraging practical adoption and thoughtful risk management.

Stay up-to-date with Shane Emmons and his work with Swipe.AI.

Follow our show