In the ever-evolving landscape of AI and automation, building a product that truly resonates with users is a challenging yet rewarding odyssey. In a recent episode of the Startup Project podcast, Nataraj engaged in a fascinating conversation with Jacob Bank, the founder and CEO of Relay.App, shedding light on the intricate product development journey of an AI agent building platform. This blog post delves into the key product development insights gleaned from their discussion, offering valuable lessons for product managers, engineers, and entrepreneurs navigating the complex terrain of AI-first product creation.
The Winding Road to Product Clarity:
Relay.App’s journey is a testament to the power of iteration and the importance of listening to the market. As Bank candidly shared, the company’s early days were marked by a period of experimentation and exploration. Founded in 2021 with the vision of enhancing cross-tool coordination using AI, the team initially ventured down multiple paths, building eight or nine different product prototypes, each exploring different facets of the core concept. This period of “wandering in the desert” was crucial in honing their understanding of customer needs and refining their product vision.
The initial thesis, predating the widespread adoption of LLMs, centered on using AI to bridge the gaps between disparate tools. However, the breakthrough came when Relay.app focused on capturing repeated tasks that combined automated components with human judgment. This shift led to the development of a workflow tool positioned between Zapier-style automation and Asana-style task management.
The “Duct Tape” Dilemma and the Pivot to AI Agents:
While the workflow product garnered some traction, the team recognized a crucial limitation: positioning themselves as an automation tool inadvertently limited their audience. The label “no-code workflow automation” often attracts a niche segment of tech-savvy users, while the broader opportunity lies in empowering every business to leverage AI for increased productivity.
This realization spurred a strategic evolution, transitioning Relay.app from an AI-powered automation platform to an AI agent building platform. This shift wasn’t merely semantic; it represented a fundamental change in product philosophy. Instead of simply connecting tools, Relay.app aimed to provide a platform where users could create intelligent agents that proactively work on their behalf.
Integrations: A Core Competency, Not an Afterthought:
A recurring theme throughout the conversation was the critical importance of integrations. Bank emphasized that integrations are not a mere checkbox feature but a skilled labor requiring top-tier engineering talent. Building robust and reliable integrations with a wide array of tools is essential for AI agents to effectively perform their tasks.
Relay.app currently boasts around 120 native integrations and is strategically working toward expanding this number to 300-500. The focus is on providing comprehensive coverage across essential business tool categories, including email, calendar, messaging, CRM, and marketing automation. Bank’s belief is that agents will only be as useful as their ability to interact with the existing tools in your workflow.
Human-in-the-Loop: Building Trust and Control:
As AI becomes increasingly integrated into our workflows, the role of human oversight remains paramount. Bank emphasized the necessity of a human-in-the-loop mechanism, allowing users to review and provide feedback on the agent’s planned actions before they are executed.
This approach not only builds trust in the AI system but also allows for continuous learning and improvement. By incorporating user feedback, the agent can refine its behavior and better align with human intent. Furthermore, should an AI deviate, it is important to create workflows in which a user can course-correct or take-over in real-time. The balance of delegation and human-interaction are vital for establishing true AI augmentation.
Product-Led Content and the Power of Community:
Relay.app’s go-to-market strategy revolves around product-led content, showcasing the tangible benefits of AI agents through compelling use cases. Bank himself actively creates content, including LinkedIn posts and YouTube tutorials, demonstrating how users can build AI agents to solve specific problems.
This approach not only drives product awareness but also fosters a thriving community of users who share their own creations and insights. By empowering users to create and share templates, Relay.app has created a virtuous cycle of product adoption and community engagement. This product-led content drives organic growth by educating and empowering users.
The Future of Product Development: AI-Powered Teams:
Bank envisions a future where product teams are leaner, more agile, and empowered by AI. Instead of relying on large teams with specialized roles, he believes that individuals will increasingly take on a “player-coach” role, combining strategic vision with practical execution.
This shift is enabled by AI agents, which can automate mundane tasks and free up human employees to focus on higher-level thinking and problem-solving. The key lies in identifying the right tasks for AI automation and designing workflows that seamlessly integrate human expertise.
Lessons Learned and the Path Forward:
Jacob Bank’s product development journey with Relay.App offers valuable insights for anyone building AI-first products. The importance of iteration, customer feedback, robust integrations, human-in-the-loop design, and product-led content cannot be overstated.
As the AI landscape continues to evolve, product teams must embrace a flexible and adaptable approach, constantly refining their products and strategies to meet the ever-changing needs of their users. By focusing on building trustworthy and valuable AI agents, they can unlock new levels of productivity and innovation.
Ultimately, Relay.App’s experience underscores the importance of moving beyond hype and focusing on delivering tangible value. By embracing a user-centric approach and prioritizing robust integrations, human oversight, and product-led growth, product teams can successfully navigate the challenges of the AI revolution and build products that truly transform the way we work.
Nataraj is a Senior Product Manager at Microsoft Azure and the Author at Startup Project, featuring insights about building the next generation of enterprise technology products & businesses.
Listen to the latest insights from leaders building the next generation products on Spotify, Apple, Substack and YouTube.