Natural Language Processing For Chatbots

The real challenge in AI today isn’t just building an agent—it’s scaling it reliably in production. An AI agent that works in a demo often breaks when handling large, real-world workloads. Why? Because scaling requires a layered architecture with multiple interdependent components. Here’s a breakdown of the 8 essential building blocks for scalable AI agents: 𝟭. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀 Frameworks like LangGraph (scalable task graphs), CrewAI (role-based agents), and Autogen (multi-agent workflows) provide the backbone for orchestrating complex tasks. ADK and LlamaIndex help stitch together knowledge and actions. 𝟮. 𝗧𝗼𝗼𝗹 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 Agents don’t operate in isolation. They must plug into the real world:  • Third-party APIs for search, code, databases.  • OpenAI Functions & Tool Calling for structured execution.  • MCP (Model Context Protocol) for chaining tools consistently. 𝟯. 𝗠𝗲𝗺𝗼𝗿𝘆 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 Memory is what turns a chatbot into an evolving agent.  • Short-term memory: Zep, MemGPT.  • Long-term memory: Vector DBs (Pinecone, Weaviate), Letta.  • Hybrid memory: Combined recall + contextual reasoning.  • This ensures agents “remember” past interactions while scaling across sessions. 𝟰. 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀 Raw LLM outputs aren’t enough. Reasoning structures enable planning and self-correction:  • ReAct (reason + act)  • Reflexion (self-feedback)  • Plan-and-Solve / Tree of Thought These frameworks help agents adapt to dynamic tasks instead of producing static responses. 𝟱. 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗕𝗮𝘀𝗲 Scalable agents need a grounding knowledge system:  • Vector DBs: Pinecone, Weaviate.  • Knowledge Graphs: Neo4j.  • Hybrid search models that blend semantic retrieval with structured reasoning. 𝟲. 𝗘𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻 𝗘𝗻𝗴𝗶𝗻𝗲 This is the “operations layer” of an agent:  • Task control, retries, async ops.  • Latency optimization and parallel execution.  • Scaling and monitoring with platforms like Helicone. 𝟳. 𝗠𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴 & 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 No enterprise system is complete without observability:  • Langfuse, Helicone for token tracking, error monitoring, and usage analytics.  • Permissions, filters, and compliance to meet enterprise-grade requirements. 𝟴. 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 & 𝗜𝗻𝘁𝗲𝗿𝗳𝗮𝗰𝗲𝘀 Agents must meet users where they work:  • Interfaces: Chat UI, Slack, dashboards.  • Cloud-native deployment: Docker + Kubernetes for resilience and scalability. Takeaway: Scaling AI agents is not about picking the “best LLM.” It’s about assembling the right stack of frameworks, memory, governance, and deployment pipelines—each acting as a building block in a larger system. As enterprises adopt agentic AI, the winners will be those who build with scalability in mind from day one. Question for you: When you think about scaling AI agents in your org, which area feels like the hardest gap—Memory Systems, Governance, or Execution Engines?

🔥 Can We Build Inclusive Agentic Systems Without Inclusive Training Data? When I first heard people talk about agentic AI — machines that can reason, decide, and act on our behalf — I was fascinated. But then one question hit me hard: How can we expect inclusive intelligence from exclusive data? I see this every day. The more I use these systems, the clearer it becomes — they speak one cultural language fluently, and stumble on the rest. The logic feels Western. The tone feels corporate. The empathy feels… selective. Let’s break down why that matters: → Most training data still comes from English-speaking, digitally rich nations. → The behaviors encoded reflect a small slice of humanity. → The missing perspectives aren’t “edge cases” — they’re billions of people. Now imagine giving that system agency. A biased chatbot can misinform. A biased agent can act — negotiate, reject, decide — without ever seeing the full picture of humanity it represents. So what’s the way forward? In my opinion, we can’t fix this with PR statements or prompt engineering — we need infrastructure-level inclusion: ✅ Build decentralized data pipelines where local communities own their voice and context. ✅ Incentivize global annotation networks that reflect cultural nuance. ✅ Create regulatory sandboxes for testing fairness dynamically, not statistically. ✅ And most importantly — give non-English regions a stake in how foundational models evolve. Because inclusion isn’t a “nice to have.” It’s an engineering challenge. If we don’t solve it now, agentic AI won’t just replicate bias — it’ll automate it. So I’ll ask again: can an AI truly act for everyone if it was only ever trained to understand some? #AIethics #AgenticAI #BiasInAI #Inclusion #ResponsibleAI #FutureofAI

It turns out that when you ask ChatGPT about values (things like trust, authority, fairness, etc.), it answers like someone from Finland, not Jordan. Or from the Netherlands, but not Ghana. I came across this nugget in the research paper below on cultural bias and LLM alignment. It's from September 2024 (ancient history in AI time), but many of the threads it pulls on are still present in the most recent model updates. The authors tested GPT-4o, 4-turbo, 4, 3.5-turbo, and 3 against survey data from 107 countries and found that all models exhibited cultural values similar to those of English-speaking and Protestant European countries. This matters because when millions of people use AI to write emails or make decisions, they're getting cultural filters that may not match how they actually think or build trust. These AI-generated responses influence communication, but also the level of interpersonal trust between communicators. And small cognitive biases accumulate and shape systems over time. The fix? Researchers found you can reduce bias by being explicit about cultural context in prompts. That's a good start, but we need systems designed from the beginning around different cultural values, not ones that just tolerate them. Culture shapes how people think, communicate, and work. If the tools we're building don't account for that, we're just scaling someone else's worldview. Link to research in comments. #AIBias #CulturalDiversityInAI #ResponsibleAI

Few-shot Text Classification predicts the label of a given text after training with just a handful of labeled data. It's a powerful technique for overcoming real-world situations with scarce labeled data. SetFit is a fast, accurate few-shot NLP classification model perfect for intent detection in GenAI chatbots. In the pre-ChatGPT era, Intent Detection was an essential aspect of chatbots like Dialogflow. Chatbots would only respond to intents or topics that the developers explicitly programmed, ensuring they would stick closely to their intended use and prevent prompt injections. OpenAI's ChatGPT changed that with its incredible reasoning abilities, which allowed an LLM to decide how to answer users' questions on various topics without explicitly programming a flow for handling each topic. You just "prompt" the LLM on which topics to respond to and which to decline and let the LLM decide. However, numerous examples in the post-ChatGPT era have repeatedly shown how finicky a pure "prompt" based approach is. In my journey working with LLMs over the past year+, one of the most reliable methods I've found to restrict LLMs to a desired domain is to follow a 2-step approach that I've spoken about in the past: https://lnkd.in/g6cvAW-T 1. Preprocessing guardrail: An LLM call and heuristical rules to decide if the user's input is from an allowed topic. 2. LLM call: The chatbot logic, such as Retrieval Augmented Generation. The downside of this approach is the significant latency added by the additional LLM call in step 1. The solution is simple: replace the LLM call with a lightweight model that detects if the user's input is from an allowed topic. In other words, good old Intent Detection! With SetFit, you can build a highly accurate multi-label text classifier with as few as 10-15 examples per topic, making it an excellent choice for label-scarce intent detection problems. Following the documentation from the links below, I could train a SetFit model in seconds and have an inference time of <50ms on the CPU! If you're using an LLM as a few- or zero-shot classifier, I recommend checking out SetFit instead! 📝 SetFit Paper: https://lnkd.in/gy88XD3b 🌟 SetFit Github: https://lnkd.in/gC8br-EJ 🤗 SetFit Few Shot Learning Blog on Huggingface: https://lnkd.in/gaab_tvJ 🤗 SetFit Multi-Label Classification: https://lnkd.in/gz9mw4ey 🗣️ Intents in DialogFlow: https://lnkd.in/ggNbzxH6 Follow me for more tips on building successful ML and LLM products! Medium: https://lnkd.in/g2jAJn5 X: https://lnkd.in/g_JbKEkM #generativeai #llm #nlp #artificialintelligence #mlops #llmops

What’s missing in conversational AI? The ability to plan responses across turns strategically to achieve goals. Most conversational AIs: • Focus on single responses • Lack strategic, long-term goals • Miss out on real human connection New UC Berkeley publications are contributing to the game: 𝗤-𝗦𝗙𝗧 (Q-Learning via Supervised Fine-Tuning) • Adapts Q-learning to train language models • Adds long-term planning directly into responses • Helps AIs respond with strategy, not just reaction 𝗛𝗶𝗻𝗱𝘀𝗶𝗴𝗵𝘁 𝗥𝗲𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 • Replays past conversations to find better responses • Learns from the past to improve future replies • Guide smarter conversational strategies Applications? • 𝗠𝗲𝗻𝘁𝗮𝗹 𝗛𝗲𝗮𝗹𝘁𝗵 𝗦𝘂𝗽𝗽𝗼𝗿𝘁: Builds trust, helping users feel heard. • 𝗘-𝗰𝗼𝗺𝗺𝗲𝗿𝗰𝗲: Remembers past chats to close sales. • 𝗖𝗵𝗮𝗿𝗶𝘁𝘆: Guides conversations with empathy, boosting donations. Together, these methods will allow CAI to be goal-oriented, plan strategically, adapt, and connect with users.

I've tested over 20 AI agent frameworks in the past 2 years. Building with them, breaking them, trying to make them work in real scenarios. Here's the brutal truth: 99% of them fail when real customers show up. Most are impressive in demos but struggle with actual conversations. Then I came across Parlant in the conversational AI space. And it's genuinely different. Here's what caught my attention: 1. The Engineering behind it: 40,000 lines of optimized code backed by 30,000 lines of tests. That tells you how much real-world complexity they've actually solved. 2. It works out of the box: You get a managed conversational agent in about 3 minutes that handles conversations better than most frameworks I've tried. 3. Conversation Modeling Approach: Instead of rigid flowcharts or unreliable system prompts, they use something called "Conversation Modeling." Here's how it actually works: 1. Contextual Guidelines: ↳ Every behavior is defined as a specific guideline. ↳ Condition: "Customer wants to return an item" ↳ Action: "Get order number and item name, then help them return it" 2. Controlled Tool Usage: ↳ Tools are tied to specific guidelines ↳ No random LLM decisions about when to call APIs ↳ Your tools only run when the guideline conditions are met. 3. Utterances Feature: ↳ Checks for pre-approved response templates first ↳ Uses those templates when available ↳ Automatically fills in dynamic data (like flight info or account numbers) ↳ Only falls back to generation when no template exists What I Really Like: It scales with your needs. You can add more behavioral nuance as you grow without breaking existing functionality. What's even better? It works with ALL major LLM providers - OpenAI, Gemini, Llama 3, Anthropic, and more. For anyone building conversational AI, especially in regulated industries, this approach makes sense. Your agents can now be both conversational AND compliant. AI Agent that actually does what you tell it to do. If you’re serious about building customer support agents and tired of flaky behavior, try Parlant.

Customer service chatbots: most overhyped use case for Gen AI? 🤖 Customer service chatbots are often the first application that comes to mind when people think of #GenAI. After all, what could be better than an AI that understands customer needs and responds helpfully, 24/7? However, as exciting as the promise is, we must be realistic about the challenges involved in developing and operating customer facing chatbots: 1. Fine-tuning a large language model (LLM) and / or leveraging retrieval augmented generation (RAG) requires high-quality, labelled, and organised customer service data. Most companies have yet to assemble such datasets. 📚 2. Serving GenAI chatbots at scale can be costly, especially if conversations aren’t volume restricted and / or limited to specific topics. Without guardrails, customers can use the chatbot for any conversation. 😱 3. LLM security vulnerabilities like prompt injection and model poisoning are major concerns for deploying customer facing chatbots. ☠️ 4. LLMs can produce different outputs for similar prompts. Minimising variability requires human oversight and providing customers with templated prompts, thereby limiting the user experience. 📊 5. Similarly, closed source LLMs change over time, resulting in different outputs for the same prompts. Lack of internal control / governance over such changes makes it hard to anticipate new behaviours. 👽 6. In heavily regulated industries like financial services and healthcare, Gen AI chatbots must walk a fine line between assisting customers and providing financial or health advice, which only certified professionals should give. 👩⚕️ 7. And what if the customer loses out because of a chatbot? Who is accountable - the customer, the company, or the AI provider? This and other questions are yet to be addressed by governments and regulators. In the UK, FCA's Consumer Duty will likely make the company accountable for customer losses caused by AI. 🏛️ Should companies abandon hope of using Gen AI in customer service? Not at all! But the better use cases in 2024 will be low(er) stakes applications like content generation and search, FAQs or virtual assistants, augmenting human agents rather than fully automating customer interactions. What are your experiences implementing Gen AI chatbots? Are you optimistic or pessimistic about Gen AI for customer service? #GenerativeAI #Chatbot #AI #AIforGood Image: Petr Vaclav & Playground v2, “Chatborg”, 2024

Human conversation is interactive. As others speak you are thinking about what they are saying and identifying the best thread to continue the dialogue. Current LLMs wait for their interlocutor. Getting AI to think during interaction instead of only when prompted can generate more intuitive and engaging Humans + AI interaction and collaboration. Here are some of the key ideas in the paper "Interacting with Thoughtful AI" from a team at UCLA, including some interesting prototypes. 🧠 AI that continuously thinks enhances interaction. Unlike traditional AI, which waits for user input before responding, Thoughtful AI autonomously generates, refines, and shares its thought process during interactions. This enables real-time cognitive alignment, making AI feel more proactive and collaborative rather than just reactive. 🔄 Moving from turn-based to full-duplex AI. Traditional AI follows a rigid turn-taking model: users ask a question, AI responds, then it idles. Thoughtful AI introduces a full-duplex process where AI continuously thinks alongside the user, anticipating needs and evolving its responses dynamically. This shift allows AI to be more adaptive and context-aware. 🚀 AI can initiate actions, not just react. Instead of waiting for prompts, Thoughtful AI has an intrinsic drive to take initiative. It can anticipate user needs, generate ideas independently, and contribute proactively—similar to a human brainstorming partner. This makes AI more useful in tasks requiring ongoing creativity and planning. 🎨 A shared cognitive space between AI and users. Rather than isolated question-answer cycles, Thoughtful AI fosters a collaborative environment where AI and users iteratively build on each other’s ideas. This can manifest as interactive thought previews, real-time updates, or AI-generated annotations in digital workspaces. 💬 Example: Conversational AI with "inner thoughts." A prototype called Inner Thoughts lets AI internally generate and evaluate potential contributions before speaking. Instead of blindly responding, it decides when to engage based on conversational relevance, making AI interactions feel more natural and meaningful. 📝 Example: Interactive AI-generated thoughts. Another project, Interactive Thoughts, allows users to see and refine AI’s reasoning in real-time before a final response is given. This approach reduces miscommunication, enhances trust, and makes AI outputs more useful by aligning them with user intent earlier in the process. 🔮 A shift in human-AI collaboration. If AI continuously thinks and shares thoughts, it may reshape how humans approach problem-solving, creativity, and decision-making. Thoughtful AI could become a cognitive partner, rather than just an information provider, changing the way people work and interact with machines. More from the edge of Humans + AI collaboration and potential coming.

I'm now spending around 40-50% of my time with clients on AI. Polishing prompts, setting up workflows. Here's the top 3 most common mistakes I see: 1. Trying to provide too much information in the context window. What's too much? 𝗥𝗲𝗱𝘂𝗻𝗱𝗮𝗻𝘁 𝗰𝗼𝗻𝘁𝗲𝗻𝘁: Repeating the same information multiple times or including verbose explanations that could be summarised. 𝗜𝗿𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝗱𝗲𝘁𝗮𝗶𝗹𝘀: Information unrelated to the task at hand that dilutes what's important. 𝗘𝘅𝗰𝗲𝘀𝘀𝗶𝘃𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲𝘀: Providing 10+ examples when 2-3 would sufficiently illustrate the concept 𝗨𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲𝗱 𝗱𝘂𝗺𝗽𝘀: Large blocks of unformatted text, logs, or data without clear organisation 𝗙𝘂𝗹𝗹 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝘄𝗵𝗲𝗻 𝗲𝘅𝗰𝗲𝗿𝗽𝘁𝘀 𝘀𝘂𝗳𝗳𝗶𝗰𝗲: Including entire papers or articles when only specific sections are relevant 𝗞𝗲𝘆 𝗶𝗻𝗱𝗶𝗰𝗮𝘁𝗼𝗿𝘀 𝘆𝗼𝘂'𝘃𝗲 𝗵𝗶𝘁 "𝘁𝗼𝗼 𝗺𝘂𝗰𝗵": • The model struggles to find relevant details buried in noise • Response quality degrades due to information overload • Important instructions get lost in the volume 2. Being either too loose or too prescriptive. Some clients operate within rigid systems (like optimising for pre-defined feeds or API outputs). So they don't understand that large language models operate best when provided with - natural language examples. On the too loose spectrum: • "Be helpful and accurate" (no specifics on HOW) • "Write in a professional tone" (what does professional mean?) • "Keep responses appropriate length" (what's appropriate?) • No examples of desired outputs • Vague quality criteria 3. Asking the AI to see the future. Not understanding that the AI is drawing on what's readily available in it's dataset. That being everything it's ingested on the internet. It isn't 'thinking' and able to come up with innovative solutions to niche areas it has little context on. Which ones I did miss?

The advancement of artificial intelligence, especially the development of sophisticated chatbots, has significantly changed how we find and share information. While these chatbots exhibit remarkable proficiency with human language—evident in their ability to craft compelling stories, mimic political speeches, and even produce creative works—it’s crucial to recognize their limitations. They are not perfect. In fact, chatbots are not only prone to mistakes but can also generate misleading or entirely fabricated information. These fabricated responses often appear indistinguishable from credible, evidence-based data, creating a serious challenge for informed decision-making and constructive dialogue. At the heart of these chatbots are large language models (LLMs), which function by predicting words based on massive datasets. This probabilistic mechanism enables them to produce logical, coherent text. However, it also means they are inherently prone to errors or "hallucinations." When chatbots are designed to sound authoritative, a mix of accurate and fabricated information can inadvertently contribute to the spread of both misinformation and disinformation. This risk becomes particularly alarming in areas like political communication or public policy, where persuasive language can easily slip into manipulation. Even with decades of advancements, modern AI technologies are still essentially advanced imitations of human conversation. These systems remain largely opaque "black boxes," whose internal operations are often not fully understood, even by their creators. These innovations have yielded groundbreaking applications for customer support, digital assistants, and creative writing, they also amplify the danger of users being misled by inaccuracies. From both regulatory and ethical perspectives, the rise of chatbots capable of fabricating information demands urgent attention. The responsibility for creating safeguards cannot exclusively lie with the companies that develop and benefit from these tools. Instead, a comprehensive, collaborative approach is critical. This approach should include greater transparency, stringent fact-checking mechanisms, and international cooperation to ensure that these powerful AI systems are used to educate and inform rather than mislead or deceive.