Smart Chatbot App is a full-stack NLP system designed to explore adaptive inference, dynamic learning, and conversational AI. It combines a BERT-based intent classifier with a GPT-2 generative module and introduces a novel meta-learning layer called Dynamic Input Activation Network (DIAN). DIAN enables the model to introspectively evaluate its internal states and adjust behavior at runtime via learned intuition coefficients. This architecture simulates confidence modulation and supports lightweight online learning, providing a flexible research platform for dynamic neural systems and hybrid intent-generation pipelines.
Source code: github.com/nbursa/smart-chatbot-app
Conversational agents have become a key application domain for natural language processing (NLP), yet most systems rely on static models that do not evolve with interaction. While transformer-based architectures such as BERT and GPT-2 have improved intent recognition and text generation, they typically lack mechanisms for introspection or adaptive behavior.
The Smart Chatbot App project was initiated to address this limitation by combining traditional NLP pipelines with a dynamic meta-learning layer. At its core, the system fuses a BERT-based intent classifier, a GPT-2 text generator, and an original architectural concept: the Dynamic Input Activation Network (DIAN). DIAN acts as an "intuition module" that tracks internal layer states, detects representational drift, and adjusts confidence coefficients on the fly.
This hybrid design serves both practical and research purposes, enabling experimentation with online learning, neural confidence, and context-aware response generation—all within a modular, full-stack implementation.
Conversational AI is rapidly adopted in sectors such as e-commerce, banking, and hospitality. Over 65% of online interactions are now assisted by chatbots (Statista, 2024). However, most systems rely on static intent classification, lacking real-time self-adjustment.
Smart Chatbot App introduces adaptive inference into this landscape, offering a novel, scalable model for evolving user needs and unpredictable query patterns.
This project investigates the following core questions:
This project intersects several active research areas in natural language processing and deep learning architecture design.
Transformer-based models like BERT [Devlin et al., 2018] have become standard for intent classification tasks due to their contextual embeddings and transfer learning capabilities. Meanwhile, GPT-2 [Radford et al., 2019] and its successors have demonstrated strong generative performance for conversational agents.
However, most of these models operate as fixed systems post-training, lacking mechanisms for real-time introspection or online behavioral adjustment. Recent efforts in meta-learning and adaptive networks (e.g., MAML, dynamic routing) attempt to address these gaps.
Smart Chatbot App introduces a new architectural layer—Dynamic Input Activation Network (DIAN)—that enables models to track internal activation patterns and self-regulate through learned intuition coefficients. This design shares conceptual similarities with confidence-aware learning and self-attention but is implemented as a post-BERT feedforward layer modulating network trust during inference.
The system builds upon these research threads to explore how adaptive neural behavior can enhance real-time NLP applications.
The Smart Chatbot App follows a modular three-layer design:
bert-base-uncased model. The pooled output is passed to a custom neural component, IntuitionNN, which is a multi-layer feedforward network with optional skip connections and gated control layers.
Figure: The BERTIntuitionModel class integrates HuggingFace BERT with a custom IntuitionNN component.
intuition_layer that estimates expected internal activation statesintuition_coefficients that are updated at runtime based on internal deltascompare_and_adjust) that softly adjusts weights over time
Figure: The compare_and_adjust() function compares layer activations with expectations and adjusts coefficients and weights online.
These assumptions guide architecture and performance expectations in current deployment.
The custom dataset used in Smart Chatbot App was manually constructed from short user queries, conversational utterances, and labeled intents. Preprocessing steps included:
BertTokenizer from HuggingFaceNo imputation or synthetic generation was performed. The final dataset was split into 80% training and 20% validation, with random shuffling.
After each user interaction:
intuition_coefficients are updated using a custom introspection mechanismThis methodology enables the agent to learn, adapt, and evolve during actual usage while maintaining modular control and separation of concerns across the system.
Figure: Flow diagram illustrating how DIAN processes activations, compares expectations, and updates coefficients during inference.
To evaluate the performance and behavior of the system, we conducted a series of small-scale experiments focused on intent classification accuracy, online adaptation impact, and coefficient drift tracking.
A custom dataset was created with 5 intent categories (e.g., question, command, greeting, feedback, request). Each example was a short natural-language utterance. The data was split into 80% training and 20% validation subsets and tokenized using the BertTokenizer.
BERTIntuitionModel (BERT + IntuitionNN)plot_loss.py)The model achieved high accuracy on intent classification. Notably, the intuition layer adjusted weights more aggressively on uncertain predictions, improving recall. Unexpectedly, intuition adjustments tended to converge faster on more ambiguous classes (e.g., general queries), indicating potential alignment with implicit intent variance.
These insights suggest that DIAN not only tracks learned intent but may amplify latent uncertainty, acting as a soft regularize.
While this study focuses on the novel intuition-driven architecture (DIAN), direct comparisons with the following baselines are planned:
Future work will benchmark Smart Chatbot App performance across these models using accuracy, F1-score, and average response alignment.
To validate the findings, we conducted basic statistical tests:
These results provide confidence in the observed performance improvements.
In production, the Smart Chatbot App would require the following monitoring strategy:
These steps help ensure long-term accuracy and reliability of the system.
The Smart Chatbot App demonstrates a modular, full-stack architecture for building adaptive conversational agents with real-time learning capabilities. By integrating BERT for intent classification, GPT-2 for generative responses, and the novel DIAN layer for introspective modulation, the system enables lightweight behavioral adaptation during usage.
DIAN (Dynamic Input Activation Network), introduced as part of this work, provides a meta-layer that simulates neural intuition by tracking internal activations and adjusting coefficients at runtime. This opens new possibilities for exploring confidence modeling, dynamic behavior, and meta-learning within real-world NLP pipelines.
While the system remains in an experimental stage, the results show promising signs of adaptability and responsiveness. Future work includes scaling the dataset, isolating session memory per user, conducting broader comparative evaluations, and deploying the model in semi-production conversational settings.
This project contributes to the growing exploration of adaptive AI systems—where agents not only respond, but evolve.
Initial deployment in a small coffee-shop ordering simulation showed effective query resolution with real-time learning. For instance, after receiving multiple ambiguous inputs (“I want something sweet”), the intuition layer gradually adapted to prioritize dessert drinks.
Failures included overfitting on repeated phrasing, which was mitigated by increasing dropout in IntuitionNN.
These experiments guided architectural tweaks and validated DIAN’s practical utility.
Full implementation available on GitHub:
🔗 github.com/nbursa/smart-chatbot-app