Leveraging LLMs for Dynamic IoT Systems Generation through Mixed-Initiative Interaction

Bassam Adnan1, Sathvika Miryala1, Aneesh Sambu1, Karthik Vaidhyanathan1, Martina De Sanctis2, Romina Spalazzese3
1IIIT Hyderabad, India, 2GSSI, L'Aquila, Italy, 3Malmo University, Sweden
Paper arXiv Video Code

Three-Pass Dialogue Flow

Three-Pass Dialogue Flow
Progressive Identification of User Goals and Service Parameters enabling Goal-Driven Architecture

Abstract

IoT systems face significant challenges in adapting to user needs, which are often under-specified and evolve with changing environmental contexts. To address these complexities, users should be able to explore possibilities, while IoT systems must learn and support users in the process of providing proper services, e.g., to serve novel experiences. The IoT-Together paradigm aims to meet this demand through the Mixed-Initiative Interaction (MII) paradigm that facilitates a collaborative synergy between users and IoT systems, enabling the co-creation of intelligent and adaptive solutions that are precisely aligned with user-defined goals.

Method: Three-Pass Dialogue Framework

Our system employs a three-pass approach to understand and fulfill user needs:

Pass 1: Contextual Awareness

Builds a picture by gathering environmental data (weather, traffic) along with user details (location, preferences) initially.

Pass 2: Goal Refinement

Conducts a two-way conversation using smart suggestions and follow-up questions, helping match what you want with what's available.

Pass 3: Service Generation

Puts together a package of services based on our earlier chat, and checks if they work for you asking for confirmation- if not, we start fresh from Pass 1.

Backend Generation

Backend Generation Image 1
Backend Generation Image 2
Backend Generation Image 3
Backend Generation Image 4

As shown in figure above, upon the detection of missing services, (1) a promt is passed by the Query Refiner, which matches the requirements of the prompt to existing services. If such a service is not found, then (2) the databases are utilized to create a new service. The Query Refiner prepares a second prompt for the Service Generator to create a new service. This new services is added to the entry list of the Service Manager (4).

Results

Goal Parser Performance

Our evaluation demonstrates the system's ability to interpret both concrete and ambiguous user queries:

Query Analysis Examples

Type User Query Services Parameters
Concrete "I'd love to visit Charminar and understand its history. Wonder if it gets too crowded on weekends?"
  • historical_info
  • crowd_monitor
 
{
        "historical_info": {
          "site_name": "charminar"
        },
        "crowd_monitor": {
          "location_name": "charminar"
        }
      }
Ambiguous "Love anything artsy with a story behind it. Places where I can learn about local culture while enjoying the atmosphere."
  • exhibition_tracker
  • historical_info
  • restaurant_finder
 
{
        "exhibition_tracker": {
          "exhibition_type": ["art", "cultural"]
        },
        "historical_info": {
          "site_name": "salar jung museum"
        },
        "restaurant_finder": {
          "cuisine_type": "hyderabadi"
        }
      }

Model Performance Metrics

Performance comparison between DeepSeek-V2.5 and GPT-4o-mini across query categories:

Model Category Precision Recall F1
GPT-4o-mini Ambiguous 0.683 0.795 0.730
Concrete 0.467 0.773 0.559
Overall 0.523 0.778 0.603
DeepSeek-V2.5 Ambiguous 0.681 0.788 0.725
Concrete 0.492 0.830 0.591
Overall 0.554 0.816 0.635

System Performance Analysis

Token Consumption Graph
Token consumption scaling with increasing number of services
Token Distribution
Token distribution during application generation

Smart City Case Study

Inspired by IIIT Hyderabad's Smart City Living Labs integration of IoT sensors monitoring key campus parameters, where the network includes temperature readings in different locations of the campus, air quality monitors across outdoor spaces, and specialized water quality sensors at water bodies and drainage systems. We consider a scenario where visitors need multiple separate applications to access information about restaurant availability, classroom occupancy, and other campus services - a fragmentation we aim to unify through integrated solutions that scale to larger urban environments.

Our implementation builds upon this sensor infrastructure to provide a range of services, including event notifications, historical site information systems, crowd monitoring, air quality assessment tools, exhibition tracking, restaurant recommendation engines, ticket purchasing platforms, travel planning tools, and water quality monitoring services. To evaluate the system's effectiveness, we conducted user studies (n=15) with participants from diverse academic backgrounds, including Ph.D. students, Electronics/Communications Engineering students, and Computer Science students. Each participant interacted with the system for 10-15 minutes, with feedback collected through an integrated form in the user interface. The results showed high satisfaction rates with an average rating of 4.1/5 for service accuracy and 4.2/5 for service relevance.

BibTeX

@inproceedings{adnan2024leveraging,
  title={Leveraging LLMs for Dynamic IoT Systems Generation through Mixed-Initiative Interaction},
  author={Adnan, Bassam and Miryala, Sathvika and Sambu, Aneesh and Vaidhyanathan, Karthik and De Sanctis, Martina and Spalazzese, Romina},
  booktitle={},
  year={2024}
}