Generating Human Daily Activities with LLM for Smart Home Simulator Agents

International Conference · 2024

Generating Human Daily Activities with LLM for Smart Home Simulator Agents

Keywords

Large Language ModelsAgent-Based ModelingSmart Home SimulationDigital TwinCyber-Physical Systems

Haruki Yonekura , Fukuharu Tanaka , Teruhiro Mizumoto , Hirozumi Yamaguchi

The 20th International Conference on Intelligent Environments (IE2024)

Award Best Short Paper Award

DOI: 10.1109/IE61493.2024.10599909

Abstract

This paper presents a novel approach to smart home simulation by integrating Large Language Models (LLMs) for generating human daily schedules and activities of smart home simulator agents. The use of LLMs aims to reduce the complexity of user settings and increase the variety and generality of generated activities. This study explores the potential of LLMs to simulate human-like activity-generating by exploiting their experiential knowledge and adaptability. In addition, this paper discusses fine-tuning techniques for LLMs to optimize their performance within a simulator. By utilizing LoRA and task-specific fine-tuning datasets, while maintaining the ability to generate proper activities, we achieve as much as 4.3% performance improvement about the number of queries. Overall, the integration of LLMs into smart home simulators offers promising opportunities for enhancing the intelligence and adaptability of virtual agents in smart home environments.

Research Note

This research proposes a method for simulating residents' decision-making and daily activities in a smart home environment using a Large Language Model (LLM). In recent years, Digital Twins and Agent-Based Modeling (ABM) have gained attention for real-world applications. However, existing agents often have simplified behavioral patterns, making it difficult to fully replicate the diversity of human behavior. By leveraging LLMs, this study aims to develop a more human-centric and flexible approach to modeling decision-making processes.

Published Paper

  • 米倉晴紀, 田中福治, 水本旭洋, 山口弘純. (2023). スマートホームシミュレータにおける大規模言語モデルを用いた生活行動の自動生成に関する検討. 研究報告モバイルコンピューティングと新社会システム (MBL)2023(28), 1-8. https://ipsj.ixsq.nii.ac.jp/records/228980
  • H. Yonekura, F. Tanaka, T. Mizumoto and H. Yamaguchi, "Generating Human Daily Activities with LLM for Smart Home Simulator Agents," 2024 International Conference on Intelligent Environments (IE), Ljubljana, Slovenia, 2024, pp. 93-96, https://ieeexplore.ieee.org/document/10599909


A Platform for Digitalizing Knowledge of Regional Communities

A Platform for Digitalizing Knowledge of Regional Communities

Japan Science and Technology Agency (JST) CREST
「基礎理論とシステム基盤技術の融合によるSociety 5.0のための基盤ソフトウェアの創出」領域

A Simulation-based Framework for Dynamic Light Pollution Prediction in Urban Air Mobility

Ying Chieh Wang, Tatsuya Amano, Hirozumi Yamaguchi

2026 IEEE International Conference on Smart Computing (SmartComp), Messina, Italy, 2026, pp. 136-143

DOI 10.1109/SmartComp69968.2026.00029

Urban Air MobilityUAM +2

Environment-Aware Distributed Scheduling for Emergency LoRa Networks

Yuto Inaba, Tatsuya Amano, Akihito Hiromori, Hirozumi Yamaguchi

2026 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), SPT-IoT 2026, pp. 1366–1371

DOI 10.1109/PerComWorkshops68308.2026.11585469

Disaster CommunicationLoRa +4

A Lightweight Vision-Language Model for Disaster Image Summarization

Hibiki Yoshizaki, Akira Uchiyama, Akihito Hiromori, Mineo Takai, Hirozumi Yamaguchi

2026 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), PerconAI 2026, pp. 1203–1208

DOI 10.1109/PerComWorkshops68308.2026.11585419

Semantic CommunicationDisaster Response +4

Physics-Integrated Deep Learning for Urban Landslide Prediction

Ren Ozeki, Hamada Rizk, Hirozumi Yamaguchi

2026 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), URBSENSE 2026, pp. 1094–1099

DOI 10.1109/PerComWorkshops68308.2026.11585337

Landslide PredictionPhysics-Integrated Learning +3

A Simulation Framework for Precision Formation Flying of Massive Satellite Swarms

Tatsuya Amano, Akihito Hiromori, Hirozumi Yamaguchi, Sumio Morioka

2026 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), PerVehicle , pp. 230–235

DOI 10.1109/PerComWorkshops68308.2026.11585321

Satellite Formation FlyingDistributed Simulation +4

Ray-Tracing-Driven Pattern-Based Vehicle Recognition in ISAC Radar

Heetae Jin, Akira Uchiyama

2026 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), PerRad 2026, pp. 328–333

DOI 10.1109/PerComWorkshops68308.2026.11585327

ISACBeyond 5G +4