Generación y Ajuste Dinámico de Behavior Trees mediante LLMs para el Control de un Robot Social

  1. Merino Fidalgo, Sergio 1
  2. Zalama Casanova, Eduardo 1
  3. Gómez García-Bermejo, Jaime 1
  4. Duque Domingo, Jaime 1
  1. 1 Universidad de Valladolid
    info
    Universidad de Valladolid

    Valladolid, España

    ROR https://ror.org/01fvbaw18

    Geographic location of the organization Universidad de Valladolid
Journal:
Jornadas de Automática
  1. Mulero Martínez, Juan Ignacio (coord.)
  2. Baños Torrico, Alfonso (coord.)
  3. Torres Sánchez, Roque (coord.)

ISSN: 3045-4093

Year of publication: 2025

Issue: 46

Type: Article

DOI: 10.17979/JA-CEA.2025.46.12071 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

Abstract

Large Language Models (LLMs) have become key tools for generating flexible and context-aware robotic behaviors. However, adapting to unforeseen events and ensuring robust task completion remain significant challenges. This work presents a system that uses LLMs and Behavior Trees (BTs) to enable a social robot to generate, execute, and adapt task plans based on natural language instructions. By combining a BT planner with a failure interpretation module, the system dynamically adjusts BTs in response to execution errors or environmental changes. Unlike static BT-based methods, our approach detects problems and proposes alternatives or requests clarifications from the user in real time, improving human-robot interaction. We validate the system across various real-world scenarios, demonstrating its effectiveness in enhancing flexibility and resilience in dynamic environments.

Bibliographic References

  • Ao, J., Ren, Z., Su, H., Zhu, J., 2024. Llm-as-bt-planner: Large language models for behavior tree based robotic task planning. arXiv preprint arXiv:2402.08013.
  • Ben-Ari, M., Mondada, F., 2018. Finite state machines. Elements of Robotics, 55–61. DOI: 10.1007/978-3-319-62533-1 4
  • Biggar, O., Zamani, M., Shames, I., 2020. A principled analysis of behavior trees and their generalisations. arXiv preprint arXiv:2008.11906.
  • Brand, D., Zafiropulo, P., 1983. On communicating finite-state machines. Journal of the ACM (JACM) 30 (2), 323–342.
  • Cao, Y., Rajeswaran, A., Grover, A., Feinberg, V., 2023. Robot task planning with large language models via behavior tree generation. arXiv preprint arXiv:2310.06796.
  • Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., Xie, X., 3 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology 15, 39. DOI: 10.1145/3641289
  • Colledanchise, M., Ögren, P., 8 2017. Behavior trees in robotics and ai: An introduction. Behavior Trees in Robotics and AI. DOI: 10.1201/9780429489105
  • De la Cruz, P., Piater, J., Saveriano, M., 2020. Reconfigurable behavior trees: Towards an executive framework meeting high-level decision making and control layer features. arXiv preprint arXiv:2007.10663.
  • Deng, J., Lin, Y., 1 2022. The benefits and challenges of chatgpt: An overview. Frontiers in Computing and Intelligent Systems 2, 81–83. DOI: 10.54097/FCIS.V2I2.4465
  • Ghzouli, R., Berger, T., Johnsen, E. B., Dragule, S., Wasowski, A., 2020. Behavior trees in action: A study of robotics applications. In: Proceedings of the 13th ACM SIGPLAN International Conference on Software Language Engineering. ACM, pp. 127–140. DOI: 10.1145/3426425.3426942
  • Ghzouli, R., Berger, T., Johnsen, E. B., Dragule, S., Wasowski, A., 2023. Behavior trees and state machines in robotics applications. IEEE Transactions on Software Engineering 49 (1), 1–14. DOI: 10.1109/TSE.2021.3115347
  • Iovino, M., Scukins, E., Styrud, J., Ögren, P., Smith, C., 2021. A survey of behavior trees in robotics and ai. Robotics and Autonomous Systems 136, 103710. DOI: 10.1016/j.robot.2020.103710
  • Isla, D., 2005. Handling complexity in the halo2 ai. Game Developers Conference.
  • Izzo, S., Ardulov, V., Scioni, E., Mohan, R., Cheng, X., He, Y., Hager, G. D., 2024. Btgenbot: Behavior tree generation using compact language models. arXiv preprint arXiv:2402.00560.
  • Li, M., Lin, L., Lin, Z., Li, X., Du, Y., 2024. A study of behavior tree generation from natural language with large language models. arXiv preprint arXiv:2403.12658.
  • Lykov, A., Ravichandar, H., Klee, S., Park, D., Nagarajan, P., Troniak, D., Liu, Z., Jenkins, R., Khante, A., Chai, J. Y., et al., 2023. Llm-mars: Leveraging large language models for multi-turn, multi-modal, multi-scene human-robot interaction. arXiv preprint arXiv:2310.02291.
  • Mahadevan, R., Wang, S., Thomaz, A. L., 2024. Generative language models for expressive robot social behaviors. arXiv preprint arXiv:2402.09013.
  • Merino-Fidalgo, S., Casanova, E. Z., García-Bermejo, J. G., Domingo, J. D., 6 2025. Generación de behavior trees mediante llm para el control de un robot social. Simposios del Comité Español de Automática (CEA) 1. URL: https://ingmec.ual.es/ojs/index.php/RBVM25/article/view/10
  • Mishra, B., Kertesz, A., 2020. The use of mqtt in m2m and iot systems: A survey. IEEE Access 8, 201071–201086. DOI: 10.1109/ACCESS.2020.3035849
  • Ögren, P., Sprague, C. I., 5 2022. Behavior trees in robot control systems. Annual Review of Control, Robotics, and Autonomous Systems 5, 81–107. DOI: 10.1146/ANNUREV-CONTROL-042920-095314
  • OpenAI, 2024. Introducing chatgpt — openai. Accessed: 26-12-2024. URL: https://openai.com/index/chatgpt/
  • Pezzato, C., Hernandez Corbato, C., Bonhof, S., Wisse, M., 2020. Active inference and behavior trees for reactive action planning and execution in robotics. arXiv preprint arXiv:2011.09756.
  • Sánchez-Girón, C., Gómez, M. G., Domingo, J. D., García-Bermejo, J. G., Casanova, E. Z., 2024. Integración convnext-yolo mediante cvv para detectar ca´ıdas en robot social. Jornadas de Automática. DOI: https://doi.org/10.17979/jacea.2024.45.10788
  • Temi, 2024. Introducing temi robot v3. Accessed: 03-01-2025. URL: https://www.robotemi.com/product/temi-sales-contact/
  • Wang, T., Patel, R., Qian, Y., Mataric, M. J., 2024. Srlm: Social robot language modeling for goal-conditioned navigation. arXiv preprint arXiv:2401.11317.
Data source: Dialnet