ChatGPT utilized by Microsoft to instruct robots and drones
To see if people can use ChatGPT to instruct robots without knowing programming languages or understanding robotics systems, Microsoft has conducted research.
In a blog post, Microsoft stated that the key challenge is teaching ChatGPT how to solve problems. Things considered include the laws of physics and the context of the operating environment. Also, they emphasized how the robot’s physical actions can change the state of the world.
It added, “It turns out that ChatGPT can do a lot by itself, but it still needs some help. Our technical paper describes a series of design principles that can be used to guide language models toward solving robotics tasks. These include, and are not limited to, special prompting structures, high-level APIs, and human feedback via text.”
Microsoft researchers investigated its ability to generate code, mostly in Python. It benefited robotics scenarios such as zero-shot planning and code generation. This happened after ChatGPT was given access to object-detection and object-distance data via application interfaces
The AI chatbot can generate code as it was trained on a large amount of code and written text. According to the researchers, the system can solve coding problems and debug programs. Also, it can respond to dialogue and seek clarification.
In addition, ChatGPT was tested as a language-based interface. It occurred between a non-technical user and a drone using these dialogue and clarification capabilities in mind.
The GPT-3, LaMDA, and Codex demonstrated good results, however, when it comes to robotics planning and coding, ChatGPT stands out as “a potentially more versatile robotics tool because it incorporates natural language and code generation models with dialogue flexibility,” the researchers note. They said,
“ChatGPT asked clarification questions when the user’s instructions were ambiguous and wrote complex code structures for the drone such as a zig-zag pattern to visually inspect shelves.”
Microsoft tested ChatGPT to use a robotic arm to move blocks around to form the Microsoft logo. This happened with writing an algorithm for a drone to reach a point without crashing into obstacles, and whether ChatGPT can decide where a robot should go based on sensor feedback in real-time.
