Bob-E: A remote presence Robot controlled through a VR headset

The Bob-E Robot is our experimental hardware. We are proud that we were able to build it. We have designed our own 3D printed parts, we have built our own electronic circuit boards, and have writter all the software for the microcontrollers and higher level control. Lower level software was written in C++, higher level control software was written in C#.Net. The robot is controlled by a Raspberry Pi mini computer, where the C# code runs on mono. The cameras, and the microcontrollers managing the motors and sensors are connect through USB. It was a fun project, it brought a lot of enjoyment to our team.

Bob-E, a friendly VR robot, that allows you to look around, move around and enjoy being there

About this project


Bob-E is a new virtual reality (VR) robot designed and built by Ozeki specifically to provide free movement and head tracking. It will change the way you think about telecommunication and remote presence forever. Bob-E provides a truly immersive experience that allows you to take part in activities like never before.

The original goal of building Bob-E was to allow our friends and colleagues to take part in activities from a remote location. The side effect was that we created a friend. Bob-E is more than a robot controlled by a human, he is a member of the community. No matter who controls it, he is fun to be with. He is always around, and he has an amazing personality.

Once you take control of Bob-E, you will feel what it is like, when people smile at you, and how you will become a closer friend to those around you. You will learn and experience things you would not learn or experience if you were there as a human.

The team

robot builder and developer team
Figure 1 - The Bob-E team

The only thing you will miss, is that you were not there with us from the beginning of this project. While we built Bob-E we had the best times of our lives. When Bob-E was created, the software developers, the electrical engineers and the 3D designers were all teasing and pushing each other, and this activity provided a great number of opportunities for fun and inspired outstanding achievement.

We are now more passionate about building robots, then we were ever before. We have learnt to work together amazingly well. It would be great if you could join us for a day at our office and experience the atmosphere.

Bob-E provides a 3D VR experience

When you start to use Bob-E, you will be amazed at how easy it is to operate. You simply have to put your smartphone (Android or iPhone) into the VR headset and turn your head to look around. You will also have a small bluetooth controller (a joystick) in your hands that allows you to move Bob-E around the room.

When you turn your head, the cameras of Bob-E will track every movement synchronously in real time. The robot offers fast, real time stereo 3D vision, that allows you to see what the robot sees just as if you were using your own eyes.

Amazing speed

The key feature of real time head tracking is possible, because Bob-E's cameras are super fast. The cameras follow your head movements at an amazing speed of 0.19 seconds / 60 degrees. If you think about it: it is less then a 1/5th of a second for an average head movement. We haven't heard about any other device that even comes close.

robot mimics the movement of the head
Figure 2 - Bob-E mimics the movement of the head


The experience is also supported by Bob-E's physical design. Bob-E's cameras are exactly 65mm apart, just like the human eyes. This provides a wonderful, realistic 3D vision when you look through the VR headset. It is a true 3D experience.

robot eye distance
Figure 3 - Bob-E eye distance

How can Bob-E be used

Bob-E is an excellent tool if you wish to use it in the office, or at an event and take part in activities from a remote location. It provides an amazing virtual presence capability. It allows you to go where you wish and look at what you want. The motors are powerfull and the 3D vision capabilities and the optics / resolution of the cameras are designed to allow you to

Capabilities

  • Read the text of documents on the desk
  • Watch the contents of the computer screen
  • Read sketches and text on a whiteboard
  • Enjoy a presentation
  • Participate in random discussions
  • Be able to go up to somebody
  • Be able to look around freely
  • Carry a payload of 5kg

    Private use

  • Participate in family events
  • Entertain friends
  • Education
  • Dating and creating friendships

    Business use

  • General office collaboration
  • Inspections and control
  • Conferences
  • Events, parties
  • Security

    Technical insights

    When you get your hands on Bob-E you will see, that it's technology is state of the art. Bob-E's head movements are controlled by data collected from the motion sensors of the smartphone in the VR headset. These sensors track the head movement, and allow the software to instruct Bob-E through the network to turn his head.

    You will love that Bob-E can move around. To instruct it to go forward, backward, or to turn, a small wireless controller (a joystick) can be used. This controller talks with the mobile phone through Bluetooth.

    robot control and communication
    Figure 4 - Bob-E's control and communication

    Feature list

    Wireless network: When you use Bob-E, you connect to it from your smartphone through Wi-Fi. The default settings for Bob-E is to create it's own Wi-Fi network. Bob-E will show up as a network on your mobile device. You also have the option to use it in Wi-Fi client mode. For example it can connect to the mobile Internet connection shared by a mobile phone.

    robot wi fi ap
    Figure 5 - Bob-E Wi-Fi AP

    The following video shows the bootup procedure of Bob-E, and the Wi-Fi connection setup:

    When you setup the wireless communication, you will also have to connect the bluetooth joystick to your mobile phone. The following video shows this process:

    Head tracking: Bob-E's head tracking uses the motion sensors of the mobile phone. These sensors can be accessed through the webbrowser, so no app needs to be installed. The following video shows how you can connect to the user interface of Bob-E and how you can turn on head tracking. Note that the first step of head tracking is called synchronization. During this proces the initial orientation of the mobile phone is pushed to Bob-E, so the camera orientation can be set.

    Movement: Bob-E's movement can be controlled with the joystick. When you push the joystick forward it will go forward, when you push it back it will go backward. To turn, you can use the joystick to turn the front wheel. (You can also adjust the acceleration and the top speed of Bob-E in the configuration section of the GUI)

    use the joystick to turn the front wheel left or right
    Figure 6 - Use the joystick to turn the front wheel left or right

    The following video demonstrates two modes for turning the front wheel. You can use the GUI on the mobile phone to make high precision movements. You may use the joystick to make fast movements. Notice that the webbrowser GUI captures the commands from the Bluetooth joystick, and highlights the corresponding arrows in blue, when a command from the joystick is received.

    To go forward or to reverse, both the graphical user interface and the joystick can be used. The graphical user interface can offer more precise, the joystick a more natural movement.

    use the joystick to go forward or backward
    Figure 7 - Use the joystick to go forward or backward



    Display: Bob-E's display can be used to display images. You may upload images to Bob-E using a webbrowser, and you may use the soft button on the joystick to switch between uploaded images. You may display a map, or a QR code at airport checkin, or any other visual information.

    display images
    Figure 8 - Display images

    How will Bob-E become your friend

    One of the first thing you will notice, when you start to use Bob-E is, that he is the right height to look you in the eyes while you are sitting. This height is also optimal to read the documents on the desk. It is very funny when you control Bob-E and you turn your head and look at somebody after reading a page. People don't expect the robot to be able to quickly turn it's head and look them in the eyes. It was very funny, when Bob-E did this on one of our meetings, Peter, one of the team member was so suprised, he fell of his chair when Bob-E turned to him. It caused great laughter.

    robot can look you in the eye
    Figure 9 - Bob-E can look you in the eye

    When you use Bob-E, you will also enjoy that after everybody get's used to the robot moving around, you can sneak behind somebody and watch his computer screen. Unless they are prepared, they will not notice. This will give you the opportunity to see what happens in the office, what people are working on.

    Another nice feature is that you can make comments very politely using body languague. You can use Bob-E's head to make gestures, such as the head nod up and down to say 'yes', and the head nod from side to side to say 'no'.

    robot is watching the display
    Figure 10 - Bob-E is watching the display

    Once Bob-E becomes part of the team, it is hard to imagine life without him. Bob-E is so much fun to have around, especially, it the person controlling it has a sense of humor.

    it is hard to imagine life without using robot
    Figure 11 - It is hard to imagine life without Bob-E

    People interacting with Bob-E quickly treat him as a friend, because he is always interested in new ideas and listens carefully.

    robot reads text on a tablet
    Figure 12 - Bob-E reads text on a tablet

    Bob-E as a waiter

    We regret to tell you that some of our team members are lazy. It was an early requirement in our office to equip Bob-E with a tray to carry drinks around. The first tray we used was not good enough beacuse glasses were sliding around in it. The second was too small, it could only carry one glass. The third attempt was perfect. Now we have a non-slip tray that can carry several glasses. To see it in action check out the following video:

    Under the hood

    If you are interested in technology, you might be interested to know that Bob-E is operated by a next generation operating system called Bob-E OS. This system was designed and implemented from scratch by our software developers. It took more than two years to build it. This operating system allows full hardware control, parallel sensor and actuator management, it provides audio and video support and has outstanding communication capabilities. This operating system manages Bob-E's motion and vision by controlling it's stereo 3D cameras.

    In terms of hardware Bob-E has powerful components, check out the technical details:

    robot rechnical details
    Figure 13 - Bob-E Robot Technical Details

    The Bob-E package

    If you decide to purchase Bob-E, you will enjoy all the benefits it offers. The complete package includes: Bob-E robot, a VR headset and a bluetooth controller. Please note, it does not include a mobile phone. You will have to put your own Android mobile or iPhone into the headset, but it supports every mobile with a webbrowser and a gyroscope.

    robot package contents
    Figure 14 - Bob-E package contents


    We would also like you to see that the biggest reward for us in building this product was that we could be part of this project. We hope that you can be part of this project as well, by making a purchase. If you are unsure, check out the following video about our marketing campaign:

    We will include a Non-Slip tray kit with all Bob-E Robots, because we are sure you would also like to carry drinks around with Bob-E. The Non-Slip tray kit consists of a round 290 mm diameter x 50 mm high non-slip tray, and a strong mounting kit that allows you to mount it on Bob-E. The tray has a brushed aluminium effect finish with a non-slip coated surface for greater ease of carrying and deep sides, designed with health and safety in mind, meaning that even at the busiest of times Bob-E can carry multiple drinks safely and securely.

    robot non slip tray kit
    Figure 15 - Bob-E's Non-Slip Tray Kit

    Frequently Asked Questions (FAQ)

    Can Bob-e handle rough terrain? Bob-E can go up slopes up to 10% in elevation, it cannot go on sandy beaches and tall grass fields. It cannot handle mud.

    What is the maximum battery life? One fully-charged battery will give Bob-E 30-120 minutes of continuous operation, obviously it is usage dependent. 70% of power is pulled by Bob-e's motors. Constant communication or human interaction will have very little impact on battery life! The battery can be easily replaced to extend operation time.

    Will there be an SDK (Software Development Kit available)? - We will provide a well documented API that exposes all the core functionality of the robot. It is our intention to build a community of developers, so you can expect great software developer support.

    Can the camera resolution be upgraded? - Bob-E uses standard USB cameras. You can replace the cameras to higher resolution. You must note, that higher resolution cameras will require more processing power and bandwidth. This will have a bad effect on battery life and might cause problems on lower bandwidth (mobile Internet) connections.

    Can Bob-e carry extra load? - The Bob-E motors used for movement are equipped with a gear box, so they are quite powerful. You can add an extra 5kg of weight to Bob-E without any issues.

    More information