Category: Robotics

No comments

Reinforcement Learning with Sim2Real on the Turtlebot3 platform (experiments)

  By , , , , , , , , , ,

While creating the concept of a new University module that has the students do a project with the Turtlebot3 robots and RL, a few ideas emerged. While evaluating different robot simulation tools for Reinforcement Learning, one in particular caught my eye: the Godot plugin “godot_rl_agents” by edbeeching. This was the result of a paper creating a bridge from Godot to RL libraries like StableBaselines3.

After trying the plugin it was clear that good result can be achieved quickly, there emerged the idea that students might be more encouraged learning the whole software stack when it involves a popular game engine instead of a “random” simulator with its own proprietary structure and configuration. So now it had to be proven that Sim2Real works with this Situation.

A friend modelled a “digital twin” of a turtlebot3, as the existing open source models usually used were very unoptimized and would hinder performance of actual training. It was purposefully minimal, but with accents to make it recognizable.

At first there was an example with just driving to a target point based on the map. No sensors needed.
Simulation:

This was the first result:

The robot visible moves pretty badly in this clip. The reason which was later found: When the sent velocity commands would result in a jerky movement, the controller kind of rejects it and sometimes only does a part of the movement. Or sometimes no movement at all. To counteract this, the input has to be smoothed out beforehand to resist rejection from the controller.

Here is the next experiment with the lerp in mind:

This was the result:

The video shows that the robots performance can definitely be improved regarding stability and sensor calculations. Another big problem is also very visible here in that the small metal caster on the back of the turtlebot is very incompatible with the labs’ carpet flooring. This will be mitigated in the future with wooden plates that will make up the box’s floor.

No comments

Master Thesis: Dog Training

  By , , , ,

My master thesis topic was “Verbal training of a robot dog“. In this thesis I have created a program stack that tries to simulate real dog training. There are a few pre programmed actions the dog can perform, and it can either “anonymize” the actions or have a few preloaded commands for them. The usual training was done without any previous knowledge and from scratch.

The robot dog used was the Unitree Go1 with various python libraries. The implementation can be found here on the GitHub repo: https://github.com/MisterIXI/verbal-dog-training

A training step goes likes this:
– (optional: Hotword recognition “Hey Techie!” to await actual speaking intent)
– Speech recognition with “Whisper” (Open-source Speech-to-text from OpenAI)
– Check if command has confirmed matches
– Check if command has a small Levenshtein distance to a confirmed command (like “sit” and “sat”)
– Query the local LLM which command could be used
– If the LLM fails or picks a confirmed negative, a random action is rolled from the remaining actions
– The dog executes the picked action
– The dog awaits Feedback: Listens to “Yes” & “Correct” for positive, and “No” & “Wrong” for the negative feedback
– The picked Command + Action Combo is memorized
– The Loop repeats

The end result was a soft success. The training itself had to rely on quite a bit of randomness, since a very weak and small LLM was used which could’ve accelerated the process immensely. The same goes for the speech recognition, which failed a lot of times and resulted in bogus text recognized. With the stronger models it worked way better, but the calculation time was reduced from practically real time to up to 30 seconds, which was unacceptable in this case.

No comments

Turtlebot3 Rebuild – Integrating a Raspi and Jetson nano in one robot

  By , , , ,

The university I was a student at has a bunch of Turtlebot3 for student projects. And they got upgraded with a Jetson nano instead of the stock Raspberry Pi it had originally. After one student project on the new platform, it was clear that the Jetson nanos were good for calculating AI stuff, but terrible for reliability. So there was a plan to bring back the Raspberry Pis, but also keep the Jetson Nanos.

This was the initial rebuild:

This setup now has:
– two motors
– the OpenCR Board 1.0 for motor and battery control
– A Raspberry Pi 4 4gb as the Robot brain with Ubuntu 22.04 and ROS Humble
– A Jetson Nano for AI and Image operations
– Various sensors (ultrasonic, button, fisheye camera, 2d lidar, imu)

This prototype later got specific instructions what to screw in where, and short enough cables so they could be managed inside the robot. Another caveat that later popped up was that the power cables for the jetson nano were too thin, so a brown out occurred on power spikes. This was remedied by using the original barrel jack cables cut open and soldered in the place of the cables.

Here an example of the build instructions:

Fetch Machine

  By , , ,

While being active in the community of Letsrobot I got an offer to try a product of Bare Conductive in trade with a documented project I do for their own blog. I got really interested in the product, so I agreed. It did take me a few months to actually finish it, but I have finally completed the project and got featured on both bare conductive’s site as well as adafruit’s one.

The product in question is the “Pi Cap” which is a hat for the Raspberry pi and adds a bunch of Capacitive touch and distance sensors. I wanted to make a product with my robots I built with LetsRobot in mind. Often enough I had people ask me to toss a ball across the room so they could fetch it, kinda like a pet. That’s what gave me the idea of a automatic fetch machine – like the ones that shoot tennis balls for dogs so they can play with themselves.

I ran into quite a few problems along the way. To list the biggest ones:

  1. Detection of the ball/robot: I thought I could make the capacitive sensors pick up the robot and motors, but that didn’t work at all. Also I needed the ball to be picked up somehow and didn’t needed a way to make that work as well.
  2. Creating the 3D models: While I already got some experience with smaller and less complicated models and thought I could create such models with some work. It took way longer than I thought, but eventually I managed to finish them.

Below a small gallery of the different steps I completed.

A summary of how it works: The script in the background is written in python and runs on the raspberry in the background. The Pi Cap senses the ball with a cable to a little aluminum foil piece which touches the ball when inserted into the loading arm. The arm is designed in such a way that the ball always rolls in the right spot to make contact. Once it senses something the script pauses for a few seconds to give the robot time to get out of the way. After that it sends a new positon for the loading servo to the servo microcontroller. This causes the ball to roll into the shooting head. After spinning up the two DC motors placed to accelerate the ball, the script moves the second servo to push the ball into the two motors. I also integrated a small LED to indicate the script sensing the ball and doing stuff.

Here is my last test:

And here a video of it in action:

Cardbot V2 stabilization with 3D Printing!

  By , , ,

Once I got my 3D Printer working and had it dialed in, I wanted to do this overdue upgrade. Over time the tape got loose over and over and the motors started sagging so hard, that the cardboard dragged on the ground. This was one of my first designs I created and printed.

These two prints made the bot usable again since it fixed the dragging body as well as stopped the Camera head from constantly falling down.

 

Sparky rewamp (V2)

  By , ,

I had a few issues with the current way Sparky was built. Mainly the the fact that the power bank did nearly not fit and had the motor axis sometimes grind against the hull of the power bank. That is obviously quite a flaw – especially since that also impaired Spary’s movement sometimes. I decided I wanted to get a fixed battery pack, inspired by other creators in the LetsRobot community.

And while I were at it, I wanted to have a better speaker setup.

Here a before/after comparison:

It should be quite obvious that the new revision is now more pleasant to look at. Also the big speaker block could be fit behind the camera and centered, which also improved driving straight. The gripper got the pads to reach all the way to the floor. (I later trimmed the front edges of those so they wouldn’t get caught up on ledges as often) The battery has been replaced with a Li-Ion pack that outputs 12V. With this addition I used two step down converters, but gained the ability to have components that required over 5V – also charging and switching the robot off got way easier.

Here a gallery with more angles and steps.

Cardbot V2

  By , ,

Alongside Sparky’s upgrade I thought I want to iterate with what I learned so far on cardbot as well. I made a few changes already. The biggest one being the Tilt head I implemented for a time.

For that I first tested an old Mindstorms LEGO motor that I had lying around.

And once I confirmed that it worked, I strapped it below the cardboard cover and expanded the hole for the camera in the front. The result was a camera with the ability to look up on command. (since that is a simple DC motor I limited that to a “look up” function that lasted around a second) bundled with the LED I strapped on the Webcam, this improved the field of vision of Cardbot immensely.

Also note on this picture: I needed to add a second Motorhat so it could run the LED + LEGO DC Motor. Luckily those hats are stackable up to 64 in theory because they’re based on l2c.

But this didn’t last long. I wanted to improve it even more. What bothered me the most was that Cardbot didn’t have a gripper like Sparky did already. So I gave Cardbot a whole rework.

First I had to change the Headlight wiring, since I wanted to control that with a pin of the RasPi. That way I wouldn’t need a second motorhat solely for the LED which would be obviously overkill. So I wired in a transistor to feed the power source seperately from the RasPi pin to protect that.

This time I added a Pan/Tilt head for the camera and added the gripper seen in the pictures. For the gripper I needed to make space since it is designed to get pulled by the actuator rod and as a solution I put everything a bit higher up. This way I could also create my own screw holes since the material I used is soft plastic. So I just used the hobbyist knife to carve out screwholes.

And I had to switch to a normal RasPi camera module to be able to move it around like that. while it worked like a charm, I soon noticed that the ziptie and tape mounts on this bot are not the best solution…

Catapult for Sparky

  By , , , ,

In effort to make interactive “Toys” for my robots I created with a catapult built for my LEGO Mindstorms robot back in the day.

I used a LEGO DC motor with the cable I prepared for Cardbot initially, and used a role of tape to weight down the base of the contraption. On the Breadboard I used an ultrasonic distance sensor to enable the robot to shoot the catapult. The logic is handled by the arduino and the simple H bridge to power the DC motor externally to protect the arduino.

Here a small video of it in action: