Contents
  1. 1 Environment Setup Issues
  2. 1.1 MuJoCo-Related Issues

1 Environment Setup Issues

(1) Compilation Error After Installing mujoco-py: Error compiling Cython file

Problem Details:

performance hint: /home/mahaofei/anaconda3/envs/reskill_new/lib/python3.7/site-packages/mujoco_py/cymj.pyx:67:5: Exception check on 'c_warning_callback' will always require the GIL to be acquired.
Possible solutions:
	1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
	2. Use an 'int' return type on the function to allow an error code to be returned.
performance hint: /home/mahaofei/anaconda3/envs/reskill_new/lib/python3.7/site-packages/mujoco_py/cymj.pyx:104:5: Exception check on 'c_error_callback' will always require the GIL to be acquired.
Possible solutions:
	1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
	2. Use an 'int' return type on the function to allow an error code to be returned.

Error compiling Cython file:
------------------------------------------------------------
...
    See c_warning_callback, which is the C wrapper to the user defined function
    '''
    global py_warning_callback
    global mju_user_warning
    py_warning_callback = warn
    mju_user_warning = c_warning_callback
                       ^
------------------------------------------------------------

/home/mahaofei/anaconda3/envs/reskill_new/lib/python3.7/site-packages/mujoco_py/cymj.pyx:92:23: Cannot assign type 'void (const char *) except * nogil' to 'void (*)(const char *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (const char *) except * nogil'.

Error compiling Cython file:
------------------------------------------------------------
...
    See c_warning_callback, which is the C wrapper to the user defined function
    '''
    global py_error_callback
    global mju_user_error
    py_error_callback = err_callback
    mju_user_error = c_error_callback
                     ^
------------------------------------------------------------

Solution:

Reference: Stack Overflow and GitHub issue

pip install "cython<3"

(2) ERROR: GLEW initalization error: Missing GL version

Problem Details:

/home/mahaofei/anaconda3/envs/reskill_new/lib/python3.7/site-packages/gym/envs/registration.py:64: UserWarning: register(timestep_limit=100) is deprecated. Use register(max_episode_steps=100) instead.
  warnings.warn("register(timestep_limit={}) is deprecated. Use register(max_episode_steps={}) instead.".format(timestep_limit, timestep_limit))
Creating window glfw
ERROR: GLEW initalization error: Missing GL version

Press Enter to exit ...Killed

Solution:

export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so

RuntimeError: Failed to initialize OpenGL when using opencv after setting up the gym environment At present, it appears that gym rendering and opencv display cannot be used at the same time.

  • To use opencv, run unset LD_PRELOAD;
  • To use gym rendering, run export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so.

(3) MujocoException: Got MuJoCo Warning: Nan, Inf or huge value in QACC at DOF X. The simulation is unstable.

Cause:

When using mujoco for reinforcement learning training, the error above may occur, causing training to terminate early.

This happens because, while env.step is executing, model interpenetration or constraint conflicts occur between models in the environment (the robot itself, or the robot and other objects), preventing MuJoCo from running the simulation.

Solution 1: Add the Joint Attributes damping and armature (Effective!)

Reference: https://github.com/google-deepmind/mujoco/issues/989

After limiting the joint rotation angles, the normal algorithm could train for more epochs, but the error still occurred later.

After checking the normals, none of the joints on my gripper had damping configured. Add damping and armature. Note that these two attributes must be added to every joint, including those on the robot and on various objects.

<joint name="robotiq_2f_85_right_driver_joint" range="0 0.834" damping="0.1" armature='0.01'/>

After that, the error never occurred again during training.

Note: The damping and armature parameters may need to be adjusted, especially for relatively small components such as grippers. If the damping is set too high here, the joints may be unable to move.

Other Method 2: Limit the Joint Range (Somewhat Effective)

For my robotic arm model, the original range of motion was set very wide because my target task is grasping. During debugging, I also found that the robotic arm would collide with itself or with the table in many situations.

My method was to manually control the robotic arm in the mujoco environment and move it to the limits of the task space, then use the command below to print the value of each joint at those limit positions.

print(sim.data.qpos[model.jnt_qposadr])

Determine the value range of each joint based on all the limit positions.

Then, in the model’s .xml file, go to <wholebody> and modify each <joint>’s range, for example:

<!-- 初始 -->
<!-- <joint name="joint1" pos="0 0 0" axis="0 0 1" armature="1.5708" limited="true" range="-3.14159 3.14159" damping='200' /> -->
<!-- 测试关节范围(抓取) -->
<joint name="joint1" pos="0 0 0" axis="0 0 1" armature="1.5708" limited="true" range="-1.57079 1.57079" damping='200' />

After testing, changing each joint’s range only increased the number of iterations that could run, but it still did not solve the problem.

Other Method 3: Modify the Reward Function (Somewhat Effective)

Because the error occurs when the robot’s action goes beyond the workspace or a collision occurs, this indicates that the learning algorithm has not learned an appropriate action and has caused the robot to move incorrectly.

Therefore, consider modifying the reward function to add penalties for leaving the workspace. For example, for grasping, apply a penalty if the xyz position of the gripper’s end exceeds certain coordinates, and also apply a penalty if the gripper is too far from the object.

This method is somewhat effective.

Other Method 4: Reduce solrel (Not Tried)

Reference: https://github.com/google-deepmind/mujoco/discussions/63

Other Method 5: Switch to the RK4 Integrator (Ineffective)

Reference: https://github.com/google-deepmind/mujoco/issues/168

Other Method 6: Check the Environment Configuration (Effective)

Check whether the environment’s reset function is configured correctly. After checking, I found that the problem in my code was that I reset the environment and every joint on the robotic arm, but did not reset mocap. As a result, if the robotic arm’s end effector had drifted by the end of the previous simulation, it would drift very far at the start of the next one, making the error above more likely as the simulation proceeded.