How to use exported .pth file about omniisaacgymenvs-dofbotreacher

[j3soon]

If you simply want to run the policy without modifications, follow the Sim2Real section in the README. The instructions should be pretty straightforward if you have already setup Isaac Sim and the real-world Dofbot.

[DJT777]

Well, what I want to do is get the policy deployed out of Isaac Sim and into a jetson Nano in the real world. Basically after training a policy in Isaac Sim, then use it on a Nano.

I've figured out how to that with RL Game's notebook here:

https://github.com/Denys88/rl_games/blob/master/notebooks/train_and_export_onnx_example_continuous.ipynb

However, now I need to know what the outputs of the model are, I'm assuming it's the mu tensor. what order of joints is the a2c model providing? Is it 1,2,3,4,5,6?

[DJT777]

I'd also like to ask if the form linked with exporting ONNX looks correct, and how the ONNX model would be used.

I'm currently trying to export it and use it like this:

import onnxruntime as ort
import numpy as np

# Load the ONNX model
sess = ort.InferenceSession('/home/jetson/Excolligere/dofbotreacherDefault.onnx')

# Get the input and output names of the model
input_name = sess.get_inputs()[0].name
output_name = sess.get_outputs()[0].name

# Get the input names and shapes
input_info = sess.get_inputs()
output_info = sess.get_outputs()a

for i in input_info:
    print("Input name:", i.name)
    print("Input shape:", i.shape)
    
for i in output_info:
    print("Output name:", i.name)
    print("Output shape:", i.shape)
    
    
import numpy as np

print(np.__version__)

# Create the individual arrays
jointPos = np.zeros(6, dtype=np.float32)
jointVel = np.zeros(6, dtype=np.float32)

#put in coordinates here
goalPos = np.array([0.2, .07, .1], dtype=np.float32)

goalRot = np.zeros(4, dtype=np.float32)
goalRotRel = np.zeros(4, dtype=np.float32)
prevAct = np.full(6, 0, dtype=np.float32)

# Concatenate the arrays into a single array
input_data = np.concatenate((jointPos, jointVel, goalPos, goalRot, goalRotRel, prevAct))
input_dict = {input_name: input_data.reshape(1, -1)}

# Run the model on the input data
output = sess.run(None, input_dict)

import math
import numpy as np

pi = math.pi

dof_angle_limits = [
        (-90, 90, False),
        (-90, 90, False),
        (-90, 90, False),
        (-90, 90, False),
        (-90, 180, False),
        (-30, 60, True),
]

servo_angle_limits = [
        (0, 90),
        (45, 90),
        (45, 90),
        (45, 90),
        (45, 90),
        (45, 180),
    ]

servo_angles = [90] * 6

for i, pos in enumerate(output[0][0]):
    if i == 5:
        # Ignore the gripper joints for Reacher task
        continue
    # Map [L, U] to [A, B]
    L, U, inversed= dof_angle_limits[i]
    A, B = servo_angle_limits[i]
    angle = np.rad2deg(float(pos))
    if not L <= angle <= U:
        print("The {}-th simulation joint angle ({}) is out of range! Should be in [{}, {}]".format(i, angle, L, U))
        angle = np.clip(angle, L, U)
    servo_angles[i] = (angle - L) * ((B-A)/(U-L)) + A # Map [L, U] to [A, B]
    if inversed:
        servo_angles[i] = (B-A) - (servo_angles[i] - A) + A # Map [A, B] to [B, A]
    if not A <= servo_angles[i] <= B:
        raise Exception("(Should Not Happen) The {}-th real world joint angle ({}) is out of range! Should be in [{}, {}]".format(i, servo_angles[i], A, B))

print(servo_angles)

Arm.Arm_serial_servo_write6(servo_angles[0], servo_angles[1], servo_angles[3], servo_angles[5], servo_angles[4], servo_angles[5], 500)

However, the model is not correctly reaching. What information would I need about using the model in order to export it and use it like this?

[DJT777]

I'd also like to ask where in the code can I see how predictions are being made. I've updated my code to use the mean and std. deviation calculated from the exported model, but my predictions are still not the same as the simulations:

import numpy as np

# Create the individual arrays
jointPos = np.zeros(6)
jointVel = np.zeros(6)
goalPos = np.array([-.1949, .1820, .2187])
goalRot = np.zeros(4)
goalRotRel = np.zeros(4)
prevAct = np.full(6, 0)

# Define a function to clip the actions
def clip_actions(actions):
    return np.clip(actions, -1.0, 1.0)

    # Concatenate the arrays into a single array
input_data = np.concatenate((jointPos, jointVel, goalPos, goalRot, goalRotRel, prevAct), dtype=np.float32)
input_dict = {input_name: input_data.reshape(1, -1)}



while True:
    input_data = np.concatenate((jointPos, jointVel, goalPos, goalRot, goalRotRel, prevAct), dtype=np.float32)
    input_dict = {input_name: input_data.reshape(1, -1)}
    actions = []
    output = sess.run(None, input_dict)
    # Run the model on the input data
    mus = output[0][0]
    sigmas = output[1][0]
    for mu, sigma in zip(mus, sigmas):
        #print("mu is " + str(mu))
        #print("sigma is " + str(sigma))
        sigma = np.exp(sigma)
        action = np.random.normal(mu, sigma)
        action = clip_actions(action)
        #print(str(action))
        actions.append(action)
        # Print the output
    prevAct = np.array(actions, dtype=np.float32)
    joinPos = prevAct
    print(str(actions)+"\n")

[DJT777]

Denys88/rl_games#226

Solved