dtorre38 / mujoco_opencv Goto Github PK
View Code? Open in Web Editor NEWIntegrating opencv with mujoco.
Integrating opencv with mujoco.
Hi,
Thanks for integrating mujoco with opencv and sharing the same. I am facing one issue with respect to depth rendering, Earlier, I was using mujoco-viewer library to capture the depth frame. Now I am trying to extract the same with your code.
I am calling
depth, offscreen_viewport = get_frame(model, data, opt, scene, context, 'chaser_cam', loc_x=viewport_width - frame_width, loc_y=viewport_height - frame_height, width=frame_width, height=frame_height)
in my simulation loop. The get_frame
is modified as follows.
def get_frame(model, data, opt, scene, context, camera_name, loc_x, loc_y, width=640, height=480):
# 1. Create a rectangular viewport in the upper right corner for example.
offscreen_viewport = mj.MjrRect(int(loc_x), int(loc_y), width, height)
# 2. Specify a different camera view by updating the scene with mjv_updateScene.
# Set the camera to the specified view
camera_id = mj.mj_name2id(model, mj.mjtObj.mjOBJ_CAMERA, camera_name)
offscreen_cam = mj.MjvCamera()
offscreen_cam.type = mj.mjtCamera.mjCAMERA_FIXED
offscreen_cam.fixedcamid = camera_id
mj.mjv_updateScene(model, data, opt, None, offscreen_cam, mj.mjtCatBit.mjCAT_ALL.value, scene)
# 3.Render the scene in the offscreen buffer with mjr_render.
depth = np.zeros((height*width*3, 1), dtype=np.uint8) # Placeholder for depth data
mj.mjr_render(offscreen_viewport, scene, context)
# 4. Read the depth with mjr_readPixels.
mj.mjr_readPixels(None, depth, offscreen_viewport, context)
# 5. Depth processing
# Shift nearest values to the origin.
depth-= depth.min()
# Scale by 2 mean distances of near rays.
depth= depth/ 2*depth[depth<= 1].mean()
# Scale to [0, 255]
depth= 255*np.clip(depth, 0, 1)
# 6. Call mjr_drawPixels using the rectangular viewport you created in step 1.
# mj.mjr_drawPixels(depth, None, offscreen_viewport, context)
return frame, offscreen_viewport
But depth is not getting rendered. Can you please help with this ?
Thanks and Regards
Karthik R
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.