mayacakmak / se2 Goto Github PK
View Code? Open in Web Editor NEWControl interfaces for manipulating SE2 configurations
License: BSD 2-Clause "Simplified" License
Control interfaces for manipulating SE2 configurations
License: BSD 2-Clause "Simplified" License
Codepen had a nice automatic compilation of .pug files into .html. Perhaps sure GitHub has some plugin that helps do that? If either we might want to create some sort of compilation script (possibly with instructions to install pug on computer) or we can also switch to editing HTML directly and remove the pug files.
I see progress on the 3d-interface branch but haven't been able to test it. @kaviMD what is the status? What remains to be done? How could I help?
Here is the completed video for one of the interfaces:
https://youtu.be/-bwEiM_2Ur8
Any feedback/sticking point/missing things before I move onto making the other 7 videos? @kaviMD
Once videos are finalized we need to update instructions.html to (1) get which interface the participant is using from the URL, (2) display the correct video for each interface, and (3) figure out the pipeline from MTurk to the full set of interfaces (probably different HITs for each interface, then each HIT has a different link with the URL parameters).
Currently the interface has 'exact' targets, where the SE2 object (dot and line specified by an x,y position and theta rotation) needs to be taken to an exact target pose (even though there is a small threshold around the exact pose). We would like to be able to specify flexible targets, where the goal is to bring the object within some range near a given SE2 pose. We can do this by changing that small threshold but we need the visualization of the target to reflect the thresholds.
The practice screen randomly specifies the target after each trial. For the systematic data collection we need to systematically vary:
We need to keep the total duration of the experiment within a certain range to avoid fatigue, so we can decide how many variations of each of these we want to do based on how long a single session tends to take for the slowest interface.
Sampling variations: Rather than having fixed/exact variations we might want to have a complete set of exact options and add a little bit of noise to everything (center_x, center_y, center_theta, thresh_xy, thresh_theta => note that the variations are in terms of dist_xy, dist_theta, thresh_xy, thresh_theta, but noise in the former would result in less recognizable patterns).
Look into security rules on Firebase
We can move discussion about data to this issue.
One initial TODO:
We probably need to tell people to use Chrome on a desktop. Could we automatically detect if people are on a mobile device and not let them participate? @kaviMD
I have looked into both how Google recommends storing data in Firebase, and what kinds of events and data the programming is currently generating. Google recommends mainly to keep the structure as flat as possible, and to denormalize it when needed.
This structure is generally what I came up with. It has 4 main datatypes:
{
// Users contains only meta info about each user such as session start time
// and any other information we need to save about them
// This is stored under each user's unique ID
"users": {
"u1": {
"date": "Wed Jun 10 2020",
"time": "11:45:13 GMT-0700 (Pacific Daylight Time)",
"timestamp": 1591814713656
},
"u2": { ... },
"u3": { ... }
},
// Each interface type is split up into users by user ID
// which each contain a list of cycle IDs
"interfaces": {
"arrow-press/release": {
"u1": {
"c1": {
"totalCycleTime": 2, // Seconds
"numberOfClicks": 4
},
"c2": { ... },
"c3": { ... },
},
"u1": { ... },
"u2": { ... }
},
"drag-click": { ... },
"panel-press/release": { ... }
},
// Cycles and actions are separated from users
// so we can still query individual events easily
"cycles": {
"c1": {
"a1": {
"date": "Thu Jun 27 2019",
"eventName": "ring-release",
"newState": "cursor-free",
"prevState": "rotating",
"time": "16:40:27 GMT-0700 (Pacific Daylight Time)",
"timeStamp": 1561678827981
},
"a2": { ... },
"a3": { ... }
},
"c2": { ... },
"c3": { ... }
}
}
User, Action, and Cycle ID's would probably randomly generated by Firebase, I just used easier to read strings here.
This structure should allow us to fairly easily and efficiently query cycles by either interface or by user. We might want to edit the structure if we will need to query the database differently. I would love any feedback you have on this. If it sounds good to you, then I can start implementing it!
Since our meeting, I have implemented two more ways to display and interact with a URDF in a browser.
The first, is pure js, it uses a three.js (a 3d rendering library for javascript) and a URDF loader. This is the code: https://github.com/mayacakmak/se2/tree/urdf-visualization-testing/urdf-visualization/only-js
The second is a bit more complicated. When the website is initialized, it opens up anywhere a ton of ROS instances, each with their own visualizer, simulation, etc. Then, whenever a user visits the website, it assigns them a specific instance which they can interact with. The website also detects when they leave and opens up that instance for another user to interact with. The number of users we can support is limited by how many ros instances the server is capable of running at once, but it does open the door to having one gazebo simulation per user or something like that. This is the code: https://github.com/mayacakmak/se2/tree/urdf-visualization-testing/urdf-visualization/multiple-ros-instances. The backend is written in Flask with socket-io for communication and detecting when a user connects or disconnects.
Basic instruction, link to study, place to enter completion code.
@kaviMD Creating a separate thread for study 2 related data stuff. The first MTurk batch is not running so soon we should have some data to look at.
I know you've updated the data model a bit in on Firebase. Is the data processing code under se3/
also updated? What needs to be done there?
Data we'll want to extract from the logs:
Other ideas?
Todos:
While most controls do not allow the object to get outside the workspace, the panel control can send the object off limits--we need to implement check so the object doesn't move once it hits the walls of the workspace.
The panel interface was implemented to support clicks (single small move of the object) as well as press-hold-release (for longer continuous moves). The latter is used much more than the former. A click-based alternative to that is that when the arrow is clicked once, the object starts moving; when it is clicked again, it stops moving. Should be reasonable to implement following the template of the other interfaces, but will require some re-discovery.
First version completed by Kavi, some formatting edits by Maya.
Revising list of questions in the Google doc.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.