Mac OS X Touch screen support about touchscript

[valyard]

I don't know how Mac touch works. I think there was native touch support or something. I've never used a touch screen with a Mac, to be honest.

If you are building for Web Player, I think you can't use sockets in this case so you can't use TUIO to get data from a touch screen either.

TouchScript supports TUIO or native Unity touch events. And I unfortunately don't know if you can use native touch events on Mac. I would bet that you can't (8

[madhuatlink]

Thanks for the reply.

That means even if I implement in Windows for Web player I won't be able to support multitouch because TUIO won't work on the Web player. Is that correct?

[valyard]

Correct.

Listening sockets
You cannot create listening sockets in the webplayer, it cannot act as a server. Therefore webplayers cannot communicate with each other directly (peer 2 peer). When using TCP sockets you can only connect to remote endpoints provided it is allowed through the socket policy system.

http://docs.unity3d.com/Documentation/Manual/SecuritySandbox.html

[david-hodgetts]

websockets might be an idea to pursue.
You would need a proxy server that receives the tuio messages and sends them to the browser via websockets. Javascript on the browser would handle the websocket and forward the messages to the unity player.

As i am writing i discover this ... might be worth a try
regards

[madhuatlink]

Thanks valyard for the replies and Thanks david-demainlalune for the info. I'll give it a try.

Earlier I tried this https://github.com/fajran/npTuioClient
Using that I was able to get multi touch support for browser. But the issue with the Unity web player was when I perform a multi touch that happens to the browser, not to the unity object. For example if I want to zoom a object in the Unity web player and if I pinch zoom it, it zooms the browser window, not the unity object.

[madhuatlink]

If I'm using TouchScript on my Unity project to enable mutli touch and if I use a java script file like above on the browser side and read the tuio objects, can you please tell me how I can forward it to the TouchScript(Unity side) so the multi touch functionality to work?
Thanks

[david-hodgetts]

Never worked with unity's web player, but i gather you can call Unity functions from javascript.

To get the data to the Touchscript system, i guess you would want to write a new InputProvider modeled on the existing TuioInput class. That class would be connected to the functions handling the javascript calls. I might give it a try this this weekend.

Hope this helps

[madhuatlink]

Thanks david-demainlalune. That'll we great if you can add a InputProvider to support this.
This is my current scenario. I have a Unity scene with a 3D model. In that I have used TouchScript to handle multitouch. I have added Tuio Input as Input source. When I build the Unity project as Mac or Windows(with Windows7 input source) standalone and run it on a Multi touch screen everything works well. But when I run the same thing using the web player it doesn't recognize any touch event. Then when I add the Mouse Input as the the Input source and run in the Web player. It recognizes single touches but not the multi touch events like pinch zoom.
Yesterday after seeing your reply I also thought of adding something to Touchscript to handle the JS calls. But when I checked in the Unity project the Touch Input scripts were coming from the dlls. Wasn't sure whether I"ll be able to modify the Touchscript code in the source and make it work. That's why I posted the previous question. So if you can add a new Input Provider to work with the Web player it'll be really helpful.
Thanks

[david-hodgetts]

Just out of curiosity, is there a particular reason you need to publish to the webplayer?

[madhuatlink]

My application(in Mac) needs have some html stuff(have to use html5). There we'll have some html bullets on one side and on the other side of the UI we need to show the 3D models related to these bullets depending on the bullet user selects. That's why I'm trying to do this with the Web player. This has to run on a Mac machine with a OS X touch screen. I couldn't find a way to load a Unity app in a Mac app. Therefore the best option was to use the Web player. But if this is not going to work, I might have to do the whole app as a Unity app. That's my other option at the moment.
If you get a chance to add a new InputSource please share the code with us. Otherwise I'll have to give up the idea of using Web player will have to look how to do this as a Unity app.
Thanks for spending your time on this. Really appreciate it.

[david-hodgetts]

ok, got it too work : )

There are two projects. The first is the full demo. It relies on Node.js and Tuio.js to proxy the tuio messages to the browser. The messages are then sent to a TuioWebSocketInput component in Unity. This is adapted from the TuioInput Touchscript input provider. You should see touchscript debug circles on tuio activity (tested with a tuio simulator);

The second project holds the unity project. If you want to integrate this in your own build. You will need the TuioWebSocketInput file and the LitJson dependency (the component uses json to deserialize the messages from the browser).

That being said. Although this was a fun week end project for me, i am not sure i would recommend going this route. I wonder if you would not be better off using a standalone unity build and use coherent-ui for your UI instead.

regards

[madhuatlink]

Thank you very much david-demainlalune. I'll play with your projects and see. :-)

I would like to know why you said that you would not recommend going this route. What is the reason for that?

[david-hodgetts]

no problem : )

This solution adds a layer with the node server and the browser to unity communication, This means more lag, and more things that can go wrong in a long running environment. Also if you are not going for a fullscreen unity player because you need html peripheral elements, you might have to fiddle with the calibration.

hope this helps

[valyard]

Thanks for helping, David.
Mind if I add this script to the main branch?

[david-hodgetts]

not at all : )
It might need some documentation to avoid people getting confused over its use case. Also it requires a proxy server to forward the tuio messages over websocket, and some javascript in the browser to talk to the Input Provider. This might have to be explained somewhere, or you can point to the example project.. Tell me if I can help.

and btw, many thanks for open-sourcing and managing TouchScript : )

[madhuatlink]

I got your demo running. And also I was able to add multi touch support to 3D model in Unity Web player using your sample Unity project. Thanks again for your help David. :-)
(As you said we'll have to decide whether we'll go on this route or whether to build a Unity App for my Work)
And it'll be good to share this with others, I'm sure there are more people who might be looking to add multi touch support in the Web player.

[david-hodgetts]

I am happy to have been of help : )

[madhuatlink]

Hi, I started doing this project as a standalone Unity app and everything seems to work fine upto now, where I tested the multi touch using the Mac trackpad with TongSeng TUIO wrapper. But recently we bought a Mac Touch screen (SMART Board 8055i). Now I need to capture the touch events from this screen and send them my unity app TouchScript TUIO input. But I don't have much idea how to do this. Do you have any plans of reading Mac inputs in TouchScript in the near future? If not please share your ideas on the best way to handle this. They have a api to detect the touches. But that's from a Cocoa app. I'm not clear how to do it from my unity app. And then how to convert it to TUIO. Any ideas on this will be really helpful. Thanks.

[valyard]

I don't see TUIO support in the specs: http://www.smarttech.ru/documents/smart_board_8055i_specification.pdf
Do they have a proprietary library to get touch input from this device? Because I don't know how to handle touch input on OS X.

You need to find out if there's a custom library. If there is, it's possible to use it as a custom input provider. The problem is that I won't be able to test it.

[madhuatlink]

Thanks for the reply.
Yes it looks like they don't support TUIO. They have a SDK. And they have some methods like below to identify multi touch gestures.
void CSBSDK2EventHandlerImpl::OnXYDown(int x, int y, int z, int iPointerID)
{
[m_pEventHandler OnXYDown:x y:y z:z iPointerID:iPointerID];
}

void CSBSDK2EventHandlerImpl::OnXYMove(int x, int y, int z, int iPointerID)
{
[m_pEventHandler OnXYMove:x y:y z:z iPointerID:iPointerID];
}

void CSBSDK2EventHandlerImpl::OnXYUp(int x, int y, int z, int iPointerID)
{
[m_pEventHandler OnXYUp:x y:y z:z iPointerID:iPointerID];
}

I have their SDK pdf and a sample code in Xcode which recognize multi touch gestures. But it seems I can't attach it here.(only images are allowed)
Do you think I can integrate this to touch script? It'll be great if we can do that.
Thanks

Update - I uploaded the pdf file and the sample code here.
https://drive.google.com/folderview?id=0B-yR7x2xK0GdUGxUMG9rZVJJYk0&usp=sharing

[madhuatlink]

Any help on this is really appreciated, as I'm stuck on this at this point.