Implement a better gecko runtime about browserhtml

[paulrouget]

From @gordonbrander on December 19, 2014 21:27

Is it possible to make the rendering engine pluggable to some degree (see #50) so we can use this codebase as a client to experiment with Servo?

[paulrouget]

From @Gozala on December 19, 2014 21:31

So I had a conversation with Myk about things I believe are missing from webrt to run firefox.html just as well as HTMLRunner does.

1. WebRT seem to ignore simple permissions most notably:
   https://github.com/paulrouget/firefox.html/blob/master/manifest.webapp#L12
   It looks like issue is that preference dom.mozBrowserFramesEnabled is not set to true.

   Turns out webrt team is implementing permissions one by one below are some examples:

   • https://bugzilla.mozilla.org/show_bug.cgi?id=892833
   • https://bug901284.bugzilla.mozilla.org/attachment.cgi?id=785462

   They will be happy to take patches to enable browser permission or any other that is currently missing.

2. We need to draw in titlebar, that is currently not supported by WebRT. I see two ways to go about it:

   1. Add new permission like "no-titlebar" or alike.
   2. Add some custom element / attribute in the markup to disable the titlebar.

   I got following response:

   i'm not a module owner/peer for DOM/WebAPI, so i can help with the process, but i can't actually approve it (and it's further complicated by the desire of the WebAPI folks to standardize their APIs for the web in general). that sounds like a good idea! it's a lot easier to explain what we want to do when we can demo working code

3. Zipping files on every refresh is slow and not ideal. We need a better workflow for development. I have successfully used custom app protocol implementation to avoid zip files. Myk is open to adding new CLI argument along to the ones that already exist that would use this protocol implementation to load app from the passed directory rather than zip.

Plan

Given this conversation I think I'd like have a fork that would resolve all these and then try to land them into nightly. More specifically

1. I assume it's matter of just flipping prefs during startup based of manifest.
2. Matter of integrating things from (shell.xul)[https://github.com/paulrouget/gecko-dev/compare/htmlrunner#diff-6333f326d01148db9494ab0c0b554ce4R1] into webapp.xul & making it opt-in somehow.
3. Just need to patch webapprt/CommandLineHandler

Benefits

1. Web app will actually become debug-able.
2. We could actually create a page that would let regular firefox users install firefox.html and play with it.

[paulrouget]

From @Gozala on December 19, 2014 21:33

I'd be also interested in making a node-webkit based version of HTMLRunner so we could even start talking how we could have firefox.next on iPlatfroms.

[paulrouget]

Is it possible to make the rendering engine pluggable to some degree (see #50) so we can use this codebase as a client to experiment with Servo?

Afaik, no. I don't really know how that would work.

[paulrouget]

From @gordonbrander on December 19, 2014 22:56

How costly would it be to fork and hack in Servo at a later date? Is it a good idea to try to build a Servo client on top of HTMLRunner or would that be too far away from the scope of this project?

[paulrouget]

We need to draw in titlebar, that is currently not supported by WebRT.

Also, I'd be interested to know how the WebRT binary works compared to Firefox's. nsBrowserApp.cpp vs. webapprt.mm for example?

Zipping files on every refresh is slow and not ideal. We need a better workflow for development. I have successfully used custom app protocol implementation to avoid zip files. Myk is open to adding new CLI argument along to the ones that already exist that would use this protocol implementation to load app from the passed directory rather than zip.

Gaia team has some great ideas of how to handle that. Basically, think about a custom app protocol, but instead, we'd use a service worker that would handle files location, files update and cache. And they have working patches. It is not handled at the runtime level but in the app. This is a thing they want to do for B2G, and I really think we should stay as close as possible to B2G (to play with service workers, use the Maple branch).

---

If we can improve WebappRT to cover what htmlrunner does today, that would be awesome, and I also believe it's the way to go as long as we can also ship a standalone version of it.

Here are some concerns though:

• can we ship a standalone binary?
• if so, how can we handle updates?
• I don't know how different are the windows generated by webappRT and by firefox.exe. For example, the window generated by B2G creates a GPU crash on osx if you draw in the titlebar.
• what about all the pref that are not related to permissions (webcomponents or CSP)? I'm not 100% sure it's just a matter to turn them on a startup (but maybe it is)

More requirements will emerge as I talk to more people, but it sounds like we should go that way.

[paulrouget]

How costly would it be to fork and hack in Servo at a later date? Is it a good idea to try to build a Servo client on top of HTMLRunner or would that be too far away from the scope of this project?

It doesn't really work that way. HTMLRunner is just Gecko. Servo can open a window and draw web content. And that's all we need. The missing pieces of Servo (app management, permissions, apis, ...) can't be "imported" from HTMLRunner.

[paulrouget]

I'd be also interested in making a node-webkit based version of HTMLRunner so we could even start talking how we could have firefox.next on iPlatfroms.

This should not be a priority.

[paulrouget]

From @Gozala on December 19, 2014 23:21

Gaia team has some great ideas of how to handle that. Basically, think about a custom app protocol, but instead, we'd use a service worker that would handle files location, files update and cache. And they have working patches. It is not handled at the runtime level but in the app. This is a thing they want to do for B2G, and I really think we should stay as close as possible to B2G (to play with service workers, use the Maple branch).

I'm not sure these are alternatives I think of these more as orthogonal to each other, but I maybe wrong as I don't know all the details. Also I'd like something that I can start using asap & if we can get something better down the road that's even better.

can we ship a standalone binary?

Apps generated by webapprt are just a thin shells that use firefox runtime that was used to install them. And yes I agree having a way to bundle runtime with binary is definitely something that we would need. I imagine that could be done, but I have not actually looked what it would take to do that.

If it turns out too hard we can always keep HTMLRunner around which would just leverage more of webapprt I guess.

if so, how can we handle updates?

Well I guess we would have to answer how do we distribute apps first.

I don't how different are the windows generated by webappRT and by firefox.exe. For example, the window generated by B2G creates a GPU crash on osx if you draw in the titlebar.

I believe webapptrt just uses firefox binary that installed it so it will behave same as the firefox that installed it.

what about all the pref that are not related to permissions (webcomponents or CSP)?

I think we could just translate them into permissions to enable them when app is installed.

I'm not 100% sure it's just a matter to turn them on a startup (but maybe it is)

I think some of them are written into a profile during installation of webapprt, I actually don't know how permissions are translated to prefs yet, but I'm going to look into that as they clearly handle some of them.

[paulrouget]

From @Gozala on December 19, 2014 23:22

Edited response above as some replies end up quoted.

[paulrouget]

We would be tweaking WebappRT a lot.

So if we want a different installation mechanism, and a different app location, a different UI (no menu, draw in titlebar) and different preferences, and we consider having standalone binaries and updates, what does WebappRT bring us, compared to, let's say, B2G desktop?

[paulrouget]

Gaia team has some great ideas of how to handle that. Basically, think about a custom app protocol, but instead, we'd use a service worker that would handle files location, files update and cache. And they have working patches. It is not handled at the runtime level but in the app. This is a thing they want to do for B2G, and I really think we should stay as close as possible to B2G (to play with service workers, use the Maple branch).

I'm not sure these are alternatives I think of these more as orthogonal to each other, but I maybe wrong as I don't know all the details. Also I'd like something that I can start using asap & if we can get something better down the road that's even better.

You're absolutely right. It's orthogonal. We still need an installation process. Live updating and packaging doesn't have to be handled at the runtime level though.

I'm not sure how that should look like. I see plenty of options. I would ask our Gaia engineers (thinking of Vivien and Fabrice) for recommendations as they have been working on these things for years now.

[paulrouget]

From @Hywan on December 20, 2014 12:16

How to follow the progress of this very interesting issue?

[paulrouget]

@Hywan as for now, it's just being discussed here.

[paulrouget]

I think we should also ask @ochameau.

[paulrouget]

Also I'd like something that I can start using asap & if we can get something better down the road that's even better.

We already have something we can use today. HTMLRunner, even if ugly, works. I'd prefer us to agree (us + B2G people) on the right way to do it before building another runtime.

I don't how different are the windows generated by webappRT and by firefox.exe. For example, the window generated by B2G creates a GPU crash on osx if you draw in the titlebar.

I believe webapptrt just uses firefox binary that installed it so it will behave same as the firefox that installed it.

I'd like to understand the differences between B2GLoader.cpp, webapprt.cpp and nsBrowserApp.cpp.

what about all the pref that are not related to permissions (webcomponents or CSP)?

I think we could just translate them into permissions to enable them when app is installed.

Doesn't really make sense. Permissions and prefs are very different things.

I actually don't know how permissions are translated to prefs yet

They are not.

[paulrouget]

Actually, being/looking 100% native is not a goal. For example, dialogs, popups and context menus don't need to be native widgets.

[paulrouget]

From @donaldpipowitch on December 22, 2014 9:46

My 2cents: I'd like being 100% native from a functional perspective (e.g. like using Native menu keyboard shortcuts as in nwjs/nw.js#367), but not from a GUI perspective (if it isn't necessary).
I really like an approach like Polymers re-implementation of <select> or the implementation of <marquee> with Web Components in Blink-in-JS. We could use all the nice web specs like WAI-ARIA and stay cross-platform.

As an example I would like to build checkboxes, radios, etc. with just web technologies and let people opt-in to native elements via -moz-appearance, if they really want to. I worked a lot with <form>s recently and the different approaches of mixing native GUI elements with non-native elements is very annoying. Try to zoom into a page with a "native" checkbox in Firefox. I'd expect it to behave like a vector graphic, but it gets blurred/pixelated. The <select>, too, and you can't just opt-out with -moz-appearance: none; as the select-icon will still be there and still be pixelated. I know native elements have a lot benefits, but from this perspective a custom Web Element using a SVG and some nice themeable CSS would be nicer. FYI: Every browser behaves differently: Safari seems to use vectors, but not while changing the state of a checkbox. Chrome switches the style of a checkbox from native to custom on a different zoom level than 100%.

[paulrouget]

From @evilpie on December 22, 2014 11:48

One big difference with a non-native menu is that it can't properly overlap the window. This probably has some accessibility implications, but it also means to menu is clipped to the bounds of the window.

[paulrouget]

From @donaldpipowitch on December 22, 2014 12:22

Maybe this can be avoided if a real native window/pane is opened, but its content is rendered with web technology. (In my opinion its one of the goals of projects like these to find out what separates web based apps from native apps. Why shouldn't it be possible for a webpage to open a pane which exceeds the browser window as some kind of independent renderable document fragment? I can rebuild something similar to this with node-webkit.)

Anyway... it if is necessary to go native for various reasons, I'm fine with that (as in the example mentioned above with native menu keyboard shortcuts for node-webkit). I then would focus on adding a nice JS API to achieve this. (One could argue this is already done for context menus.)

[paulrouget]

@donaldpipowitch, agreed with what you're saying.

We don't have to choose between full native or full HTML, but we should use HTML whenever we can. We will find situations where we should really use native mechanisms (osx menus and window drag for example), let's identify them as we build Firefox.html.

For the context menu, let's build it as a floating <div>, we can always revisit this decision later.

[paulrouget]

From @donaldpipowitch on December 23, 2014 6:21

@paulrouget Say I'd like to implement a JS API to open a native window/pane where do I need to look into? Just https://github.com/paulrouget/gecko-dev? This is a fairly big project, could you guide me somehow to most relevant files where I need to look into?

[paulrouget]

From @Gozala on December 23, 2014 7:27

One prototype that @ZER0 build for add-on sdk used following approach. We introduced custom elements that add-ons could add to the add-on page, like toolbar-button for example. Chrome code was observing mutations to the add-on page document and would reflect those custom elements in the browser chrome ui, so in case of toolbar-button it would add buttons to the browser toolbar. If user would click browser button, we would dispatch click event on the HTML toolbar-button element etc... I think this would be a great approach to use here as well. We could start with custom HTML markup and then gradually implement support of native widgets by extending HTMLRunner or something platfrom itself.

[paulrouget]

@donaldpipowitch Somehow Firefox.html would notify the runtime that it needs a floating window. Then, there are 2 approaches. 1) use a xul:panel in shell.xul/js. 2) open the panel from C++. See http://mxr.mozilla.org/mozilla-central/search?string=dropdownframe

2 looks better to me as it doesn't require XUL.

We should start with a floating <div> for now.

[paulrouget]

From @nt1m on December 29, 2014 15:58

• "Native [...] overscroll effect"
  This is a nice effect that we should definitively use, but it is Yosemite specific unfortunately, and therefore, we'll end up with something non-native on other platforms which is not a real problem, but I'm not sure how much it will affect perf.
• Ways to build floating content
  We need a way to create floating content (with/without titlebars), that can float outside the window area, as mentioned in previous comments. This will be useful for menus and context menus, since we need to be able to overlap other windows.
• Interaction between mozbrowser content and app itself
  This is probably long term, but it's nice to consider early. We should have APIs for that, especially if we want to support add-ons, or themes in the future.
  One important thing is having drag and drop support between the content and chrome. Ths is important for many things, including opening files inside the browser, having a customization mode similar to Firefox's, ...

[paulrouget]

We need a way to create floating content (with/without titlebars), that can float outside the window area, as mentioned in previous comments. This will be useful for menus and context menus, since we need to be able to overlap other windows.

Is it really that necessary?

[gordonbrander]

This sounds like a UX decision we should defer until we have a solid UX/UI direction.

[paulrouget]

This sounds like a UX decision we should defer until we have a solid UX/UI direction.

Agreed.

[paulrouget]

Tracked by bug 1115098.

[Hywan]

https://bugzil.la/1115098