Difference between revisions of "X3D V4 Open Meeting"

Latest revision as of 14:28, 15 June 2016

X3D V4.0 Open Workshop / Meeting June 8th 2016

Topics

Note: The red question marks at the end of each question are place-holders for any short answers to emerge from the discussions.

• What level of X3D integration into HTML5 do we want?
  • Do we want to be fully integrated like SVG? Yes
• Do we want/need a DOM spec? If so: Yes
  • Which DOM version should it be based on? ???
  • Do we want to fully support all DOM/HTML features? ???
• Do we want to maximize the backwards compatibility of V4.0 with V3.3? Or break away completely? Maximize compatibility
  • Do we want to retain SAI? Yes
• What features do we want? For example, ???
  • How is animation to be handled? The X3D way of TimeSensor and ROUTEs, or an HTML way, such as CSS3 animations, or else JavaScript? ???
  • How is user interaction to be handled? The X3D way of Sensors, or the HTML way with event handlers? ???
  • Do we need any different nodes? One example might be a mesh node? ???
  • Do we want Scripts and Prototypes in HTML5? ???
  • How do we want to handle styling? ???
• What profile(s) do we need for HTML? ???

Attendees and contributors

E-mail contributors: Mike Aratow, Don Brutzman, John Carlson, Leonard Daly, Yves Piguet, Andreas Plesch, Philipp Slusallek

Meeting Attendees: Vince Marchetti, Dick Puk, Leonard Daly, Anita Havele, Christophe Mouton, Nicholas Polys, Roy Walmsley

Apologies: Don Brutzman, Andreas Plesch

Discussion Capture

Note: This section is intended for the capture of discussion points.

HTML integration and backwards compatibility

General strategy is to integrate into HTML and keep backwards compatibility, without throwing away existing capabilities.

XML3D Design

Recent XML3D design work is investigating the use of Web Components

XML3D a layered structure, built on Node.js and a DOM module.

X3D Player in the Browser

Can we get to a situation where X3D presentation is built into the browser or will we always need Cobweb/X3DOM or something similar?

Currently browser writers are not interested in adding 3D. Therefore, always need a JavaScript engine to be used.

Node Set

Small core node set may be required

Other nodes can be built from the core set

Easier to implement a core set into JavaScript.

"Core" is essentially an API. XML3D use a three.js library. XFlow for data handling - a modified version. Shade.js - fully programmable shader - automatically combines lighting and shading.

Node.js/DOM layer/X3D combination

Could map X3D on top of a game engine

Define X3D as a high level layer on top of any rendering engine, game or otherwise.

How to specify

DOM yes.

SAI / DOM-HTML could be some incompatibilities.

SAI - internal and external interactions.

Could use node.js

HTML5 scripting will probably be used.

DOM scripting and SAI scripting serving two different purposes.

Could there be mappings for different types of implementations.

Major version change - that could mean compatibility breaks.

Example of FORTRAN - after FORTRAN4 introduced a new standard FORTRAN77. Similar to change of VRML to X3D.

CSS and styling

Fraunhofer are believed to indicate that there may be performance issues.

XML3D uses CSS.

CSS implementation in browsers need some fixes, particularly in regard to Web Components.

XML3D allows direct changes and CSS, but that may change.

Business case

What are the business issues?

How much use of X3D is there outside of the web applications. Is that community important? One example cited is Octaga player.

A lot of X3D will not change.

What is the future browser language? Will it be HTML in five years time?

Application development - mobile apps run as binaries, but are often developed from HTML5/CSS/JavaScript using PhoneGap.

MAR and VR Incorporation

Is there anything in this area that might affect our path?

Stereo rendering. VR platform.

MAR still not sufficiently clearly defined.

The Korean and German proposals have been largely integrated.

Conclusions

Maintain the design approach? Keep separate specifications. Probably need an encoding, and a DOM spec.

Can we integrate into the DOM and keep existing without too many incompatibilities.

DOM believed to have weaknesses in 3D support.

Need to generate a set of actions.

HTML Profile, might be in essence be the same as the "core nodes" discussed earlier.

Understanding of X3DOM and Cobweb implementations.

Event models - asynchronous changes with respect to rendering.

Prior e-mail contributions:

Note 1: Contributions are presented in chronological order.

Note 2: Links within the preformatted sections do not appear. So any used have been added as references at the end of the contribution.

Contribution 1

From Leonard Daly

I think the bigger question of what should be done with X3D. Is X3D solely going to exist within HTML or will X3D have a separate life inside and outside of HTML.

If the life is solely within HTML, then the questions below become inclusive of all X3D. If there are separate existences, then the first question is what is the cross-compatibility between X3D/HTML and X3D/other.

Contribution 2

From Don Brutzman (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004788.html)

Relevant working-group references follow.  A lot of excellent work has been accomplished already.

	X3D Version 4
	http://www.web3d.org/x3d4

	Web3D Consortium Standards Strategy
	http://www.web3d.org/strategy

	X3D Graphics Standards: Specification Relationships
	http://www.web3d.org/specifications/X3dSpecificationRelationships.png

	X3D Version 4.0 Development
	http://www.web3d.org/wiki/index.php/X3D_version_4.0_Development

A 5-10 minute quicklook discussion across these resources might help.  We are pretty far up X3D4 Mountain already!

The posted discussion-topics list is a good start for renewed activity, and an important way to keep track of everyone's many valuable ideas.  Suggestion: create some kind of topics-discussion page, probably easily linked off the preceding wiki page.

My general inputs for each of these topics are guiding questions:

a. What do the HTML5/DOM/CSS/SVG/MathML specifications actually say?

b. How is cross-language HTML page integration actually accomplished, as shown in best practices by key exemplars?

c. What is the minimal addition needed to achieve a given technical goal using current X3D capabilities?

Editorial observation: the word "want" appears 9 times in this list...  Understandable from common usage, but not a very good way to achieve consensus over a long-term effort.  Also not very useful for measuring successful resolution.

Pragmatic engineering rephrase: "what problem are you trying to fix?"

Over 20 years of successful working-group + community efforts can guide us in these endeavors - we know how to succeed together.  An effective path for building consensus is to:
- define goals that are illustrated by use cases,
- derive technical requirements,
-  perform gap analysis, and then
-  execute loosely coordinated task accomplishment according to each participant's priorities.

How to execute each specification addition: write prose, create examples, implement, evaluate. Repeat until done, topic by topic.

References:

• X3D Version 4
• Web3D Consortium Standards Strategy
• X3D Graphics Standards: Specification Relationships
• X3D Version 4.0 Development

Contribution 3

From Andreas Plesch (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004793.html)

The discussion on introducing an id field seemed to point towards the need to have fuller integration in the sense that it is difficult to isolate features. It may be necessary to define a x3d dom similar to the svg dom, with the corresponding interfaces. svg is very successful on the web but it took a long time to arrive there.

x3dom has a dual graph approach. There is the x3d graph and in parallel the page dom graph which are kept in sync but are both fully populated. Johannes  Behr would know better how to explain the concept.
It looks like FHG decided that x3dom is now considered community (only?) supported. This probably means it will be out of sync as newer web browsers arrive, or webgl is updated.

I explored Aframe a bit more. It will be popular for VR. It is still in flux and evolves rapidly. The developers (mozilla) focus on its basic architecture (which is non-hierarchical, a composable component system) and expects users to use javascript to develop more advanced functionality (in the form of shareable components). So it is quite different, fun for developers, and for basic scenes easy for consumers. Since most mobile VR content at this point is basic (mostly video spheres and panos), it is a good solution for many.

(As a test I also implemented indexedfaceset as an Aframe component, and it was pretty easy - after learning some Three.js. So it would be possible to have x3d geometry nodes on top of aframe. Protos, events and routes are another matter but also may not be impossible).

There is still space for x3d as a more permanent, and optionally sophisticated 3d content format on the web.

Event system: My limited understanding is that on a web page, the browser emits events when certain things happen. Custom events can also be emitted by js code (via DOM functions) for any purpose. (All ?) events have a time stamp and can have data attached. Then, events can be listened to. There is no restriction to listening, eg. all existing events are available to any listener. A listener then invokes a handler which does something related to the event. js code can consume, cancel, or relay events as needed (via DOM functions). It is not unusual that many events are managed on a web page. events can be used to guarantee that there is a sequence of processing.

So how does the x3d event system relate ? There is a cascade, and directivity. How long does an event live ? one frame ? Until it fully cascaded through the scene graph ?

Since x3dom and cobweb are currently the only options, from a practical stand point a question to ask may be this: what is needed to make x3dom and cobweb easy to use and interact with on a web page ? Typically, the web page would provide an UI, the connection to databases or other sources of data, and the x3d scene is responsible for rendering, and interacting with the 3d content. For VR, the UI would need to be in the scene, but connections and data sources would still be handled by the web page.

Cobweb in effect allows use of the defined SAI functions. Is it possible to define a wrapper around these functions to allow a DOM like API (createElement, element.setAttribute .. element = null) ? It may be since they are similar anyways and it would go a long way. But it still would not be sufficient to let other js libraries such as D3.js or react control and modify a scene since they would expect x3d nodes to be real DOM elements.

VR: A current issue is control devices. It would be probably useful to go over the spec. and see where there is an implicit assumption that mouse or keyboard input is available. VR HMDs have different controls (head position and orientation(pose), one button) and hand held controllers (gamepads, special sticks with their own position/orientations) or the tracked hands themselves become more popular. In VR, you do want to your hands in some way.

Perhaps, it makes sense to have <Right/LeftHand/> nodes paralleling <Viewpoint/> with position/orientation fields which can be routed to transforms to manipulate objects ? How  a browser would feed the <Hand> nodes would be up to the browser. InstantReality has a generic IOSensor.

Contribution 4

From Philipp Slusallek

I am not sure that I will be able to join the meeting, so let me present some of our ideas in this context by email already. As you might know we have done a lot of work on declarative 3D -- in the alternate universe called XML3D. The reason for this unfortunate split, lies in the fact that it has been rather difficult to bridge the gap between our ideas of a minimum and generic extensions to HTML-5 to enable declarative 3D while staying as close as possible to the current Web technology stack on the one side and the need for backward compatibility that people strived for in X3D(OM).

Now, there may be a quite interesting these two world views could be resolved with each other. The basic idea for that was already proposed and discussed between us and the X3DOM group: Merging X3DOM and XML3D by identifying the (significant and large!) common core and a layering X3D/XMl3D compatible interfaces on top of this (or eventually identifying common set of Dec3D elements). Unfortunately, this idea was not really picked up by anyone back then.

However, with newer technology like WebComponent (a version of prototypes in HTML5) this now becomes a highly interesting and much more practical option. We are actually exploring this right now. First results look very promising and a first paper on this has just been accepted for Web3D this year.

At the core of this approach we are using a clean interface to rendering engines, where we use Three.js a default option. Other engines like game engines or ray tracing etc. would be alternatives. (We are also exploring server-based real-time ray tracing based on a generic real-time synchronization layer between scene descriptions as one example.)

On top of this is a slightly refined generic data handling layer that is derived from our Xflow. Xflow has been tremendously useful for us and makes data management significantly easier than with the specialized nodes in X3D. It is the perfect building block for WebComponents. We are also integrating into this layer a programmable data processing engine that is derived from our flexible shade.js compiler for programmable shading.

Everything on top of this is essentially fair game for WebComponents (similar to a-frame, but with mny more options). The more specialized and often domain-specific nodes from X3D would be prime examples for this. We have actually started to implemented some of the basic X3D nodes already, plus the non-core XML3D nodes. Pretty much along the lines of what we discussed with X3DOM several years ago. Given the powerful underlying engine, it actually becomes rather straight forward to implement these nodes as Web Components, especially the X3DOM subset.
But we are just starting.

We are even exploring having a public repository of WebComponents that people could develop independently and that get automatically loaded when referenced in a scene (subject to some security policy, of course).
Talk about leveraging the power of the distributed web :-).

We are finalizing the paper for the final version right now but can make a preprint available as soon as this is ready. People would be more than welcome to help design and develop this further. Maybe this could also be an interesting basis for some of the work on X3D V4?

Contribution 5

From Don Brutzman (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004794.html)

As repeated a few days ago, I remain keen for us to establish some kind of X3D v4 wiki page.  We need to collaboratively show how these different categories of concepts can be defined along with their associated pros and cons.  Email and meeting discussions can then grow those sections effectively, to show that use-case goals and design requirements can be defined and met.

Wondering if the x3dom dual-graph page dom structures align with the draft work on Shadow DOM at W3C.
https://www.w3.org/TR/shadow-dom

I'm hoping that John Carlson gets a chance to look at whether his JSON prototype expander might be adaptable as part of x3dom - that would be pretty valuable.  Although Fraunhofer has outstanding prototype support in their Instant Reality engine, it has never been clear why that code can't simply be applied in x3dom as well.  Certainly a consistent approach would seem to make both codebases more coherent and maintainable for them.  If Fraunhofer refuses to adapt or release the Instant Reality prototype code then we will just have to do it ourselves... John's work seems like a big step in that direction.

Also thanks for pointing out important questions about Fraunhofer's stewardship of the X3DOM project.  It will be good to learn more about their intentions so that our community can align effectively.  There has been no handoff.

Regarding the X3D event system: a changing value of any field in any node can be sent as an input to any field in any node, as long as types strictly match (apples to apples).  Rephrased: a ROUTE passes a time-stamped value from one node to another. Internal to an X3D scene graph, that has been implemented dozens of times.  Seems extremely simple.

External to an X3D scene graph, meaning via a browser using the Scene Access Interface (SAI), mechanisms are similarly well defined.  Since the DOM is string based, and since any X3D event value can be expressed as a string, it seems like we have a straight connect-the-dots approach awaiting us.  Forgive me for using a four-letter word, but if interested individuals might actually _read_ the HTML5/DOM and X3D specifications, then the answers to most implementation & alignment questions are likely spelled out for us.

Reference:

• Shadow DOM

Contribution 6

From Philipp Slusallek

Please do not go the route (sic!) of a string-based interface for implementing X3D routes. Yes, the DOM has a generic string-based interface, which is really important in general. But not for efficiently handling big 3D data. Any DOM node can additionally provide a "binary"
JS API as well, ideally using Typed Aarrays in JS.

Converting to strings and back will cause huge overhead and will rule out any GPU-based computation and acceleration. The latter is a must in today's environments, especially on mobiles. You do not want to create this overhead for large arrays of vertices or such and have to parse all the numbers again and again. It can also cause numerical inaccuracies in the conversion that may lead to inconsistencies in the binary representation, which can cause gaps in supposedly closed geometry.

BTW, this is exactly why we have created Xflow: To efficiently be able to specify generic typed data arrays (available as GPU buffers in the engine as early as possible), flexibly composite individual buffers into sets of buffers (<data> elements that are define all the data the input for efficient draw calls), and also to process the data as necessary along the way (e.g. flexible animation, image processing, procedural shading, transitions, etc.).

Xflow is actually much more powerful than routes and it fits much better to HTML5 -- in my opinion at least. Funded by Intel we are just extending Xflow to automatically make use of e.g. SIMD instructions (via
SIMD.js) and other JS acceleration techniques. We are also looking at WebAssembly here for better performance even if not going to the GPU.

Contribution 7

From Andreas Plesch (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004798.html)

Wondering if the x3dom dual-graph page dom structures align with the draft work on Shadow DOM at W3C.
https://www.w3.org/TR/shadow-dom

The shaw-dom spec is being upstreamed to the dom spec.

https://dom.spec.whatwg.org/
 
This is a good resource, since it also explains the web page DOM in detail.

I am not sure if the shaw-dom aligns with the dual graph approach because the shadow-dom is still a dom structure and not a scene graph. I think it may be still necessary to define a x3d dom and interfaces similar to https://developer.mozilla.org/en-US/docs/DOM/DOM_Reference#SVG_interfaces

I'm hoping that John Carlson gets a chance to look at whether his JSON prototype expander might be adaptable as part of x3dom - that would be pretty valuable.  Although Fraunhofer has outstanding prototype support in their Instant Reality engine, it has never been clear why that code can't simply be applied in x3dom as well.  Certainly a consistent approach would seem to make both codebases more coherent and maintainable for them.  If Fraunhofer refuses to adapt or release the Instant Reality prototype code then we will just have to do it ourselves... John's work seems like a big step in that direction
 
Well, I would not have a good idea on how to implement prototypes in x3dom. Not sure if the Instant Reality code would really help. One idea may be to see if there is an existing declarative way to register new custom elements using web components, and extend that. But I do not think there is.
https://www.w3.org/TR/custom-elements/

 
Also thanks for pointing out important questions about Fraunhofer's stewardship of the X3DOM project.  It will be good to learn more about their intentions so that our community can align effectively.  There has been no handoff.

Well, there have been no fixes or updates to the code base on github for a long time and a recent issue response suggested that FHG considers x3dom community supported.
 
Regarding the X3D event system: a changing value of any field in any node can be sent as an input to any field in any node, as long as types strictly match (apples to apples).  Rephrased: a ROUTE passes a time-stamped value from one node to another. Internal to an X3D scene graph, that has been implemented dozens of times.  Seems extremely simple. 

External to an X3D scene graph, meaning via a browser using the Scene Access Interface (SAI), mechanisms are similarly well defined.  Since the DOM is string based, and since any X3D event value can be expressed as a string, it seems like we have a straight connect-the-dots approach awaiting us.  Forgive me for using a four-letter word, but if interested individuals might actually _read_ the HTML5/DOM and X3D specifications, then the answers to most implementation & alignment questions are likely spelled out for us.

https://dom.spec.whatwg.org/#events would be HTML5 side of events.
Is the statement at https://dom.spec.whatwg.org/#action-versus-occurance compatible with http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/concepts.html#Events ?

It will be great to get our wiki organized for clarity and go-forward action on these important X3D version 4 design issues.

Thanks for letting us know about your other explanations - very interesting!  8)

Yeah, VR is not really part of this discussion (although it could be a V4 topic) .
 
Looking forward to continuing, sustainable evolution and progress together.

I think if there is way to open spec. drafting more along the lines of other www spec. efforts on github, there is a chance to attract a wider community for more man power which would be much needed.

References:

• Shadow DOM
• Shaw-DOM spec
• SVG Interfaces
• custom elements
• HTML5 events
• action vs occurrence
• X3D Events

Contribution 8

From John Carlson (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004808.html)

Before we can answer these questions, we need to know who the V4.0 targetted audience is.  Who are V4.0’s users?  Purely web developers and users?  Geo?  Medical?  CAD? H-Anim? Which working groups are involved?

Contribution 9

From Leonard Daly (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004817.html)

    X3D needs to run in the HTML5/DOM environment. A few nodes need to be removed, but all capabilities remain.
    Preliminary proposed V4 document at: http://tools.realism.com/specification/x3d-v40

I am going to start my position with a response to a question asked by John Carlson on a different list (x3dom-users): are we adding HTML5 capabilities to X3D or 3D (X3D in particular) capabilities to HTML5?
HTML is the dominant environment world-wide. It provides text, image, 2D graphics (SVG), video, and other capabilities. The size of the HTML5 development community far exceeds the total of the entire X3D community. Forcing HTML5 into X3D is a losing game right from the start -- whether you want all of HTML5 or just a portion. So in my mind, the only choice with a future is to add 3D to HTML5. Running in the HTML5 environment means full integration with the HTML5 DOM (or later versions when they happen). BTW, there are already a number of non-Web3D Consortium efforts to do so. We are not out in front of the effort and are about to be made irrelevant. There is no more time for delays or debates.
So now that the environment is settled, it is important to identify what in current X3D (V3.3) is incompatible with HTML5. There are only three obvious features - Script node, event handling, and case sensitivity. There are other capabilities that are dependent on these capabilities -- I'll discuss those later.

Starting with the easiest one first - case sensitivity. HTML5 is case insensitive. Relaxing X3D's rules on that allows existing X3D code to run in a browser. If everything gets converted to lower-case prior to handling (except quoted strings), then there is not a problem.

There is an obvious naming incompatibility with Script -- the name. HTML5 is already using that name. Under my initial condition there cannot be an X3D Script node. That does not mean all scripting functions are given up. HTML5 provides a wonderful script interface a more flexible structure. In X3D, Scripts are meant to process events, so the function argument is always an event (except for X3D-Script internal functions). Functions in HTML5 are a lot more flexible and can include events, objects, scalars, arrays, etc. So there is no loss of functionality by giving up X3D-Script.

Event handling is different between HTML5 and X3D. In X3D events are "routed" from one node to another. They allow one part of the scene graph to "talk" to another part. In HTML5, events "bubble-up" from the originator to the event through any handler that may be attached to any parent node of the originator until the event is cancelled. In all of my design work on V4 I have not found any instance where HTML5-Scripts could not provide the same functionality as X3D-Script+ROUTE. It requires a little different mind-set, but the HTML5 mind-set is very familiar to JavaScript programmers and other front-end developers. I also believe that a graphical development interface can be built that completely simplifies the differences.

The biggest issue I have seen with event handling is scene graph updating. X3D updates the scene graph once all non-looping events in the cascade have completed. After the scene graph is updated, a new frame is rendered. This can cause a large delay between rendered frames. HTML5 renders as it goes. Rendering happens asynchronously to changes to the DOM. There is no concept of accessing the DOM before or after all events for that frame. X3D worlds that depend on that feature will probably not be able to be ported to X3D V4.

Summarizing the three incompatibilities - with the exception of some event processing, none will prevent X3D from doing what it currently can do and all can be easily migrated to the HTML5 environment.


There are a number of features that I think should not be included in X3D V4, but these are just features and not fundamental capabilities. These include all nodes that generate geometry (e.g., Extrusion, ElevationGrid, Text) with the exception of simple solids and perhaps a couple of additions. My view here is based on the availability of free modeling software (e.g., Blender) that does all of the above, and a lot more efficiently than X3D can. Also by not including these nodes, the resultant models will look better.

Lastly (for now), I believe that there is no purpose for a PROTO node. Without a X3D-Script node, PROTOs just become convenience generators. To replace that feature, I am proposing a MACRO node that takes X3D and does string substitution prior to inserting the result into the scene graph (and DOM). I have a partial implementation of this for X3DOM. 

Summarizing: The Consortium needs to get out and lead the way for 3D on the web (and this includes VR) or it will be by-passed and left with the relics of history like blinking text, and Flash. The environment must be HTML5/DOM and X3D must stay current with the web environment. There will always be someone who needs something specialized that does not use a web environment, but those will be individual cases and not worth significant volunteer efforts.

Reference:

• V4 preliminary proposal

Contribution 10

From John Carlson (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004818.html)

•   What level of X3D integration into HTML5 do we want?
o      Do we want to be fully integrated like SVG?

In the spirit of openness, I say may the best implementation win. That means that there should not be full integration.  However, there should be a standard.


•   Do we want/need a DOM spec? If so:
o      Which DOM version should it be based on?

HTML5???

o      Do we want to fully support all DOM/HTML features?

We should document what tags and attributes we are adding.   If not DOM, what declarative language do you want to use? CSS may be an option

•   Do we want to maximize the backwards compatibility of V4.0 with V3.3? Or break away completely?
o      Do we want to retain SAI?

Not retain SAI.   Not be backwards compatible.


•   What features do we want? For example,
o      How is animation to be handled? The X3D way of TimeSensor and ROUTEs, or an HTML way, such as CSS3 animations, or else JavaScript?

HTML way

o      How is user interaction to be handled? The X3D way of Sensors, or the HTML way with event handlers?

HTML way and user can add own event emitters.  We can also add JavaScript emitters which can be added as attributes.

o      Do we need any different nodes? One example might be a mesh node?

We need a way to provide templates or macros, which might be provided through picking an existing one.

o      Do we want Scripts and Prototypes in HTML5?

No.


o      How do we want to handle styling?

By declaratively addressing the fragment shader on objects.


•   What profile(s) do we need for HTML?

IDK

Contribution 11

From John Carlson (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004819.html)

X3D needs to run in the HTML5/DOM environment. A few nodes need to be removed, but all capabilities remain.
Preliminary proposed V4 document at: http://tools.realism.com/specification/x3d-v40

I am going to start my position with a response to a question asked by John Carlson on a different list (x3dom-users): are we adding HTML5 capabilities to X3D or 3D (X3D in particular) capabilities to HTML5?
HTML is the dominant environment world-wide. It provides text, image, 2D graphics (SVG), video, and other capabilities. The size of the HTML5 development community far exceeds the total of the entire X3D community. Forcing HTML5 into X3D is a losing game right from the start -- whether you want all of HTML5 or just a portion. So in my mind, the only choice with a future is to add 3D to HTML5. Running in the HTML5 environment means full integration with the HTML5 DOM (or later versions when they happen). BTW, there are already a number of non-Web3D Consortium efforts to do so. We are not out in front of the effort and are about to be made irrelevant. There is no more time for delays or debates.
I think the only DOM items that are required are canvas and script.  These can be moved to X3D if name conflicts are resolved.


So now that the environment is settled, it is important to identify what in current X3D (V3.3) is incompatible with HTML5. There are only three obvious features - Script node, event handling, and case sensitivity. There are other capabilities that are dependent on these capabilities -- I'll discuss those later.

Did you just throw out Cobweb which runs successfully in several browsers?




Starting with the easiest one first - case sensitivity. HTML5 is case insensitive. Relaxing X3D's rules on that allows existing X3D code to run in a browser. If everything gets converted to lower-case prior to handling (except quoted strings), then there is not a problem.

Not a problem.



There is an obvious naming incompatibility with Script -- the name. HTML5 is already using that name. Under my initial condition there cannot be an X3D Script node. That does not mean all scripting functions are given up. HTML5 provides a wonderful script interface a more flexible structure. In X3D, Scripts are meant to process events, so the function argument is always an event (except for X3D-Script internal functions). Functions in HTML5 are a lot more flexible and can include events, objects, scalars, arrays, etc. So there is no loss of functionality by giving up X3D-Script.

Linkage of Scripts with ROUTEs is given up.   Data binding has to be reinvented, or borrowed.  Cobweb has shown that Scripts can be available in HTML5 browsers.



Event handling is different between HTML5 and X3D. In X3D events are "routed" from one node to another. They allow one part of the scene graph to "talk" to another part. In HTML5, events "bubble-up" from the originator to the event through any handler that may be attached to any parent node of the originator until the event is cancelled. In all of my design work on V4 I have not found any instance where HTML5-Scripts could not provide the same functionality as X3D-Script+ROUTE. It requires a little different mind-set, but the HTML5 mind-set is very familiar to JavaScript programmers and other front-end developers. I also believe that a graphical development interface can be built that completely simplifies the differences.

Agreed.



The biggest issue I have seen with event handling is scene graph updating. X3D updates the scene graph once all non-looping events in the cascade have completed. After the scene graph is updated, a new frame is rendered. This can cause a large delay between rendered frames. HTML5 renders as it goes. Rendering happens asynchronously to changes to the DOM. There is no concept of accessing the DOM before or after all events for that frame. X3D worlds that depend on that feature will probably not be able to be ported to X3D V4.
What does WebGL do separate from HTML5?  We really need to find out what WebGL/Canvas does before jumping straight into DOM and HTML5.


Summarizing the three incompatibilities - with the exception of some event processing, none will prevent X3D from doing what it currently can do and all can be easily migrated to the HTML5 environment.


There are a number of features that I think should not be included in X3D V4, but these are just features and not fundamental capabilities. These include all nodes that generate geometry (e.g., Extrusion, ElevationGrid, Text) with the exception of simple solids and perhaps a couple of additions. My view here is based on the availability of free modeling software (e.g., Blender) that does all of the above, and a lot more efficiently than X3D can. Also by not including these nodes, the resultant models will look better.

Lastly (for now), I believe that there is no purpose for a PROTO node. Without a X3D-Script node, PROTOs just become convenience generators. To replace that feature, I am proposing a MACRO node that takes X3D and does string substitution prior to inserting the result into the scene graph (and DOM). I have a partial implementation of this for X3DOM. 

I don’t think we should spend effort generating our our implementation of templates, unless done for efficiency.   Underscore, handlebars, meteor, angular, mustache  etc all present templating frameworks which are likely highly superior to MACROs at this point.  If all we are doing is manipulating DOM, we should use developed frameworks that other developers use, not invent our own templates.  If we are manipulating WebGL, then we should look at the WebGL or GL frameworks for what is done with templating.  Has anyone googled webgl templating or webgl macros?  Is that what we are ultimately doing, providing WebGL macros?


Summarizing: The Consortium needs to get out and lead the way for 3D on the web (and this includes VR) or it will be by-passed and left with the relics of history like blinking text, and Flash. The environment must be HTML5/DOM and X3D must stay current with the web environment. There will always be someone who needs something specialized that does not use a web environment, but those will be individual cases and not worth significant volunteer efforts.

I believe our focus should be on providing a good component library or framework or integrated with an existing DOM templating system(s) as examples.   My favorite is D3.js, but it may be too JavaScript heavy for some users.  D3.js also leverages several extendable data models for tables, trees and graphs.  I think these should be leveraged to show that capabilities can be quickly built up to build apps.  We should make *H-Anim* and geo work well on the web.  My gut is saying don’t build our own templating system.  I think we should use existing data mechanisms (JSON ala meteor templates) to provide data to the templating system.  If we want to provide a way to send XML and WRL to a templating system, this should be investigated, but no one will likely use it (I think everyone has moved to REST/JSON).  If we can’t use X3DOM and Angular together, then perhaps that problem needs to be investigated with the Angular team.

For a not perfect sample of Meteor 1.2’s templating system with X3DOM look at the bottom of the following link.   {{ }} is what gets replaced, and the new tag is <template> and there are loops.

https://github.com/coderextreme/jsondemons/blob/master/client/jsondemons.html

I too believe that DOM should be used as the lingua franca in the short term, and if people need more performance, they can move to WebGL, PlayCanvas and Three.JS.  I think JSON may be superior in speed to DOM however (run some tests?), and we should investigate a WebGL X3D JSON Loader that skips DOM.


http://psteeleidem.com/the-javascript-templating-landscape/

Here’s are the options of things to focus on:

1.  Providing declarative abstractions for the graphics card, since WebGL exposes the graphics card.  This means fragment shader, textures, lighting and materials. (CSS4+images+canvas)
2.  Provide a barebones structure for the scenegraph.  transforms OR groups, but not both. (DOM)
3.  Provide ways to generate simple generic geometry to build objects, like spheres which can be used by shaders, with configurable # of faces.  The surfaces can be manipulated with vertex shaders.  Provide ways to declaratively specify unique geometry (perlin noise, etc.)—Geometry Shader.
4.  Allow for more complex geometry specified by Blender. (SRC)
5.  Provide a way to apply fragment shaders and vertex shaders to objects (DOM).
6.  Provide a way to place and orient objects in the scenegraph.(DOM)
7.  H-Anim and physics.
8.  Geo.
9. VR
10. Mobile
11.  Templates or Macros of the above. (DOM).  I think the only way our own templating system is going to fly is if we apply it to HTML5 and SVG as well.
12.  Proper support for HTML5 events on X3D objects.

Wish List:

13. SVG Extrusions for 3D printing Blind visualization.

This sounds like a lot.   We need to figure out what we already have and apply resources where necessary.

There may be new items to standardize in 1) reflection, refraction, prismatic and DVD effects 3) easy ways to subdivide boxes and spheres (already in X3DOM, I think) 8) great circle interpolator 9)  eye and motion control 10) standardized shader variable names.

Here’s how X3DOM needs to be extended to meet the above focus items:

1. Add a templating system { CHOOSE ONE OR MORE, EVALUATE }
2. Add JSON { MEDIUM EFFORT }
3. Standardized shader variables. { PROBABLY EASY }
4. Access to CSS  from X3DOM (through id, style and class attributes).  { BIG EFFORT }
5. Add VR devices (this should come for free from the browser vendors)


The last 4 of these apply to Cobweb as well.  Perhaps those should be the ones we focus on.  If we want a JSON templating system, we’ll have to do that as well.

References:

• json demons
• JavaScript templating landscape

Contribution 12

From Philipp Slusallek

Let me focus on the list at the end.

> Here’s are the options of things to focus on:
> 
> 1.  Providing declarative abstractions for the graphics card, since 
> WebGL exposes the graphics card.  This means fragment shader, 
> textures, lighting and materials. (CSS4+images+canvas)

The only fully programmable option of I am aware of that does not use the low-level mechanism of glsl is shade.js from our group. It is full integrated into XML3D for rendering as well as Xflow for generic handling of the input data, but it could be used also in other frameworks with some effort. It uses a subset of JS for specifying shaders and offers to compile them into glsl for both both forward and deferred rendering. We also have a backend to ray tracing and globillum renderers.

See http://xml3d.org/papers/ for the paper and XML3D.org for the implementation.

> 2.  Provide a barebones structure for the scenegraph.  transforms OR 
> groups, but not both. (DOM)

This is what I promoted in my email a bit earlier. We are then using webComponents on top of this core, which seems to be the best way forward in the Web technology stack and is really powerful.

> 3.  Provide ways to generate simple generic geometry to build objects, 
> like spheres which can be used by shaders, with configurable # of faces.
>  The surfaces can be manipulated with vertex shaders.  Provide ways to 
> declaratively specify unique geometry (perlin noise, etc.)—Geometry Shader.

Xflow offers exectly this capabilities. We already showed how (in suitable cases) the processing can be fully integrated into the vertex shader.

In a project with Intel we are currently working on extensing the shade.js compiler to also generate code that could, for example, use SIMD.js for processing that is not suitable for the GPU.

> 4.  Allow for more complex geometry specified by Blender. (SRC)

I would be a bit careful here but this should be discussed.

Our approach would be to specify this domain-specific functionality in a WebComponent, whcih would also encapsulate the conversion into renderable geometry based on the input data.

This functionality would not have to be in the core but could be loaded on demand.

> 5.  Provide a way to apply fragment shaders and vertex shaders to 
> objects (DOM).

See shade.js, we even merge the vertex shading code with the other code that comes from the scene description  (light iterators, shadowing, etc.). Shade .js then is able to extract all code int the specific stages automaticalls (using dependency analysis). As a result some code for fragment shading might get pulled into setting uniform variables in JS, if there is no varying data associated with the geometry that requires this code to be in the fragment shader stage.

> 6.  Provide a way to place and orient objects in the scenegraph.(DOM)

> 7.  H-Anim and physics.
> 8.  Geo.
> 10. Mobile

Many of these could be nicely implemented as separate WebComponents.

> 9. VR

WebVR is getting quite mature and we are currently integrating this in XML3D. Seems rather straight forward. Suitable VR authoring is another story, through.

> 11.  Templates or Macros of the above. (DOM).  I think the only way 
> our own templating system is going to fly is if we apply it to HTML5 
> and SVG as well.

WebComponents is the HTML5 native way to here I would say.

> 12.  Proper support for HTML5 events on X3D objects.

Already implemented in XML3D and being integrated into the new common structure.

> 12a: I would add full support for CSS for 3D elements. We have shown
how this should be done in a paper at last years Web3D. There are a few issues with proper support from CSS still waiting to be addressed, though.

> Wish List:
> 
> 13. SVG Extrusions for 3D printing Blind visualization.

Could probably be done as another domain-specific WebComponent on top of the core.

> This sounds like a lot.   We need to figure out what we already have and
> apply resources where necessary.

As you see a lot of the functionality is already available (in the current XML3D and X3DOM) and we are putting a proof of concept to bring both of them together on top of a unified core.

Reference:

• XML3D papers

Contribution 13

From Mike Aratow (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004820.html)

Thank you for that concise, clear and pragmatic summary of a possible bright future for an X3D V4.

From a user and business standpoint, your approach puts X3D out in front and provides a path towards wide adoption and sustainability.

I fully support your strategy.

Contribution 14

From Yves Piguet (http://web3d.org/pipermail/x3d-public_web3d.org/2016-June/004821.html)

Thanks to all those who've expressed their view.

I'm less sure about the need for a close integration between X3D and HTML5 at the DOM level. To me, 3D looks like it's at the same level as other multimedia such as video. The justification for MathML is much stronger, with the need to have inline math and style it with css the same way as surrounding text. Control of X3D from scripts in HTML pages, or pushing information from X3D to HTML, is useful but doesn't require DOM integration. I wouldn't mind it if it wasn't going to break compatibility with v3 in major ways.

Which user case would it address? Why would a new X3D v4 be accepted rather than "modern" 3D approaches if compatibility is broken and deemed useless anyway?