Skip to content

Commit 0c7fff3

Browse files
tharkumtharkum
committed
Add WEBGL_element_texture (#2508)
 * Add new HTMLElementTexture object and new tokens for TEXTURE_ELEMENT_WEBGL texture target.
1 parent d48fd7f commit 0c7fff3

File tree

1 file changed

+298
-0
lines changed

1 file changed

+298
-0
lines changed

‎extensions/proposals/WEBGL_element_texture/extension.xml‎

Lines changed: 298 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,298 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<proposal href="proposals/WEBGL_element_texture/">
3+
<name>WEBGL_element_texture</name>
4+
5+
<contact> <a href="https://www.khronos.org/webgl/public-mailing-list/">WebGL
6+
working group</a> (public_webgl 'at' khronos.org) </contact>
7+
8+
<contributors>
9+
<contributor>Byungseon Shin, LG Electronics</contributor>
10+
11+
<contributor>Andrei Volykhin, LG Electronics</contributor>
12+
13+
<contributor>Mark Callow, HI Corporation</contributor>
14+
15+
<contributor>Members of the WebGL working group</contributor>
16+
</contributors>
17+
18+
<number>NN</number>
19+
20+
<depends>
21+
<api version="1.0"/>
22+
</depends>
23+
24+
<overview>
25+
<p>This extension defines a new object, the <code>HTMLElementTexture</code>,
26+
that can be used to efficiently transfer a sequence of image frames from
27+
a producer which out of control the WebGL into a WebGL texture.
28+
This is done via a new texture target, <code>TEXTURE_ELEMENT_WEBGL</code>
29+
which can only be specified as being the consumer of an image stream from a
30+
producer element.</p>
31+
32+
<p>There is no support for most of the functions that manipulate other
33+
texture targets (e.g. you cannot use <code>*[Tt]ex*Image*()</code>
34+
functions with <code>TEXTURE_ELEMENT_WEBGL</code>). Also,
35+
<code>TEXTURE_ELEMENT_WEBGL</code> targets never have more than a single
36+
level of detail. Because of these restrictions, it is possible to
37+
allow sources which have internal formats not otherwise supported by WebGL,
38+
such as planar or interleaved YUV data to be WebGL texture target siblings.</p>
39+
40+
<p>The extension extends GLSL ES with a new <code>samplerElementWebGL</code> type
41+
and matching sampling functions that provide a place for an implementation
42+
to inject code for sampling non-RGB data when necessary without degrading performance
43+
for other texture targets. Sampling a <code>TEXTURE_ELEMENT_WEBGL</code> via a sampler of
44+
type <code>samplerElementWebGL</code> always returns RGBA data.</p>
45+
46+
<p>Sampling a WebGL texture will return an RGBA vector in the same
47+
colorspace as the source image. If the source image is stored in YUV
48+
(or some other basis) then the YUV values will be transformed to RGB values.
49+
(But these RGB values will be in the same colorspace as the original image.
50+
Colorspace here includes the linear or non-linear encoding of the samples.
51+
For example, if the original image is in the sRGB color space then the RGB value
52+
returned by the sampler will also be sRGB, and if the original image is stored
53+
in ITU-R Rec. 601 YV12 then the RGB value returned by the sampler will be
54+
an RGB value in the ITU-R Rec. 601 colorspace)</p>
55+
56+
<p>Sampling a WebGL texture which is not associated with any
57+
<code>HTMLElementTexture</code> object will return a sample value of (0,0,0,1).</p>
58+
59+
<p>Each <code>TEXTURE_ELEMENT_WEBGL</code> texture object may require up
60+
to 3 texture image units for each texture unit to which it is bound.
61+
The number of texture image units required by a bound texture object
62+
can be queried using <code>getTexParameter</code> with target set
63+
to the texture target in question, value set to
64+
<code>REQUIRED_TEXTURE_ELEMENT_IMAGE_UNITS_WEBGL</code>, and ActiveTexture
65+
set to the texture unit to which the texture object is bound.</p>
66+
67+
<p><code>HTMLElementTexture</code> provides a commands for <em>latching</em> an
68+
image frame into the consuming texture as its contents and retrieving additional
69+
information as a timestamp or a transformation matrix.</p>
70+
71+
<p>This extension does not cover the details of how a <code>HTMLELementTexture</code>
72+
object could be created from a producer element.
73+
Different kinds of producers works differently and described in additional externsion
74+
specifications which will extend this one.
75+
Example of producer element specification:
76+
WEBGL_video_texture</p>
77+
78+
<features>
79+
<glsl extname="WEBGL_element_texture">
80+
<stage type="vertex"/>
81+
82+
<stage type="fragment"/>
83+
84+
<type name="samplerElementWEBGL"/>
85+
86+
<function name="texture2D" type="vec4">
87+
<param name="sampler" type="samplerElementWEBGL"/>
88+
89+
<param name="coord" type="vec2"/>
90+
</function>
91+
92+
<function name="texture2DProj" type="vec4">
93+
<param name="sampler" type="samplerElementWEBGL"/>
94+
95+
<param name="coord" type="vec3"/>
96+
</function>
97+
98+
<function name="texture2DProj" type="vec4">
99+
<param name="sampler" type="samplerElementWEBGL"/>
100+
101+
<param name="coord" type="vec4"/>
102+
</function>
103+
104+
</glsl>
105+
</features>
106+
</overview>
107+
108+
<idl xml:space="preserve">
109+
[
110+
NoInterfaceObject
111+
] interface HTMLElementTexture : EventTarget {
112+
void updateTexImage();
113+
114+
double getTimestamp();
115+
void getTransformMatrix(Float32Array matrix);
116+
117+
double getLastAvailableTimestamp();
118+
119+
attribute EventHandler onframeavailable;
120+
};
121+
122+
[
123+
NoInterfaceObject
124+
] interface WEBGL_element_texture {
125+
const GLenum TEXTURE_ELEMENT_WEBGL = 0x9248;
126+
const GLenum SAMPLER_ELEMENT_WEBGL = 0x9249;
127+
const GLenum TEXTURE_BINDING_ELEMENT_WEBGL = 0x924A;
128+
const GLenum REQUIRED_TEXTURE_ELEMENT_IMAGE_UNITS_WEBGL = 0x924B;
129+
};
130+
</idl>
131+
132+
<!-- new functions -->
133+
134+
<newfun>
135+
<p>On <code>HTMLElementTexture</code>:</p>
136+
137+
<function name="updateTexImage" type="void">
138+
Update the texture image to the most recent frame from the image stream
139+
and release the previously-held frame.
140+
This may cause some frames of the stream to be skipped.
141+
Sampling the <code>WebGLTexture</code>,
142+
that is the <code>HTMLElementTexture</code>'s <em>consumer</em>, will
143+
return values from the latched image.
144+
The image data is guaranteed not to change as long as the image is latched.
145+
</function>
146+
147+
<function name="getTimestamp" type="double">
148+
Retrieve the timestamp associated with the texture image set by the most
149+
recent call to updateTexImage. This timestamp represent the time of frame
150+
relative to start of the producer timeline, the time when
151+
the frame was produced (the Media Stream Counter in OpenML),
152+
not the time when frame was received by application.
153+
This timestamp is in seconds, and is normally monotonically increasing.
154+
The timestamp should be unaffected by time-of-day adjustments and
155+
should be strictly monotonic but for some image producers (media)
156+
may be reset when the position is set.
157+
The specific meaning and zero point of the timestamp depends
158+
on the source providing images to the <code>HTMLELementTexture</code>.
159+
Unless otherwise specified by the image source, timestamps cannot
160+
generally be compared across <code>HTMLElementTexture</code> instances,
161+
or across multiple application invocations. It is mostly useful for
162+
determining time offsets between subsequent frames.
163+
</function>
164+
165+
<function name="getTransformMatrix" type="void">
166+
<param name="matrix" type="Float32Array"/>
167+
Retrieve the 4x4 texture coordinate transform matrix associated with
168+
the texture image set by the most recent call to updateTexImage.
169+
This transform matrix maps 2D homogeneous texture coordinates
170+
of the form (s, t, 0, 1) with s and t in the inclusive range [0, 1]
171+
to the texture coordinate that should be used to sample that location
172+
from the texture. Sampling the texture outside of the range
173+
of this transform is undefined.
174+
The matrix is stored in column-major order.
175+
</function>
176+
177+
<function name="getLastAvailableTimestamp" type="double">
178+
Retrieve the timestamp associated with the most recent texture image
179+
which could be provided by a producer.
180+
</function>
181+
</newfun>
182+
183+
<dl class="methods">
184+
<dt><code class="attribute-name">onframeavailable</code> of type
185+
<code>EventHandler</code></dt>
186+
187+
<dd>The <code>onframeavailable</code> handler is executed when a new frame
188+
could be latched from image stream.</dd>
189+
</dl>
190+
191+
<!-- new tokens -->
192+
193+
<newtok>
194+
<p>On <code>WEBGL_element_texture</code>:</p>
195+
196+
<p>The meaning and use of these tokens is similar as described in <a
197+
href="https://www.khronos.org/registry/OpenGL/extensions/OES/OES_EGL_image_external.txt">OES_EGL_image_external</a>.</p>
198+
199+
<function name="bindTexture" type="void">
200+
<param name="target" type="GLenum"/>
201+
<param name="texture" type="WebGLTexture?"/>
202+
<code>TEXTURE_ELEMENT_WEBGL</code> is accepted as a target by the
203+
<code>target</code> parameter of <code>bindTexture()</code>
204+
</function>
205+
206+
<function name="getActiveUniform" type="WebGLActiveInfo?">
207+
<param name="program" type="WebGLProgram?"/>
208+
<param name="index" type="GLuint"/>
209+
<code>SAMPLER_ELEMENT_WEBGL</code> can be returned in the
210+
<code>type</code> field of the <code>WebGLActiveInfo</code> returned by
211+
<code>getActiveUniform()</code>
212+
</function>
213+
214+
<function name="getParameter" type="any">
215+
<param name="pname" type="GLenum"/>
216+
<code>TEXTURE_BINDING_ELEMENT_WEBGL</code> is accepted by
217+
the <code>pname</code> parameter of <code>getParameter()</code>
218+
</function>
219+
220+
<function name="getTexParameter*" type="any">
221+
<param name="target" type="GLenum"/>
222+
<param name="pname" type="GLenum"/>
223+
<code>REQUIRED_TEXTURE_ELEMENT_IMAGE_UNITS_WEBGL</code> is accepted
224+
as the <code>pname</code> parameter of <code>getTexParameter*()</code>
225+
</function>
226+
</newtok>
227+
228+
<samplecode xml:space="preserve">
229+
230+
<p> This a fragment shader that samples a element texture.</p>
231+
<pre>
232+
#extension GL_WEBGL_element_texture : require
233+
precision mediump float;
234+
varying vec2 v_texCoord;
235+
236+
uniform samplerElementWEBGL uSampler;
237+
238+
void main(void) {
239+
gl_FragColor = texture2D(uSampler, v_texCoord);
240+
}
241+
</pre>
242+
243+
<p>This shows application that renders HTML*Element using proposed extension.</p>
244+
<pre>
245+
var customElement = document.getElementById("...");
246+
var customTexture = gl.createTexture();
247+
var elementTexture;
248+
249+
function initialize() {
250+
var ext = gl.getExtension('WEBGL_element_texture');
251+
if (ext === null)
252+
return;
253+
254+
elementTexture = ext.create*Texture(customElement, customTexture);
255+
elementTexture.onframeavailable = function() { ... };
256+
}
257+
258+
function update() {
259+
gl.bindTexture(ext.TEXTURE_ELEMENT_WEBGL, customTexture);
260+
elementTexture.updateTexImage();
261+
gl.bindTexture(ext.TEXTURE_ELEMENT_WEBGL, null);
262+
}
263+
264+
function render() {
265+
.....
266+
267+
gl.activeTexture(gl.TEXTURE0);
268+
gl.bindTexture(ext.TEXTURE_ELEMENT_WEBGL, customTexture);
269+
270+
.....
271+
272+
gl.drawArrays(gl.TRIANGLE_STRIP, 0, 4);
273+
}
274+
</pre>
275+
276+
<p>Application renders each video frames into WebGL canvas based on game-loop pattern.</p>
277+
<pre>
278+
initialize();
279+
280+
while (true) {
281+
update();
282+
processInput();
283+
render();
284+
}
285+
</pre>
286+
287+
</samplecode>
288+
289+
<tests/>
290+
291+
<issues/>
292+
293+
<history>
294+
<revision date="2017/09/19">
295+
<change>Initial revision.</change>
296+
</revision>
297+
</history>
298+
</proposal>

0 commit comments

Comments
 (0)