Add support for multires SHT and thumbnail previews.

Spherical harmonic transform (SHT) previews are display instead of a solid
background color before the base multires tiles are loaded. If a thumbnail is
provided, it will be displayed instead of the SHT preview. If the thumbnail is
provided as a Base64-encoded string, the SHT preview may be briefly displayed,
since image loading is asynchronous, while the SHT preview is loaded
synchronously.

doc/json-config-parameters.md

@@ -492,6 +492,23 @@ This specifies the size in pixels of the full resolution cube faces the image
 tiles were created from.
 
 
+#### `shtHash` (string)
+
+Specifies the spherical-harmonic-transform-based preview hash. This is rendered
+instead of the background color before the base set of cube faces are loaded.
+
+
+#### `equirectangularThumbnail` (string or HTMLImageElement or ImageData or ImageBitmap)
+
+Specifies a equirectangular preview thumbnail to be rendered instead of the
+background color or SHT hash before the base set of cube faces are loaded. This
+image can either be specified as a Base64-encoded string or as an object that
+can be directly uploaded to a WebGL texture, e.g., `ImageData`, `ImageBitmap`,
+`HTMLImageElement`, `HTMLCanvasElement` objects. If a Base64-encoded string is
+used, the image size should be kept small, since it needs to be loaded with the
+configuration parameters.
+
+
 
 ## Dynamic content specific options
 

doc/sht-hash.md

@@ -0,0 +1,65 @@
+# Spherical harmonic transform hash
+
+This document specifies a spherical harmonic transform (SHT) hash, which is
+intended to be a compact method of encoding a spherical panorama preview. It is
+based on the [BlurHash specification](https://github.com/woltapp/blurhash/blob/master/Algorithm.md)
+for DCT-based 2D image previews.
+
+There are three steps for creating a SHT hash:
+
+1. Calculate real spherical harmonic transform coefficients
+2. Encode with compact binary encoding
+3. Encode binary data as Base-83-encoded string
+
+Spherical harmonics form an orthogonal basis for representing a function on the
+sphere. Combined with coefficients for each harmonic, they can be used to
+represent a frequency-space approximation of such a function, without boundary
+effects. There are multiple normalization conventions for spherical harmonics;
+the $4\pi$ convention is used here. Since JavaScript does not natively support
+complex numbers, real harmonics with separate real sine and cosine coefficients
+are used.
+
+Spherical harmonics, $Y_{\ell m}$, are defined for $\ell \in \mathbb{Z}^+$,
+with $|m| \leq \ell$. For each $Y_{\ell m}$, there is a corresponding
+$f_{\ell m}$ coefficient. When these coefficients are represented in a pair of
+matrices (one for the sine coefficient and one for the cosine coefficients)
+with rows indexed by $\ell$ and columns indexed by $m$, the upper triangle of
+the matrix is zero. Additionally, the $\ell = m = 1$ coefficient in the sine
+matrix is always 1 in the $4\pi$ normalization convention, which is why it is
+used here. Spherical harmonic coefficients are calculated separately for each
+color channel. Once calculated, the sine and cosine coefficients are stored in
+1D arrays using row-first ordering with the upper triangle of the matrices
+excluded. The coefficient arrays also excludes the first row and column of
+the coefficient matrices since their contents are always zero, except for the
+$\ell = m = 1$ sine coefficient, which is always 1 (as previously mentioned).
+The 1D cosine coefficient array is appended to the 1D sine coefficient array,
+for each of the color channels.
+
+The maximum coefficient magnitude is then found across the color channels, and
+this value is used to normalize the coefficients in the range $[-1, 1]$. The
+normalized coefficients are then multiplied by 9 and converted to integers,
+thereby quantizing the coefficients as integer values. These signed integers in
+the range $[-9, 9]$ are then converted to unsigned integers in the range
+$[0, 18]$ by adding 9. For each coefficient, the color channel values are
+packed into a single number in the range $[0, 6859]$ using
+$R \cdot 19^2 + G \cdot 19 + B$. This number is then Base-83-encoded into a
+pair of characters. Color is encoded and decoded using gamma-compressed sRGB
+values, for simplicity.
+
+The final SHT hash string is constructed by combining the Base-83-encoded
+coefficients with a prefix. The first character in the prefix contains the max
+$\ell$ value for the coefficients, encoded as Base 83. For Pannellum, this is
+currently fixed at $\ell = 5$. The next character contains the maximum
+coefficient value, which was used in the normalization. The value is divided by
+255 to normalize it to the range $[0, 1]$. This value is then multiplied by 82
+and quantized as an integer, before being Base-83 encoded. For $\ell = 5$ this
+string is 74 characters in length.
+
+
+## Base 83
+
+A custom Base-83 encoding is used. Values are encoded individually, using one
+or two digits, and concatenated together. Multiple-digit values are encoded in
+big-endian order, with the most-significant digit first.
+
+The character set used is `0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~`.

readme.md

@@ -35,7 +35,7 @@ Configuration parameters are documented in the `doc/json-config-parameters.md` f
 For final deployment, it is recommended that one use a minified copy of Pannellum instead of using the source files in `src` directly. The easiest method is to download the most recent [release](https://github.com/mpetroff/pannellum/releases) and use the pre-built copy of either `pannellum.htm` or `pannellum.js` & `pannellum.css`. If you wish to make changes to Pannellum or use the latest development copy of the code, follow the instructions in the _Building_ section below to create `build/pannellum.htm`, `build/pannellum.js`, and `build/pannellum.css`.
 
 ### Using `generate.py` to create multires panoramas
-To be able to create multiresolution panoramas, you need to have the `nona` program installed, which is available as part of [Hugin](http://hugin.sourceforge.net/), as well as Python with the [Pillow](https://pillow.readthedocs.org/) package. Then, run
+To be able to create multiresolution panoramas, you need to have the `nona` program installed, which is available as part of [Hugin](http://hugin.sourceforge.net/), as well as Python 3 with the [Pillow](https://pillow.readthedocs.org/) and [NumPy](https://numpy.org/) packages. The [pyshtools](https://shtools.github.io/SHTOOLS/) Python package is also recommended. Then, run
 
 ```
 python3 generate.py pano_image.jpg

src/js/libpannellum.js

@@ -36,6 +36,7 @@ function Renderer(container) {
     container.appendChild(canvas);
 
     var program, gl, vs, fs;
+    var previewProgram, previewVs, previewFs;
     var fallbackImgSize;
     var world;
     var vtmps;
@@ -99,6 +100,18 @@ function Renderer(container) {
             gl.deleteProgram(program);
             program = undefined;
         }
+        if (previewProgram) {
+            if (previewVs) {
+                gl.detachShader(previewProgram, previewVs);
+                gl.deleteShader(previewVs);
+            }
+            if (previewFs) {
+                gl.detachShader(previewProgram, previewFs);
+                gl.deleteShader(previewFs);
+            }
+            gl.deleteProgram(previewProgram);
+            previewProgram = undefined;
+        }
         pose = undefined;
 
         var s;
@@ -470,7 +483,10 @@ function Renderer(container) {
             }
 
             // Set parameters for rendering any size
-            gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
+            if (imageType != "cubemap" && image.width <= maxWidth && haov == 2 * Math.PI)
+                gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.REPEAT);
+            else
+                gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
             gl.texParameteri(glBindType, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
             gl.texParameteri(glBindType, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
             gl.texParameteri(glBindType, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
@@ -512,6 +528,124 @@ function Renderer(container) {
             program.nodeCache = [];
             program.nodeCacheTimestamp = 0;
             program.textureLoads = [];
+
+            if (image.shtHash || image.equirectangularThumbnail) {
+                // Create vertex shader
+                previewVs = gl.createShader(gl.VERTEX_SHADER);
+                gl.shaderSource(previewVs, v);
+                gl.compileShader(previewVs);
+
+                // Create fragment shader
+                previewFs = gl.createShader(gl.FRAGMENT_SHADER);
+                gl.shaderSource(previewFs, fragEquirectangular);
+                gl.compileShader(previewFs);
+
+                // Link WebGL program
+                previewProgram = gl.createProgram();
+                gl.attachShader(previewProgram, previewVs);
+                gl.attachShader(previewProgram, previewFs);
+                gl.linkProgram(previewProgram);
+
+                // Log errors
+                if (!gl.getShaderParameter(previewVs, gl.COMPILE_STATUS))
+                    console.log(gl.getShaderInfoLog(previewVs));
+                if (!gl.getShaderParameter(previewFs, gl.COMPILE_STATUS))
+                    console.log(gl.getShaderInfoLog(previewFs));
+                if (!gl.getProgramParameter(previewProgram, gl.LINK_STATUS))
+                    console.log(gl.getProgramInfoLog(previewProgram));
+
+                // Use WebGL program
+                gl.useProgram(previewProgram);
+
+                // Look up texture coordinates location
+                previewProgram.texCoordLocation = gl.getAttribLocation(previewProgram, 'a_texCoord');
+                gl.enableVertexAttribArray(previewProgram.texCoordLocation);
+
+                // Provide texture coordinates for rectangle
+                if (!texCoordBuffer)
+                    texCoordBuffer = gl.createBuffer();
+                gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
+                gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1,1,1,1,1,-1,-1,1,1,-1,-1,-1]), gl.STATIC_DRAW);
+                gl.vertexAttribPointer(previewProgram.texCoordLocation, 2, gl.FLOAT, false, 0, 0);
+
+                // Pass aspect ratio
+                previewProgram.aspectRatio = gl.getUniformLocation(previewProgram, 'u_aspectRatio');
+                gl.uniform1f(previewProgram.aspectRatio, gl.drawingBufferWidth / gl.drawingBufferHeight);
+
+                // Locate psi, theta, focal length, horizontal extent, vertical extent, and vertical offset
+                previewProgram.psi = gl.getUniformLocation(previewProgram, 'u_psi');
+                previewProgram.theta = gl.getUniformLocation(previewProgram, 'u_theta');
+                previewProgram.f = gl.getUniformLocation(previewProgram, 'u_f');
+                previewProgram.h = gl.getUniformLocation(previewProgram, 'u_h');
+                previewProgram.v = gl.getUniformLocation(previewProgram, 'u_v');
+                previewProgram.vo = gl.getUniformLocation(previewProgram, 'u_vo');
+                previewProgram.rot = gl.getUniformLocation(previewProgram, 'u_rot');
+
+                // Pass horizontal extent
+                gl.uniform1f(previewProgram.h, 1.0);
+
+                // Create texture
+                previewProgram.texture = gl.createTexture();
+                gl.bindTexture(glBindType, previewProgram.texture);
+
+                // Upload preview image to the texture
+                var previewImage, vext, voff;
+                var uploadPreview = function() {
+                    gl.useProgram(previewProgram);
+
+                    gl.uniform1i(gl.getUniformLocation(previewProgram, 'u_splitImage'), 0);
+                    gl.texImage2D(glBindType, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, previewImage);
+
+                    // Set parameters for rendering any size
+                    gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.REPEAT);
+                    gl.texParameteri(glBindType, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
+                    gl.texParameteri(glBindType, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
+                    gl.texParameteri(glBindType, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
+
+                    // Pass vertical extent and vertical offset
+                    gl.uniform1f(previewProgram.v, vext);
+                    gl.uniform1f(previewProgram.vo, voff);
+
+                    gl.useProgram(program);
+                };
+                if (image.shtHash) {
+                    previewImage = shtDecodeImage(image.shtHash);
+                    // Vertical extent & offset are chosen to set the top and bottom
+                    // pixels in the preview image to be exactly at the zenith and
+                    // nadir, respectively, which matches the pre-calculated Ylm
+                    vext = (2 + 1 / 31) / 2;
+                    voff = 1 - (2 + 1 / 31) / 2;
+                    uploadPreview();
+                }
+                if (image.equirectangularThumbnail) {
+                    if (typeof image.equirectangularThumbnail === 'string') {
+                        if (image.equirectangularThumbnail.slice(0, 5) == 'data:') {
+                            // Data URI
+                            previewImage = new Image();
+                            previewImage.onload = function() {
+                                vext = 1;
+                                voff = 0;
+                                uploadPreview();
+                            };
+                            previewImage.src = image.equirectangularThumbnail;
+                        } else {
+                            console.log('Error: thumbnail string is not a data URI!');
+                            throw {type: 'config error'};
+                        }
+                    } else {
+                        // ImageData / ImageBitmap / HTMLImageElement / HTMLCanvasElement
+                        previewImage = image.equirectangularThumbnail;
+                        vext = 1;
+                        voff = 0;
+                        uploadPreview();
+                    }
+                }
+
+                // Reactivate main program
+                gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertBuf);
+                gl.vertexAttribPointer(program.vertPosLocation, 3, gl.FLOAT, false, 0, 0);
+                gl.useProgram(program);
+            }
         }
 
         // Check if there was an error
@@ -562,6 +696,10 @@ function Renderer(container) {
             gl.viewport(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight);
             if (imageType != 'multires') {
                 gl.uniform1f(program.aspectRatio, canvas.clientWidth / canvas.clientHeight);
+            } else if (image.shtHash) {
+                gl.useProgram(previewProgram);
+                gl.uniform1f(previewProgram.aspectRatio, canvas.clientWidth / canvas.clientHeight);
+                gl.useProgram(program);
             }
         }
     };
@@ -688,6 +826,41 @@ function Renderer(container) {
             gl.drawArrays(gl.TRIANGLES, 0, 6);
         
         } else {
+            // Draw SHT hash preview, if needed
+            var drawPreview = typeof image.shtHash !== 'undefined'
+            if (drawPreview && program.currentNodes.length >= 6) {
+                drawPreview = false;
+                for (var i = 0; i < 6; i++) {
+                    if (!program.currentNodes[i].textureLoaded) {
+                        drawPreview = true;
+                        break;
+                    }
+                }
+            }
+            if (drawPreview) {
+                gl.useProgram(previewProgram);
+                gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
+                gl.vertexAttribPointer(previewProgram.texCoordLocation, 2, gl.FLOAT, false, 0, 0);
+                gl.bindTexture(gl.TEXTURE_2D, previewProgram.texture);
+
+                // Calculate focal length from vertical field of view
+                var vfov = 2 * Math.atan(Math.tan(hfov * 0.5) / (gl.drawingBufferWidth / gl.drawingBufferHeight));
+                focal = 1 / Math.tan(vfov * 0.5);
+
+                // Pass psi, theta, roll, and focal length
+                gl.uniform1f(previewProgram.psi, yaw);
+                gl.uniform1f(previewProgram.theta, pitch);
+                gl.uniform1f(previewProgram.rot, roll);
+                gl.uniform1f(previewProgram.f, focal);
+
+                // Draw using current buffer
+                gl.drawArrays(gl.TRIANGLES, 0, 6);
+
+                gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertBuf);
+                gl.vertexAttribPointer(program.vertPosLocation, 3, gl.FLOAT, false, 0, 0);
+                gl.useProgram(program);
+            }
+
             // Create perspective matrix
             var perspMatrix = makePersp(hfov, gl.drawingBufferWidth / gl.drawingBufferHeight, 0.1, 100.0);
             
@@ -756,7 +929,7 @@ function Renderer(container) {
                 program.textureLoads.shift()();
 
             // Draw tiles
-            multiresDraw();
+            multiresDraw(!image.shtHash);
         }
         
         if (params.returnImage !== undefined) {
@@ -837,13 +1010,15 @@ function Renderer(container) {
     
     /**
      * Draws multires nodes.
+     * @param {bool} clear - Whether or not to clear canvas.
      * @private
      */
-    function multiresDraw() {
+    function multiresDraw(clear) {
         if (!program.drawInProgress) {
             program.drawInProgress = true;
             // Clear canvas
-            gl.clear(gl.COLOR_BUFFER_BIT);
+            if (clear)
+                gl.clear(gl.COLOR_BUFFER_BIT);
             // Determine tiles that need to be drawn
             var node_paths = {};
             for (var i = 0; i < program.currentNodes.length; i++)
@@ -1408,6 +1583,153 @@ function Renderer(container) {
         canvas.width = Math.round(canvas.width / 2);
         canvas.height = Math.round(canvas.height / 2);
     }
+
+    // Data for rendering SHT hashes
+    var shtB83chars = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~',
+        shtYlmStr = 'Bf[ff4fff|ffff0fffffBo@Ri5xag{Jmdf2+WiefCs@Ll7+Vi]Btag6' +
+        '[NmdgCv=Ho9;Qk;7zWiF_GsahDy:ErE?Mn$5+SkS_AyWiD#-CuJ[Iqp6;Nnx?7*SlE$' +
+        '*BxR@FtPA?Jq+%7:NnF*zAzn?CwIG@Ft-Y9?IrG+vA%w:AzGR?Cx*IF@EuI,nA+$*9%' +
+        'Gu:A#xCR?ByJ-VB-*wA+J**9*ZBv:9%L.QD.*aB.O.v9-MF+$8,O:MG:*OD;a:UB:IO' +
+        ':n9:Q:KJ;#IG=u-KE=Hs:MC?T:IO=wEL?#%FJ@K**FI@Y;HV=pDU?*sCS@S.uCR[m;H' +
+        'p=VDq?*SCs@s.QCt[r:Iw=OEz?#IF$@#*HF%@u:K$;KI+=uEK-=*sCM:?w:M+:HO.;a' +
+        'CU;:%OCn?:z.Q..Ha;.ODv?-yFG$@,$-V;-Hw=+JH*?*lBP:?%%,n=+J*?%GQ:=#NCt' +
+        '?;y++v=%O:=zGt?:xHI,@-u,*z=zX?:wI+@,tEY??%r-$*;xt@,tP=?$qG%[:xn.#-:' +
+        'u$[%qp];xnN?[*sl.y:-r-?yn$^+sks_=yoi:v=*o?;uk;[zoi,_+skh:s@zl[+pi];' +
+        'tkg][xmhg;o@ti^xkg{$mhf|+oigf;f[ff_fff|ffff~fffff',
+        shtMaxYlm = 3.317,
+        shtYlm = [];
+
+    /**
+     * Decodes an integer-encoded float.
+     * @private
+     * @param {number} i - Integer-encoded float.
+     * @param {number} maxVal - Maximum value of decoded float.
+     * @returns {number} Decoded float.
+     */
+    function shtDecodeFloat(i, maxVal) {
+        return Math.pow(((Math.abs(i) - maxVal) / maxVal), 2) * (i - maxVal > 0 ? 1 : -1);
+    }
+
+    /**
+     * Decodes encoded spherical harmonic transform coefficients.
+     * @private
+     * @param {number} val - Encoded coefficient.
+     * @param {number} maxVal - Maximum value of coefficients.
+     * @returns {number[]} Decoded coefficients; one per color channel [r, g, b].
+     */
+    function shtDecodeCoeff(val, maxVal) {
+        var quantR = Math.floor(val / (19 * 19)),
+            quantG = Math.floor(val / 19) % 19,
+            quantB = val % 19;
+        var r = shtDecodeFloat(quantR, 9) * maxVal,
+            g = shtDecodeFloat(quantG, 9) * maxVal,
+            b = shtDecodeFloat(quantB, 9) * maxVal;
+        return [r, g, b];
+    }
+
+    /**
+     * Decodes base83-encoded string to integers.
+     * @private
+     * @param {string} b83str - Encoded string.
+     * @param {number} length - Number of characters per integer.
+     * @returns {number[]} Decoded integers.
+     */
+    function shtB83decode(b83str, length) {
+        var cnt = Math.floor(b83str.length / length),
+            vals = [];
+        for (var i = 0; i < cnt; i++) {
+            var val = 0;
+            for (var j = 0; j < length; j++) {
+                val = val * 83 + shtB83chars.indexOf(b83str[i * length + j]);
+            }
+            vals.push(val);
+        }
+        return vals;
+    }
+
+    /**
+     * Renders pixel from spherical harmonic transform coefficients.
+     * @private
+     * @param {number[]} flm - Real spherical harmonic transform coefficients.
+     * @param {number[]} Ylm - 4pi-normalized spherical harmonics evaluated for ell, m, and lat.
+     * @param {number} lon - Longitude (radians).
+     * @returns {number} Pixel value.
+     */
+    function shtFlm2pixel(flm, Ylm, lon) {
+        var lmax = Math.floor(Math.sqrt(flm.length)) - 1
+
+        // Precalculate sine and cosine coefficients
+        var cosm = Array(lmax + 1),
+            sinm = Array(lmax + 1);
+        sinm[0] = 0;
+        cosm[0] = 1;
+        sinm[1] = Math.sin(lon);
+        cosm[1] = Math.cos(lon);
+        for (var m = 2; m <= lmax; m++) {
+            sinm[m] = 2 * sinm[m - 1] * cosm[1] - sinm[m - 2];
+            cosm[m] = 2 * cosm[m - 1] * cosm[1] - cosm[m - 2];
+        }
+
+        // Calculate value at pixel
+        var expand = 0,
+            cosidx = 0;
+        for (var i = 1; i <= lmax + 1; i++)
+            cosidx += i;
+        for (var l = lmax; l >= 0; l--) {
+            var idx = Math.floor((l + 1) * l / 2);
+            // First coefficient is 1 when using 4pi normalization
+            expand += idx != 0 ? flm[idx] * Ylm[idx - 1] : flm[idx];
+            for (var m = 1; m <= l; m++)
+                expand += (flm[++idx] * cosm[m] + flm[idx + cosidx - l - 1] * sinm[m]) * Ylm[idx - 1];
+        }
+
+        return Math.round(expand);
+    }
+
+    /**
+     * Renders image from spherical harmonic transform (SHT) hash.
+     * @private
+     * @param {string} shtHash - SHT hash.
+     * @returns {ImageData} Rendered image.
+     */
+    function shtDecodeImage(shtHash) {
+        if (shtYlm.length < 1) {
+            // Decode Ylm if they're not already decoded
+            var ylmLen = shtYlmStr.length / 32;
+            for (var i = 0; i < 32; i++) {
+                shtYlm.push([]);
+                for (var j = 0; j < ylmLen; j++)
+                    shtYlm[i].push(shtDecodeFloat(shtB83decode(shtYlmStr[i * ylmLen + j], 1), 41) * shtMaxYlm);
+            }
+        }
+
+        // Decode SHT hash
+        var lmax = shtB83decode(shtHash[0], 1)[0],
+            maxVal = (shtDecodeFloat(shtB83decode(shtHash[1], 1), 41) + 1) * 255 / 2,
+            vals = shtB83decode(shtHash.slice(2), 2),
+            rVals = [],
+            gVals = [],
+            bVals = [];
+        for (var i = 0; i < vals.length; i++) {
+            var v = shtDecodeCoeff(vals[i], maxVal);
+            rVals.push(v[0]);
+            gVals.push(v[1]);
+            bVals.push(v[2]);
+        }
+
+        // Render image
+        var lonStep = 0.03125 * Math.PI;
+        var img = [];
+        for (var i = 31; i >= 0; i--) {
+            for (var j = 0; j < 64; j++) {
+                img.push(shtFlm2pixel(rVals, shtYlm[i], (j + 0.5) * lonStep));
+                img.push(shtFlm2pixel(gVals, shtYlm[i], (j + 0.5) * lonStep));
+                img.push(shtFlm2pixel(bVals, shtYlm[i], (j + 0.5) * lonStep));
+                img.push(255);
+            }
+        }
+        return new ImageData(new Uint8ClampedArray(img), 64, 32);
+    }
 }
 
 // Vertex shader for equirectangular and cube

utils/multires/Dockerfile

@@ -5,8 +5,9 @@ FROM ubuntu:20.04
 
 ENV DEBIAN_FRONTEND noninteractive
 RUN apt-get update && apt-get install -y --no-install-recommends \
-    python3 python3-dev python3-pil hugin-tools \
+    python3 python3-dev python3-numpy python3-pip python3-pil hugin-tools \
  && rm -rf /var/lib/apt/lists/*
+RUN pip3 install pyshtools
 
 ADD generate.py /generate.py
 ENTRYPOINT ["python3", "/generate.py"]

utils/multires/generate.py

@@ -1,7 +1,8 @@
 #!/usr/bin/env python3
 
-# Requires Python 3.2+ (or Python 2.7), the Python Pillow package,
-# and nona (from Hugin)
+# Requires Python 3.2+, the Python Pillow and NumPy packages, and
+# nona (from Hugin). The Python pyshtools package is also needed for creating
+# spherical-harmonic-transform previews (which are recommended).
 
 # generate.py - A multires tile set generator for Pannellum
 # Extensions to cylindrical input and partial panoramas by David von Oheimb
@@ -35,6 +36,9 @@ import math
 import ast
 from distutils.spawn import find_executable
 import subprocess
+import base64
+import io
+import numpy as np
 
 # Allow large images (this could lead to a denial of service attack if you're
 # running this script on user-submitted images.)
@@ -47,6 +51,57 @@ except KeyError:
     # Handle case of PATH not being set
     nona = None
 
+
+genPreview = False
+try:
+    import pyshtools as pysh
+    genPreview = True
+except:
+    sys.stderr.write("Unable to import pyshtools. Not generating SHT preview.\n")
+
+def img2shtHash(img, lmax=5):
+    '''
+    Create spherical harmonic transform (SHT) hash preview.
+    '''
+    def encodeFloat(f, maxVal):
+        return np.maximum(0, np.minimum(2 * maxVal, np.round(np.sign(f) * np.sqrt(np.abs(f)) * maxVal + maxVal))).astype(int)
+
+    def encodeCoeff(r, g, b, maxVal):
+        quantR = encodeFloat(r / maxVal, 9)
+        quantG = encodeFloat(g / maxVal, 9)
+        quantB = encodeFloat(b / maxVal, 9)
+        return quantR * 19 ** 2 + quantG * 19 + quantB
+
+    b83chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~"
+
+    def b83encode(vals, length):
+        result = ""
+        for val in vals:
+            for i in range(1, length + 1):
+                result += b83chars[int(val // (83 ** (length - i))) % 83]
+        return result
+
+    # Calculate SHT coefficients
+    r = pysh.expand.SHExpandDH(img[..., 0], sampling=2, lmax_calc=lmax)
+    g = pysh.expand.SHExpandDH(img[..., 1], sampling=2, lmax_calc=lmax)
+    b = pysh.expand.SHExpandDH(img[..., 2], sampling=2, lmax_calc=lmax)
+
+    # Remove values above diagonal for both sine and cosine components
+    # Also remove first row and column for sine component
+    # These values are always zero
+    r = np.append(r[0][np.tril_indices(lmax + 1)], r[1, 1:, 1:][np.tril_indices(lmax)])
+    g = np.append(g[0][np.tril_indices(lmax + 1)], g[1, 1:, 1:][np.tril_indices(lmax)])
+    b = np.append(b[0][np.tril_indices(lmax + 1)], b[1, 1:, 1:][np.tril_indices(lmax)])
+
+    # Encode as string
+    maxVal = np.max([np.max(r), np.max(b), np.max(g)])
+    vals = encodeCoeff(r, g, b, maxVal).flatten()
+    asstr = b83encode(vals, 2)
+    lmaxStr = b83encode([lmax], 1)
+    maxValStr = b83encode(encodeFloat([2 * maxVal / 255 - 1], 41), 1)
+    return lmaxStr + maxValStr + asstr
+
+
 # Subclass parser to add explaination for semi-option nona flag
 class GenParser(argparse.ArgumentParser):
     def error(self, message):
@@ -90,6 +145,8 @@ parser.add_argument('-q', '--quality', dest='quality', default=75, type=int,
                     help='output JPEG quality 0-100')
 parser.add_argument('--png', action='store_true',
                     help='output PNG tiles instead of JPEG tiles')
+parser.add_argument('--thumbnailsize', dest='thumbnailSize', default=0, type=int,
+                    help='width of equirectangular thumbnail preview (defaults to no thumbnail; >512 not recommended)')
 parser.add_argument('-n', '--nona', default=nona, required=nona is None,
                     metavar='EXECUTABLE',
                     help='location of the nona executable to use')
@@ -99,6 +156,7 @@ parser.add_argument('-d', '--debug', action='store_true',
                     help='debug mode (print status info and keep intermediate files)')
 args = parser.parse_args()
 
+
 # Create output directory
 if os.path.exists(args.output):
     print('Output directory "' + args.output + '" already exists')
@@ -229,6 +287,22 @@ if not args.debug:
         if os.path.exists(os.path.join(args.output, face)):
             os.remove(os.path.join(args.output, face))
 
+# Generate preview (but not for partial panoramas)
+if haov < 360 or vaov < 180:
+    genPreview = False
+if genPreview:
+    # Generate SHT-hash preview
+    shtHash = img2shtHash(np.array(Image.open(args.inputFile)))
+if args.thumbnailSize > 0:
+    # Create low-resolution base64-encoded equirectangular preview image
+    img = Image.open(args.inputFile)
+    img = img.resize((args.thumbnailSize, args.thumbnailSize // 2))
+    buf = io.BytesIO()
+    img.save(buf, format='JPEG', quality=75, optimize=True)
+    equiPreview = bytes('data:image/jpeg;base64,', encoding='utf-8')
+    equiPreview += base64.b64encode(buf.getvalue())
+    equiPreview = equiPreview.decode()
+
 # Generate config file
 text = []
 text.append('{')
@@ -252,6 +326,10 @@ if args.autoload:
     text.append('    "autoLoad": true,')
 text.append('    "type": "multires",')
 text.append('    "multiRes": {')
+if genPreview:
+    text.append('        "shtHash": "' + shtHash + '",')
+if args.thumbnailSize > 0:
+    text.append('        "equirectangularThumbnail": "' + equiPreview + '",')
 text.append('        "path": "/%l/%s%y_%x",')
 text.append('        "fallbackPath": "/fallback/%s",')
 text.append('        "extension": "' + extension[1:] + '",')

utils/multires/readme.md

@@ -16,11 +16,14 @@ or [Docker](https://www.docker.com/) can be used to avoid this installation.
 ### Option 1: with local dependencies
 
 The `generate.py` script depends on `nona` (from [Hugin](http://hugin.sourceforge.net/)),
-as well as Python with the [Pillow](https://pillow.readthedocs.org/) package. On Ubuntu,
-these dependencies can be installed by running:
+as well as Python 3 with the [Pillow](https://pillow.readthedocs.org/) and
+[NumPy](https://numpy.org/) packages. The [pyshtools](https://shtools.github.io/SHTOOLS/)
+Python package is also recommended. On Ubuntu, these dependencies can be
+installed by running:
 
 ```bash
-$ sudo apt install python3 python3-pil hugin-tools
+$ sudo apt install python3 python3-pil python3-numpy python3-pip hugin-tools
+$ pip3 install --user pyshtools
 ```
 
 Once the dependencies are installed, a tileset can generated with: